Hi, we're using Prince to generate high quality user configurable books.

During development it has been working fine (and providing beautiful output!), but when testing on the full data set we are encountering an error:

prince: internal error: incorrect_stream_length(1247746, -4293719550)

2^32 - 4293719550 = 1247746, so we're wondering if we're up against a hard limit here – the numbers seem to indicate a rollover on a 32-bit / 4Gb addressing limitation – or if there is anything we could / should try, like running on a system with more memory?

Yes this is a rare privilege, perhaps the first time we have ever seen this error in the wild!

Have your PDF files been approaching 2GB in size? If so that is really quite impressive, although I suppose embedding a few thousand large images will get you there.

The simplest workaround would be to use a 64-bit Prince package.

I love it when your already have a solution.

Jim Albright
Wycliffe Bible Translators

Thank you for the quick response – we'll check on Monday if we are running the 32-bit version and upgrade.

We're building an A4 3000 page book with quite a few double page spreads at 200DPI so we've been expecting brittleness to emerge at some point. Fingers crossed!

It turns out we're running the 64-bit version on our server, file gives:

/usr/lib/prince/bin/prince: ELF 64-bit LSB  executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=27b4d3102a2442b397b6cf1950f5a4c62f5931fa, stripped

and:

dpkg --list | grep prince
ii  prince                                 10r7-1                              amd64        formatter for converting XML and HTML into PDF

We're also getting the exact same error locally under OS X with the 64-bit version.

It consumes 9Gb of assets, starts building the PDF then falls over with the internal error: incorrect_stream_length above after a couple of minutes. Locally when running on OS X we see approx 4Gb on disk when this happens.

Would be very grateful for any suggestions. If you want to look into this specifically we can provide the html document that causes the behaviour.

Oops, it appears that deep in the code there is an assumption that PDF files will not exceed 4GB in size, even on 64-bit platforms. We will need to fix this ASAP!

Fantastic! Thank you for looking into this.

I think you will like the end result, it's pretty comprehensive. Would be a pity if we had needed to put artificial constraints on it. We don't know of any other solution that can create the requisite quality so it's great that you located this.

Do you expect it to be a quick fix?

We should have an updated build available in a couple of days. I haven't been able to reproduce the incorrect_stream_length error, but I have seen PDF files over 4GB in size generated with a garbled cross reference table due to length overruns. This is now fixed, and hopefully that should be sufficient.

Excellent, will we see a new build appear in the list of downloads? You can also get in touch with me on email if you want the URL to validate the fix with.

This issue should now be fixed in the 64-bit latest builds.

@mikeday Since you noted how rare it is to see this error "in the wild", I thought you might like to know that we also encountered this error (because we were running a version of Prince before you fixed it) and that after updating to Prince 12 it ran successfully with a final file size of 7.6 GB and a page count of 79,619. Generated from a 159 MB HTML file.

Prince deserves immense kudos in my opinion for being able to handle something that size.

(And yes, I don't think a PDF this size makes any sense - but it's for a customer, and that's what they say they want).

That's great! I just hope it doesn't get printed out on paper too often!