I've been asked to do a feasibility study to dynamically generate PDFs running into thousands of pages; photo-heavy, lots of tables etc. No-matter how you dissect it; this is a pretty hefty task.

So I'm wondering what approach would be best? My initial thought is to break the document down into smaller chunks then join them back together using pdftk but I realise that I have a few issues:

* Page Numbers being reset
* Table of Contents not functioning (how could you link to a page that doesn't exist?)
* Visible "joins" (where the end of each pdf chunk has whitespace where content would normally reside)

The list goes on, but these ones explain the ilk of issue.

Is there a best practice for this conundrum?

Have you tried just doing it in Prince XML? Have plenty of RAM available.

Jim Albright
Wycliffe Bible Translators

I have indeed!

Using the free version of prince it runs into memory issues. after 30 minutes of processing I get my document; I need to be able to monitor this to make sure it hasn't crashed. I was thinking I could run a chunk, validate it then do another pass. This way I could see if it's functioning OK...

The free version is equal to the paid version except for the displayed logo. The logo disappears when you print. I would suggest monitoring the log file. You have several options for displaying errors/warnings.

The other thing to do is get the latest, fastest computer. That is a straight forward way of making things faster but I am assuming you have that if you are working on LARGE documents.

Prince is just a solid piece of software and improving with every release. The last crash of Prince was caused by illegal characters in the document but that is now trapped in version 9.2.

Consider just taking a coffee break and don't worry about the software. If you can get Prince to crash, rest assured that the Prince team takes that quite personally and will get if fixed ASAP.

Jim Albright
Wycliffe Bible Translators

Are you running Prince on a 64-bit machine with plenty of RAM?

Tried it on a 64-bit machine with 1.7Gb of RAM... should we be looking for more?

I like the idea of chunking the process, keeping RAM low, appending to a file and giving progress feedback - speed is not really a concern, but a resource-hog would be!

You will be better off with a newer computer. Mine is two years old 64 byte, 8Gb of RAM and quad core. And mine was not a super expensive computer. Since your time is worth something, just figure that you could figure out how to do the chunking in a month, but how much is a month of you time worth?

When you are asking to do really difficult tasks, get the best tool possible.

Jim Albright
Wycliffe Bible Translators

Jim has a good point, 8GB RAM is about $100, and makes a great christmas present.

I've done 500 pages of A4 300dpi images, and 2,000 pages of complex cross-referenced A4 content, and never noticed anything other than proportionate speed decreases.

Now, trying to actually read those documents on an iPad, or get them printed ... that's another matter entirely ☺

FTR, I compiled a 2,000 page PDF in 1 minute 50 seconds, from a 32MB .html file. And it wasn't dummy content.

That times seems reasonable to me. Watch as computers get faster. So what processor speed/configuration and memory?

Jim Albright
Wycliffe Bible Translators