Is this why Data Merge PDFs are “throttled back”?

Some time ago, I wrote an article about the difference between Data Merge PDF Export vs regular PDF export. The article highlighted the difference between the two PDF exports, but not why the data merge PDF export appears to have been throttled. Admittedly, this seemed more of a curiosity than anything else, and no further research was undertaken.

dmmystery001

The Data Merge PDF export dialog box. The yellow highlight shows items that can’t be checked.

In October 2018, the issue was once again raised by a reader who asked about preparing tagged PDFs via the Data Merge palette for the purposes of Section 508 accessibility – a way tagging a PDF so that PDF-reading software with accessibility features can assist users with limited accessibility in various ways, such as the example of tagging an image with a description that can be read aloud for users with little- or no vision.

As I couldn’t offer an explanation, all I could do was point to my previous article and submit a request to the InDesign Uservoice team to allow the Export to PDF to have all features of a regular PDF.

Then one month later, I fielded a query on the Adobe InDesign Forums concerning Interactive PDF and Data Merge Conflict where a user had hyperlinks in an interactive PDF where data merge fields were present, even though they weren’t linking to anything.

dmmystery002

Notice the tooltip displayed by the name “Zolly” – text that was a Data Merge text field.

That reminded me of an article from InDesignSecrets about Ghost Hyperlinks explaining that data merge fields can also be shown in the hyperlinks panel, meaning merge fields are hyperlinks themselves.

dmmystery003

The highlighted field in the art is the data merge field, as well as hyperlink 2.

However, Data Merge can’t export to interactive PDF from the data merge panel, so the only way I could replicate the forum poster’s issue was to prepare an InDesign data merge file, navigate to a relevant record,  then go to the file menu and export an Interactive PDF of the visible record on the page.

It was at that moment it occurred to me that THIS may be the reason that the hyperlinks checkbox is greyed out from the “export to PDF” dialog box in the data merge palette – because the merge fields are – in effect – hyperlinks themselves, and having hyperlinks enabled would also make the variable text clickable in the resulting PDF, despite having no actual link to go to. It would also explain why data merge does not offer an “export to interactive PDF” option from the data merge palette.

So, despite having a “eureka” moment of solving why the exports were different, it also made me realise that – without overhauling the way data merge works in InDesign – it may be unlikely to directly export PDFs with hyperlinks or interactivity via the Data Merge Panel anytime soon.

It is worth noting that if the data merge is exported to an InDesign file first (as explained in my workaround linked at the start of the article), and then exported to an interactive PDF, these issues do not occur. However, it is double-handling.

Still, this is frustrating in a world moving online, particularly for marketers who want to prepare unique PDFs that contain:

  • Hyperlinks, not just for navigating to URLs, but within the PDF itself, such as footnotes or page navigation;
  • Interactivity such as form fields for the purposes of surveys and feedback; and
  • Accessibility, not only from the legal standpoint of Section 508 conformance, but the genuine desire to engage with people with limited accessibility.

On that note, isn’t it about time to update the data merge feature so that exports are no longer limited to throttled-back PDFs or InDesign files as their only options? As an addition, how about the ability to print a merge directly from a data merge file, a feature that has been available in Microsoft Word for nearly thirty years.

Or what about the ability to merge to uniquely named files based on each record, something that I’ve already prepared a script for but would welcome as part of the InDesign user interface. And why stop at full-throttle PDFs… how about image formats such as jpg or png, or other formats such as html? Statistics from an InDesignSecrets poll show that print PDFs account for just under half of the participants’ usual file output exports.

Advertisements

Data Merging to specific pages

I was recently faced with the following brief:

I have a database that need to be run through the Data Merge to Single Record script. Some of them need only page 1 and some need page 2. Is there a way to add another column in the database and let InDesign know if it should export page 1 or 2? 

My original thoughts were that this could not be done, but then I realised there is a new feature added in Adobe InDesign CC 2018.1 to the PDF export: Create Separate PDF Files.

mergemerge001

I decided to create a new PDF export style and ensured that the Create Separate PDF Files checkbox was turned on, and make sure that the numbering is based on page numbers.

mergemerge002

Wondering if this would work, I created the following scenario – thank you graphics to be emailed to contributors to this site, based upon three available contribution values. The InDesign file is 3 pages long, has two different page sizes, and is linked to a Data Merge of 1,000 records that has – among its fields – a field called “Variable” that has one of 3 values in the field – 5, 10 or 25. Page 1 is the five dollar campaign, page 2 is the ten dollar campaign, and page 3 is the twenty five dollar campaign.

mergemergesplash

For the sake of this demonstration, I’m going to try with records 1-25. Running the Data Merge to Single Record script available from the downloads or scripts pages of this site, I make sure that the variable field is part of the filename, but ensure that it is the last part of the filename:

mergemerge003

I let the script run, and then once complete, the Results folder appears as follows:

mergemerge004

So right now, there are myriad folders and files, so this looks like a complete mess and failure. However, using the search function at the top right of the finder window, I’m able to find what I’m after. I’d like to find all of the five dollar PDFs first, so I type in 5_01.pdf into the search field, ensuring I’m searching on the Result folder. Why 5_01.pdf? Well, the 5 was the variable in the variable field, and 01 was the page number associated in the InDesign file to the five dollar campaign.

mergemerge005

So this found 11 records. I’ll move those records into a folder elsewhere and look for the other PDFs in a similar fashion: 10_02.pdf and 25_03.pdf

I now have all the records required, and can remove the remaining records from the Results folder, so that’s great for creating single page PDFs by using one InDesign file for multiple campaigns.

Modifying the technique for JPG or PNG

If I want the files as JPG or PNG instead, I could use the Data Merge to Single Record script and merge to InDesign files without splitting the files into single records, and then use Peter Kahrel’s Batch Convert script to export to JPG or PNG.

It is then possible to perform a similar search that was used to separate the earlier campaigns.

Doing the modified technique in less steps

Or, if you want to run one script instead of two, perhaps consider having a look at the Data Merge to Single Records PRO version:

mergemerge010

Run the script and then perform the search that was used to separate the earlier campaigns.

If you’re interested in the Data Merge to Single Records PRO version, please contact me via the contact page.

Another Data Merge script: Data merge to batches

This latest script compensates for a feature that should be available in InDesign’s Data Merge feature, but simply doesn’t work.

The problem:

Take for example the following Data Merge file where we want to export a custom range, but prepare PDFs in batches of 50s for production purposes. Each record is one page long.

dmlots001

When the Data Merge feature is used in Adobe InDesign, it is possible to merge to several new InDesign files that contain a maximum amount of records per file. The same dialog box is present when merging directly to PDF as well.

dmlots002

Unfortunately, the Record Limit per Document checkbox may as well be there for decoration, because it doesn’t work. Instead of 13 PDFs being created with the maximum size of each PDF being 50 records long, a PDF the size of the complete merge file is created.

dmlots003

The workaround

It is possible to split the document into ranges 50 pages per PDF, but it has to be done in Acrobat. From here, click the Organize Pages button.

dmlots004

This will show the Organize Pages toolbar.

dmlots005

From here, click on the split icon to show the split pages portion of the toolbar.

dmlots005

With the split pages toolbar now visible, choose the appropriate split by dropdown and edit the amount required. In this case, we need “number of pages” and 50 pages.

dmlots005b

The files now need to be saved somewhere. Click the Output Options button to show the Output Options dialog.

dmlots005c

Select a destination for the files and any additional filenaming information and click OK.

dmlots006

Once back at the regular toolbar, click the big blue split button.

dmlots005

The task will run and then present a dialog box once it is complete.

dmlots007

On an example such as the one demonstrated, this isn’t an onerous task. However, if working in a production setting where PDF page lengths can be tens of thousands or longer, this is inconvenient and unacceptable.

The solution

Rather than use the Adobe Acrobat solution, I would prefer that the original dialog box worked correctly. One option is to let the Adobe InDesign Engineers know that it should be fixed. A link to the direct request can be found here.

Until it is fixed, I’ve written a free script specifically for this purpose.

dmlots008

Using the earlier example, the same settings will be keyed in.

dmlots009

Once keyed in, click OK. A progress bar will let you know how the merge is going.

dmlots010

An alert will let you know when the merge is complete. The files are then saved to the selected destination with no extra splitting required.

dmlots011

Interested in this script? It can be downloaded from here.

Of interest: New Data Merge techniques and quote bug

In the latest Colecandoo Youtube episode, four Data Merge specific features are covered, namely:

  • Adding faux-returns to a data merge field to split over lines, and subsequent limitations of this technique;
  • Using GREP styles to swap a character for a glyph during a Data Merge;
  • Highlighting field codes so that they are easier to see when not showing live data; and
  • A bug that occurs when a double-quote is at the start of any field in a Data Merge text file.

Faux returns within a field

The faux-returns technique is written about elsewhere, so rather than spoil their presentations, please read the articles directly from the appropriate sources:

I’m a fan of this trick, but emphasise that this is a workaround rather than a long-term solution, given that formatting is limited and there are more appropriate ways of accomplishing this task such as dedicated plug-ins or an XML workflow.

Swap characters for glyphs

Daniel Solis also features a clever trick to swap phrases with glyphs during a Data Merge that uses both GREP styles and ligatures. Again, rather than simply repeat the technique, please see his original video here.

A similar method can be employed using Indiscripts’ Indyfont script, but rather than swapping phrases with glyphs, will swap single characters.

Highlight Field Codes

The video also shows a method for highlighting field-codes when Data Merge is not in the preview mode. It relies on the [Basic Paragraph] style using a GREP style that contains a large highlight, and that any other styles in the document are based off of the [Basic Paragraph] style. It also means the document has to be styled correctly.

I’d demonstrated this technique following a real-world example of a live file where fields were very hard to see, and the file had to have an offset shell printed with variable data printed afterwards, so making sure the shell had no variable data on it was crucial. Using this technique would make finding the field codes much easier to see.

Double Quote bug

Also featured in the video was an issue that once again arose from a real-world example where a customer had provided a database that had double-quotes at the beginning of fields, but no closing quotes, resulting in rather unusual results.

 

%d bloggers like this: