Data Merge to Uniquely-Named INTERACTIVE PDFs

In this episode of Colecandoo, I’ll demonstrate several ways to data merge to uniquely named interactive PDFs. The first method uses the data merge to single records script that I released in 2015 and can be downloaded here.

myscript

This demonstration features an InDesign file that is a survey for a package tour company. It contains form elements such as check boxes, radio buttons, a combo box, text box and a submit button. It is also a Data Merge document and contains two text fields within the first paragraph.

surveyitself

With my script, this should be a simple task, but as I click on the PDF export preset dropdown, I notice that I don’t have an option for interactive PDF. Why is this? Well put simply, the script works by calling upon the two ways that a Data Merge can normally be exported – to a newly merged InDesign file, or to a PDF.

As described on Colecandoo before, PDF export from Data Merge is neither a print PDF nor interactive, but it’s own style. Read the full article here.

Method One

But I said it could be done, so what’s the trick? Ultimately, we have to run my script to merge to InDesign files first, and once the folder of InDesign files is generated, use another script from Peter Kahrel, namely BatchConvert.

batchprocess

This script is an amazing utility created by Peter Kahrel that I have written about for InDesignSecrets. It takes a folder of InDesign files and can convert them to a variety of formats, including – for our purposes – interactive PDF. Simply point the script to the folder of InDesign files that were made initially, then point the script to a folder where the interactive files should save save to. Choose the output option as PDF interactive, and then run the script. That’s the first way.

Method Two

The second method is identical to the first method in that files are initially merged to InDesign files, and again uses the batch convert script. The difference is that rather than export to PDF interactive, files remain as InDesign files. Instead, there is a checkbox at the bottom of the user interface that allows another script to run during the batch. From here, I’m going to choose a script I’ve written for this express purpose – it will create an interactive PDF with the same name as the ID file but will save it to a folder called interactive PDFs on my desktop. So that’s the second method.

exportoption

Method Three

The third method demonstrates a sneak-peek at the PRO version of the data merge to unique names script.

proscript

The interface doesn’t look too much different to the previous script, with one exception – the option to run a script during an InDesign export. From this new option in the user interface, simply select the script that I used in method two. Choose some fields for the filenames, the range, and click OK. That’s the third method.

Method Four

The last method demonstrates a sneak-peek at another alternate version of the data merge to unique names script. Unlike the other methods shown, this method is by far the most direct, as it adds “PDF interactive” directly to the user interface.

extscript

To accomplish this task, choose the save location, choose the “PDF interactive” radio button, choose some fields for the filenames, the range, and click OK. That’s the fourth method.

Sidenote about Document Fonts

One issue not addressed in the video is the issue of potential font substitution while creating the interactive PDFs. This comes about because all four techniques rely on creating an InDesign file first that is removed from the original merge file, and may not have access to the fonts used by the original merge file. I’m running Extensis Suitcase font management software so I know the fonts will always be active until I turn them off, but for those relying on other solutions such as the Document Fonts folder, beware of this issue. I’ve written about this for InDesignSecrets.

An added bonus

One thing about the PDFs made during the demonstration was that the text in the dropdown field didn’t suit the formatting of the survey. Formatting of text-related form fields can’t really be controlled in InDesign except for the point size. However, I’ve made an Acrobat Action that I can run not just to this file, but all files in a folder. This action will convert the font in the text and combo boxes to Helvetica and make them 12 point. It’s worth noting that while it’s possible to change the font to whatever is on your system, other users may not have those fonts, so be conscious about this before using the action. Helvetica, Times, Symbol and Courier are present in Adobe Acrobat.

I’ve made this Acrobat Action available from my downloads page as well.

For those after a more robust solution, perhaps consider Form Magic from ID-Extras.

So there you have it, four ways to create uniquely named interactive PDFs from Adobe InDesign. If you’re interested in purchasing the upgraded versions of the data merge to unique names scripts shown in this video, contact me directly via my contact page.

Data Merging into the same Text Frame

From time to time, I hear the question along the lines of “how to I get Data Merge to put the next records in the same textframe during an InDesign Data Merge?” Well, if you’re expecting to do this procedure in one step, you are going to be disappointed because there is no one-click, turnkey solution to do this in Adobe InDesign itself.

That does not mean that accomplishing this task is impossible, in fact it can be done, and in Episode 16 of the Colecandoo Youtube videos, I outline several methods of how to accomplish this task, such as:

  • Merge in Microsoft Word beforehand and import into InDesign as a regular text import;
  • Merge in InDesign using the multiple record feature and use one of three scripts to re-thread the text so that it can be combined into one textframe;
  • Use a script from Ozalto that performs this task with almost no effort;
  • Venture into the rabbit-hole that is XML publishing; or
  • Consider one of the half-dozen paid plug-ins that are dedicated to the production of catalogues.

The video demonstrates several scripts that are absolutely essential for this task, as well as a bonus script that will help in everyday InDesign use:

This is a question that comes up often on various forums, and I feel that by showing how to accomplish this task through a video tutorial that it makes it easier to understand why this task needs to be done differently to Data Merge campaigns related to direct mail items or the creation of business cards.

Data Merge PDF Export vs regular PDF Export

From time to time, I receive feedback on my Data Merge to Single Records script saying that – when exporting to PDF – spreads or hyperlinks won’t work. There is a reason for this and it has nothing to do with the script, but a behaviour (or bug) of how Data Merge exports to PDF.

A regular user of Adobe InDesign will be aware that there are two different kinds of PDF export – print and interactive. What users may not be aware of is that there is a third, more subtle method of PDF export, and that is PDF export during Data Merge.

On the surface, the PDF export looks the same, but the devil is in the detail. When merging directly to a PDF directly from the Data Merge palette (or using the Data Merge to Single Record script), there are options that cannot be chosen. These are:

  • The ability to merge to an interactive PDF
  • The page range (not the record range)
  • Spreads
  • Create Tagged PDF
  • Create Acrobat Layers
  • Hyperlinks
pdfdiff1

The export PDF dialog box via Data Merge. Note that several options are greyed out.

With the exception of being unable to check the spreads checkbox, the other limitations don’t usually pose too many issues when creating print-ready PDFs. I don’t know the reasons why these options are unable to be checked and in my mind it doesn’t make any sense that they should not be able to be selected at all.

But I digress… if interactive PDFs need to be created using Data Merge (for example, a survey where the participant was directly named during the Data Merge) then this cannot be done via the export to PDF directly from Data Merge.

There is a workaround

The solution is to merge to an InDesign file, and then export the merged InDesign file to an interactive PDF.

This situation presented itself in a real-world example where a school wanted to provide an interactive form to the parents of the students. The form would contain the student’s name, class number and teacher from the Data Merge panel, but other components of the document were interactive form fields. The PDFs had the name of the student and the teacher as the filename. The solution was as follows:

  1. Use the Data Merge to Single Record script to merge to InDesign files to a new location using the name of the student and the teacher in the filename.
  2. pdfdiff2

    Using the Data Merge to Single Records script to merge as InDesign files.

  3. Once created into a new directory, select a single file at random from that directory and ensure that it opens correctly without any font substitution. Export this single file to an interactive PDF using the settings you would like to use for the remaining files, and click OK.
  4. Open the interactive PDF that was created to make sure it opens as intended. If so, close the PDF and delete it (the reason will become apparent in a moment).
  5. Close all documents and run the Batch Convert script from Peter Kahrel. If you can’t get to the script palette from here because regular palettes are hidden and replaced with a dialog showing recent documents, go to the Window Menu, Utilities, then Scripts.
  6. pdfdiff3

    Peter Kahrel’s Batch Convert script is used to convert the InDesign files to Interactive PDF files.

  7. Once Peter’s script is open, choose the input folder by pointing to the folder containing the newly created InDesign files. The three checkboxes underneath this option should be unchecked.
  8. Choose an output folder by pointing to a destination folder for the interactive PDFs.
  9. Choose the source format as InDesign and the target format as PDF (interactive).
  10. Click OK

Peter’s script will now begin to work through the files exporting them to interactive PDFs. While this technique works, it’s annoying to have to double-handle files. One other frustration with this method is that the interactive export settings have to be initially created from a “sacrificial” PDF. To be fair, this has nothing to do with Peter’s script, but rather a “feature” (or lack of…) by InDesign. Unlike the print PDF presets, there is no ability to create interactive PDF presets.

This is a feature I would like to see and if you feel the same way, make sure to tell the Adobe InDesign team at their feature request/bug report form. In fact, I personally wouldn’t stop there… instead of having to go through this entire procedure, I’d let them know that Data Merge’s features need to be expanded and improved upon so that workarounds like this should not be necessary.

 

Alternate ways to merge with Creative Cloud

Recently, I was contacted by Lance from Nova Printing in New York, concerning a technique I’d demonstrated but had not provided a tutorial for.

illyvdp1

The technique was one Mike Rankin had demonstrated with live type on a path (chapter 18 of InDesignSecrets Guide to Graphic FX).

illyvdp2

Lance was more interested in creating perspective text going from left to right, not top to bottom as per the previous example – something that couldn’t be done in InDesign. I knew this was something that could be done in Adobe Illustrator, and that Illustrator does have a feature similar to Data Merge, but my experiences with it had been clunky. Nevertheless I persevered and to my surprise, was shown that other Illustrator users were not only using this feature, but had improved upon it.

Improved Data Merge procedure in Illustrator

illyvdp3

In a nutshell, the technique involves a javascript from Vasily Hall to use a txt or csv file and then export using a batch sequence, rather than import an XML into Illustrator. This technique is written about in greater detail elsewhere – one piece from John Garrett of Hypertransitory sums it up the best. He explains the technique in full on his website, but He also has a full course on Linkedin Learning that explains the procedure in thorough detail.

The script’s author, Vasily Hall, also writes about how this quite powerful script came to be and shows a real-world workflow.

There is another thorough step by step on this procedure by fellow Australian Stephen Marsh of Prepression.

Photoshop can merge too!

illyvdp4

Really? Well, it too has a feature similar to Data Merge, and once again it is well documented on the web, but only when looking for the term “data driven graphics”.

Once again, another blogger has done the heavy lifting here… check out Daniel Hedrick’s explanation of Data Driven Graphics.

Advantages:

Improved effects to live type

Photoshop and Illustrator are able to create a wider variety of effects to live type. Any live type can also become variable type, meaning that boring old “Dear Your Name” letters are a thing of the past.

Toggle visibility

Unlike Adobe InDesign, these two techniques can allow a variable data object’s visibility to be turned on or off based on the database.

One-to-many database relationship to make graphs

Within the advanced Illustrator technique, it is possible to have a main data file call upon smaller data files saved separately – something not possible in the other techniques.

Workaround to handle returns in a database

Using the advanced Illustrator technique, it is possible to output carriage returns in a database. This is done by substituting the carriage returns in the database with double backslashes.

Cons:

Exclusivity

Wouldn’t it be great if one AI or PS file that contained the required variable data could then be placed into InDesign and data merged from there? Yes, it would be fantastic, but alas it does not exist, at least without a paid plug-in. To accomplish this feat otherwise, the graphic has to be merged out from AI or PS first, and then added as a column in the database that InDesign will reference. While it can be done, it is done at the expense of inconvenience and time.

Time taken to process records

Unlike data merging in InDesign, using the other techniques take more time to output each unique file. This is a serious consideration when processing databases longer than 100 records.

Inconsistency between applications

Each data driven application has different ways of importing or exporting, and knowing what can do what is frustrating. Two can toggle layers, one can’t; one can produce QR codes on the fly, the other two can’t; two can output to single files, while InDesign can only do this with the Data Merge to single record script that I made in 2015. What boggles me is why two of the applications use txt or csv to import, while Illustrator relies on XML (unless you are using Vasily’s script). More importantly, none of the applications will handle an Excel file – an application I would argue that data would be more likely to have created in the first place!

Still lacks features of Microsoft Word

When I can write that Microsoft Word has features that InDesign doesn’t, it’s a sad day. Alas, Microsoft Word’s Mail merge has advantages over InDesign’s Data Merge, such as:

  • Filtering records;
  • The “next record” feature so that multiple records can appear in the same text-frame;
  • Handle returns within a database without ruining the output;
  • Link to an Excel file

These are to mention just a few.

Attitude of Adobe itself

During the InDesignConference 2016 in Washington DC at the all-attendee session with the Adobe Engineers, fellow Australian Melissa Grant asked a question about what future improvements will be made to Data Merge. To my bemusement, their response was that they didn’t think it was widely used feature and that it wouldn’t be upgraded… that was until a show of hands in the room revealed that over half of attendees in the session were using it. With the exception of adding variable QR codes, InDesign’s Data Merge feature remains the same since ten years ago. My opinion is that if more functionality was added to Data Merge, more people would use it.

What next?

For now, I’m still using Adobe InDesign to handle my Data Merging needs, as much of the merging I’m doing is functional rather than artistic (e.g. address blocks, consecutive numbering). I do plan on investing some time to research the Illustrator and Photoshop techniques, so expect some videos in the future about these methods.

 

%d bloggers like this: