Swapping paragraphs during Data Merge

A recent post on the adobe forums raised an old question which I’d been happy to consign to the too-hard basket, and that question was how to import text as part of a data merge. While text can’t be imported into a merge à la importing pictures into a data merge (i.e. can import image files such as JPG, EPS, PDF, TIFF… but not text files such as DOC, RTF etc) I knew there had to be a workaround.

The workaround came in the form of using a field from the data merge to trigger a GREP style. To demonstrate this, I have created an example in the form of this case study:

An electronics store is preparing a mailing of its statements, along with a with compliments slip which is specifically targeted to each customer. The store has prepared a database with an ID to correspond with the statements, the client’s first name, and a code that corresponds to the campaign, which happens to be:

  1. Good payers
  2. Online shoppers
  3. Recently filled out an in-store survey
  4. Long time customers
  5. Dormant customers
  6. Due for a phone upgrade

Each merge has its own text, so the typical way of handling this would be to perform six separate merges, but because the with compliments slips have to go with statements that are printed in ID order and the client does NOT WANT the ID to appear on the with compliments slip, the merge must be prepared once and in order, so this is not an option.

Instead, I have created an InDesign file with two layers – a base with the static data, and then a layer that I have labelled 1 (to represent the good payers). For now, the text for the first campaign is added, along with the letters |xxxx| at the start of each paragraph, and the text is formatted using a paragraph style called changingpara1 – so far, it contains nothing out of the ordinary… yet. This will make more sense soon.

The other campaigns are literally duplicates of layer “1” but named appropriately (2,3,4,5,6) and contain their appropriate content. For the ease of creation, only the base and layer being made are viewed at any time. Until the data is imported, I’m just using first names.

Using the data merge palette, the data “namesdb.txt” is imported, and the |xxxx| portions are replaced with |<<number>>| but still retain their “|” at the start and end of the field code. Again, for ease of working with the layers, only the base and layer being made are viewed at any time.

As Heston Blumenthal would say… this is where the magic happens. With no text selected, two character styles need to be created:

Hide

  • Horizontal scale to -1, and
  • no fill nor stroke

Black

  • Black fill only

Now, viewing only layer “1”, the text box is selected and the paragraph style assigned to it is changed, namely:

  • No fill nor stroke, and

  • GREP styles

apply Black style,

to text: ^\|1\|.+

(this will find the start of the line and |1| followed by everything in the paragraph)

apply Hide style,

to text: ^\|1\|

(this will find the start of the line and the |1| only so it can be hidden)

But something unexpected has happened… the type has disappeared! This should happen… so far… pushing on, view only layer two, click on the textbox and in the paragraph styles palette, duplicate changingpara1 BUT change its name to changingpara2 and change the GREP styles this way:

apply Black style,

to text: ^\|2\|.+

(this will find the start of the line and |2| followed by everything in the paragraph)

apply Hide style,

to text: ^\|2\|

(this will find the start of the line and the |2| only so it can be hidden)

Repeat this for the remaining layers, so that layer “3” text has the “changingpara3” paragraph style that, in turn, has the GREP styles which show/hide text with the letters |3| at the start of the paragraph, and so on for the other layers.

After doing this, make sure all the layers are visible and turn the “preview” on the Data Merge palette. Suddenly, the text seems to come to life, responding to the values in the “numbers” field.

The picture above represents the records in the data merge and how they change from record to record. Alternatively, watch the video of how a similar campaign was assembled:

As usual, all files associated with this demonstration are available in the merged PDF file which can be downloaded here.

 

 

 

Advertisements

Improving on the default scripts

 

One of the features that I like about InDesign is the ability to automate repetitive tasks using scripts. Since discovering this feature, using scripts have become part of my everyday work routine.

InDesign ships with 20 scripts, but with few exceptions the same scripts have been shipped with InDesign for the past six releases.

The scripts are (as described by Adobe itself on its website):

AddGuides.jsx – Adds guides around selected object/objects.

AddPoints.jsx – Adds points to the paths of the selected object or objects.

AdjustLayout.jsx – Moves objects by specified distances on right/left pages.

AlignToPage.jsx – Aligns objects to specified positions on a page.

AnimationEncyclopedia.jsx – Automatically creates buttons with different animation properties.

BreakFrame.jsx – Removes a selected text frame and its contents from a story.

CornerEffects.jsx – redraws the path of the selected item or items using a variety of corner effects. Corner effects can be applied to selected points on the path.

CreateCharacterStyles.jsx – Defines a complete character style based on the selected text.

CropMarks.jsx – Adds crop and/or registration marks around the selected object or objects.

ExportAllStories.jsx – Exports all stories in a document to a series of text files.

FindChangeByList.jsx – Performs a series of common text find/change operations by reading a tab-delimited text file.

ImageCatalog.jsx – Places all graphics in a specified folder in a “contact sheet” layout.

MakeGrid.jsx – Creates a grid by subdividing or duplicating the selected object or objects.

Neon.jsx – Applies a “blend” effect to the selected object or objects.

PathEffects.jsx – Changes the position of path points on the selected object or objects to add a creative effect.

PlaceMultipagePDF.jsx – Places all pages of a PDF.

SelectObjects.jsx – Selects objects on the active spread by their object type.

SortParagraphs.jsx – Sorts the paragraphs in the selection alphabetically.

SplitStory.jsx – Splits the text frames in the selected story into separate, unlinked text frames.

TabUtilities.jsx – Applies tab stops and indents to the selected text.

But after six releases, some of these scripts have either become redundant or improved upon elsewhere.

Scripts that have become redundant thanks to the user interface are:

  • AlignToPage.jsx; and
  • CornerEffects.jsx

Both scripts perform tasks which are just as easy to do in the actual user interface. Since InDesign introduced Live Corners, the CornerEffects script is actually more of a burden to use given that the script interface is hard to understand.

Also, objects made using the CornerEffects script do not hold their rounded corners once scaled.

Scripts that ship with InDesign but have been improved by other users:

  • AdjustLayout.jsx;
  • FindChangeByList.jsx; and
  • MakeGrid.jsx

As each version of InDesign had different ways of interpreting javascript, the AdjustLayout behaved differently from version to version. Fed up with this, a co-author of Real World InDesign CS6 Ole Kvern has fixed this script and it can be downloaded here

For users working in millimetres, the MakeGrid script had one quibble that was it would always display measurements in points, but again the script has been fixed by Gerard Singelmann and can be downloaded here (this contains several scripts in German, the MakeGrid script is the one called Rahmen zerschneiden.jsx).

The FindChangeByList script itself hasn’t been improved upon, but the way of entering the find/replace values into the text file where the script gets its values from has. Kasyan Servetsky has a script called Recordfindchange which works by entering the values into the find/replace dialog box, then running the script which creates a text file which can be copied and entered into the text file being referenced by the FindChangeByList.jsx script. Kasyan’s script can be downloaded here.

Scripts that have been made obsolete by scripts made by others:

  • CropMarks.jsx
  • ExportAllStories.jsx
  • PlaceMultipagePDF.jsx
  • SortParagraphs.jsx
  • SplitStory.jsx

The CropMarks Script had issues given that by default, it did not account for bleed and would put crop-marks within the gutters. An alternative by Loïc Teixeira resolves these two issues and can be downloaded here. 

The ExportAllStories script still works but Kris Coppieters of Rorohiko has made a much better export. This one is a paid feature but if text exporting from InDesign is to be done often, is well worth it and can be downloaded here.

The PlaceMultipagePDF has been made quite obsolete by a much better script by Scott Zanelli called MultiPageImporter.jsx which not only imports PDFs but also allows page rotation, scaling, offsets, reverse page order… it’s massive! This script is one which is so good it even has its own manual. More information on the script can be downloaded here.

SortParagraphs again is a good script but Peter Kahrel has made a much better version.

SplitStory again works well but there is just a better version out there by Adi Ravid called StorySplitter which rather than just splitting the stories also has a dialog box with a few options. Again, that script can be found here.

If there any of the default scripts which have been made obsolete that I haven’t covered in this article, please let me know.

 

 

 

 

Making Picture Indexes with Captions

This idea had its beginnings with a colleague asking me how to create a caption above an image that would show the filename so that he knew which pictures were placed in a montage, without the need to click on the links panel. It made me wonder if a similar technique could be used to “tag” pictures so that they could then be referenced in an index, such as:

  • Image index
  • Advertisers index
  • Figures index

The principle is simple: apply captions to images, with the captions featuring a paragraph style that has no fill or stroke. The caption references the Title field from the captions dialog box of InDesign. Once all images have had captions applied, then a Table of Contents is run and any duplicates are consolidated.

The complications:

  • Adding data to the metadata information to dozens, possibly hundreds or thousands of images;
  • Consolidating duplicates once a Table of Contents is run.

Part One: Applying the metadata to the caption.

To begin with, the items to be referenced by the index have to contain metadata that can be referenced by the caption. Using the “name” reference will contain the suffix of the filename which can be fixed in further processing, but would be ideal if it did not contain the suffixes.

For this, a different reference will be required from the caption setup, but the items to be referenced have to contain the correct information in the metadata. This requires Bridge to edit the metadata. For the purposes of this article, the caption is going to reference the “Document Title” field in Bridge. This information can be added to this one picture by placing the cursor in the Document Title field of the IPTC (IIM, legacy) portion of the metadata, but this is a solution if there are few pictures.

For a solution which will convert filenames to Document Title names, this is done with scripting. The script can be found in the following link and when installed into Bridge (which is MUCH HARDER than installing scripts into InDesign) will display a new option in Bridge under the tools menu: Add Filename to Title.

Any items selected in Bridge can now have the Document Title reflect the filename without the suffix if the items are highlighted in Bridge and then apply the new menu item “Add Filename to Title”

Once the image is relinked and the caption definition changed to “Title” (this is how InDesign references what Bridge calls “Document Title”) the caption can be applied.

So that the caption doesn’t interfere with any artwork on the page, it will be given a paragraph style which has no fill or stroke in the font; and is placed so that it is on the image but grouped with the picture so it is unobtrusive.

This caption will now act as “tagged” so that its Title can be referred to in a Table of Contents.

Part 2: Consolidating duplicates once Table of Contents is run.

In the next part of this example, captions have been applied captions to hundreds of pictures in a pictorial index but now need to appear sorted by the names of the images of the products. The first image shows a general Table of Contents that was run using the paragraph style in the caption.

The list is in the order that the pictures appeared in the book, but is not in page order. To do this, the data could be:

On this occasion, the contents will be copied and pasted into Excel, then sorted using Excel’s sort function, then copied and re-pasted back into InDesign.

This is better, but there are a series of entries at the end which are duplicated which are “imagetocome” followed by page numbers which they appear.

To do this, two GREPs need to be run. The first GREP is:

Find:

^(.+)\r(\1)

replace:

$1

run this GREP on the story containing the contents until 0 results are returned.

This will remove exact duplicates through the index but will keep close duplicates (e.g. same name but different page number)

Then, run this GREP:

Find:

^(.+)(\t)(.+)\r(\1)(\t)

Replace:

$1$2$3,

run this GREP on the story containing the contents until 0 results are returned.

and this will remove close duplicates by consolidating them onto the same line.

The reality is that the entry “imagetocome” was deleted anyway but worth demonstrating.

“No fills” GREP Styles – Part 2: The Glitch of the Merge

This is part 2 of the “No fills” GREP styles articles. In this post, the “No fills” GREP Styles are used to overcome a Data Merge glitch within InDesign when the “remove blank lines” checkbox is selected. An example has been created which contains an Email, Direct number and Mobile number. Here is how the file looks without any data in it.

Here is how the file looks with the first record.

Good so far. Won’t win any awards, but all the fields are populated. How about the next record?

Now there is an issue. There is no number for the Direct field but the word Direct implies that there is. This looks untidy, it would be good if this line would disappear.  “Remove Blank Lines for Empty Fields” should solve this.

OK, this hasn’t solved the situation. It must consider the word “Direct” to be content of the line, regardless if there is data or not. What if the prefixes (Email, Direct, Mobile) weren’t in the type as such but were bullets for example? Easy enough to make them using Paragraph Styles, so this will be demonstrated using the Email field. This will be done on a new page to compare results.

This Paragraph Style will be duplicated twice to make styles for the Direct line and the Mobile line, with the prefix being the only thing to change amongst the Paragraph Styles.

So this should work. How does the first record look?

Good. How about the second record?

Excellent. The word “Direct” has disappeared as intended and the Mobile line is now underneath the Email line. How about the third record?

Bad. The line which should start with the word “Direct” now starts with the word “Email”. It is almost as if InDesign’s “Remove Blank Lines for Empty Fields” feature has deleted the line but inadvertently taken the Email line formatting. This is a glitch, but occurs during the preview only.

However, it could be a costly glitch, especially if the glitch is inadvertently triggered via the Preview.

So is there an alternative way to do this so that this issue is avoided? GREP Styles. Again, this will be done on a new page to compare results.

The following character styles need to be set up. Apart from the font type and size, the settings required are as follows:

STYLE Firstword White nothing
General Based on [None] Based on [None] Based on [None]
Character Colour Fill: Red Fill: Paper Fill: None

Now the following paragraph styles need to be set up. Again, apart from the font type and size, the settings required are as follows:

STYLE Lines Direct disappear
GREP styles Apply: nothing

to text: ^\+.+

Apply: nothing

To text: ^1800.+

Apply: white

Direct\t.+

Apply: firstword

To text: Direct\t(?=\+|\d)

Character colour Fill: Paper Fill: None

Set the text so that only Email and Mobile have their prefixes showing, but with the placeholders in place. Note the position of the “Direct” field – this is on purpose.

Next, copy and paste in place the text onto a new layer. Format the text so that the first line is a hard return and that the next line is formatted as pictured.

 

Once done, begin with record one

That worked. How about the second one?

Good, but the third one fouled up before. Is it good now?

FINALLY! How about record nine… this doesn’t start the way the other numbers do

FANTASTIC! The GREP style took this into consideration when it was created.

Yes, this example is not a turn-key solution to EVERY Data Merge glitch, but the use of “no fill” GREP styles can be, provided the GREP styles are carefully tailored to the situation. This post is simply to demonstrate the power of GREP styles with “No fill”.

As usual, all necessary files can be found as attachments within the PDF of the file here. The PDF was made prior to “activating” the glitch, but can easily be done in the InDesign file by toggling back and forth through pages in the Data Merge preview.

%d bloggers like this: