Better Infographics for Data Merge with Chartwell Bars

While speaking at the 2016 PEPCON in San Diego along with Co-presenter David Creamer on the topic of Data Publishing, I presented an older tip that allows shapes to change size based on numerical values that appear in Data Merge. The tip requires the Chartwell typeface, particularly the Chartwell bars font. I’d mentioned at the time that while it was a novel tip, I didn’t have a practical purpose for it. I’d also mentioned in my presentation about using knockout groups in InDesign to hide information and had demonstrated it using my “Parkway Drive” demonstration where it is used to hide parts of a sign that changes size, but again felt there should be a better use of this tip.

However, it was on my 15 hour flight from Los Angeles to Melbourne where I thought of a new and much more practical purpose – creating infographics. I also thought about getting some sleep, but that was a fool’s errand!

Once I arrived home, I tested out the theories I had during the flight, and while the results were mixed, I was happy with what had been achieved.

Ultimately, I have created three techniques for anyone making infographics. In all instances, I’ve colored the chartwell bars font as black so that the technique can be demonstrated, but in application the type (and its spacer) would be given the color “none”:

1 – Infographics as scaleable shapes.

01image

This uses the method described in an earlier indesignsecrets.com article that I have written. Rather than rewrite the tip, the link to that article is here. The point of difference is that the shape being transformed into an infographic is what is being scaled.

There are some drawbacks to this method.

First, the shape has to allow the chartwell bars font to expand from the left to the right without getting caught on any part of the shape, so not every shape will work. Bottles that were used in the example were fine because they meet this criteria.

02image

Second, there is a lack of precision, especially concerning low numbers as the graphic scales. This appears to be because there is a minimum size that the graphic can shrink to.

03image

2 – Data that is pasted into a vector

04image

This method works the same way with the exception that the data is in a rectangular shaped textframe that is pasted into the target shape, and also given a 2mm spacer object to allow low figures to be presented. The 2mm spacer is a 2mm square that is an inline object before the figures in the chartwell font.

05image

For anyone wondering why such an odd technique was used to add 2mm to the frame, I had tried using a 2mm left align or a 2mm inset space in the shape itself but these presented issues.

3 – Hiding an image underneath

06image

This works the same way as method 2 with the exception that rather than being pasted into the graphic, it is pasted above the graphic. An additional anchored object that is larger than the infographic is then pasted after the figures in chartwell bars and given very specific values in the anchored object dialog box, along with being given the fill color of paper and a multiply effect of 0% from the effects panel. The frame with the values is then grouped with the infographic that is underneath and the “knockout group” checkbox is ticked.

07image

To make the effect more impressive, an “after” graphic is added that is the same size as the infographic but has different properties to make the difference in the values clear to the reader.

Moving forward

By itself, these techniques aren’t that impressive if creating one-off graphics, but if preparing infographics for variable data (whether for a catalogue or direct mail) I’m sure that readers will find these methods quite useful. These are not the only infographics tricks I have recently discovered, so watch this space.

Advertisements

Pre-Sort Mail Pressure

Many articles on this blog feature advice for creating Variable Data Printing (VDP), but this post will focus on preparing VDP letters using Australia Post’s Pre-Sort Mail service. While the advice may not apply to everybody, there may be some information within the article that could still be relevant. With that out of the way, it is important to discuss what the Pre-Sort Mail Service is.

What is Pre-Sort Mail?

Australia Post offers many mailing services such as Clean Mail, Print Post and Acquisition Mail, but Pre-Sort Mail specifically refers to the delivery of barcoded mail throughout Australia.

What is the significance of Pre-Sort Mail?

Ultimately it is price and speed. As of 1 August 2014, an individual posting an addressed DL sized envelope under 125g from one Australian destination to another will pay 70 cents to post that letter (full rates of mail can be found at http://auspost.com.au/parcels-mail/pricing-updates.html). Pre-Sort Mail offers businesses a discount on their postage, provided that:

  • There are more than 300 items of addressed mail within Australia in one lodgement;
  • That the mail has been barcoded and lodged according to Australia Post’s standards.

With many items of post being substituted for email, one would ask what the importance of printed mail:

  • Conventional mail is tangible, something an individual can hold.
  • It confirms the street address of the receiver (e.g. letters that are marked return to sender will indicate if the receiver has moved).
  • It can’t be blocked with a spam-filter.

How does it work?

On the surface, “barcoded mail” would imply that the only process is to add a barcode to the mail… if only it were that simple. In fact the procedure is more complicated. The full procedure can be found here http://auspost.com.au/media/documents/presort-letters-service-guide-jun14.pdf but a summary of the process that mostly involves a prepress operator is as follows:

  • The use of dedicated barcoding software to compare the client’s database to the Australia Post database. This applies a barcoded Delivery Point IDentifier (DPID – effectively an 8-digit number that represents a street address… think of it akin to a phone number, without using the actual phone number of that address) to clients’ addresses that match Australia Post’s addresses, and leaves the remainder unbarcoded. That should be the end of it, but sadly no… there is more.
  • Using this same dedicated software, creating a manifest that lists what letters are to be sent to specific mailing locations (not postcodes – one of 54 specific locations that receive the mail and then distribute the mail to their postcodes). The software then creates mailing tags for the cardboard or plastic tubs that will hold the finished articles for mailing. This is to identify the tubs to Australia Post employees who then send the tubs to the appropriate mailing locations.
  • Once the data is exported from the dedicated software, the data then has to be “mail merged” (or Data Merged via InDesign) but it must be in the same order as the manifest. This creates many production headaches such as how to split the merge for the appropriate destinations, dealing with “spoils” (letters that are damaged during their production) etc.

What are the pressure points?

  • The dedicated barcoding software. It isn’t cheap, and this leads to many businesses reconsidering the idea of barcoding their own mail in-house, given the return on investment of mail savings is eaten by the subscription fee to the dedicated software. The software tends to be Windows Operating System specific and requires ongoing updates from Australia Post to maintain current address data.
  • Quality of a customer’s database. Items such as soft returns, address fields where the suburb-state-postcode details are in one field instead of three separate fields can hamper not just the dedicated barcoding software, but its import into InDesign. Similarly, values that need to be presented in a set format (such as dollar values, or names needing to be title-cased rather than UPPER cased) need to be resolved before importing the data into InDesign. Another trap is the length of fields – for example, a design feature that allows for most full names that would be 15-25 letters long, but names in the database that can be 35-45 letters long may not fit the space required, unless the square peg round hole trick is used.
  • The strict rules set down by Australia Post for lodgement, such as the height, width, clear zones and allowable skew of the barcode. These rules also apply to the appearance of the envelope, particularly if an address is being printed onto the envelope instead of using a window-faced envelope.
  • The speed of the lodgement. This will determine what postage paid imprint is to be used on the items to be mailed.

The last pressure point is the one that will catch out many customers and sales representatives alike. Since its introduction on 2 June 2014, Australia Post has introduced two speeds to business mail: Priority and Regular. Apart from the lodgement, the other way that this is indicated to Australia Post staff is the imprint graphic on the top right hand corner of the envelope.

What this effectively means for customers is that instead of having one variety of business envelope stationery, they now need to have two varieties for the different delivery speeds, unless the customer wants to stick to one variety of stationery, and this will lock them into a set delivery speed. At the same time, printers and mailing houses have to be aware of this when asking clients for a delivery deadline, especially if envelopes supplied by the client are at a different delivery speed to the requested lodgement speed.

What do I need to remember?

It is possible to save money on your postage by using the Pre-Sort Mail program via a Printing company or Mailing House. My employer offers this service, but it is worth asking a few questions in advance:

  • Can they barcode letters from the database I’m using, and what is the best way to supply the data?
  • If I have a set date I would like the letters in the hands of customers, when should I have my data and letter ready to begin the campaign?
  • What items can be variable on the letters that I send? Is it just type, or can I have graphs, barcodes such as QR codes, or images tailored to each letter?

 

Variable QR codes? Sort of possible…

splash

UPDATE 2014-07-22: Since the release of Adobe InDesign CC 2014, variable QR codes via Data Merge is now possible. A post will be written about this feature once it has been thoroughly tested, but in the meantime this article has been edited to reflect the update.

Since the invention of QR codes, a burning question has been “how to incorporate these barcodes into a Data Merge?” There are lots of ways to generate one-off QR codes such as:

  • QR code generating websites;
  • An InDesign Javascript written by Jongware that is suitable for CS4-6;
  • Built into InDesign CC and above; and
  • Third party plug-ins that offer one-off creation as a “taster”.

But what if there are 30, 300, 3,000 or 30,000+ codes that need to be made as a direct mail campaign? Creating 30,000 QR codes is not a task that anyone would want to do individually.

So can Adobe InDesign, fresh from the shelf, create a Data Merge with variable QR codes? Apart from the latest release of Adobe InDesign CC 2014, no – not without the use of a third party plug-in. However, this workaround does the next best thing: Creates lots of QR codes all at once, export them as uniquely named PDFs for reference in a Data Merge, and then use the built-in Data Merge feature from InDesign.

The following example is a business card for fictional clothing manufacturer “Mean Jeans”. The client would like QRcodes that feature the staff member’s email address, and if no email address appears then no QR code needs to appear. The client has supplied the database in Microsoft Excel.

There is a way to automate this task thanks to three scripts

  • QRcode.jsxbin by Jongware (as previously mentioned). When this script is used on its own, a user interface appears asking the user for the text to be coded and then a level of error correction. Jongware did allow other scripts to call upon the QRcode script, and that leads into the next script;
  • A slight modification of a script supplied by an Adobe Forums user by the handle of sergemca. The original script (found in the same Adobe Forum link as Jongware’s QRcode.jsxbin) by sergemca searched an open InDesign document for any textboxes that contained the starting words MECARD: and would then convert the contents of the complete textbox into a QR code and then apply formatting such as scaling and rotation. As this example is creating QRcodes from email addresses, the modification in this example searches an open InDesign document for any textboxes that contain the starting word mailto: . Modifications to this script are best dealt with by the scripting forum of the Adobe InDesign forum.
  • PDFStyleExporter.jsxbin by Loic Aigon. This script has featured on Colecandoo before and it is used to split a large InDesign file into single page PDFs with unique names. While researching this story I have noticed that Loic will update this script in due course, so stay tuned.

To create variable QR codes:

  1. Open the Excel file and create two fields in the database in addition to the other fields that need to appear: order, and ‘@QRcode (the ‘ will disappear in Excel after it is typed, this is intentional). In the order column, use the autofill function of Excel to add sequential numbers to this column. Leave the contents of the @QRcode field for now.1qr
  2. Save the Excel file but also save it as a “Windows Formatted Text .txt” file, and give it a name that reflects that this database is purely for the QRcoding only e.g. forcodingonly.txt2qr
  3. In Adobe InDesign, create a new file that will be used to create the QR codes first. In this example, a file that is business card size has been created (90x55mm). Once open, go to the Data Merge panel and select the data source as the recently created txt file. Once the data is available in the Data Merge panel, create two text boxes. One with the word mailto: followed by the <email> field, and another text box with the <order> field. The document should look like this3qr
  4. The order text will provide the filename in a later script, but it also must appear in the file but not get in the way of the QRcode graphic that will appear later. To make sure that this text will be live but not output in the resulting QRcode, select the textbox containing the <order> field, give the text a paragraph style called “order” – the only change is that it has no fill and stroke. Also, align the text box to be in the centre of the page. The document should now look like this4qr
  5. From the Data Merge panel, select “Create Merged Document” and merge to a new single page document.5qr
  6. This will create a large document that will contain one mailto: address per page. It is important at this stage that any mailto: addresses that don’t have email addresses in the text boxes be deleted (or else the QRcode script will make QRcodes that contain the word “mailto:” only and clearly won’t work). To delete these addresses open the Find/Change dialog box and in the GREP panel type in the word mailto:$ , leave the change field blank (as well as the formatting fields) and click Change All.6qr
  7. The document is ready to have the QR codes applied. To do this, the altered version of sergemca’s script needs to be run.
var _d = app.documents[0];
var _allStories = _d.stories;
for(var nx=_allStories.length-1;nx>=0;nx--){
    var _storyAllTextFrames = _allStories[nx].textContainers;
    for(var mx=_storyAllTextFrames.length-1;mx>=0;mx--){
         _storyAllTextFrames[mx].select(); // change page
         if(_storyAllTextFrames[mx].contents.indexOf('mailto:')==0){
             var obj = app.doScript(new File(app.activeScript.path+'/qrcode.jsxbin'), ScriptLanguage.JAVASCRIPT, [_storyAllTextFrames[mx].contents, 3], UndoModes.ENTIRE_SCRIPT);
             _storyAllTextFrames[mx].contents = "";
         };
    };
};

7qr

If done successfully, the InDesign file should now contain QRcodes in place of the text that contained the email addresses.

To export these QRcodes with their unique names

  1. Run the script PDFStyleExporter.jsxbin (called PDFExportCropper.jsxbin on my machine)8qr
  2. In the user interface that appears, select the paragraph style “order” and leave the rest of the dialog box as it is. In the PDF options dropdown field, select the destination to save the resulting PDFs. Once done, click the Export button.9qr

Again, if done successfully, the resulting PDF QR codes should now save to the nominated directory.

There are other ways to export single page PDFs but Loic’s script is used in the example in case a reference other than a sequential number is to be used, such as a person’s name, phone number, etc.

To incorporate these QR codes into the data merge:

  1. Return to the Excel file and in the @QRcode field, use the CONCATENATE function to take the order number and apply the .pdf suffix to it. The formula to use is: =CONCATENATE(A2,”.pdf”)10qr
  2. Save the Excel file and again, also save it as a “Windows Formatted Text .txt” file, and give it a name that reflects that this database has been qrcoded QRcoding only e.g. qrcoded.txt
  3. In Adobe InDesign, create (or open) the card that needs the QRcodes applied. Once ready for the data, go to the Data Merge panel and select the data source as the newly created txt file. Place the name, address references etc as necessary, and create a frame for the QR codes to appear.11qr
  4. Apply the qrcode field to this frame and go to the Data Merge panel and select Content Placement Options. In the dropdown field “Image Placement” select the fitting “Fill frames proportionally”.12qr
  5. Instead of using the “preview” function, go to the Data Merge panel and select the Export to PDF function and export one record only. In the example, page 7 was chosen at random.13qr
  6. Once satisfied that the merge looks like it will work, again use the Export to PDF function to export the entire merge to a PDF.

For the barcodes to fit the image, the PDFs need to import PDFs based on the bounding box. If the images are not fitting the frame properly, an additional script originally by Dave Saunders (but improved upon by Marc Autret) will allow the import option to change. The script is available from this forum and once loaded and run into InDesign, the option to select is “Content All Layers”

14qr

So there it is. It is worth noting that this is a workaround and not a direct live Data Merge solution. There are limitations to this solution:

  • Resulting codes can’t be colorised on-the-fly;
  • If the database changes, this will mean repeating the entire process, rather than simply updating the data in the data merge file once, and removing all  QRcodes created previously.
  • Because the codes are not human readable without a decoder, there is added emphasis to check, recheck and check again to make sure the merge is behaving properly.

While this is a workaround, there is no doubt that a turnkey solution is preferable. There are enough third party providers making variable barcode solutions for Adobe InDesign. The full list is available here. From memory here is a list of third party providers that provide variable QR codes as part of a complete VDP package:

  • Rorohiko’s Tada QR;
  • XMPie’s uDirect;
  • Meadows Publishing Software
  • Cacidi LiveMerge;
  • Teacup Software’s BarcodeMaker;
  • DirectSmile;
  • Objectif Lune’s Printshop Mail;

Several edits since this was first published:

  • Added a list of third party providers who provide variable QR codes and moved a reference to an individual one earlier on in the article into that list;
  • Fixed type within field chevrons that did not appear when the article was published;
  • Acknowledged that variable QR codes via Data Merge are now possible in Adobe InDesign CC 2014

I’m gonna knock you out, my printer didn’t knock you out…

An earlier post “To Overprint or not to Overprint, Black is the question” explains how the colour labelled [Black] in InDesign behaves, and when solid black ink should and should not knock out of the colours behind it.

Paying attention to this advice and applying it to artwork should result in a good printed reproduction, correct? While the answer should be yes, there is one more level of control of black appearance and overprints, and that is in the hands of the printing company and their output software.

OFFSET EXAMPLE

Let us look at this following example:

ko1

This card is set up for a Black plus spot output for an offset press. The Black is only overprinting on the text as misregistration would be noticeable here, but the Black elsewhere is knocking out so that the colour does not look muted through the yellow.

However, despite best intentions and checking the separations both in InDesign and Acrobat, the card has printed like this (effect is exaggerated for the screen):

ko2

So what has happened? The separations were correct, they were checked in both InDesign and Acrobat! It turns out that the Raster Image Processor (RIP) software that the commercial printer uses to image the design onto the printing plates has its own settings. Here are some example screenshots from AGFA’s Apogee X system and Fuji’s XMF system respectively about the overprinting of black:

apogee

xmf

In both screenshots above, the respective RIP software CAN honor the settings that were in the initial PDF and not apply its own preferences, but in the instance of the business card, the RIP settings overrode the PDF settings and chose to overprint all instances of 100% black, regardless what swatches were chosen in InDesign.

DIGITAL PRINT EXAMPLE

Using the same artwork, the card was printed via a colour copier, but this time the result was as follows:

ko3

So what happened here? The while the solid black looks good, where the black in the top line meets the vignette looks rather weak, and there are is a lighter black around the travel agent. What is going on?

Again, the RIP software has manipulated the artwork with unintentional results. Unlike printing directly to a desktop printer, most digital printers will print to a RIP where the file can be imposed, colour adjusted and printed in whatever order the prepress operator sees fit.

Using the EFI Fiery RIP, there is a little-known feature of the RIP that changes the way black is displayed that can produce unexpected results, and that is in the color settings dialog box and it is “Pure Black On”.

Screen Shot 2013-06-03 at 2.55.28 PM

This setting takes every instance of 100K and ramps the colour to a “super black” as opposed to using the black toner only. Again, this setting can be changed, but when this card was printed, the defaults were unchanged resulting in this unwanted appearance.

This setting only applies to vectors and text AFTER the PDF is flattened into postscript. This is visible where the rich black abruptly changes to the muted black. One look at the flattener preview in InDesign confirms that areas of flat black in that image were as a result of the flattening.

ko4

THE RESULT?

This small example shows how changing the client’s intended black overprints can have unwanted consequences. For prepress operators it is a wake-up call to make sure that the RIP defaults will maintain the clients’ expected results; and for designers or publishers it is worth understanding that even the treatment of black overprint is an important topic.

%d bloggers like this: