To be, or CSV, that is the delimiter

From time to time I receive emails requesting support for some of the scripts that I offer through the site. Since InDesign began to support semicolon separated text files for data merge, one particular issue began to receive more requests than normal.

The emails were consistent in nature – users had downloaded the data merge single or pro script and ran the script on files they had prepared. Instead of users being able to select from the fields to the left, all of the fields appeared in one line.

This behaviour usually occurs when the script runs a data merge database that had a CSV extension, but was actually separated by semicolons rather than commas. I’d explain this back to the user and ask them to try a different CSV export from Excel, or use my preferred file export of UTF-16 text from Excel.

However, many users who had exported from Excel to CSV said that this did not change the issue and the problem persisted. Usually the problem was that – despite choosing CSV from Excel’s export options, the software was still using semicolons as a delimiter rather than commas. Luckily, exporting to UTF-16 text usually resolved the issue.

On that note, I was uncomfortable with this issue and tried to replicate an Excel export from CSV that would use semicolons as delimiters rather than commas, but I couldn’t replicate this behaviour. But then I stumbled across the following article.

In short, the article says that Excel uses the user’s locale to determine what delimiter to use for CSV files. In short, if you use a comma to separate a dollar value from cents rather than a full-stop, then a CSV will likely export with semicolon delimiters rather than commas.

Adjusting this setting is not so simple, especially for Mac users like me – the adjustment is to change a system preference that uses the appropriate currency format, but that changes lots of other related information, so this isn’t an option.

Ultimately, if you are using the data merge to single record script, and are doing so with data exported from Excel, I highly recommend that you do so with a UTF-16 Unicode Text format.

I’ll admit this was a phenomenon I was unfamiliar with, and somewhat frustrated that a file format that itself stands for comma separated variables – isn’t actually separated by commas but is in fact separated by semicolons… depending on what system locale your computer is set to and that Microsoft Excel obeys.

%d bloggers like this: