Encodings and line breaks
The encodings Trusty CSV reads and writes, how to fix accents that come out wrong, which encoding Excel likes, and the two kinds of line break.
Encodings
A text file is bytes, and its encoding says which characters they are. When it opens a file, Trusty CSV reads:
- UTF-8, the encoding of the web and of most programs today;
- Western (Windows 1252), which Excel and older Windows programs write for Western European languages, when a file isn’t valid UTF-8;
- UTF-16, little or big endian, in files that start with a byte order mark.
A byte order mark (BOM) is a few bytes at the very start of a file that say its encoding. The status bar says whether a file has one (UTF-8 with BOM), and saving keeps it, or its absence.
Changing the encoding
A click on the encoding in the status bar, or File: Change File Encoding… in the command palette, lists two kinds of choices:
- Save with… changes the encoding the file is saved with the next time you save it, and marks the tab as having unsaved changes. Save with UTF-8 with BOM is the one to choose for a file you’ll open in Excel: without the BOM, Excel on Windows reads UTF-8 files as Windows 1252 and shows
éasé. - Reopen with… reads the file again with another encoding. If accents or other letters look wrong when a file opens, it was written in another encoding than the one detected: reopen it with the right one. These choices are there while the tab has no unsaved changes.
If a file has characters that the chosen encoding can’t hold (a Chinese name in Windows 1252, say), saving asks before replacing them with ?, and offers to save in UTF-8 instead.
Line breaks
Windows ends lines with two characters, CRLF; macOS, Linux and the web, with one, LF. Trusty CSV detects which one a file uses, shows it in the status bar, and keeps it when saving.
It keeps track of two separately: the line breaks between rows, and those inside values (between quotes). Excel, for instance, writes CRLF between rows and LF inside a cell, and a file saved by Trusty CSV still has both, as Excel wrote them.
A click on LF or CRLF in the status bar, or File: Change End of Line Sequence… in the command palette, changes the line breaks between rows (and those inside values, if they were the same). The tab is then marked as having unsaved changes; the change is made when you save.