Showing Nonprintable Characters in Filenames (Unix Power Tools, 3rd Edition)

8.12. Showing Nonprintable Characters in Filenames

From time to time, you may get filenames with nonprinting characters, spaces, and other garbage in them. This is usually the result of some mistake -- but it's a pain nevertheless.

If you're using a version of ls that uses -q by default (and most do these days), the ls command gives you some help; it converts all nonprinting characters to a question mark (?), giving you some idea that something funny is there.[36] For example:

[36]Even in lses that use it, the -q option is the default only when ls's standard output is a terminal. If you pipe the output or redirect it to a file, remember to add -q.

% ls
ab??cd

This shows that there are two nonprinting characters between ab and cd. To delete (or rename) this file, you can use a wildcard pattern like ab??cd.

WARNING: Be careful: when I was new to Unix, I once accidentally generated a lot of weird filenames. ls told me that they all began with ????, so I naively typed rm ????*. That's when my troubles began. See Section 14.3 for the rest of the gruesome story. (I spent the next day and night trying to undo the damage.) The moral is: it's always a good idea to use echo to test filenames with wildcards in them.

If you're using an ls that came from System V Unix, you have a different set of problems. System V's ls doesn't convert the nonprinting characters to question marks. In fact, it doesn't do anything at all -- it just spits these weird characters at your terminal, which can respond in any number of strange and hostile ways. Most of the nonprinting characters have special meanings -- ranging from "don't take any more input" to "clear the screen." [If you don't have a System V ls, but you want this behavior for some reason, try GNU ls with its -N option. -- JP]

To prevent this, or to see what's actually there instead of just the question marks, use the -b option.[37] This tells ls to print the octal value of any nonprinting characters, preceeded by a backslash. For example:

[37]On systems that don't support ls -b, pipe the ls -q output through cat -v or od -c ( (Section 12.4) to see what the nonprinting characters are.

% ls -b
ab\013\014cd

This shows that the nonprinting characters have octal values 13 and 14, respectively. If you look up these values in an ASCII table, you will see that they correspond to CTRL-k and CTRL-l. If you think about what's happening -- you'll realize that CTRL-l is a formfeed character, which tells many terminals to clear the screen. That's why the regular ls command behaved so strangely.

Once you know what you're dealing with, you can use a wildcard pattern to delete or rename the file.

-- ML