Batch Editing Gotcha: Editors Fail on Big Files (Unix Power Tools, 3rd Edition)

20.8. Batch Editing Gotcha: Editors Fail on Big Files

People use the ed editor with script files to make global edits. But many versions of ed can't edit large files. The ex editor is usually better, but it has limits, too. How large is "large"? That depends on your version. Most eds I've seen can't handle more than about 100,000 characters.

There are no limits on sed (Section 34.1), although you'll need to save its output somehow (Section 34.4), and your editing script may have to be changed to work with sed.[56] Here's what you'll see when ed fails:

[56]By default, ed commands apply to the current line. sed commands are global. Also, relative line addresses like -5 don't work in sed.

% cat edscr 
s/Unix/UNIX/g
w
% ed - words < edscr
?
%

The ? is ed's "verbose" way of telling you that something's wrong. This obscure message is especially bad if you write a shell script that edits multiple files in a loop; you may not notice the error or be able to tell which file had the problem. Be sure your script checks for errors!

Unfortunately for programmers, ed may not return an error status that you can test. There are workarounds, though. When the ed - command succeeds, it doesn't display anything. The simplest way to find errors is to check for any output on stdout or stderr. This chunk of a Bourne shell script shows how (your filename is in the shell variable $filename (Section 35.9)):

2>&1 Section 36.16, [ ] Section 35.26, $? Section 35.12

edout="`ed - $filename < edscr 2>&1`"
if [ -n "$edout" -o $? -ne 0 ]
then
   echo "$edout" 1>&2
   echo "QUITTING: 'ed - $filename < edscr' failed?!?" 1>&2
   exit 1
fi

-- JP