14.2.2. Profiling and Performance Tools

In order to use gprof with a program, you must compile the program using the -pg option with gcc. This adds profiling information to the object file and links the executable with standard libraries that have profiling information enabled.

Having compiled the program to profile with -pg, simply run it. If it exits normally, the file gmon.out will be written to the working directory of the program. This file contains profiling information for that run and can be used with gprof to display a table of statistics.

As an example, let's take a program called getstat, which gathers statistics about an image file. First, we compile getstat with -pg, and run it:

papaya$ getstat image11.pgm > stats.dat 
papaya$ ls -l gmon.out 
-rw-------   1 mdw      mdw         54448 Feb  5 17:00 gmon.out 
papaya$

To examine the profiling data, we run gprof and give it the name of the executable and the profiling file gmon.out :

papaya$ gprof getstat gmon.out

gprof output is rather verbose, so you may want to redirect it to a file or pipe it through a pager. It comes in two parts. The first part is the "flat profile," which gives a one-line entry for each function, listing the percentage of time spent in that function, the time (in seconds) used to execute that function, the number of calls to the function, and other information. For example:

Each sample counts as 0.01 seconds. 
  %   cumulative   self              self     total            
 time   seconds   seconds    calls  ms/call  ms/call  name     
 45.11     27.49    27.49       41   670.51   903.13  GetComponent 
 16.25     37.40     9.91                             mcount 
 10.72     43.93     6.54  1811863     0.00     0.00  Push 
 10.33     50.23     6.30  1811863     0.00     0.00  Pop 
  5.87     53.81     3.58       40    89.50   247.06  stackstats 
  4.92     56.81     3.00  1811863     0.00     0.00  TrimNeighbors

The second part of the gprof report can help us here. It gives a detailed "call graph" describing which functions called other functions and how many times they were called. For example:

index % time    self  children    called     name 
                                                 <spontaneous> 
[1]     92.7    0.00   47.30                 start [1] 
                0.01   47.29       1/1           main [2] 
                0.00    0.00       1/2           on_exit [53] 
                0.00    0.00       1/1           exit [172]

The call graph normally displays the children as well as the parents of each function in the graph. Here, we can see that start called the functions main, on_exit, and exit (listed below the line for start). However, there are no parents (normally listed above start); instead, we see the ominous word <spontaneous>. This means that gprof was unable to determine the parent function of start; more than likely because start was not called from within the program itself but kicked off by the operating system.

Skipping down to the entry for GetComponent, or function-under-suspect, we see the following:

index % time    self  children    called     name 
                0.67    0.23       1/41          GetFirstComponent [12] 
               26.82    9.30      40/41          GetNextComponent [5] 
[4]     72.6   27.49    9.54      41         GetComponent [4] 
                6.54    0.00 1811863/1811863     Push [7] 
                3.00    0.00 1811863/1811863     TrimNeighbors [9] 
                0.00    0.00       1/1           InitStack [54]

GetComponent itself requires over 27.49 seconds to run; only 9.54 seconds are spent executing the children of GetComponent (including the many calls to Push and TrimNeighbors!). So it looks as though GetComponent and possibly its parent GetNextComponent need some tuning; the oft-called Push function is not the sole cause of the problem.

gprof also keeps track of recursive calls and "cycles" of called functions and indicates the amount of time required for each call. Of course, using gprof effectively requires that all code to be profiled is compiled with the -pg option. It also requires a knowledge of the program you're attempting to profile; gprof can only tell you so much about what's going on. It's up to the programmer to optimize inefficient code.

Use of calls is simple: you tell it the names of the source files to map out, and a function-call tree is displayed. For example:

papaya$ calls scan.c 
    1   level1 [scan.c] 
    2           getid [scan.c] 
    3                   getc 
    4                   eatwhite [scan.c] 
    5                           getc 
    6                           ungetc 
    7                   strcmp 
    8           eatwhite [see line 4] 
    9           balance [scan.c] 
   10                   eatwhite [see line 4]

14.2.3. Using strace

Take the "Hello, World!" program given earlier in the chapter. Running strace on the executable hello gives us:

papaya$ strace hello 
execve("./hello", ["hello"], [/* 49 vars */]) = 0
mmap(0, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,\
 -1, 0) = 0x40007000
mprotect(0x40000000, 20881, PROT_READ|PROT_WRITE|PROT_EXEC) = 0
mprotect(0x8048000, 4922, PROT_READ|PROT_WRITE|PROT_EXEC) = 0
stat("/etc/ld.so.cache", {st_mode=S_IFREG|0644, st_size=18612,\
 ...}) = 0
open("/etc/ld.so.cache", O_RDONLY)      = 3
mmap(0, 18612, PROT_READ, MAP_SHARED, 3, 0) = 0x40008000
close(3)                                = 0
stat("/etc/ld.so.preload", 0xbffff52c)  = -1 ENOENT (No such\
 file or directory)
open("/usr/local/KDE/lib/libc.so.5", O_RDONLY) = -1 ENOENT (No\
 such file or directory)
open("/usr/local/qt/lib/libc.so.5", O_RDONLY) = -1 ENOENT (No\
 such file or directory)
open("/lib/libc.so.5", O_RDONLY)        = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3"..., 4096) = 4096
mmap(0, 770048, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = \
0x4000d000
mmap(0x4000d000, 538959, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_\
FIXED, 3, 0) = 0x4000d000
mmap(0x40091000, 21564, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_\
FIXED, 3, 0x83000) = 0x40091000
mmap(0x40097000, 204584, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_\
FIXED|MAP_ANONYMOUS, -1, 0) = 0x40097000
close(3)                                = 0
mprotect(0x4000d000, 538959, PROT_READ|PROT_WRITE|PROT_EXEC) = 0
munmap(0x40008000, 18612)               = 0
mprotect(0x8048000, 4922, PROT_READ|PROT_EXEC) = 0
mprotect(0x4000d000, 538959, PROT_READ|PROT_EXEC) = 0
mprotect(0x40000000, 20881, PROT_READ|PROT_EXEC) = 0
personality(PER_LINUX)                  = 0
geteuid()                               = 501
getuid()                                = 501
getgid()                                = 100
getegid()                               = 100
fstat(1, {st_mode=S_IFCHR|0666, st_rdev=makedev(3, 10), ...}) = 0
mmap(0, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,\
 -1, 0) = 0x40008000
ioctl(1, TCGETS, {B9600 opost isig icanon echo ...}) = 0
write(1, "Hello World!\n", 13Hello World!
)          = 13
_exit(0)                                = ?
papaya$

The first call execve starts the program itself. All the mmap, mprotect, and munmap calls come from the kernel's memory management and are not really interesting here. In the three consecutive open calls, the loader is looking for the C library and finds it on the third try. The library header is then read and the library mapped into memory. After a few more memory-management operations and the calls to getuid, geteuid, getgid, and getegid, which retrieve the rights of the process, there is a call to ioctl. The ioctl is the result of a tcgetattr library call, which the program uses to retrieve the terminal attributes before attempting to write to the terminal. Finally, the write call prints our friendly message to the terminal and exit ends the program.

The calls to munmap (which unmaps a memory-mapped portion of a file) and brk (which allocates memory on the heap) set up the memory image of the running process. The ioctl call is the result of a tcgetattr library call, which retrieves the terminal attributes before attempting to write to it. Finally, the write call prints our friendly message to the terminal, and exit ends the program.

strace sends its output to standard error, so you can redirect it to a file separately from the actual output of the program (usually on standard output). As you can see, strace tells you not only the names of the system calls, but also their parameters (expressed as well-known constant names, if possible, instead of just numerics) and return values.

14.2.4. make and imake

One solution is to use imake, an extension to make based on the use of the C preprocessor. imake is simply a makefile generator: you write an Imakefile that imake converts to a robust makefile. imake is used by programs in the X Window System distribution but is not limited to use by X applications.

We should note at this point that imake can simplify the process of writing makefiles, especially for compiling C programs. However, make is more generally applicable than imake for this task. For example, you can use make to automatically format documents using groff or TeX. In this case, you need the flexibility of make alone, and imake may not be the best solution.

Here is a sample Imakefile that builds two programs, laplacian and getstat. At the top of the Imakefile, options for the entire compilation are specified (imake has its own defaults for these, but they aren't always useful). Following that, variables are defined for each program to be compiled, and the imake macros AllTarget and NormalProgramTarget create makefile rules for compiling these programs:

# Linker options: 
LDOPTIONS = -L/usr/local/lib -L../lib  
# The C compiler to use: 
CC = gcc 
# Flags to be used with gcc: 
CFLAGS = -I. -I$(HOME)/include -g  
# Local and system libraries to link against: 
LOCAL_LIBRARIES = -lvistuff 
SYS_LIBRARIES = -lm 
 
# Specify the sources in the SRCS variable, and the corresponding object 
# files in the variable LAP_OBJS. 
SRCS = laplacian.c laplacian-main.c 
LAP_OBJS = laplacian.o laplacian-main.o 

# Create rules for building laplacian. 
AllTarget(laplacian) 
NormalProgramTarget(laplacian,$(LAP_OBJS),,$(LOCAL_LIBRARIES),\
$(SYS_LIBRARIES)) 
 
# Do the same thing for getstat. Note that SRCS can be redefined for each 
# target, but LAP_OBJS can't, so we use a unique name for each target. 
SRCS = getstat.c getstat-main.c component.c 
GS_OBJS = getstat.o getstat-main.o component.o 

AllTarget(getstat) 
NormalProgramTarget(getstat,$(GS_OBJS),,$(LOCAL_LIBRARIES),\
$(SYS_LIBRARIES))

Note that we must use a different variable for the object files for each target, although SRCS can be redefined for each.

In order to translate the Imakefile into a makefile, use the command xmkmf. xmkmf will simply run imake with the options to do the translation correctly, using the default imake macros (such as AllTarget and NormalProgramTarget). You can then issue make to compile the program:

papaya$ xmkmf 
mv Makefile Makefile.bak 
imake -DUseInstalled -I/usr/X386/lib/X11/config 
papaya$

If you find imake too complex for your taste, other "makefile makers" are available as well, such as ICmake, which generates makefiles using a macro language similar to C.

14.2.5. Using Checker

Checker is not just a replacement for malloc and friends. It also inserts code into your program to verify all memory reads and writes. It is very robust and therefore somewhat slower than the regular malloc routines. Checker is meant to be used during program development and testing; once all potential memory-corrupting bugs have been fixed, you can link your program with the standard libraries.

For example, take the following program, which allocates some memory and attempts to do various nasty things with it:

#include <malloc.h> 
int main() { 
  char *thememory, ch; 
 
  thememory=(char *)malloc(10*sizeof(char)); 
   
  ch=thememory[1];     /* Attempt to read uninitialized memory */ 
  thememory[12]=' ';   /* Attempt to write after the block */ 
  ch=thememory[-2];    /* Attempt to read before the block */ 
}

From Checker: 
        Memory access error 
        When Reading at address 0x10033 
        inside the heap 
        1 bytes after the begin of the block 

From Checker: 
        Memory access error 
        When Writing at address 0x1003e 
        inside the heap 
        2 bytes after the end of the block 
From Checker: 
        Memory access error 
        When Reading at address 0x10030 
        inside the heap 
        2 bytes before the begin of the block

14.2.6. Interface Building Tools

The classic X programming model has attempted to be as general as possible, providing only the bare minimum of interface restrictions and assumptions. This generality allows programmers to build their own interface from scratch, as the core X libraries don't make any assumptions about the interface in advance. The X Toolkit Intrinsics (Xt) provides a rudimentary set of interface widgets (such as simple buttons, scrollbars, and the like), as well as a general interface for writing your own widgets if necessary. Unfortunately this can require a great deal of work for programmers who would rather use a set of pre-made interface routines. A number of Xt widget sets and programming libraries are available for Linux, all of which make the user interface easier to program.

Before you start developing with XView or Motif, a word of caution is in order. XView, which was once a commercial product of Sun Microsystems, has been dropped by the developers and is no longer maintained. Also, while some people like the look, the programs written with XView look very nonstandard. Motif on the other hand is still being actively developed (albeit rather slowly), but has some problems itself. First, programming with Motif can be frustrating, because it is difficult, error prone, and cumbersome since the Motif API was not designed according to modern GUI API design principles. Also, Motif programs tend to run very slowly.

But there are other widget sets and interface libraries for X like:

Many programmers are finding that building a user interface, even with a complete set of widgets and routines in C, requires much overhead and can be quite difficult. This is a question of flexibility versus ease of programming: the easier the interface is to build, the less control the programmer has over it. Many programmers are finding that prebuilt widgets are adequate enough for their needs, so the loss in flexibility is not a problem.

One of the problems with interface generation and X programming is that it is difficult to generalize the most widely used elements of a user interface into a simple programming model. For example, many programs use features such as buttons, dialog boxes, pull-down menus, and so forth, but almost every program uses these widgets in a different context. In simplifying the creation of a graphical interface, generators tend to make assumptions about what you'll want. For example, it is simple enough to specify that a button, when pressed, should execute a certain procedure within your program, but what if you want the button to execute some specialized behavior the programming interface does not allow for? For example, what if you wanted the button to have a different effect when pressed with mouse button 2 instead of mouse button 1? If the interface-building system does not allow for this degree of generality, it is not of much use to programmers who need a powerful, customized interface.

If you like the Tk toolkit, but do not like the programming language Tcl, you will be delighted to hear that you can use Tk with other languages as well; it has become the GUI toolkit of choice for the scripting languages Python and Perl, too.

14.2.7. Revision Control Tools--RCS

Why bother with revision control? Many large projects require some kind of revision control in order to keep track of many tiny complex changes to the system. For example, attempting to maintain a program with a thousand source files and a team of several dozen programmers would be nearly impossible without using something like RCS. With RCS, you can ensure that only one person may modify a given source file at any one time, and all changes are checked in along with a log message detailing the change.

When you check in a file with RCS, revisions are added to the RCS file, and the original file is deleted by default. In order to access the original file, you must check it out from the RCS file. When you're editing a file, you generally don't want someone else to be able to edit it at the same time. Therefore, RCS places a lock on the file when you check it out for editing. A locked file may only be modified by the user who checks it out (this is accomplished through file permissions). Once you're done making changes to the source, you check it back in, which allows anyone working on the project to check it back out again for further work. Checking out a file as unlocked does not subject it to these restrictions; generally, files are checked out as locked only when they are to be edited but are checked out as unlocked just for reading (for example, to use the source file in a program build).

RCS automatically keeps track of all previous revisions in the RCS file and assigns incremental version numbers to each new revision that you check in. You can also specify a version number of your own when checking in a file with RCS; this allows you to start a new "revision branch" so that multiple projects can stem from different revisions of the same file. This is a good way to share code between projects but also to assure that changes made to one branch won't be reflected in others.

Here's an example. Take the source file importrtf.c, which contains our friendly program:

#include <stdio.h> 
 
int main(void) { 
  printf("Hello, world!"); 
}

papaya$ ci importrtf.c 
importrtf.c,v  <--  importrtf.c 
enter description, terminated with single '.' or end of file: 
NOTE: This is NOT the log message! 
>> Hello world source code 
>> . 
initial revision: 1.1 
done 
papaya$

If you use RCS often, you may not like all of those unsightly importrtf.c,v RCS files cluttering up your directory. If you create the subdirectory RCS within your project directory, ci and co will place the RCS files there, out of the way from the rest of the source.

In addition, RCS keeps track of all previous revisions of your file. For instance, if you make a change to your program that causes it to break in some way and want to revert to the previous version to "undo" your changes and retrace your steps, you can specify a particular version number to check out with co. For example:

papaya$ co -l1.1 importrtf.c 
importrtf.c,v  -->  importrtf.c 
revision 1.1 (locked) 
writable importrtf.c exists; remove it? [ny](n): y  
done 
papaya$

/* $Header: /work/linux/hitch/programming/tools/RCS/rcs.tex 1.2 1994/12/04 15:19:31 mdw Exp mdw $ */

Other keywords exist as well, such as $Author$ $Date$, and $Log$ (the latter keeps a complete record of the log entries for each revision embedded in the source file).

Many programmers place a static string within each source file to identify the version of the program after it has been compiled. For example, within each source file in your program, you can place the line:

static char rcsid[] = "\@(#)$Header$";

RCS has several other programs in its suite, including rcs, used for maintaining RCS files. Among other things, rcs can give other users permission to check out sources from an RCS file. See the manual pages for ci, co, and rcs for more information.

14.2.8. Revision Control Tools--CVS

By default, CVS works with full directory trees. That is, each CVS command you issue affects the current directory and all the subdirectories it contains, including their subdirectories and so on. This recursive traversal can be switched off with a command-line option, or you can specify a single file for the command to operate on.

When you as a developer have checked out a local directory tree, all the files are writable. You can make any necessary changes to the files in your personal workspace. When you have finished local testing and feel sure enough of your work to share the changes with the rest of the programming team, you write any changed files back into the central repository by issuing a CVS commit command. CVS then checks whether another developer has checked in changes since you checked out your directory tree. If this is the case, CVS does not let you check your changes in, but asks you first to take the changes of the other developers over to your local tree. During this update operation, CVS uses a sophisticated algorithm to reconcile ("merge") your changes with those of the other developers. There are cases in which this is not automatically possible. In this case, CVS informs you that there have been conflicts and asks you to resolve those. The file in question is marked up with special characters so that you can see where the conflict has occurred and decide which version should be used. Note that CVS makes sure that conflicts can only occur in local developers' trees. There is always a consistent version in the repository.

14.2.9. Patching Files

patch is a program that makes context-dependent changes in a file in order to update that file from one version to the next. This way, when your program changes, you simply release a patch file against the source, which the user applies with patch to get the newest version. For example, Linus Torvalds usually releases new Linux kernel versions in the form of patch files as well as complete source distributions.

A nice feature of patch is that it applies updates in context; that is, if you have made changes to the source yourself, but still wish to get the changes in the patch file update, patch usually can figure out the right location in the original file to apply the changes to. This way, your versions of the original source files don't need to correspond exactly to those that the patch file was made against.

/* hello.c version 1.0 by Norbert Ebersol */ 
#include <stdio.h>  
 
int main() { 
  printf("Hello, World!"); 
  exit(0); 
}

Let's say that you were to update this source, as in the following:

/* hello.c version 2.0 */ 
/* (c)1994 Norbert Ebersol */ 
#include <stdio.h>  
 
int main() { 
  printf("Hello, Mother Earth!\n"); 
  return 0; 
}

If you want to produce a patch file to update the original hello.c to the newest version, use diff with the -c option:

papaya$ diff -c hello.c.old hello.c > hello.patch

In many cases, you'll want to update not only a single source file but an entire directory tree of sources. patch allows many files to be updated from a single diff. Let's say you have two directory trees, hello.old and hello, which contain the sources for the old and new versions of a program, respectively. To make a patch file for the entire tree, use the -r switch with diff:

papaya$ diff -cr hello.old hello > hello.patch

Now, let's move to the system where the software needs to be updated. Assuming that the original source is contained in the directory hello, you can apply the patch with:

papaya$ patch -p0 < hello.patch

The -p0 switch tells patch to preserve the pathnames of files to be updated (so it knows to look in the hello directory for the source). If you have the source to be patched saved in a directory named differently from that given in the patch file, you may need to use the -p option. See the patch manual page for details about this.

14.2.10. Indenting Code

Take this terribly formatted source:

double fact (double n) { if (n==1) return 1; 
else return (n*fact(n-1)); } 
int main () { 
printf("Factorial 5 is %f.\n",fact(5)); 
printf("Factorial 10 is %f.\n",fact(10)); exit (0); }

#include <math.h> 
 
double 
fact (double n) 
{ 
  if (n == 1) 
    return 1; 
  else 
    return (n * fact (n - 1)); 
} 
void 
main () 
{ 

  printf ("Factorial 5 is %f.\n", fact (5)); 
  printf ("Factorial 10 is %f.\n", fact (10)); 
  exit (0); 
}