Extending Perl (Using C from Perl) (Programming Perl)

21.3. Extending Perl (Using C from Perl)

If you want to use C source code (or a C library) from Perl, you need to create a library that can be either dynamically loaded or statically linked into your perl executable. (Dynamic loading is usually preferred, to minimize the number of different perl executables sitting around being different.) You create that library by creating an XS file (ending in .xs) containing a series of wrapper subroutines. The wrapper subroutines are not Perl subroutines, however; they are in the XS language, and we call such a subroutine an XSUB, for "eXternal SUBroutine". An XSUB can wrap a C function from an external library, a C function elsewhere in the XS file, or naked C code in the XSUB itself. You then use the xsubpp utility bundled with Perl to take the XS file and translate it into C code that can be compiled into a library that Perl will understand.

Assuming your operating system supports dynamic linking, the end result will be a Perl module that behaves like any other module written in 100% pure Perl, but runs compiled C code under the hood. It does this by pulling arguments from Perl's argument stack, converting the Perl values to the formats expected by a particular C function (specified through an XSUB declaration), calling the C function, and finally transferring the return values of the C function back to Perl. These return values may be passed back to Perl either by putting them on the Perl stack or by modifying the arguments supplied from the Perl side. (If your system doesn't support dynamic linking, you have another hoop to jump through, and we'll discuss that in the next section.)

The previous description is a somewhat simplified view of what really happens. Since Perl allows more flexible calling conventions than C, XSUBs can do much more in practice, such as checking input parameters for validity, throwing exceptions, returning undef or an empty list, calling different C functions based on numbers and types of the arguments, or providing an object-oriented interface. Again, see the perlxs and perlxstut manpages.

21.3.1. XS and XSUBs

XS is a convenience: there's nothing stopping you from writing glue code directly in C and linking it into your Perl executable. However, this would be tedious, especially if you need to write glue for multiple C functions, or if you're not familiar with the Perl stack discipline and other arcana. XS lets you write a concise description of what should be done by the glue, and the XS compiler xsubpp handles the rest.

For people who don't find XS convenient enough, the SWIG system automatically generates simple XSUBs. See http://www.swig.org for more information.

The XS language allows you to describe the mapping between a C function and a Perl function. It also allows you to create a Perl function that is a wrapper around pure C code that you write yourself. When XS is used merely to map between C and Perl, the XSUB declaration is almost identical to a declaration of a C function. In such circumstances, a tool called h2xs (bundled with Perl) is able to translate an entire C header file into a corresponding XS file that provides glue to the C functions and macros.

The xsubpp tool creates the constructs necessary to let an XSUB manipulate Perl values and the glue necessary to let Perl call the XSUB.

An XS file begins with any C code you want to include, which will often be nothing more than a set of #include directives. After a MODULE keyword, the remainder of the file should be in the XS "language", a combination of XS directives and XSUB definitions. We'll see an example of an entire XS file soon, but in the meantime here is a simple XSUB definition that allows a Perl program to access a C library function called sin(3). The XSUB specifies the return type (a double length floating-point number), the function name and argument list (with one argument dubbed x), and the type of the argument (another double):

double
sin(x)
    double x

More complicated XSUBs will often contain other bits of XS code. Each section of an XSUB starts with a keyword followed by a colon, such as INIT: or CLEANUP:. However, the first two lines of an XSUB always contain the same data: a description of the return type and the name of the function and its parameters. Whatever immediately follows these is considered to be an INPUT: section unless explicitly marked with another keyword. The various keywords are all explained in the perlxs manpage, which you should read to learn about everything you can do with XSUBs.

If your system does not have the capability to dynamically load shared libraries, you can still use XSUBs, but you must statically link the XSUBs with the rest of Perl, creating a new Perl executable (to sit around and be different). The XSUB build mechanism will check the system and build a shared library if possible, or else a static library. Optionally, it can build a new statically linked executable with that static library linked in. (But you might want to delay that to bundle all your new extensions into a single executable sitting around being the same, as it were.)

If your system can link libraries dynamically but you still want to build a statically linked executable, you can run make perl instead of make in the following examples. You should then run make test_static instead of make test to test your extension.

The xsubpp program also needs to know how to convert from Perl's data types to C's data types. Often it can guess, but with user-defined types you may need to help it out by specifying the conversion in a typemap file. The default conversions are stored in PATH-TO-PERLLIB/ExtUtils/typemap.

The typemap is split into three sections. The first section, labeled TYPEMAP, tells the compiler which of the code fragments in the following two sections should be used to map between C types and Perl values. The second section, INPUT, contains C code specifying how Perl values should be converted to C types. The third section, OUTPUT, contains C code specifying how to translate C types into Perl values.

21.3.2. Creating Extensions

A proper extension consists of several files: one containing the XS code, plus other supporting files that help Perl figure out what to do with the XS code. You can create all of these files by hand, but it's easier to use the h2xs tool, which creates a skeletal extension that you can then flesh out:

h2xs -A -n Mytest

This creates a directory named Mytest, possibly under ext/ if that directory exists in the current directory. Six files will be created in the Mytest directory: MANIFEST, Makefile.PL, Mytest.pm, Mytest.xs, test.pl, and Changes. We describe the first four below.

MANIFEST

The MANIFEST file contains the names of all the files just created in the Mytest directory. If you add more files to your extension and intend to distribute it to the wide world, add the filenames here. This is tested by some systems to ensure that your distribution is complete.

Makefile.PL

This is a Perl program that generates a Makefile (which is then passed to make or an equivalent). Makefile.PL is described further in "Creating CPAN Modules" in Chapter 22, "CPAN".

Mytest.pm

Users will use this module when they want to load your extension. You're expected to fill in the blanks in the skeletal module created for you by h2xs:

package Mytest;

use strict;
use warnings;

require Exporter;
require DynaLoader;

our @ISA = qw(Exporter DynaLoader);
# Items to export into callers namespace by default. Note: do not export
# names by default without a very good reason. Use EXPORT_OK instead.
# Do not simply export all your public functions/methods/constants.
our @EXPORT = qw(

);
our $VERSION = '0.01';

bootstrap Mytest $VERSION;

# Preloaded methods go here.

# Autoload methods go after __END__, and are processed by the autosplit program.

1;
__END__
# Below is the stub of documentation for your module. You better edit it!

Most extension modules will require the Exporter and DynaLoader extensions. After setting @ISA (for inheritance) and @EXPORT (to make functions available to the package using the module), the initialization code tells Perl to bootstrap the XS code. Perl then dynamically links the shared library into the perl process at run time.

Mytest.xs

The Mytest.xs file contains the XSUBs that tell Perl how to pass data to the compiled C routines. Initially, Mytest.xs will look something like this:

#include "EXTERN.h"
#include "perl.h"
#include "XSUB.h"

MODULE = Mytest                PACKAGE = Mytest

Let's edit the XS file by adding this to the end of the file:

void
hello()
    CODE:
        printf("Hello, world!\n");

When you run perl Makefile.PL, the Makefile that make needs will be created:

% perl Makefile.PL
Checking if your kit is complete...
Looks good
Writing Makefile for Mytest

Running make will now produce output that looks something like this (some long lines have been shortened for clarity and some extraneous lines have been deleted):

% make
umask 0 && cp Mytest.pm ./blib/Mytest.pm
perl xsubpp -typemap typemap Mytest.xs >Mytest.tc && mv Mytest.tc Mytest.c
cc -c Mytest.c
Running Mkbootstrap for Mytest ()
chmod 644 Mytest.bs
LD_RUN_PATH="" ld -o ./blib/PA-RISC1.1/auto/Mytest/Mytest.sl -b Mytest.o
chmod 755 ./blib/PA-RISC1.1/auto/Mytest/Mytest.sl
cp Mytest.bs ./blib/PA-RISC1.1/auto/Mytest/Mytest.bs
chmod 644 ./blib/PA-RISC1.1/auto/Mytest/Mytest.bs
Manifying ./blib/man3/Mytest.3

We assume that the make program that Perl uses to build programs is called make. Instead of running make in these examples, you may have to substitute whatever make program Perl has been configured to use. You can find out what that program is with:

% perl -V:make

Running make created a directory called blib (for "build library") in the current working directory. This directory will contain the shared library that we will build. Once we're sure we know what we're doing, we can install it from that directory into its final resting place. Until then, we'll have to explicitly add the blib directory to Perl's @INC array by using the ExtUtils::testlib module. If we now create a file called hello that looks like this:

use ExtUtils::testlib;   # adds blib/* directories to @INC
use Mytest;
Mytest::hello();

we can burrow down from Perl into C:

% perl hello
Hello, world!

Once your extension is complete and passes all its tests, you can install it with make install.

You will need write permission for your Perl library. (If you don't have permission, you can specify another directory as shown in "Installing CPAN modules" in Chapter 22, "CPAN".)

21.3.3. XSUB Input and Output

Continuing with the previous example, we'll add a second XSUB, which takes a single numeric argument as input and returns 0 if the number is even, or 1 if the number is odd:

int
is_even(x)
        int x
    CODE:
        RETVAL = (x % 2 == 0);
    OUTPUT:
        RETVAL

The list of output parameters occurs at the very end of the function, just after the OUTPUT: directive. The use of RETVAL tells Perl that you wish to send this value back as the return value of the XSUB. Had we wanted the function to modify its input parameter, we would have used x in place of RETVAL.

We can rebuild our new shared library with the same steps as before, generating a Makefile from the Makefile.PL file and running make.

In order to verify that our extension works, we'll create a test suite in test.pl. This file is set up by h2xs to mimic the test script that Perl itself has. Within the script, you may run tests to confirm that the extension behaves properly, printing ok when it does and not ok when it doesn't. Change the print statement in the BEGIN block of test.pl to print "1..4\n";, and add the following code to the end of the file:

print Mytest::is_even(0) == 1 ? "ok 2" : "not ok 2", "\n";
print Mytest::is_even(1) == 0 ? "ok 3" : "not ok 3", "\n";
print Mytest::is_even(2) == 1 ? "ok 4" : "not ok 4", "\n";

The test script will be executed when you type make test.

21.3.4. Using Functions from an External C Library

So far, our two examples haven't relied on any C code outside of the XS file. Now we'll use some functions from the C math library:

void
round(arg)
        double  arg
    CODE:
        if (arg > 0.0) {
                arg = floor(arg + 0.5);
        } else if (arg < 0.0) {
                arg = ceil(arg - 0.5);
        } else {
                arg = 0.0;
        }
    OUTPUT:
        arg

Note that the round we define above does not return a value, but instead changes the value of its argument in place.

The floor(3) and ceil(3) functions are part of the C math library. If you were compiling a C program and needed to link in the math library, you'd append -lm to the command line, so that's what you put into the LIBS line in Makefile.PL:

'LIBS'  => ['-lm'],   # Link in the 'm' math library

Generate the Makefile and run make. Change the BEGIN block to run nine tests and add the following to test.pl:

$i = -1.5; Mytest::round($i); print $i == -2.0 ? "ok 5" : "not ok 5", "\n";
$i = -1.1; Mytest::round($i); print $i == -1.0 ? "ok 6" : "not ok 6", "\n";
$i = 0.0; Mytest::round($i); print $i == 0.0 ? "ok 7" : "not ok 7", "\n";
$i = 0.5; Mytest::round($i); print $i == 1.0 ? "ok 8" : "not ok 8", "\n";
$i = 1.2; Mytest::round($i); print $i == 1.0 ? "ok 9" : "not ok 9", "\n";

Running make test should now print out that all nine tests are okay.

The perlxstut documentation bundled with Perl has several more examples of Perl extensions, including an example that uses h2xs to automatically make an entire C library available to Perl.