Callback Functions Between C and Perl

2017-02-21

Everything you ever want to do with xs is documented somewhere in perlxs, perlguts, perlapi, perlxstypemap, and perlcall. Figuring out where it’s documented, and how it relates to everything else, is the hard part.

In my case, the part I wanted was in perlcall, in the section “Strategies for Storing Callback Context Information”. What I wanted to do was to glue together the callbacks for the Perl interface in Games::Chipmunk to the Chipmunk 2D physics library.

From Perl, we want to be able to say things like this:

cpBodySetVelocityUpdateFunc( $ballBody, sub {
    my ($body, $gravity, $damping, $dt) = @_;
    cpBodyUpdateVelocity( $body, $gravity, $damping, $dt );
    return;
});

The C version of cpBodySetVelocityUpdateFunc() does not know what a Perl sub is. It sees an SV, which happens to contain a bunch of things that the Perl interpreter can execute as a Perl sub. What we need to do is hand off a C function to the callback, and then use that to grab our SV and use call_sv() to call it.

If C supported closures, this would be easy. C does not support closures.

Some C libraries with callbacks have a parameter that will pass whatever data you give it directly into the callback later on. Chipmunk has this on a few functions, but not consistently.

One thing the Chipmunk libraries do give us is a pseduo-Object Oriented interface, where we pass the associated datastructure in as the first parameter on every function (Object Oriented languages move this parameter to the left of the function call). This gives us something we can grab on to for getting the SV we need that stores our Perl sub.

The “object” is a pointer, and pointers are just numbers. Numbers can be looked up in a hash. So we’ll make a bunch of global hashes, one for each set of callbacks, and use the address as the lookup key and the SV as the value.

The examples below use Perl’s context macros, which means those global hashes are still thread-safe. You can read about them in perlxs, under the section “Safely Storing Static Data in XS”. Their use won’t be detailed here.

Using cpBodySetVelocityUpdateFunc as an example, we start with the xs declaration:

void
cpBodySetVelocityUpdateFunc( body, velocityFunc )
    cpBody *body
    SV* velocityFunc
  PREINIT:
    dMY_CXT;
  CODE:
    hv_store(
        MY_CXT.bodyVelocityFuncs,
        (char*)&body,
        sizeof(body),
        velocityFunc,
        0
    );

    cpBodySetVelocityUpdateFunc(
        body,
        (cpBodyVelocityFunc) __perlCpBodyVelocityFunc
    );

We’re taking an SV* and assuming it holds a reference to the sub. We store it in the HV* bodyVelocityFuncs, which is initialized elsewhere. Using (char*)&body (C is a true bastion of type safety), we convert the address of the cpBody* into a char pointer, which the hash can use as a key. Lastly, we call the actual cpBodySetVelocityUpdateFunc() in the C library, and pass it our own C function as a callback.

That C function looks like this:

static void __perlCpBodyVelocityFunc(
    cpBody* body,
    cpVect gravity,
    cpFloat damping,
    cpFloat dt
);
static void
__perlCpBodyVelocityFunc(
    cpBody* body,
    cpVect gravity,
    cpFloat damping,
    cpFloat dt
) {
    dTHX;
    dSP;
    dMY_CXT;
    SV ** perl_func = hv_fetch(
        MY_CXT.bodyVelocityFuncs,
        (char*)&body,
        sizeof(body),
        FALSE
    );
    if( perl_func == (SV**) NULL ) {
        croak( "No cpBodyVelocityFunc found" );
    }

    PUSHMARK(SP);
    EXTEND( SP, 4 );
    PUSHs( sv_2mortal( sv_setref_pv( newSV(0), "cpBodyPtr", body ) ) );
    PUSHs( sv_2mortal( sv_setref_pv( newSV(0), "cpVectPtr", &gravity ) ) );
    PUSHs( sv_2mortal( newSVnv( damping ) ) );
    PUSHs( sv_2mortal( newSVnv( dt ) ) );
    PUTBACK;

    call_sv( *perl_func, G_VOID );
}

This goes at the top of your xs file, before any PACKAGE declarations, up with #include "EXTERN.h" and such. The perlxs doc doesn’t show this part in its examples very well, but you need to put the dTHX; call here to declare the context for a bunch of Perl macros that come later. Otherwise, you’ll get a bunch of cryptic compiler errors and spend a few hours scratching your head, until you finally come across the section in perlguts entitled “How multiple interpreters and concurrency are supported”. Like I said, everything is documented, you just won’t know where.

Going past the Perl macros, we get to hv_fetch(). This function should have gotten the same cpBody* as we got earlier, so we once again torture the type system and pretend it’s a pointer to a char array for the sake of a hash lookup key. The person who wrote this part of the interface was a two-star C programmer, so we better check that we actually got the SV we wanted by checking perl_func == (SV**) NULL.

Now we’ll need to grow the stack so we can push the subref’s arguments there. In this case, the callback received four arguments, and we want to pass all four to the subref. In this particular case, the last two arguments are easy. They’re double precision floats, which we can pass directly into a Perl SV that contains a number.

The first two are the tricky ones. They’re pointers to complex structs. In my typemap, I made cpBody* into a T_PTROBJ. This means it converts the ‘*’ into ‘Ptr’, collapses any whitespace, and uses the resulting name as the Perl class. You can make an SV contain a pointer to this C object and then carry it around in Perl. The cpVect* comes out the same way (since we take a pointer to the gravity struct).

Finally, we can call call_sv() to call the actual subref.

The good news is that Games::Chipmunk v0.3 has most of the callbacks implemented. There’s some in cpSpatialIndex that aren’t there because they don’t follow the same conventions. Still, this should be good enough to write real physics systems for games in Perl.