Underappreciated Perl: Passing File Descriptors

2014-04-22

Despite my previous benchmarks, copying the video data from the AR.Drone to another process via a pipe caused a small but perceptible delay in the output. Then I remembered an old Unix trick that we can use in Perl: passing file descriptors around as integers.

On Unix, filehandles are just integers. STDIN, STDOUT, and STDERR are respectively numbered 0, 1, and 2. Each filehandle you open after that is generally numbered sequentially. You may also remember that networking sockets are just filehandles, which is what makes this trick work for UAV::Pilot.

In Perl, filehandles can have a more complicated structure, especially with the PerlIO layer introduced in perl 5.8. Some of these are not true filehandles in the Unix sense. For instance, when you open against a scalar reference:

my $data = '';
open( my $out, '>', \$data );
print $data "Writing to scalar";

Regardless, actual files and networking sockets still have an underlying Unix file descriptor. We can get the integer with fileno():

my $fd = fileno( $fh );

We can then fork() off and exec() a new program, passing the file descriptor as an option. (Remember, system(), `backticks`, and open( ..., '|-' ) and such are just fancy ways of doing fork() and exec()). But there’s a problem, which is that Perl sets a close-on-exec flag on all filehandles by default.

This means we have to unset the flag before doing anything else:

my $flags = fcntl( $fh, F_GETFD, 0 )
    or die "fcntl F_GETFD: $!";
fcntl( $fh, F_SETFD, $flags & ~FD_CLOEXEC )
    or die "fcntl F_SETFD: $!";

(This part is not portable to Strawberry Perl on Windows. The error says that F_GETFD is not configured for this vender. I think the actual technique of passing file descriptor numbers is otherwise portable. Windows is more Unixy than it likes to admit.)

(Edit: This is all possible under Windows, but you need to use some native Win32 APIs to get it done. See this Perlmonks SoPW for details.)

Now we can fork() and exec(). We also set $SIG{CHLD} = 'IGNORE'; so we don’t have to manage zombie children.

$SIG{CHLD} = 'IGNORE';
my $child_pid = fork();
if( $child_pid ) {
    # Parent
    while(1) { sleep 10 }
}
else {
    # Child
    exec( '/path/to/program', $fd )
        or die "Could not exec: $!\n";
}

Inside the target program, you have to change the integer back into Perl’s complex filehandle, which you can do with a funny looking open() call:

open( my $in, '<&', $fd ) or die "Could not open descriptor '$fd': $!\n";

And now you can read from $in like just any other filehandle.

After doing this in UAV::Pilot, that small but perceptible delay went away.