http://www.yapcna.org/yn2014/wiki?node=Hackathons
I’d like to give people who don’t currently own a UAV the opportunity to play with one. There’s also a lot of things I’d like to see as part of UAV::Pilot that could be hacked up.
The Galaxy S5 is the first phone I’ve owned that gets good reception in my basement. Don’t know what else you want.
UAV::Pilot::Video::Ffmpeg v0.2, UAV::Pilot, UAV::Pilot::WumpusRover v0.2, and UAV::Pilot::WumpusRover::Server v0.2
These modules just got some Yak Shaving done. Fixed up their CHANGELOG, spammed license headers on everything, and added a test for POD errors.
UAV::Pilot v0.10
Fixed a regression bug (and added a test case) of FileDump taking a filehandle.
UAV::Pilot::Command will now call uav_module_quit() on the implementation libraries for cleanup purposes.
Same Yak Shaving as above.
UAV::Pilot::ARDrone v0.2
Added bin/ardrone_display_video.pl for processing the video data. This can either be over STDIN, or in a raw filehandle. This is implemented in a way that should work for both Unixy operating systems and Windows.
Added UAV::Pilot::ARDrone::Video::Stream and UAV::Pilot::ARDrone::Video::Fileno to support the above.
In the uav shell for ARDrone, there is a parameter you can pass to start_video. Calling it without a parameter has the video stream handled in an external process with the fileno() method, which keeps the stream latency down. Calling it with 1 will instead use a pipe, which has a small but often noticeable lag in the video output. Calling with 2 will use the old fashioned way, which does not open an external process. Using an external process tends to take advantage of multicore CPUs better.
Nav data now correctly parses roll/pitch/yaw. The SDL output was updated to accommodate the corrected values.
Same Yak Shaving as above.
Given that the AR.Drone’s control system sends roll/pitch/yaw parameters as floats between 1.0 and -1.0, I thought the navdata sent back things the same way. I was never getting the output I expected in the SDL nav window, though.
Then last night, I was comparing how NodeCopter parses the same nav packets and seeing completely different numbers. Turns out the AR.Drone sends back numbers in millidegrees instead.
I feel dumb for not noticing this before. OTOH, this is only “documented” in the AR.Drone source code. The actual docs tell you to look at the navdata in Wireshark and figure out the rest for yourself.
Corrections are now up on github. Should be a new release of UAV::Pilot distros coming up soon. These releases will cleanup quite a few details that I wanted to get done before YAPC, so we should be in good shape.
Hopefully, the TSA won’t bother me too much with an AR.Drone in my carry-on. Some people at my local hackerspace managed to get a whole Power Wheels Racer, including the battery, into their carry-on, so I think I’ll be good.
Despite my previous benchmarks, copying the video data from the AR.Drone to another process via a pipe caused a small but perceptible delay in the output. Then I remembered an old Unix trick that we can use in Perl: passing file descriptors around as integers.
On Unix, filehandles are just integers. STDIN, STDOUT, and STDERR are respectively numbered 0, 1, and 2. Each filehandle you open after that is generally numbered sequentially. You may also remember that networking sockets are just filehandles, which is what makes this trick work for UAV::Pilot.
In Perl, filehandles can have a more complicated structure, especially with the PerlIO layer introduced in perl 5.8. Some of these are not true filehandles in the Unix sense. For instance, when you open against a scalar reference:
my $data = '';
open( my $out, '>', \$data );
print $data "Writing to scalar";
Regardless, actual files and networking sockets still have an underlying Unix file descriptor. We can get the integer with fileno():
my $fd = fileno( $fh );
We can then fork() off and exec() a new program, passing the file descriptor as an option. (Remember, system(), `backticks`, and open( ..., '|-' ) and such are just fancy ways of doing fork() and exec()). But there’s a problem, which is that Perl sets a close-on-exec flag on all filehandles by default.
This means we have to unset the flag before doing anything else:
my $flags = fcntl( $fh, F_GETFD, 0 )
or die "fcntl F_GETFD: $!";
fcntl( $fh, F_SETFD, $flags & ~FD_CLOEXEC )
or die "fcntl F_SETFD: $!";
(This part is not portable to Strawberry Perl on Windows. The error says that F_GETFD is not configured for this vender. I think the actual technique of passing file descriptor numbers is otherwise portable. Windows is more Unixy than it likes to admit.)
(Edit: This is all possible under Windows, but you need to use some native Win32 APIs to get it done. See this Perlmonks SoPW for details.)
Now we can fork() and exec(). We also set $SIG{CHLD} = 'IGNORE'; so we don’t have to manage zombie children.
$SIG{CHLD} = 'IGNORE';
my $child_pid = fork();
if( $child_pid ) {
# Parent
while(1) { sleep 10 }
}
else {
# Child
exec( '/path/to/program', $fd )
or die "Could not exec: $!\n";
}
Inside the target program, you have to change the integer back into Perl’s complex filehandle, which you can do with a funny looking open() call:
open( my $in, '<&', $fd ) or die "Could not open descriptor '$fd': $!\n";
And now you can read from $in like just any other filehandle.
After doing this in UAV::Pilot, that small but perceptible delay went away.
I’ve been running some crude benchmarks of the UAV::Pilot video timing. As I went over in my last post, I’m planning on having the video be read from the network in one process, and have it piped out to another process for decoding and display.
I added logging statements that show the exact time (using Time::HiRes::gettimeofday()) that a video packet comes in, and then another log for when we display it on the SDL window.
The first benchmark used the existing uav_video_display that’s in the UAV::Pilot distribution, reading from the file ardrone_video_stream_dump.bin. This file is in the UAV::Pilot::ARDrone distribution and is a direct dump of the network stream from an AR.Drone’s h.264 video port. It’s primarily used to run some of the video parsing tests in that distro.
I found that on my laptop, there was a delay of 12.982ms between getting the video frame and actually displaying it. At 60fps, there is a delay of 16.667ms between each frame, so this seems quite acceptable. The AR.Drone only goes up to 30fps, anyway, but it’s nice to know we have some leeway for future UAVs.
I then implemented a new script in the UAV::Pilot::ARDrone distro that read the same video frames from STDIN. I had planned on doing this with the same file noted above, like this:
cat ardrone_video_stream_dump.bin | ardrone_display_video.pl
But this ended up displaying only the last frame of video.
My theory on why this happens is that we use AnyEvent for everything, including reading IO and telling SDL when to display a new window. Using cat like that, there’s always more data for the AnyEvent->io watcher to grab, so SDL never gets a chance until the pipe is out of data. At that point, it still has the last frame in memory, so that’s what it displays.
I tried playing around with dd instead of cat, but got the same results.
So I broke down and connected to the actual AR.Drone with nc:
nc 192.168.1.1 5555 | ardrone_display_video.pl
Which did the trick. This does mean that the results are not directly comparable to each other. We can still run the numbers and make sure the delay remains insignificant, though.
And indeed it did. It averaged out to 13.025ms. That alleviates my concern that using a pipe would introduce a noticeable delay and things can go right ahead with this approach.
I’ve been going back into the various UAV::Pilot distros and trying to figure out how to best approach putting video and nav data together. Ideally, navigation would overlay information directly, with a standalone nav display perhaps being an option.
That doesn’t work, because the video uses a YUV overlay in SDL to spit all the pixels to screen at once. Because of whatever hardware magic SDL does to make this work, drawing on top of those pixels has a nasty flicker effect. SDL2 might solve this, but there hasn’t been much movement on the Perl bindings in a number of months.
Using OpenGL to use the YUV video data as a texture might also solve this, and I suspect it’s the way to go in the long term. Trouble is, Perl’s OpenGL docs are lacking. They seem to assume you already have a solid grounding in how to use OpenGL in C, and you just want to move over to Perl. I messed with OpenGL ES on Android (Java) a while back, but I’m more or less starting fresh. Still, working through an OpenGL book in C might be a good exercise, and then I can revisit this in Perl.
(If anybody else wants to take up the OpenGL stuff, I would wholeheartedly endorse it.)
It would be nice if SDL let you create two separate windows in the same process, but it doesn’t seem to like that.
The trick that’s already implemented in the full releases is to take an SDL window and subdivide it into different drawing areas. This meant implementing a half-assed layout system. It also ended up breaking in the last release, as I called SDL::Video::update_rect() on the whole window, which caused nasty visual issues with the YUV overlay.
That part is fixed now by only updating parts of the layout that want to be updated. Now the problem is that displaying the nav and video together causes a half-second or so lag in the video. This is unacceptable in what should be a real-time output.
I think the way to go will be to fork() off and display the video and nav in separate processes. The central process will manage all incoming data from the video and nav network sockets, and then pipe it to its children. Then there are separate SDL windows in each process. The UAV::Pilot::SDL::Window interface (that half-assed layout system) will probably still be implemented, but will be effectively vestigial for now.
This might mean parsing the nav stream redundantly in both the master process and the nav process. There are still things in the master process that would need the nav data. But it’s probably not a big deal.
It’ll also mean all the video processing can be done on a separate CPU core, so that’s cool.
Another benefit: currently, closing the SDL window when using the uav shell will exit the whole shell. There are probably some SDL parameters I could play with to fix this, but with separate processes, this is no longer a problem.
As a rule, stupidity is more likely than malice. The simple proof of this is that it’s easier to be incompetent than it is to be some grand chessmaster who sees all the pieces and manipulates them at a high level. So it is with Heartbleed.
Consider what had to go wrong for this bug to be introduced:
A group that wanted to deliberately subvert OpenSSL would need all of that to go wrong. If OpenSSL had tested builds for all combinations of compile flags, Heartbleed wouldn’t have happened. If they hadn’t built a custom allocator in the first place, Heartbleed wouldn’t have happened.
Video of my presentation, Perl Encryption Primer, which is a summary of the series of blog posts I’ve been doing here.
Theo de Raadt is known for general assholery, but when he says “OpenSSL is not developed by a responsible team”, there are very good reasons for him to say that. The project has been a mess for a long time, and this Heartbleed situation has brought it all to the forefront.
It’s long past time to ditch OpenSSL. Firefox and Chrome use NSS, which seems as good an alternative as any.
(Interestingly, there was a proposal to switch Chrome to OpenSSL just a few months ago. Yeah, let’s not.)
Copyright © 2024 Timm Murray
CC BY-NC
Opinions expressed are solely my own and do not express the views or opinions of my employer.