Capturing USB2 video at 35MB/sec on Linux

Problem: many USB cameras require the full bandwidth of a USB2.0 port without any interruptions. Unfortunately the kernel power saving features in modern linux distributions periodically switch the processor into sleep states that disrupt USB data transfer and cause frames to be dropped. As an example our uEye 25fps 1.3Mpix machine-vision cameras will often drop 50% of frames, not ideal from an expensive camera. The problem is with the operating system and not the hardware. It has been documented on the uEye cameras by IDS. The same problem happens with the Kinect RGBD sensor, with 10-50% of frames frequently being lost.

We have observed this problem with Intel processors including Core 2, Core i5 and i7, and running on Ubuntu 9.10, 10.04, 10.10, 11.04. It would likely occur on any Linux distribution with kernel 2.6+.  The different sleep "C-states" on Intel's processors are discussed here. It might seem counter-intuitive, however a faster PC is likely to drop more video frames.

Verify: One way to verify if the C-states are causing the issue is to heavily load the CPU so it doesn't enter any low power modes. Start capturing video data and then run something CPU intensive, eg. `cat /dev/urandom | gzip > /dev/null` will load one or two CPU cores. If you have a 4 or 8 core processor you'll want to run this multiple times. If the video capture performance improves, this is your problem.

You can check what C-states your processor is entering using Intel's powertop command line tool (`sudo apt-get install powertop` and run `powertop`).

Some BIOSs allow you to set the maximum C-state, however the operating system (both Windows or Linux) appear to ignore this setting. For Ubuntu linux the intel_idle kernel module must be instructed to not to enter any sleep C-states below 0 (or 1). However, the module is loaded at boot and despite offering a /sys interface cannot be changed after loading:
echo 0 > /sys/module/intel_idle/parameters/max_cstate
bash: /sys/module/intel_idle/parameters/max_cstate: Permission denied

Solution: on Ubuntu you need to pass the max_cstate level as a parameter during boot. This is easily done by editing the Grub config file /etc/default/grub. Add or edit the GRUB_CMDLINE_LINUX line to say:

GRUB_CMDLINE_LINUX="processor.max_cstate=0 intel_idle.max_cstate=0"
or if you'd like to try and retain some power savings:

GRUB_CMDLINE_LINUX="processor.max_cstate=1 intel_idle.max_cstate=1"
Update your Grub menu with `sudo update-grub` and reboot.

After booting confirm the C-states have been disabled by running `powertop` again. It will either indicate the processor isn't entering the states greater than zero, or not show any C-state information. Unfortunately this will increase your PC's power usage, however that's not likely a problem if you're processing live video at 35MB/sec!

Large-scale Multi-robot Mapping in MAGIC 2010

5th IEEE Conference on Robotics, Automation and Mechatronics (RAM 2011)

Authors: Robert Reid, Thomas Bräunl

Abstract: We describe a large-scale decentralised multi-robot mapping system that outputs globally optimised metric maps in real-time. The mapping system was used by team WAMbot in the finals of the Multi-Autonomous Ground-robotics International Challenge (MAGIC 2010). Research contributions include a novel large-scale multi-robot graph-based non-linear map optimisation approach, a hybrid decentralised and distributed mapping system and novel graphics processing unit (GPU) based approaches for accelerating intensive map matching and fusion operations. Our mapping system scales linearly with map size and on commodity hardware can easily map a 500m×500m urban area. We demonstrate robust, highly efficient and accurate mapping results from two different fleets of mobile robots. Videos, maps and timing results from the MAGIC 2010 challenge are presented.

Index Terms: Large-scale multi-robot mapping, simultaneous localisation & mapping (SLAM), graph-based SLAM, distributed, decentralised, GPU map fusion, GPU map correlation.

 RAM2011 Large-scale Multi-robot Mapping (1.1MB PDF) 

 BibTeX

Videos: These videos present the output of the Mapbuilder system and play at 15 × real-time speed. They are best viewed full-screen and at 1920×1080 (click the full-screen icon in the lower right of the video and also select 1080p). The global occupancy maps are a top-down view showing obstacles, free-space and unknown areas in the environment. The submap spatial constraints are shown as blue lines. We note that tunable parameters in the mapping system were not changed when moving between environments.

MAGIC 2010 Challenge Phase 2: This video shows 7 UGVs exploring an agricultural show-ground, passing in and out of horse stables. Figure 5 in the conference paper shows the final map overlaid on aerial imagery. The 200m× 160m map is metrically accurate and correctly georeferenced by the intermittent and noisy GPS data. To quantify the RMS error in the final map we selected a set of evenly distributed features in the map and measured their linear error to the aerial image. The estimated mean linear error was 0.57m. The final map has 446 submaps, 419 ground-truth constraints and 1284 submap constraints. For the final map each execution cycle it took 21ms to incrementally optimise the entire pose-graph, and 87ms to build and output the global occupancy map. The initial submaps take some time to become connected due to occlusions and ambiguous geometry in the matching process, however after driving several meters each UGV correctly joins its current submap to the global graph. Slow drifts in submaps are observed as transient GPS noise is filtered into ground-truth data. If additional ground truth constraints had been supplied by the operator, spatial errors could have been reduced by an order of magnitude.

MAGIC 2010 Old Ram Shed Challenge: This video shows 5 UGVs exploring a 70m ×40m agricultural shed. Again the submaps take some time to become connected however once the UGVs begin moving the map is rapidly constrained and is metrically accurate. Note that barriers inside the shed were temporary and very few walls and angles were regular. No ground-truth was made available to evaluate the accuracy of this map. The final map has 260 submaps and 2170 submap constraints. For the complete map each execution cycle took 23ms to incrementally optimise the pose graph and 85ms to build and output the global occupancy map.

Cooperative Multi-Robot Navigation and Mapping of Unknown Terrain

5th IEEE Conference on Robotics, Automation and Mechatronics (RAM 2011)


Authors: Adrian Boeing, Thomas Bräunl, Robert Reid, Aidan Morgan, Kevin Vinsen

Abstract: We describe the hardware, software and sensor equipment for a fleet of mobile robots used for cooperative multi-robot navigation and mapping. This robot team was developed for the MAGIC 2010 Robotics Challenge and successfully made it through two down-selection processes to the finals, where it placed 4th. Our mobile robot design leverages off-the-shelf hardware and open source software with novel software contributions to realize cooperative navigation, mapping, reconnaissance, and surveillance tasks for a large urban environment.

 RAM2011 Cooperative Multi-Robot Navigation and Mapping (1.1MB PDF) 

 BibTeX

Backing Up Gmail and Google Apps Mail using Ubuntu

I like using cloud-based services... especially Gmail since I can sit anywhere and access all my email. But what if Google has another failure like last week? I previously used Thunderbird and IMAP to download a copy of my emails until the "All Mail" folder got too large and I couldn't be sure I was getting a complete backup.

Lifehacker did an article on backing up your gmail. The command-line option they gave (fetchmail) is clunky and requires a local MTA. A few searches and a few clicks and I had the light-weight open-source "offlineimap" program working perfectly.

OfflineIMAP has a simple mode for Gmail. It's very easy to get it to copy your "All Mail" gmail folder to a local drive. It uses the convenient maildirs format where each email becomes a text file in a folder.

First enable IMAP access for your account.

Next install offlineimap using apt:
sudo apt-get install offlineimap
Create a configuration file for your user: `nano ~/.offlineimaprc`:
[general]
accounts = Gmail
ui = Noninteractive.Basic

[Account Gmail]
localrepository = LocalMaildirRepository
remoterepository = ServerRepository

[Repository LocalMaildirRepository]
type = Maildir
localfolders = /backup/Maildir_Gmail/

[Repository ServerRepository]
type = Gmail
remoteuser = user@googleappsdomain.com
remotepass = paaasswd
realdelete = no
folderfilter = lambda folder: folder in ['[Google Mail]/All Mail','[Gmail]/All Mail']

The configuration above restricts the download to the "All Mail" folder: I didn't want to download multiple copies of emails that have more than one label.

Test your configuration by running `offlineimap`:
$ offlineimap
OfflineIMAP 6.2.0
Copyright (C) 2002 - 2009 John Goerzen
This software comes with ABSOLUTELY NO WARRANTY; see the file
COPYING for details. This is free software, and you are welcome
to distribute it under the conditions laid out in COPYING.
***** Processing account Gmail
Copying folder structure from Gmail to Maildir
Establishing connection to imap.gmail.com:993.
Syncing [Google Mail]/All Mail: Gmail -> Maildir
Copy message 1 Gmail[[Google Mail]/All Mail] -> Maildir[[Google Mail].All Mail], LocalStatus[[Google Mail].All Mail]
Copy message 2 Gmail[[Google Mail]/All Mail] -> Maildir[[Google Mail].All Mail], LocalStatus[[Google Mail].All Mail]
Copy message 3 Gmail[[Google Mail]/All Mail] -> Maildir[[Google Mail].All Mail], LocalStatus[[Google Mail].All Mail]
...

You can start/stop and resume this download. It might pay to check the permissions on ~/.offlineimaprc. My version automatically locked the file so other users cant see the password.

To schedule a 2am nightly synchronisation using crontab: `sudo nano /etc/crontab`. Add this line at the bottom of the file:
0 2 * * * your_username offlineimap