Multiclass Multimodal Detection and Tracking in Urban Environments

L. Spinello , R. Triebel and R. Siegwart

International Journal of Robotics Research (IJRR)

Keywords: multimodal people and vehicle detection, camera and laser people and car detection, sensor fusion, ISMe, Conditional Random Fields


This paper presents a novel approach to detect and track people and cars based on the combined information retrieved from a camera and a laser range scanner. Laser data points are classified by using boosted Conditional Random Fields (CRF), while the image based detector uses an extension of the Implicit Shape Model (ISM),  which learns a codebook of local descriptors from a set of  hand-labeled images and uses them to vote for centers of detected  objects. Our extensions to ISM include the learning of object parts and template masks to obtain more distinctive votes for the particular object classes. The detections from both sensors are then fused and the objects are tracked using a Kalman Filter with multiple motion models. Experiments conducted in real-world urban scenarios demonstrate the effectiveness of our approach.


Paper Preprint (PDF) Video (Divx codec) Bibtex entry


Multiclass Multimodal Detection and Tracking in Urban Environments