W. Burgard, C. Stachniss, G. Grisetti, B. Steder, R. Kümmerle, C. Dornhege, M. Ruhnke, A. Kleiner, and Juan D. Tardós.
A Comparison of SLAM Algorithms Based on a Graph of Relations.
In Proc. of the IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS). St. Louis, MO, USA, October 2009.


In this paper, we address the problem of creating an objective benchmark for comparing SLAM approaches. We propose a framework for analyzing the results of SLAM approaches based on a metric for measuring the error of the corrected trajectory. The metric uses only relative relations between poses and does not rely on a global reference frame. The idea is related to graph-based SLAM approaches in the sense that it considers the energy needed to deform the trajectory estimated by a SLAM approach to the ground truth trajectory. Our method enables us to compare SLAM approaches that use different estimation techniques or different sensor modalities since all computations are made based on the corrected trajectory of the robot. We provide sets of relative relations needed to compute our metric for an extensive set of datasets frequently used in the SLAM community. The relations have been obtained by manually matching laser-range observations. We believe that our benchmarking framework allows the user an easy analysis and objective comparisons between different SLAM approaches.

Download: Xvid Avi (3 MB)

BibTeX entry:

  author = {Burgard, W. and Stachniss, C. and Grisetti, G. and Steder, B. and
     K{\"u}mmerle, R. and Dornhege, C. and Ruhnke, M. and Kleiner, A. and
     Tard{\'o}s, Juan D.},
  title = {A Comparison of SLAM Algorithms Based on a Graph of Relations},
  booktitle = {Proc. of the {IEEE/RSJ} Int. Conf. on Intelligent Robots and Systems (IROS)},
  address = {St. Louis, MO, USA},
  month = {October},
  year = {2009}