No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Using a fiducial map metric for assessing map quality in the context of RoboCup Rescue
Abstract
Mapping is an important task for mobile robots in general and for Safety, Security, and Rescue Robotics (SSRR) in particular. It is hence one core aspect which is evaluated in the RoboCup Rescue league. But assessing the quality of maps in a simple and efficient way is not trivial, especially if no detailed, complete ground truth data of the environment is available. A new approach on map evaluation is presented here. It makes use of artificial objects placed in the environment named “fiducials”. Using the known ground-truth positions and the positions of the fiducials identified in the map, a number of quality attributes can be assigned to that map. Depending on the application domain those attributes can weighed to compute a final score. Results are presented that are based on using this method during the RoboCup Rescue competition 2010 in Singapore where maps were generated by different teams in an maze populated with fiducials. Those maps are evaluated here and compared to a human judgment, showing the effectiveness of the fiducial approach.
... Those attributes can include the coverage, the accuracy (Correctness of positions of features in the global reference frame) and topological measurements like the consistency of local groups of features. An approach using artificial Fiducials (barrels) that can calculate those attributes has been proposed in [21], [22] and is used as a tool for grading 2D maps at RoboCup Rescue competitions. This paper describes how this concept can be extended to 3D and performs some proof of concept experiments. ...
... The suggested map evaluation approach closely follows the Fiducial approach presented in [21] and [22]. For the ground level exactly the same method is used: The coverage is calculated by dividing the number of barrels identified by the number of barrels available at that height. ...
... Long-range consistency is measured for two barrel parts where the shortest robotdrivable path between places of observation of the two barrel parts is very long (over 8 pallets = 9.6 m) and short range consistency is measured for barrels with a shorter path between them. Please refer to [22] for more details. ...
For missions in Safety, Security, and Rescue Robotics (SSRR) maps are often a core deliverable. Hence it is of high interest to assess the quality of maps in a simple and efficient way. Since SSRR is mostly taking place in unstructured environments, 3D mapping has become more and more important. Here a method to evaluate the quality of 3D maps is presented that extends the previously developed 2D Fiducial approach to the third dimension. The artificial features are identified and located in 2D cross-sections of the map as well as in the 3D maps. It is then attempted to proof this concept using a ground truth map and robot generated maps from the RoboCup Rescue competition 2013 in Eindhoven.
... To ease place-based evaluation the use of artificial markers in the environment, called Fiducials, has been proposed [26,25]. In this place-based approach just the positions of those Fiducials has to be known to calculate the map attributes mentioned above. ...
... The input to our map evaluation are two dimensional grid maps, i.e., arrays of free or occupied cells [25]. We assume w.l.o.g. that any kind of "geo-referencing" information to project the maps to real-world metric coordinates, especially offsets and scalings, can be a priori extracted and stored as separate parameters. ...
... The correspondences are one to one mappings and they can then be used to identify matches between the evaluated map and the ground truth. Up to here, this is conceptually similar to previous work on map evaluation using place recognition like [35,19,26,25], which mainly differ in the computation of sim() -using different forms of natural landmarks or even artificial markers. ...
Mapping is an important task for mobile robots. The assessment of the quality of maps in a simple, efficient and automated way is not trivial and an ongoing research topic. Here, a new approach for the evaluation of 2D grid maps is presented. This structure-based method makes use of a topology graph, i.e., a topological representation that includes abstracted local metric information. It is shown how the topology graph is constructed from a Voronoi diagram that is pruned and simplified such that only high level topological information remains to concentrate on larger, topologically distinctive places. Several methods for computing the similarity of vertices in two topology graphs, i.e., for performing a place-recognition, are presented. Based on the similarities, it is shown how subgraph-isomorphisms can be efficiently computed and two topology graphs can be matched. The match between the graphs is then used to calculate a number of standard map evaluation attributes like coverage, global accuracy, relative accuracy, consistency, and brokenness. Experiments with robot generated maps are used to highlight the capabilities of the proposed approach and to evaluate the performance of the underlying algorithms.
... 2011. [Schwertfeger et al., 2011b] • Schwertfeger, S., Jacoff, A., Scrapper, C., Pellenz, J., and Kleiner, A. Evaluation of maps using fixed shapes: The fiducial map metric. In Proceedings of PerMIS. ...
... While the idea to use barrels as features that can be detected in maps originated from Mr. Jacoff, most other steps such as formalizing the approach and generating maps and results during experiments have been performed by the author. This work has been published in [Schwertfeger et al., 2010b] and [Schwertfeger et al., 2011b]. ...
... At RoboCup 2011 in Istanbul, Johannes Pellenz introduced the use of tachymeters to determine the positions of the fiducials [Schwertfeger et al., 2011b]. Bright markers, as shown in Figure 5.3, are put at the center point of the barrel circles. ...
Being able to generate maps is a significant capability for mobile robots. Measuring the performance
of robotic systems in general, but also particularly of their mapping, is important in different as-
pects. Performance metrics help to assess the quality of developed solutions, thus driving the research
towards more capable systems. During the procurement and safety testing of robots, performance
metrics ensure comparability of different robots and allow for the definition of standards.
In this thesis, evaluation methods for the maps produced by robotic systems are developed. Those
maps always contain errors, but measuring and classifying those errors is a non trivial task. The
algorithm has to analyze and evaluate the maps in a systematic, repeatable and reproducible way. The
problem is approached systematically: First the different terms and concepts are introduced and the
state of the art in map evaluation is presented.
Then a special type of mapping using video data is introduced and a path-based evaluation of
the performance of this mapping approach is made. This evaluation does not work on the produced
map, but on the localization estimates of the mapping algorithm. The rest of the thesis then works on
classical two-dimensional grid maps.
A number of algorithms to process those maps are presented. An Image Thresholder extracts
informations about occupied and free cells, while a Nearest Neighbor Remover or an Alpha Shape
Remover are used to filter out noise from the maps. This all is needed to automatically process the
maps.
Then the first novel map evaluation method, the Fiducial algorithm, is developed. In this place-
based method, artificial markers that are distributed in the environment are detected in the map. The
errors of the positions of those markers with respect to the known ground truth positions are used to
calculate a number of attributes of the map. Those attributes can then be weighted according to the
needs of the application to generate a single number map metric.
The main contribution of this thesis is the second novel map evaluation algorithm, that uses a graph
that is representing the environment topologically. This structure-based approach abstracts from all
other information in the map and just uses the topological information about which areas are directly
connected to asses the quality of the map. The different steps needed to generate this topological
graph are extensively described. Then different ways to compare the similarity of two vertices from
two graphs are presented and compared. This is needed to match two graphs to each other - the graph
from the map to be evaluated and the graph of a known ground truth map. Using this match, the same
map attributes as those from the Fiducial algorithm can be computed. Additionally, another interesting
attribute, the brokenness value, can be determined. It counts the large broken parts in the map that
internally contain few errors but that have, relative to the rest of the map, an error in the orientation
due to a singular error during the mapping process.
Experiments made on many maps from different environments are then performed for both map
metrics. Those experiments show the usefulness of said algorithms and compare their results among
each other and against the human judgment of maps.
... Those approaches have in common, that they use the pose of the detected features to determine the map quality by matching them to their counterparts in the ground truth. To ease this process, there is also the option to use artificial markers in the environment, so called fiducials [12], [13]. Just the positions of those fiducials has to be known to calculate the map attributes mentioned above. ...
... Generating the Topology Graph is a form of skeletonization. The input to the algorithm is a two dimensional grid-map which is colorized to occupied and free space [13]. After some simplification and pruning, the desired Topology Graph can be extracted from the Voronoi Diagram. ...
... Once the matching of the structures of two Topology Graphs is done this still has to be turned into a metric for assessing their similarity. Evaluating the map quality after matching the Topology Graph G of the map with the Topology Graph G 0 of a reference map is similar to the approach of the Fiducial Map Metric [13]. The map attributes are calculated as follows: ...
Mapping is an important task for mobile robots. But assessing the quality of maps in a simple, efficient and automated way is not trivial and an ongoing research topic. A new approach on map evaluation is presented here. It is based on Topology Graphs as a topological, abstracted representation of 2D grid maps. The Topology Graphs are derived from Voronoi Diagrams that get post-processed to capture the high-level spatial structures. Based on a similarity metric on vertices in Topology Graphs, the vertices can be matched across maps and spatial (dis)similarities and hence errors in the mapping can be identified and measured. More precisely, the vertex-similarity is the basis to match the structures of Topology Graphs up to the identification of subgraph isomorphisms through wave-front propagation. This allows to determine important map quality attributes up to very challenging structural elements like brokenness, i.e., the number of locally correct partitions in the candidate map and their relative placement towards each other. Experiments with real robot generated maps including examples from various teams in the RoboCup Rescue competition are used to validate the usefulness of this method for map quality assessment.
... This cluster, shown in purple in Fig. 3, includes Jacoff, Pellenz, and Suthakorn who have published multiple articles addressing performance evaluation and standard test method development for rescue robots [57]. On the other hand, authors grouped in cluster 1, shown in red in the figure, have contributed to RR from AI and computer science perspectives, namely Nejat [58], Brik [59], and Schwertfeger [60]. Moreover, the clustering of the in-network authors has captured the nationality as well as the mutual sub-fields of research and co-authorships. ...
The multidisciplinary nature of response robotics has brought about a diversified research community with extended expertise. Motivated by the recent accelerated rate of publications in the field, this paper analyzes the research trends, statistics, and implications of the literature from bibliometric standpoints. The aim is to study the global progress of response robotics research and identify the contemporary trends. To that end, we investigated the collaboration mapping together with the citation network to formally recognize impactful and contributing authors, publications, sources, institutions, funding agencies, and countries. We found how natural and human-made disasters contributed to forming productive regional research communities, while there are communities that only view response robotics as an application of their research. Furthermore, through an extensive discussion on the bibliometric results, we elucidated the philosophy behind research priority shifts in response robotics and presented our deliberations on future research directions.
... This cluster, shown in purple in Figure 3, includes Jacoff, Pellenz, and Suthakorn who have published multiple articles addressing performance evaluation and standard test method development for rescue robots [52]. On the other hand, authors grouped in cluster 1, shown in red in the figure, have contributed to RR from AI and computer science perspectives, namely Nejat [53], Brik [54], and Schwertfeger [55]. Moreover, the clustering of the in-network authors has captured the nationality as well as the mutual sub-fields of research and co-authorships. ...
The multidisciplinary nature of response robotics has brought about a diversified research community with extended expertise. Motivated by the recent accelerated rate of publications in the field, this paper analyzes the technical content, statistics, and implications of the literature from bibliometric standpoints. The aim is to study the global progress of response robotics research and identify the contemporary trends. To that end, we investigated the collaboration mapping together with the citation network to formally recognize impactful and contributing authors, publications, sources, institutions, funding agencies, and countries. We found how natural and human-made disasters contributed to forming productive regional research communities, while there are communities that only view response robotics as an application of their research. Furthermore, through an extensive discussion on the bibliometric results, we elucidated the philosophy behind research priority shifts in response robotics and presented our deliberations on future research directions.
... However, these methods have their own limitations because maps often have errors like structures appearing more than once due to localization errors. In [10], [11], high-level features like barrels for evaluation of maps both in 2D and 3D maps are applied [12]. The third group is to utilize the topology of the maps and use the matches for comparison, such as [13]. ...
This paper presents a fully hardware synchronized mapping robot with support for a hardware synchronized external tracking system, for super-precise timing and localization. Nine high-resolution cameras and two 32-beam 3D Lidars were used along with a professional, static 3D scanner for ground truth map collection. With all the sensors calibrated on the mapping robot, three datasets are collected to evaluate the performance of mapping algorithms within a room and between rooms. Based on these datasets we generate maps and trajectory data, which is then fed into evaluation algorithms. We provide the datasets for download and the mapping and evaluation procedures are made in a very easily reproducible manner for maximum comparability. We have also conducted a survey on available robotics-related datasets and compiled a big table with those datasets and a number of properties of them.
... In our previous work we used topological maps to evaluate the quality of the underlying 2D grid maps. This is interesting since all maps carry some degree of error, up to the point of severely bend or broken maps due to localization errors [8]. Our work determined the map quality by matching topological maps and measuring the error of the match [9] [10]. ...
Mapping is an important part of many robotic applications. In order to measure the performance of the mapping process we have to measure the quality of its result: the map. The map is essential for robotic algorithms like localization and path planning. Previously it was shown how matched Topology Graphs can be used for map evaluation by comparing the topology of the robot generated map to the topology of a ground truth map. In this paper we are extending the previous work by detecting open areas, for example rooms, in the 2D grid map and adding those to the topological representation. This way we can avoid the unreliable generation of paths in open areas, thus making the Topology Graph generation, and through that also the Topology Graph matching, more stable and robust. The detection applies the alpha shape algorithm for room detection.
... For that image similarity methods [6] and pixel-level feature detectors [7], [8] can be applied to the maps, but have their limitations, because maps often have errors like structures appearing more than once due to localization errors. More high level features like barrels are used for evaluation in [9], [10] and also in 3D maps in [11]. ...
This paper presents a fully hardware synchronized mapping robot with support for a hardware synchronized external tracking system, for super-precise timing and localization. We also employ a professional, static 3D scanner for ground truth map collection. Three datasets are generated to evaluate the performance of mapping algorithms within a room and between rooms. Based on these datasets we generate maps and trajectory data, which is then fed into evaluation algorithms. The mapping and evaluation procedures are made in a very easily reproducible manner for maximum comparability. In the end we can draw a couple of conclusions about the tested SLAM algorithms.
... Mapping and the evaluation of the generated maps is another research area that is important for the league. The Fiducial method for 2D grid map evaluation [12] has recently been extended to 3D maps, using data from the RoboCup Rescue competition [13]. ...
The RoboCup Rescue competitions have been initiated in 2000. To celebrate 16 years of research and development in this socially relevant initiative this article gives an overview of the experience gained during these competitions. This article provides an overview the state-of-the-art and the lessons learned from the RoboCup Rescue competitions.
... All maps have some degree of error which should be measured Schwertfeger et al. (2011). Recent work on map quality assessment matches the topology graph of a ground truth map to the topology graph of robot generated maps Schwertfeger and Birk (2015a), Schwertfeger and Birk (2013). ...
Topological maps have many applications in robotics. Matching two topological maps from the same environment can be used for map merging, place detection, map evaluation and other purposes. In this paper we present an approach to match two corresponding edges from two Topology Graphs to each other based on the actual path with which the vertices of the edges are connected in the underlying 2D grid maps. We perform experiments with two artificial maps as well as with four maps from the RoboCup Rescue WorldCup 2010.
... Recently, the use of artificial markers in the environment, so called Fiducials, has been proposed [13,14]. In this place-based approach just the positions of those Fiducials has to be known to calculate the map attributes mentioned above. ...
Mapping is an important task for mobile robots in general and in Safety, Security, and Rescue Robotics (SSRR) in particular - often maps are even a core mission deliverable for SSRR applications. The assessment of the quality of maps in a simple, efficient and automated way is hence of high interest. But it is not trivial and an ongoing research topic. Here, an overview of advances with a new approach on map evaluation is presented. This structure-based method makes use of a Topology Graph, a topological, abstracted representation of the map. The Topology Graph is constructed by generating and processing a Voronoi Diagram from a 2D grid map. Having a ground truth map, the Topology Graphs of both maps are matched using both similarity metrics on the vertices as well as structural matching of subgraph isomorphisms.
... The paper addresses only RGB-D sensors and it is not clear how the results can be extended to other sensors. Another approach is to use a fiducial map metric for assessing the quality of the created map (Schwertfeger et al., 2011). It makes use of a number of artificial objects (fiducials) which are placed in the environment at known positions. ...
In this paper we present results of an evaluation of sensors and perception algorithms in a large-scale emergency response exercise. We deployed state-of-the art sensors like Lidars and publicly available mapping approaches in a simulated car accident in a tunnel. The main goal is to investigate how well existing technologies are accepted by first responders for such scenarios. A rich sensor data set was recorded during a reconnaissance mission with a robot and later analyzed offline. We present first results of the representations generated and discuss what techniques are already accepted by responders. Finally, we raise issues that have to be tackled in order to increase the acceptance.
This paper presents a fully hardware synchronized mapping robot with support for a hardware synchronized external tracking system, for super-precise timing and localization. Nine high-resolution cameras and two 32-beam 3D Lidars were used along with a professional, static 3D scanner for ground truth map collection. With all the sensors calibrated on the mapping robot, three datasets are collected to evaluate the performance of mapping algorithms within a room and between rooms. Based on these datasets we generate maps and trajectory data, which is then fed into evaluation algorithms. We provide the datasets for download and the mapping and evaluation procedures are made in a very easily reproducible manner for maximum comparability. We have also conducted a survey on available robotics-related datasets and compiled a big table with those datasets and a number of properties of them.
A map representation method with an effective map building mechanism for representation of ruins environments is presented. Based on the hybrid metric-topological map representation, the ruins environment is described at different levels, according to the morphological characteristics in the interior of ruins environments after a seismic disaster. The whole environment is described on the basis of the global topological map, to ensure the system computing capability, ambient adaptability and friendly interaction, while building a local metric map at each topological node region, to represent clearly the irregular obstacle formed by a destructive pattern. The experiments in an artificial ruins environment demonstrate that the method can realize the simultaneous localization and mapping in complex environment, which indicates the feasibility and validity of the algorithm in practical search and rescue fields.
Robot navigation in complex, dynamic and unstructured environments demands robust mapping and localization solutions. One of the most popular methods in recent years has been the use of scan-matching schemes where temporally correlated sensor data sets are registered for obtaining a Simultaneous Localization and Mapping (SLAM) navigation solution. The primary bottleneck of such scan-matching schemes is correspondence determination, i.e. associating a feature (structure) in one dataset to its counterpart in the other. Outliers, occlusions, and sensor noise complicate the determination of reliable correspondences. This paper describes testing scenarios being developed at NIST to analyze the performance of scan-matching algorithms. This analysis is critical for the development of practical SLAM algorithms in various application domains where sensor payload, wheel slippage, and power constraints impose severe restrictions. We will present results using a high-fidelity simulation testbed, the Unified System for Automation and Robot Simulation (USARSim).
The maps generated by robots in real environment are usually incomplete, distorted, and noisy. The map quality is a quantitative performance measure of a robot's understanding of its environment. Map quality also helps researcher study the effects of different mapping algorithms and hardware components used. In this paper we present an algorithm to assess the quality of the map generated by the robot in terms of a ground truth map. To do that, First, localized features are calculated on the pre-evaluated map. Second, nearest neighbor of each valid local feature is searched between the map and the ground truth map. The quality of the map is defined according to the number of the features having the correspondence in the ground truth map. Three feature detectors are tested in terms of their effectiveness, these are the Harris corner detector, Hough Transform and Scale Invariant Feature Transform.
Mapping is an important task for mobile robots. Assessing the quality of those maps is an open topic. A new approach on map evaluation is presented here. It makes use of artificial objects placed in the environment named "Fiducials". Using the known ground-truth positions and the positions of the fiducials identified in the map, a number of quality attributes can be assigned to that map. Depending on the application domain those attributes are weighed to compute a final score. During the 2010 NIST Response Robot Evaluation Exercise at Disaster City an area was populated with fiducials and different mapping runs were performed. The maps generated there are assessed in this paper demonstrating the Fiducial approach. Finally this map scoring algorithm is compared to other approaches found in literature.
In this paper, we address the problem of creating an objective benchmark for evaluating SLAM approaches. We propose a framework
for analyzing the results of a SLAM approach based on a metric for measuring the error of the corrected trajectory. This metric
uses only relative relations between poses and does not rely on a global reference frame. This overcomes serious shortcomings
of approaches using a global reference frame to compute the error. Our method furthermore allows us to compare SLAM approaches
that use different estimation techniques or different sensor modalities since all computations are made based on the corrected
trajectory of the robot.
We provide sets of relative relations needed to compute our metric for an extensive set of datasets frequently used in the
robotics community. The relations have been obtained by manually matching laser-range observations to avoid the errors caused
by matching algorithms. Our benchmark framework allows the user to easily analyze and objectively compare different SLAM approaches.
This paper presents the map evaluation methodology developed for the Virtual Robots Rescue competition held as part of RoboCup.
The procedure aims to evaluate the quality of maps produced by multi-robot systems with respect to a number of factors, including
usability, exploration, annotation and other aspects relevant to robots and first responders. In addition to the design choices,
we illustrate practical examples of maps and scores coming from the latest RoboCup contest, outlining strengths and weaknesses
of our modus operandi. We also show how a benchmarking methodology developed for a simulation testbed effortlessly and faithfully
transfers to maps built by a real robot. A number of conclusions may be derived from the experience reported in this paper
and a thorough discussion is offered.
This paper introduces the RoboCup-Rescue Simulation Project, a
contribution to the disaster mitigation, search and rescue problem. A
comprehensive urban disaster simulator is constructed on distributed
computers. Heterogeneous intelligent agents such as fire fighters,
victims and volunteers conduct search and rescue activities in this
virtual disaster world. A real world interface integrates various sensor
systems and controllers of infrastructures in the real cities with the
virtual world. Real-time simulation is synchronized with actual
disasters, computing complex relationship between various damage factors
and agent behaviors. A mission-critical man-machine interface provides
portability and robustness of disaster mitigation centers, and
augmented-reality interfaces for rescue parties in real disasters. It
also provides a virtual reality training function for the public. This
diverse spectrum of RoboCup-Rescue contributes to the creation of the
safer social system
A quantitative assessment of the quality of a robot generated map is of high interest for many reasons. First of all, it allows
individual researchers to quantify the quality of their mapping approach and to study the effects of system specific choices
like different parameter values in an objective way. Second, it allows peer groups to rank the quality of their different
approaches to determine scientific progress; similarly, it allows rankings within competition environments like RoboCup. A
quantitative assessment of map quality based on an image similarity metric Ψ is introduced here. It is shown through synthetic as well as through real world data that the metric captures intuitive notions
of map quality. Furthermore, the metric is compared to a seemingly more straightforward metric based on Least Mean Squared
Euclidean distances (LMS-ED) between map points and ground truth. It is shown that both capture intuitive notions of map quality
in a similar way, but that Ψ can be computed much more efficiently than the LMS-ED.
USARSim is a high fidelity robot simulation tool based on a commercial game engine. We illustrate the overall structure of the simulator and we argue about its use as a bridging tool between the RoboCupRescue Real Robot League and the RoboCupRescue Simulation League. In particular we show some results concerning the validation of the system. Algorithms useful for the search and rescue task have been developed in the simulator and then executed on real robots providing encouraging results.
Various common error sources affect the quality of a map, e.g., salt and pepper noise and other forms of noise that are more
or less uniformly distributed over the map. But there also exist errors that only occur very rarely in the mapping process
but that have severe effects on the final result. They influence not only the local accuracy but also the whole spatial layout
of the map. Examples of related error sources include bump noise in the robot’s pose or residual errors in Simultaneous Localization
and Mapping (SLAM). The concept of brokenness is introduced in this article to capture the notion of structural errors in
grid maps. The map is partitioned into regions that are locally consistent with ground truth but “off” relative to each other.
Brokenness measures the number of these regions and their spatial relations. A theoretical basis is introduced to derive the
concept of brokenness in a formal way. Furthermore, it is shown how brokenness can be computed in an algorithmic way. Experiments
with maps from simulated as well as real world data are presented. They show that the metric can indeed be used to automatically
determine the structural quality of a map in a quantitative way.
In this paper we present a system to enhance the performance of feature correspondence based alignment algorithms for laser
scan data. We show how this system can be utilized as a new approach for evaluation of mapping algorithms. Assuming a certain
a priori knowledge, our system augments the sensor data with hypotheses (‘Virtual Scans’) about ideal models of objects in
the robot’s environment. These hypotheses are generated by analysis of the current aligned map estimated by an underlying
iterative alignment algorithm. The augmented data is used to improve the alignment process. Feedback between data alignment
and data analysis confirms, modifies, or discards the Virtual Scans in each iteration. Experiments with a simulated scenario
and real world data from a rescue robot scenario show the applicability and advantages of the approach. By replacing the estimated
‘Virtual Scans’ with ground truth maps our system can provide a flexible way for evaluating different mapping algorithms in
different settings.
During the initial phase of a disaster response it is essential for responders to quickly and safely assess the overall situation. The use of rescue robots that can autonomously navigate and map these environments can help responders realize this goal while minimizing danger to them. In order for rescue robots to be of service to the responders, they must be able to sense the environment, create an internal representation that identifies victims and hazards to responders, and provide an estimate of where they are and where they have been. Methods for developing a stable navigation solution are based on sensors that can be broadly classified into two approaches, absolute (exteroception) and relative (proprioception). Commonly, two or more of these approaches are combined to develop a stable navigation solution that is insensitive to and robust in the presence of the errors that plague partial solutions by taking into account errors in the vehicle's pose, thus bounding the uncertainty in the navigation solution. Since the capabilities and limitations of these approaches vary, it is essential for developers of robotic systems to understand the performance characteristics of methodologies employed to produce a stable navigation solution. This paper will provide quantitative analysis of two proprioceptive approaches, namely encoder-based odometry and inertial navigation system, and an exteroceptive approach namely visual odometry that uses scan matching techniques.
This thesis documents an experimental investigation into the map-building and exploration capabilities of a mobile robot. A map enables a robot to predict the state of its environment and plan its actions accordingly. This ability is essential in a wide range of practical applications. The map-building research begins with a thorough experimental evaluation of the robot's ultrasonic rangefinder, leading to a model which minimises the uncertainty caused by the wide beam and uneven signal strength of the sensor. Two types of map are used: a set of line and point features, and a grid-based free-space map. Potential features are extracted from the processed sonar data and classed as 'confirmed' if detected repeatedly. The free-space map is derived from the set of confirmed features. A distance transform algorithm is then used to plan paths on this map. This research places exceptional stress on the need for practical experimentation and quantitative, statistical, evaluation of the results. For this to be possible, it is essential to have a clearly-defined measure of map quality. A novel metric is defined which predicts the effectiveness of the robot if it were to use the map to execute a set of test tasks. This metric is shown to correspond closely to an intuitive understanding of quality. The confirmed features are used by a Kalman filter to estimate the robot's position relative to known objects. This localisation algorithm is shown to produce dramatic improvements in map quality in the later stages of exploration. Exploration strategies are tested experimentally in a range of environments and starting positions. The results are evaluated and compared statistically. The tested strategies range from totally reactive to primarily map-based. The most promising results are observed from hybrid exploration strategies which combine the robustness of reactive navigation and the directive power of map-based strategies.
The RoboCupRescue competition encourages the research on rescue robots: In a simulated disaster site, the task for the robots is to search for victims and to map the environment. The robots are either manually controlled or they explore autonomously. In addition to the number of victims, the quality of the maps and the mobility of the robots is evaluated and awarded in the competition. This paper gives an overview of the RoboCupRescue compe- tition and the test arena with its standardized test elements. Also, the mapping and exploring technique used by team resko@UniKoblenz on the autonomous robot "Robbie X" is presented. Finally, the current map scoring at the RoboCupRes- cue competition is described and a method to score the maps automatically is proposed. I. I NTRODUCTION To support first responders to find survivors in collapsed buildings (e.g. after an earth quake), rescue robots can be used to collect sensor data such as thermal and video images in areas that are inaccessible for humans. These robots are nowadays mostly remote-controlled. To relieve the operator from the exhausting task of controlling the robot, the avail abil- ity of partially or fully autonomous robots is desirable. Th ose robots must be able to explore an unstructured environment systematically. Therefore, they have to generate a map of the building to keep track of areas that were already inspected. These maps also help the first responders to find the victims that the robot has located. In the RoboCupRescue competition, such a disaster site is simulated by using standardized test elements that simulate walls, uneven ground and building stuc- tures such as stairs and ramps. The task for the (autonomous or teleoperated) rescue robots is to search for victims within this area while localizing itself and mapping the environment (14). This problem is widely known as the SLAM (Simultaneous Localization and Mapping) problem (6), (10). The team that finds the most victims and acquires the most accurate data about the victim and the building will win the competition. Special awards are given for the best autonomous robot and the robot that performs best in the part of the arena which is most difficult of access. During the RoboCup 2007 competition, the map quality was evaluated by "visual judgment": The accuracy was determined by comparing the printed maps generated by the different teams with maps of other teams and with the ground truth. In 2008, the organizers asked the teams to turn in the maps electronically in the GeoTIFF file format. Using the tags defined for GeoTIFF, the images/maps can be georeferenced and contain a scale. With this additional data, the maps can be overlayed with the ground truth map or other team's maps using a GeoTIFF map viewer. This makes the comparsion easier for a human, but a fully automated scoring is still not available yet. For an automatic scoring procedure, we propose the fol- lowing system: Given the generated map as an occupancy grid and the ground truth map, the rooms (or other significant features) are extracted from both maps. Since the rooms have characteristic properties (such as size and shape), they ca n be matched with the rooms of the other map. With these correspondencies, the best transformation to align the maps (either rotation and translation or a non-rigid transforma tion by additional warping) is determined. Finally, the remaining error is calculated. In the case of a rigid transformation, t his error is a measure for the quality of the map; for warped maps, the parameters of the warping function are also taken into account.
Disaster rescue is one of the most serious social issues that involves very large numbers of heterogeneous agents in the hostile environment. The intention of the RoboCup Rescue project is to promote research and development in this socially significant domain at various levels, involving multiagent teamwork coordination, physical agents for search and rescue, information infrastructures, personal digital assistants, a standard simulator and decision-support systems, evaluation benchmarks for rescue strategies, and robotic systems that are all integrated into a comprehensive system in the future. For this effort, which was built on the success of the RoboCup Soccer project, we will provide forums of technical discussions and competitive evaluations for researchers and practitioners. Although the rescue domain is intuitively appealing as a large-scale multiagent and intelligent system domain, analysis has not yet revealed its domain characteristics. The first research evaluation meeting will be held at RoboCup-2001, in conjunction with the Seventeenth International Joint Conference on Artificial Intelligence (IJCAI-2001), as part of the RoboCup Rescue Simulation League and RoboCup/AAAI Rescue Robot Competition. In this article, we present a detailed analysis of the task domain and elucidate characteristics necessary for multiagent and intelligent systems for this domain. Then, we present an overview of the RoboCup Rescue project.
Rescue robotics is an important steppingstone in the scientific chal- lenge to create autonomous systems. There is a significant market for res- cue robots, which has unique features that allow a fruitful combination of application oriented developments and basic research. Unlike other mar- kets for advanced robotics systems like service robots, the rescue robotics domain benefits from the fact that there is a human in the loop, which allows a stepwise transition from dumb teleoperated devices to truly au- tonomous systems. Current teleoperated devices are already very useful in this domain and they benefit from any bit of autonomy added. Human rescue workers are a scarce resource at disaster scenarios. A single op- erator should hence ideally supervise a multitude of robots. We present results from the rescue robots at the International University Bremen (IUB) in a core area supporting autonomy, namely mapping.
In the past many solutions for simultaneous localization and mapping (SLAM) have been presented. Recently these solutions have been extended to map large environments with six degrees of freedom (DoF) poses. To demonstrate the capabilities of these SLAM algorithms it is common practice to present the generated maps and successful loop closing. Unfortunately there is often no objective performance metric that allows to compare different approaches. This fact is attributed to the lack of ground truth data. For this reason we present a novel method that is able to generate this ground truth data based on reference maps. Further on, the resulting reference path is used to measure the absolute performance of different 6D SLAM algorithms building a large urban outdoor map.
Contents List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv List of Abbreviations and Symbols . . . . . . . . . . . . . . . . . xvii Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix 1 Introduction 1 1.1 The Navigation Problem . . . . . . . . . . . . . . . . . . . . 1 1.2 Why Use Sonar? . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 Choosing a Representation . . . . . . . . . . . . . . . . . . . 3 1.4 The Kalman Filter . . . . . . . . . . . . . . . . . . . . . . . 5 1.5 Data Association . . . . . . . . . . . . . . . . . . . . . . . . 10 1.6 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2 A Sonar Sensor Model 13 2.2 Previous Work . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.3 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.4 The Physics of Sonar . . . . . . . . . . . . . . . . . . . . . . 17 2.5 Predi
This article provides a comprehensive introduction into the field of robotic mapping, with a focus on indoor mapping. It describes and compares various probabilistic techniques, as they are presently being applied to a vast array of mobile robot mapping problems. The history of robotic mapping is also described, along with an extensive list of open research problems.
Mapping and Map Scoring at the RoboCupRescue Competition Quantitative Performance Evaluation of Navigation Solutions for Mobile Robots
- J Pellenz
- D Paulus
Robocuprescue interleague mapping challenge
- S Schwertfeger