Real-time 3D visual sensor for robust object recognition.
ABSTRACT This paper presents a novel 3D measurement system, which yields both depth and color information in real time, by calibrating a time-of-flight and two CCD cameras. The problem of occlusions is solved by the proposed fast occluded-pixel detection algorithm. Since the system uses two CCD cameras, missing color information of occluded pixels is covered by one another. We also propose a robust object recognition using the 3D visual sensor. Multiple cues, such as color, texture and 3D (depth) information, are integrated in order to recognize various types of objects under varying lighting conditions. We have implemented the system on our autonomous robot and made the robot do recognition tasks (object learning, detection, and recognition) in various environments. The results revealed that the proposed recognition system provides far better performance than the previous system that is based only on color and texture information.
- SourceAvailable from: Vineet Gandhi[Show abstract] [Hide abstract]
ABSTRACT: The combination of range sensors with color cameras can be very useful for robot navigation, semantic perception, manipulation, and telepresence. Several methods of combining range- and color-data have been investigated and successfully used in various robotic applications. Most of these systems suffer from the problems of noise in the range-data and resolution mismatch between the range sensor and the color cameras, since the resolution of current range sensors is much less than the resolution of color cameras. High-resolution depth maps can be obtained using stereo matching, but this often fails to construct accurate depth maps of weakly/repetitively textured scenes, or if the scene exhibits complex self-occlusions. Range sensors provide coarse depth information regardless of presence/absence of texture. The use of a calibrated system, composed of a time-of-flight (TOF) camera and of a stereoscopic camera pair, allows data fusion thus overcoming the weaknesses of both individual sensors. We propose a novel TOF-stereo fusion method based on an efficient seed-growing algorithm which uses the TOF data projected onto the stereo image pair as an initial set of correspondences. These initial “seeds” are then propagated based on a Bayesian model which combines an image similarity score with rough depth priors computed from the low-resolution range data. The overall result is a dense and accurate depth map at the resolution of the color cameras at hand. We show that the proposed algorithm outperforms 2D image-based stereo algorithms and that the results are of higher resolution than off-the-shelf color-range sensors, e.g., Kinect. Moreover, the algorithm potentially exhibits real-time performance on a single CPU.Proceedings - IEEE International Conference on Robotics and Automation 01/2012;
Conference Paper: Robust Multi-Algorithm Object Recognition Using Machine Learning Methods[Show abstract] [Hide abstract]
ABSTRACT: Robust object recognition is a crucial requirement for many robotic applications. We propose a method towards increasing reliability and flexibility of object recognition for robotics. This is achieved by the fusion of diverse recognition frameworks and algorithms on score level which use charac- teristics like shape, texture and color of the objects. Machine Learning allows for the automatic combination of the respective recognition methods’ outputs instead of having to adapt their hypothesis metrics to a common basis. We show the applicability of our approach through several real-world experiments in a service robotics environment. Great importance is attached to robustness, especially in varying environments.IEEE International Conference on Multisensor Fusion and Information Integration; 01/2012
- [Show abstract] [Hide abstract]
ABSTRACT: We propose a method for learning novel objects from audio visual input. The proposed method is based on two techniques: out-of-vocabulary (OOV) word segmentation and foreground object detection in complex environments. A voice conversion technique is also involved in the proposed method so that the robot can pronounce the acquired OOV word intelligibly. We also implemented a robotic system that carries out interactive mobile manipulation tasks, which we call “extended mobile manipulation”, using the proposed method. In order to evaluate the robot as a whole, we conducted a task “Supermarket” adopted from the RoboCup@Home league as a standard task for real-world applications. The results reveal that our integrated system works well in real-world applications. KeywordsMobile manipulation–Object learning–Object recognition–Out-of-vocabulary–RoboCup@HomeJournal of Intelligent and Robotic Systems 04/2011; 66(1):187-204. · 0.83 Impact Factor