ArticlePDF Available

Motion and Structure from Motion in a Piecewise Planar Environment

September 1988
International Journal of Pattern Recognition and Artificial Intelligence 02(03)

Authors:

National Institute for Research in Computer Science and Control

We show in this article that when the environment is piecewise linear, it provides a powerful constraint on the kind of matches that exist between two images of the scene when the camera motion is unknown. For points and lines located in the same plane, the correspondence between the two cameras is a collineation. We show that the unknowns (the camera motion and the plane equation) can be recovered, in general, from an estimate of the matrix of this collineation. The two-fold ambiguity that remains can be removed by looking at a second plane, by taking a third view of the same plane, or by using a priori knowledge about the geometry of the plane being looked at. We then show how to combine the estimation of the matrix of collineation and the obtaining of point and line matches between the two images, by a strategy of Hypothesis Prediction and Testing guided by a Kalman filter. We finally show how our approach can be used to calibrate a system of cameras.

Content uploaded by Olivier D. Faugeras

Content may be subject to copyright.

inria-00075698, version 1 - 24 May 2006

A Robust Approach to Vision-Based Terrain-Aided Localization

Article

Feb 2025

Inertia Sensor Aided Alignment for Burst Pipeline in Low Light Conditions

Preprint

Nov 2018

Merging short-exposure frames can provide an image with reduced noise in low light conditions. However, how best to align images before merging is an open problem. To improve the performance of alignment, we propose an inertia-sensor aided algorithm for smartphone burst photography, which takes rotation and out-plane relative movement into account. To calculate homography between frames, a three by three rotation matrix is calculated from gyro data recorded by smartphone inertia sensor and three-dimensional translation vector are estimated by matched feature points detected from two frames. The rotation matrix and translations are combined to form the initial guess of homography. An unscented Kalman filter is utilized to provide a more accurate homography estimation. We have tested the algorithm on a variety of different scenes with different camera relative motions. We compare the proposed method to benchmark single-image and multi-image denoising methods with favorable results.

Sparse Depth Sensing for Resource-Constrained Robots

Preprint

Mar 2017

We consider the case in which a robot has to navigate in an unknown environment but does not have enough on-board power or payload to carry a traditional depth sensor (e.g., a 3D lidar) and thus can only acquire a few (point-wise) depth measurements. We address the following question: is it possible to reconstruct the geometry of an unknown environment using sparse and incomplete depth measurements? Reconstruction from incomplete data is not possible in general, but when the robot operates in man-made environments, the depth exhibits some regularity (e.g., many planar surfaces with only a few edges); we leverage this regularity to infer depth from a small number of measurements. Our first contribution is a formulation of the depth reconstruction problem that bridges robot perception with the compressive sensing literature in signal processing. The second contribution includes a set of formal results that ascertain the exactness and stability of the depth reconstruction in 2D and 3D problems, and completely characterize the geometry of the profiles that we can reconstruct. Our third contribution is a set of practical algorithms for depth reconstruction: our formulation directly translates into algorithms for depth estimation based on convex programming. In real-world problems, these convex programs are very large and general-purpose solvers are relatively slow. For this reason, we discuss ad-hoc solvers that enable fast depth reconstruction in real problems. The last contribution is an extensive experimental evaluation in 2D and 3D problems, including Monte Carlo runs on simulated instances and testing on multiple real datasets. Empirical results confirm that the proposed approach ensures accurate depth reconstruction, outperforms interpolation-based strategies, and performs well even when the assumption of structured environment is violated.

Mapping and Localization from Planar Markers

Preprint

Full-text available

Jun 2016

Squared planar markers are a popular tool for fast, accurate and robust camera localization, but its use is frequently limited to a single marker, or at most, to a small set of them for which their relative pose is known beforehand. Mapping and localization from a large set of planar markers is yet a scarcely treated problem in favour of keypoint-based approaches. However, while keypoint detectors are not robust to rapid motion, large changes in viewpoint, or significant changes in appearance, fiducial markers can be robustly detected under a wider range of conditions. This paper proposes a novel method to simultaneously solve the problems of mapping and localization from a set of squared planar markers. First, a quiver of pairwise relative marker poses is created, from which an initial pose graph is obtained. The pose graph may contain small pairwise pose errors, that when propagated, leads to large errors. Thus, we distribute the rotational and translational error along the basis cycles of the graph so as to obtain a corrected pose graph. Finally, we perform a global pose optimization by minimizing the reprojection errors of the planar markers in all observed frames. The experiments conducted show that our method performs better than Structure from Motion and visual SLAM techniques.

Full-Parameter Online Initialization Algorithm Based on Monocular ORB SLAM3

Chapter

Mar 2025

Minimal Relative Pose Estimation Using Translation-Decoupled Rotation for Planar Scenario

Article

Jan 2025
IEEE T INSTRUM MEAS

The minimal calibrated relative pose estimation in a planar scenario, which uses four 2D-2D image correspondences, has been effectively addressed through homography decomposition. Nonetheless, homography estimation through Direct Linear Transformation suffers from vulnerability to image noise when three points are (almost) collinear. To mitigate this issue, this paper proposes four alternative minimal relative pose solvers for the planar scenario. The proposed method separates rotation estimation from translation using the coplanar line constraint. To achieve this, the line directions are expressed in terms of the relative rotation using four image point correspondences. Enforcing the line coplanarity constraint on these lines results in two distinct sets of multivariate polynomial equations. The equation systems are subsequently solved using the hidden variable technique and Gröbner bases approach, yielding four relative rotation solvers. Apart from the relative rotation recovery for the general planar scene, rotation estimation with an available common reference direction is further considered. Once the relative rotation arrives, a unified formulation retrieves the translation and plane normal. To benchmark the proposed method against existing methods, simulations and real-world experiments were conducted under various planar scenarios and camera motion patterns. The results demonstrate that the proposed methods are 2-8 times more accurate than multiple existing methods regarding rotation estimates, and the translation estimation accuracy for these methods remains similar. In cases of degeneracy where three points are (almost) collinear, the proposed solvers effectively reduce rotation errors from 90° via the homography decomposition to approximately 7°. Overall, this paper offers a promising alternative for relative pose estimation in planar scenarios.

Non-contact sensing methods for wave measurement: current research and challenge

Article

Mar 2025

Non-contact methods for measuring wave characteristics play a crucial role in wave monitoring and engineering applications, aiding in the understanding and early warning of marine environments. Over the past decade, with the rapid advancement of sensing technologies and deep learning, research on non-contact wave measurement methods has made significant progress. To gain a comprehensive understanding of the current status and challenges, this paper provides a systematic review of non-contact wave characteristic measurement methods, discussing their applications in both oceanic and laboratory settings. It introduces various effective non-contact measurement techniques, such as stereo vision, optical, and electromagnetic wave measurements, and compares and summarizes the characteristics and applicability of each method. The advantages of different sensing technologies and their applications in wave monitoring are analyzed, emphasizing the improvement in measurement accuracy and efficiency through multi-source data fusion and deep learning algorithms. The review also summarizes the strengths and limitations of deep learning-driven monocular and stereo depth measurement methods. This review aims to provide comprehensive reference and guidance for future research and applications in wave characteristic measurement technologies in both oceanic and laboratory environments.

A Review of Simultaneous Localization and Mapping for the Robotic-Based Nondestructive Evaluation of Infrastructures

Article

Full-text available

Jan 2025
SENSORS-BASEL

The maturity of simultaneous localization and mapping (SLAM) methods has now reached a significant level that motivates in-depth and problem-specific reviews. The focus of this study is to investigate the evolution of vision-based, LiDAR-based, and a combination of these methods and evaluate their performance in enclosed and GPS-denied (EGD) conditions for infrastructure inspection. This paper categorizes and analyzes the SLAM methods in detail, considering the sensor fusion type and chronological order. The paper analyzes the performance of eleven open-source SLAM solutions, containing two visual (VINS-Mono, ORB-SLAM 2), eight LiDAR-based (LIO-SAM, Fast-LIO 2, SC-Fast-LIO 2, LeGO-LOAM, SC-LeGO-LOAM A-LOAM, LINS, F-LOAM) and one combination of the LiDAR and vision-based method (LVI-SAM). The benchmarking section analyzes accuracy and computational resource consumption using our collected dataset and a test dataset. According to the results, LiDAR-based methods performed well under EGD conditions. Contrary to common presumptions, some vision-based methods demonstrate acceptable performance in EGD environments. Additionally, combining vision-based techniques with LiDAR-based methods demonstrates superior performance compared to either vision-based or LiDAR-based methods individually.

Image-Based Visual Servoing With Collision-Free Path Planning for Monocular Vision-Guided Assembly

Article

Jan 2024

The demand for robotic assembly technology is increasing with the advancement of Industry 4.0. Image-based visual servoing (IBVS) is highly anticipated for robotic assembly since it enhances robotic systems’ flexibility and accuracy. However, since IBVS drives the robot by controlling image features in image space without 3-D position information, the Cartesian spatial path cannot be pre-planned, inevitably causing collision problems during assembly. This paper proposes a novel collision-free spatial path planning method for IBVS. Through perspective transformation, it is rigorously proved that there exists a unique path node on the line connecting the camera and target, enabling zero orientation error for the part here. By defining a virtual camera, image path midpoints corresponding to this Cartesian path node are generated as intermediate goals for visual servo control. Hence, a two-stage assembly path involving leveling followed by docking is devised, with orientation adjustment completed already in the leveling path to avoid assembly collisions during docking. The translational docking path is innovatively determined by the camera position, making it flexible and predictable. Then, a controller is presented to track planned paths. By deriving a depth-independent image Jacobian matrix, depth error interference on the docking path is eliminated. The closed-loop system’s dynamic stability is rigorously proved with Lyapunov methods. Simulation and experimental results confirm the significant regulation and improvement of spatial paths for IBVS, with 100% collision avoidance. Satellite prototype assembly results further illustrate the feasibility of using the proposed method in practical scenarios.

An Evolutionary Target Pose Measurement Based on Camera Motion Compensation

Conference Paper