François Chaumette

National Institute for Research in Computer Science and Control | INRIA

This work proposes a QP-based visual servoing scheme for limiting motion blur during the achievement of a visual task. Unlike traditional image restoration approaches, we want to avoid any deconvolution step by keeping the image sequence acquired by the camera as sharp as possible. To do so, we select the norm of the image gradient as sharpness met...

In this letter, we present a visual control framework for accurately positioning feature points belonging to the surface of a 3D deformable object to desired 3D positions, by acting on a set of manipulated points using a robotic manipulator. Notably, our framework considers the dynamic behavior of the object deformation, that is, we do not assume t...

In this paper, we propose a shape visual servoing approach for manipulating a suspended cable attached between two quadrotor drones. A leader-follower control strategy is presented, where a human operator controls the rigid motion of the cable by teleoperating one drone (the leader), while the second drone (the follower) performs a shape visual ser...

To cite this version: Maxime Robic, Renaud Fraisse, Eric Marchand, François Chaumette. Vision-based rotational control of an agile observation satellite. Abstract-Recent Earth observation satellites are now equipped with new instrument that allows image feedback in real-time. Problematic such as ground target tracking, moving or not, can now be add...

Robotic systems are increasingly used to work in dynamic and/or unstructured environments and to operate with a high degree of safety and autonomy. Consequently, they are often equipped with external sensors capable of perceiving the environment (e.g. cameras) and the contacts that may arise (e.g.force/torque sensors). This paper proposes a general...

This entry presents the basic concepts of vision-based control, that is, the use of visual data to control the motions of a robotics system. It details the modeling steps allowing to achieve a visual task, the list of possible visual features that can be used as input of the control scheme, the design of basic kinematics control schemes, and hints...

This paper introduces the use of numerical optical flow (OF) in vision-based steering techniques - that control characters locomotion trajectories by using a simulation of their visual perception. In contrast with synthetic OF that was previously used, numerical OF is sensitive to the contrast of objects, and provides, for example, uncertain result...

This paper proposes a trifocal tensor-based approach for six-degree-of-freedom visual servoing. The trifocal tensor model among the current, desired, and reference views is constructed to describe the geometric relationship of the system. More precisely, to ensure the computation consistency of trifocal tensor, a virtual reference view is introduce...

Steering and navigation are important components of character animation systems to enable them to autonomously move in their environment. In this work, we propose a synthetic vision model that uses visual features to steer agents through dynamic environments. Our agents perceive optical flow resulting from their relative motion with the objects of...

This letter focuses on finding robust paths for a robotic system by taking into account the state uncertainty and the probability of collision. We are interested in dealing with intermittent exteroceptive measurements (e.g., collected from vision). We assume that these cues provide reliable measurements that will update a state estimation algorithm...

In this paper we investigate for the online genera- tion of optimal trajectories for target tracking with a quadrotor while satisfying a set of image-based and actuation constraints. We consider a quadrotor equipped with a camera (either down or front-looking) with limited field of view. The aim is to follow in a smooth but reactive way toward a mo...

In this paper, visual servoing based on photometric moments is advocated. A direct approach is chosen by which the extraction of geometric primitives, visual tracking and image matching steps of a conventional visual servoing pipeline can be bypassed. A vital challenge in photometric methods is the change in the image resulting from the appearance...

This paper proposes a control framework based on auditory perception. Generally, in robot audition, the motion control of a robot from the sense of hearing relies on sound source localization. We propose in this paper an alternative approach, aural servo, which is derived from the sensor-based control framework. In this approach, robot motions are...

This paper proposes a control framework allowing to position a robot in range and orientation with respect to a sound source from a binaural system. To this end, a sensor-based control framework is developed upon interaural level difference (ILD) cues. ILD is known to be implicitly related to the sound source azimuth but also to the sound source di...

The goal of this paper is to propose a coupling between the execution of an image-based visual servoing task and an active structure from motion strategy. The core idea is to modify online the camera trajectory in the null-space of the (main) servoing task for rendering the camera motion ‘more informative’ with respect to the estimation of the 3-D...

This paper presents image-based navigation from an image memory using a combination of line segments and feature points. The environment is represented by a set of key images, which are acquired during a prior mapping phase that defines the path to be followed during the navigation. The switching of key images is done exploiting the common line seg...

We present a deep neural network-based method to perform high-precision, robust and real-time 6 DOF visual servoing. The paper describes how to create a dataset simulating various perturbations (occlusions and lighting conditions) from a single real-world image of the scene. A convolutional neural network is fine-tuned using this dataset to estimat...

In this paper, we show that visual servoing can be formulated as an acceleration-resolved, quadratic optimization problem. This allows us to handle visual constraints, such as field of view and occlusion avoidance, as inequalities. Furthermore, it allows us to easily integrate visual servoing tasks into existing whole-body control frameworks for hu...

The aim of this paper is to show how visual servoing can help a humanoid robot to realize manipulation tasks with one and two hands, in conjunction with a gaze control. In addition, the use of vision in a closed-loop control scheme allows to accomplish these tasks with high repeatability without an accurate calibration of the robot kinematic model,...

This paper is a follow-up way to our previous works regarding audio-based control, that is an alternative method for auditory-based robot tasks. Conversely to classic methods oriented towards sound source localization, audio-based control is a sensor-based framework that does not localize the sound source. Instead, auditory features are used as inp...

This paper validates experimentally a novel approach to robot audition, sound-based control, which consists in introducing the auditory features directly as inputs of a closed-loop control scheme, that is, without any explicit localization process. The applications we present rely on the implicit bearings of the sound sources computed from the time...

This paper presents a method for image-based navigation from an image memory using line segments as landmarks. The entire navigation process is based on 2D image information without using any 3D information at all. The environment is represented by a set of reference images with overlapping landmarks, which are acquired during a prior learning phas...

This paper presents a novel approach to robot audition by performing robotic tasks with auditory cues. Unlike many previous works, we propose a control scheme, that does not require any explicit sound source localization. This ap- proach is capable of controlling all the three degrees of freedom in a plane from two microphones. Built upon the senso...

Visual servoing based on photometric data is of great interest since it does not necessitate any image processing or visual tracking steps. A vital issue in such methods is the change in the image resulting from the appearance and disappearance of portions of the scene from the camera field-of-view during the visual servo. In this paper, we propose...

In this paper we experimentally validate and compare three different methods for estimating the 3D parameters of a planar scene from a (possibly time-varying) set of feature points acquired by a moving monocular camera. The first method, based on the classical decomposition of the homography matrix, is meant to serve as a baseline condition classic...

The selection of a suitable set of visual features for an optimal performance of closed-loop visual control or Structure from Motion (SfM) schemes is still an open problem in the visual servoing community. For instance, when considering integral region-based features such as image moments, only heuristic, partial, or local results are currently ava...

One way to deal with occlusions or loss of tracking of the visual features used for visual servoing tasks is to predict the feature behavior in the image plane when the measurements are missing. Different prediction and correction methods have already been proposed in the literature. The purpose of this paper is to compare and experimentally valida...

This thesis is concerned with visual servoing, a feedback control technique for controlling camera-equipped actuated systems like robots. For visual servoing, it is essential to synthesize visual information from the camera image in the form of visual features and establish the relationship between their variations and the spatial motion of the cam...

In this paper, we illustrate the application of a nonlinear active structure estimation from motion (SfM) strategy to three problems, namely 3-D structure estimation for 1) a point, 2) a sphere, and 3) a cylinder. In all three cases, an appropriate parameterization reduces the problem to the estimation of a single quantity. Knowledge of this estima...

Moving obstacle avoidance is a fundamental requirement for any robot operating in real environments, where pedestrians, bicycles, and cars are present. In this paper, we propose and validate a framework for avoiding moving obstacles during visual navigation with a wheeled mobile robot. Visual navigation consists of following a path, represented as...

Predicting the behavior of visual features on the image plane over a future time horizon is an important possibility in many different control problems. For example when dealing with occlusions (or other constraints such as joint limits) in a classical visual servoing loop, or also in the more advanced model predictive control schemes recently prop...

Navigating within an unknown indoor environment using an electric wheelchair is a challenging task, especially if the user suffers from severe disabilities. In order to reduce fatigability and increase autonomy, control architectures have to be designed that would assist users in wheelchair navigation. We present a framework for vision-based autono...

Equipping mobile robots with an omnidirectional camera is very advantageous in numerous applications as all information about The surrounding scene is stored in a single image frame. In The given context, The present paper is concerned with detection, Tracking and following of a moving object with an omnidirectional camera. The camera calibration a...

Our objective is to extend the assistance functionalities of the cable-driven parallel robot Marionet-Assist, designed principally for transfer operation, by allowing it to grasp usual objects (knives, box of medicines, phone, ···) by using visual servoing. Our crane robot has a configuration that provides three translational d.o.f., and a camera w...

In this paper we propose a solution for coupling the execution of a visual servoing task with a recently developed active Structure from Motion strategy able to optimize online the convergence rate in estimating the (unknown) 3D structure of the scene. This is achieved by suitably modifying the robot trajectory in the null-space of the servoing tas...

Plane detection and estimation from visual data is a classical problem in robotic vision. In this work we propose a novel active strategy in which a monocular camera tries to determine whether a set of observed point features belongs to a common plane, and, if so, what are the associated plane parameters. The active component of the strategy impose...

Structure estimation from motion (SfM) is a classical and well-studied problem in computer and robot vision, and many solutions have been proposed To Treat it as a recursive filtering/estimation Task. However, The issue of actively optimizing The Transient response of The SfM estimation error has not received a comparable attention. In This paper,...

In this paper, we introduce the concept of tunable visual features for moments based visual servoing schemes. The main contribution of this work is the introduction of tunable shift points along with some effective methods to tune them. We propose two different metrics: the first metric ensures optimal response of the control to errors in the image...

This paper deals with pose estimation using an iterative scheme. We show that using adequate visual information, pose estimation can be performed iteratively with only three independent unknowns, which are the translation parameters. Specifically, an invariant to rotational motion is used to estimate the camera position. In addition, an adequate tr...

A framework is presented in this paper for the control of a multisensor robot under several constraints. In this approach, the features coming from several sensors are treated as a single feature vector. The core of our approach is a weighting matrix that balances the contribution of each feature, allowing the taking of constraints into account. Th...

h i g h l i g h t s • Image-based visual servoing from spherical projection. • Decoupled control using invariant features. • A near-linear behavior is obtained thanks to the proposed features. • The sensitivity to image noise is taken into account. a b s t r a c t This paper deals with the use of invariant visual features for visual servoing. New f...

This paper deals with pose estimation using an iterative scheme. We show that using adequate visual information, pose estimation can be performed in a decoupling the estimation of translation and rotation. More precisely, we show that pose estimation can be achieved iteratively as a function of only three independent unknowns, which are the transla...

This paper presents an image-based visual servoing for controlling the position and orientation of an unmanned aerial vehicle (UAV) using a fixed downward camera observing landmarks on the level ground. In the proposed method, the visual servoing of the image moments is used to control the vertical motion and rotation around the roll axis. In contr...

Moving obstacle avoidance is a fundamental re-quirement for any robot operating in real environments, where pedestrians, bicycles and cars are present. In this work, we design and validate a new approach that takes explicitly into account obstacle velocities, to achieve safe visual navigation in outdoor scenarios. A wheeled vehicle, equipped with a...

In this paper, a framework for visual servo control of a humanoid robot's upper body is presented. The framework is then implemented and tested on the REEM humanoid robot. The implementation is composed of 2 controllers - a head gaze control and a hand position control. The main application is precise manipulation tasks using the hand. For this, th...

In this paper, we propose a new type of visual features for visual servoing: photometric moments. These global features do not require any segmentation, matching or tracking steps. The analytical form of the interaction matrix is developed in closed form for these features. Results from experiments carried out with photometric moments have been pre...