Robust face tracking with a consumer depth camera


We address the problem of tracking human faces under vari-ous poses and lighting conditions. Reliable face tracking is a challenging task. The shapes of the faces may change dramat-ically with various identities, poses and expressions. More-over, poor lighting conditions may cause a low contrast image or cast shadows on faces, which will significantly degrade the performance of the tracking system. In this paper, we de-velop a framework to track face shapes by using both color and depth information. Since the faces in various poses lie on a nonlinear manifold, we build piecewise linear face models, each model covering a range of poses. The low-resolution depth image is captured by using Microsoft Kinect, and is used to predict head pose and generate extra constraints at the face boundary. Our experiments show that, by exploiting the depth information, the performance of the tracking system is significantly improved.

Download full-text


Available from: Fei Yang,
  • [Show abstract] [Hide abstract]
    ABSTRACT: This paper explains the process of design and implementation of a face detection and tracking system able to locate the face of autism children with low functioning through a depth camera, that establishing the distance and angle relative to a reference point, controls in real-time the movement of the head of a robotic model. The system was developed using the Microsoft Kinect® SDK library, Artificial Vision techniques and the development environment LabVIEW, and makes part of a robotic prototype oriented to therapeutic support and diagnostic of autism spectrum disorder. The results show a good system performance, so the next step will be validate its functioning with real patients and then integrate it on the final model.
    Engineering Mechatronics and Automation (CIIMA), 2013 II International Congress of; 01/2013
  • [Show abstract] [Hide abstract]
    ABSTRACT: The difficulty of vision-based posture estimation is greatly decreased with the aid of commercial depth camera, such as Microsoft Kinect. However, there is still much to do to bridge the results of human posture estimation and the understanding of human movements. Human movement assessment is an important technique for exercise learning in the field of healthcare. In this paper, we propose an action tutor system which enables the user to interactively retrieve a learning exemplar of the target action movement and to immediately acquire motion instructions while learning it in front of the Kinect. The proposed system is composed of two stages. In the retrieval stage, nonlinear time warping algorithms are designed to retrieve video segments similar to the query movement roughly performed by the user. In the learning stage, the user learns according to the selected video exemplar, and the motion assessment including both static and dynamic differences is presented to the user in a more effective and organized way, helping him/her to perform the action movement correctly. The experiments are conducted on the videos of ten action types, and the results show that the proposed human action descriptor is representative for action video retrieval and the tutor system can effectively help the user while learning action movements.
    Cybernetics, IEEE Transactions on 07/2014; 45(4). DOI:10.1109/TCYB.2014.2335540 · 3.47 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Facial features tracking is widely used in face recognition, gesture, expression analysis, etc. AAM (Active Appearance Model) is one of the powerful methods for objects feature localization. Nevertheless, AAM still suffers from a few drawbacks, such as the view angle change problem. We present a method to solve it by using the depth data acquired from Kinect. We use the depth data to get the head pose information and RGB data to match the AAM result. We establish an approximate facial 3D gird model and then initialize the subsequent frames with this model and head pose information. To avoid the local extremum, we divide the model into several parts by the poses and match the facial features with the closest model. The experimental results show improvement of AAM performance when rotating the head.
    Cybernetics and Information Technologies 09/2015; 15(3). DOI:10.1515/cait-2015-0046