Conference Paper

Automatic Face Replacement in Video Based on 2D Morphable Model

Hubei Key Lab. of Intell. Robot, Wuhan Inst. of Technol., Wuhan, China
DOI: 10.1109/ICPR.2010.551 Conference: Pattern Recognition (ICPR), 2010 20th International Conference on
Source: IEEE Xplore

ABSTRACT This paper presents an automatic face replacement approach in video based on 2D morphable model. Our approach includes three main modules: face alignment, face morph, and face fusion. Given a source image and target video, the Active Shape Models (ASM) is adopted to source image and target frames for face alignment. Then the source face shape is warped to match the target face shape by a 2D morphable model. The color and lighting of source face are adjusted to keep consistent with those of target face, and seamlessly blended in the target face. Our approach is fully automatic without user interference, and generates natural and realistic results.

  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Using generic interpolation machinery based on solving Poisson equations, a variety of novel tools are introduced for seamless editing of image regions. The first set of tools permits the seamless importation of both opaque and transparent source image regions into a destination region. The second set is based on similar mathematical ideas and allows the user to modify the appearance of the image seamlessly, within a selected region. These changes can be arranged to affect the texture, the illumination, and the color of objects lying in the region, or to make tileable a rectangular selection.
    ACM Transactions on Graphics 07/2003; 22:313-318. · 3.73 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Abstract Pasting somebody's face into an existing image with traditional photo retouching and digital image processing tools has only been possible if both images show the face from the same viewpoint and with the same illumination. However, this is rarely the case for given pairs of images. We present a system that exchanges faces across large differences in viewpoint and illumination. It is based on an algorithm that estimates 3D shape and texture along with all relevant scene parameters, such as pose and lighting, from single images. Manual interaction is reduced to clicking on a set of about 7 feature points, and marking the hairline in the target image. The system can be used for image processing, virtual try-on of hairstyles, and face recognition. By separating face identity from imaging conditions, our approach provides an abstract representation of images and a novel, high-level tool for image manipulation. Categories and Subject Descriptors (according to ACM CCS): I.3.7 [Computer Graphics]: Animation
    Computer Graphics Forum 09/2004; 23:669-676. · 1.60 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: In this paper, we present a complete system for automatic face replacement in images. Our system uses a large library of face images created automatically by downloading images from the internet, extracting faces using face detection software, and aligning each extracted face to a common coordinate system. This library is constructed off-line, once, and can be efficiently accessed during face replacement. Our replacement algorithm has three main stages. First, given an input image, we detect all faces that are present, align them to the coordinate system used by our face library, and select candidate face images from our face library that are similar to the input face in appearance and pose. Second, we adjust the pose, lighting, and color of the candidate face images to match the appearance of those in the input image, and seamlessly blend in the results. Third, we rank the blended candidate replacements by computing a match distance over the overlap region. Our approach requires no 3D model, is fully automatic, and generates highly plausible results across a wide range of skin tones, lighting conditions, and viewpoints. We show how our approach can be used for a variety of applications including face de-identification and the creation of appealing group photographs from a set of images. We conclude with a user study that validates the high quality of our replacement results, and a discussion on the current limitations of our system.
    ACM Trans. Graph. 01/2008; 27.

Full-text (2 Sources)

Available from
Oct 25, 2014