Recovering non-rigid 3D shape from image streams
Top Cited Papers
- 7 November 2002
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 2 (10636919) , 690-696
- https://doi.org/10.1109/cvpr.2000.854941
Abstract
The paper addresses the problem of recovering 3D non-rigid shape models from image sequences. For example, given a video recording of a talking person, we would like to estimate a 3D model of the lips and the full face and its internal modes of variation. Many solutions that recover 3D shape from 2D image sequences have been proposed; these so-called structure-from-motion techniques usually assume that the 3D object is rigid. For example, C. Tomasi and T. Kanades' (1992) factorization technique is based on a rigid shape matrix, which produces a tracking matrix of rank 3 under orthographic projection. We propose a novel technique based on a non-rigid model, where the 3D shape in each frame is a linear combination of a set of basis shapes. Under this model, the tracking matrix is of higher rank, and can be factored in a three-step process to yield pose, configuration and shape. To the best of our knowledge, this is the first model free approach that can recover from single-view video sequences nonrigid shape models. We demonstrate this new algorithm on several video sequences. We were able to recover 3D non-rigid human face and animal models with high accuracy.Keywords
This publication has 12 references indexed in Scilit:
- Deformable model-based shape and motion analysis from images using motion residual errorPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Separability of pose and expression in facial tracking and animationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Recovering non-rigid 3D shape from image streamsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Linear Fitting with Missing Data for Structure-from-MotionComputer Vision and Image Understanding, 2001
- A morphable model for the synthesis of 3D facesPublished by Association for Computing Machinery (ACM) ,1999
- Making facesPublished by Association for Computing Machinery (ACM) ,1998
- Synthesizing realistic facial expressions from photographsPublished by Association for Computing Machinery (ACM) ,1998
- Learning to track the visual motion of contoursArtificial Intelligence, 1995
- Shape and motion from image streams under orthography: a factorization methodInternational Journal of Computer Vision, 1992
- Eigenfaces for RecognitionJournal of Cognitive Neuroscience, 1991