Peter Sand
Abstract
We describe a method for the acquisition of deformable human geometry from silhouettes. Our technique uses a commercial tracking system to determine the motion of the skeleton, then estimates geometry for each bone using constraints provided by the silhouettes from one or more cameras. These silhouettes do not give a complete characterization of the geometry for a particular point in time, but when the subject moves, many observations of the same local geometries allow the construction of a complete model. Our reconstruction algorithm provides a simple mechanism for solving the problems of view aggregation, occlusion handling, hole filling, noise removal, and deformation modeling. The resulting model is parameterized to synthesize geometry for new poses of the skeleton. We demonstrate this capability by rendering the geometry for motion sequences that were not included in the original datasets.
Supplemental Material
- ALLEN, B., CURLESS, B., AND POPOVIĆ, Z. 2002. Articulated body deformation from range scan data. In Proceedings of the 29th annual conference on Computer graphics and interactive techniques (SIGGRAPH), 612--619. Google Scholar
Digital Library
- BLANZ, V., AND VETTER, T. 1999. A morphable model for the synthesis of 3d faces. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques (SIGGRAPH), 187--194. Google ScholarDigital Library
- BRAND, M. 2001. Morphable 3D models from video. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (CVPR), II:456--463.Google ScholarCross Ref
- BREGLER, C., HERTZMANN, A., AND BIERMANN, H. 2000. Recovering non-rigid 3D shape from image streams. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (CVPR), II:690--696.Google ScholarCross Ref
- BROOMHEAD, D., AND LOWE, D. 1988. Multivariable functional interpolation and adaptive networks. Complex Systems 2, 3, 321--355.Google Scholar
- CARR, J. C., BEATSON, R. K., CHERRIE, J. B., MITCHELL, T. J., FRIGHT, W. R., MCCALLUM, B. C., AND EVANS, T. R. 2001. Reconstruction and representation of 3d objects with radial basis functions. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques (SIGGRAPH), 67--76. Google ScholarDigital Library
- COOTES, T. F., EDWARDS, G. J., AND TAYLOR, C. J. 1998. Active appearance models. In Proceedings of the Fifth European Conference on Computer Vision (ECCV), 484--498. Google ScholarDigital Library
- GU, X., GORTLER, S. J., HOPPE, H., MCMILLAN, L., BROWN, B. J., AND STONE, A. D. 1999. Silhouette mapping. Tech. Rep. TR-1-99, Harvard.Google Scholar
- GUENTER, B., GRIMM, C., WOOD, D., MALVAR, H., AND PIGHIN, F. 1998. Making faces. In Proceedings of the 25th annual conference on Computer graphics and interactive techniques (SIGGRAPH), 55--66. Google ScholarDigital Library
- KAKADIARIS, I. A., AND METAXAS, D. 1993. 3d human body model acquisition from multiple views. In Proceedings of the 5th IEEE International Conference on Computer Vision (ICCV), 618--623. Google ScholarDigital Library
- KAYDARA. 2001. FiLMBOX Reference Guide. Kaydara Inc., Montréal, Québec.Google Scholar
- LEWIS, J. P., CORDNER, M., AND FONG, N. 2000. Pose space deformation: a unified approach to shape interpolation and skeleton-driven deformation. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques (SIGGRAPH), 165--172. Google ScholarDigital Library
- MATUSIK, W., BUEHLER, C., RASKAR, R., GORTLER, S. J., AND MCMILLAN, L. 2000. Image-based visual hulls. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques (SIGGRAPH), 369--374. Google ScholarDigital Library
- MIKIĆ, I., TRIVEDI, M., HUNTER, E., AND COSMAN, P. Human body model acquisition and tracking using voxel data. International Journal of Computer Vision. In Press. Google ScholarDigital Library
- NEBEL, J.-C., RODRIGUEZ-MIGUEL, F. J., AND COCKSHOTT, W. P. 2001. Stroboscopic stereo rangefinder. In Proceedings of the Third International Conference on 3D Imaging and Modeling, 59--64.Google ScholarCross Ref
- NELDER, J. A., AND MEAD, R. 1965. A simplex method for function minimization. Computer Journal 7, 4, 308--313.Google ScholarCross Ref
- NEVATIA, R., AND BINFORD, T. O. 1977. Description and recognition of curved objects. Artificial Intelligence 8, 1, 77--98.Google ScholarDigital Library
- PLÄNKERS, R., AND FUA, P. 2001. Articulated soft objects for video-based body modeling. In Proceedings of the 8th IEEE International Conference on Computer Vision (ICCV), I:394--401.Google ScholarCross Ref
- RUSINKIEWICZ, S., HALL-HOLT, O., AND LEVOY, M. 2002. Real-time 3d model acquisition. In Proceedings of the 29th annual conference on Computer graphics and interactive techniques (SIGGRAPH), 438--446. Google ScholarDigital Library
- SLOAN, P.-P. J., CHARLES F. ROSE, I., AND COHEN, M. F. 2001. Shape by example. In Proceedings of the 2001 symposium on Interactive 3D Graphics, 135--143. Google ScholarDigital Library
- STOKDYK, S., HAHN, K., NOFZ, P., AND ANDERSON, G., 2002. Spiderman: Behind the mask. Special Session of SIGGRAPH 2002.Google Scholar
- SZELISKI, R., AND TONNESEN, D. 1992. Surface modeling with oriented particle systems. In Proceedings of the 19th annual conference on Computer graphics and interactive techniques (SIGGRAPH), 185--194. Google ScholarDigital Library
- THEOBALT, C., MAGNOR, M., SCHUELER, P., AND SEIDEL, H.-P. 2002. Combining 2d feature tracking and volume reconstruction for online video-based human motion capture. In Proceedings of the 10th Pacific Conference on Computer Graphics and Applications, 96--103. Google ScholarDigital Library
- TORRESANI, L., AND BREGLER, C. 2002. Space-time tracking. In Proceedings of the 7th European Conference on Computer Vision (ECCV), 801--812. Google ScholarDigital Library
- VEDULA, S., BAKER, S., AND KANADE, T. 2002. Spatio-temporal view interpolation. In Proceedings of the 13th ACM Eurographics Workshop on Rendering, 65--76. Google ScholarDigital Library
- VICON. 2003. Vicon iQ Reference Manual. Vicon Motion Systems Inc., Lake Forest, CA.Google Scholar
- WANG, X. C., AND PHILLIPS, C. 2002. Multi-weight enveloping: least-squares approximation techniques for skin animation. In Proceedings of the ACM SIGGRAPH symposium on Computer animation, 129--138. Google ScholarDigital Library
- WÜRMLIN, S., LAMBORAY, E., STAADT, O. G., AND GROSS., M. H. 2002. 3d video recorder. In Proceedings of the 10th Pacific Conference on Computer Graphics and Applications, 325--334. Google ScholarDigital Library
Index Terms
- Continuous capture of skin deformation
Recommendations
MoSh: motion and shape capture from sparse markers
Marker-based motion capture (mocap) is widely criticized as producing lifeless animations. We argue that important information about body surface motion is present in standard marker sets but is lost in extracting a skeleton. We demonstrate a new ...
Data-driven modeling of skin and muscle deformation
In this paper, we present a data-driven technique for synthesizing skin deformation from skeletal motion. We first create a database of dynamic skin deformations by recording the motion of the surface of the skin with a very large set of motion capture ...
Capturing and animating skin deformation in human motion
During dynamic activities, the surface of the human body moves in many subtle but visually significant ways: bending, bulging, jiggling, and stretching. We present a technique for capturing and animating those motions using a commercial motion capture ...
Comments