Matthew Loper
Skip Abstract Section
Skip Supplemental Material SectionAbstract
We present a learned model of human body shape and pose-dependent shape variation that is more accurate than previous models and is compatible with existing graphics pipelines. Our Skinned Multi-Person Linear model (SMPL) is a skinned vertex-based model that accurately represents a wide variety of body shapes in natural human poses. The parameters of the model are learned from data including the rest pose template, blend weights, pose-dependent blend shapes, identity-dependent blend shapes, and a regressor from vertices to joint locations. Unlike previous models, the pose-dependent blend shapes are a linear function of the elements of the pose rotation matrices. This simple formulation enables training the entire model from a relatively large number of aligned 3D meshes of different people in different poses. We quantitatively evaluate variants of SMPL using linear or dual-quaternion blend skinning and show that both are more accurate than a Blend-SCAPE model trained on the same data. We also extend SMPL to realistically model dynamic soft-tissue deformations. Because it is based on blend skinning, SMPL is compatible with existing rendering engines and we make it available for research purposes.
Supplemental Material
Available for Download
zip
a248-loper.zip (151.9 MB)
Supplemental files.
- Allen, B., Curless, B., and Popović, Z. 2002. Articulated body deformation from range scan data. ACM Trans. Graph. (Proc. SIGGRAPH) 21, 3 (July), 612--619. Google Scholar
Digital Library
- Allen, B., Curless, B., and Popović, Z. 2003. The space of human body shapes: Reconstruction and parameterization from range scans. ACM Trans. Graph. (Proc. SIGGRAPH) 22, 3, 587--594. Google ScholarDigital Library
- Allen, B., Curless, B., Popović, Z., and Hertzmann, A. 2006. Learning a correlated model of identity and pose-dependent body shape variation for real-time synthesis. In Proceedings of the 2006 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, SCA '06, 147--156. Google ScholarDigital Library
- Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. SCAPE: Shape Completion and Animation of PEople. ACM Trans. Graph. (Proc. SIGGRAPH 24, 3, 408--416. Google ScholarDigital Library
- Baran, I., and Popović, J. 2007. Automatic rigging and animation of 3D characters. ACM Trans. Graph. (Proc. SIGGRAPH) 26, 3 (July). Google ScholarDigital Library
- Bogo, F., Romero, J., Loper, M., and Black, M. J. 2014. FAUST: Dataset and evaluation for 3D mesh registration. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 3794--3801. Google ScholarDigital Library
- Chang, W., and Zwicker, M. 2009. Range scan registration using reduced deformable models. Computer Graphics Forum 28, 2, 447--456.Google ScholarCross Ref
- Chen, Y., Liu, Z., and Zhang, Z. 2013. Tensor-based human body modeling. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 105--112. Google ScholarDigital Library
- 2000. CMU graphics lab motion capture database. http://mocap.cs.cmu.edu. Accessed: 2012-12-11.Google Scholar
- Corazza, S., and Gambaretto, E., 2014. Automatic generation of 3D character animation from 3D meshes, Aug. 5. US Patent 8,797,328.Google Scholar
- De Aguiar, E., Theobalt, C., Thrun, S., and Seidel, H.-P. 2008. Automatic conversion of mesh animations into skeleton-based animations. Computer Graphics Forum 27, 2, 389--397.Google ScholarCross Ref
- 2015. Dyna dataset. http://dyna.is.tue.mpg.de/. Accessed: 2015-05-15.Google Scholar
- Freifeld, O., and Black, M. J. 2012. Lie bodies: A manifold representation of 3D human shape. In European Conf. on Computer Vision (ECCV), Springer-Verlag, A. Fitzgibbon et al. (Eds.), Ed., Part I, LNCS 7572, 1--14. Google ScholarDigital Library
- Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., and Seidel, H. 2009. A statistical model of human pose and body shape. Computer Graphics Forum 28, 2, 337--346.Google ScholarCross Ref
- Hasler, N., Thormählen, T., Rosenhahn, B., and Seidel, H.-P. 2010. Learning skeletons for shape and pose. In Proceedings of the 2010 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, ACM, New York, NY, USA, I3D '10, 23--30. Google ScholarDigital Library
- Hirshberg, D., Loper, M., Rachlin, E., and Black, M. 2012. Coregistration: Simultaneous alignment and modeling of articulated 3D shape. In European Conf. on Computer Vision (ECCV), Springer-Verlag, A. F. et al. (Eds.), Ed., LNCS 7577, Part IV, 242--255. Google ScholarDigital Library
- James, D. L., and Twigg, C. D. 2005. Skinning mesh animations. ACM Trans. Graph. 24, 3 (July), 399--407. Google ScholarDigital Library
- Kavan, L., and Žára, J. 2005. Spherical blend skinning: A real-time deformation of articulated models. In Proceedings of the 2005 Symposium on Interactive 3D Graphics and Games, ACM, New York, NY, USA, I3D '05, 9--16. Google ScholarDigital Library
- Kavan, L., Collins, S., Žára, J., and O'Sullivan, C. 2008. Geometric skinning with approximate dual quaternion blending. ACM Transactions on Graphics (TOG) 27, 4, 105:1--105:23. Google ScholarDigital Library
- Kavan, L., Collins, S., and O'Sullivan, C. 2009. Automatic linearization of nonlinear skinning. In Proceedings of the 2009 Symposium on Interactive 3D Graphics and Games, ACM, New York, NY, USA, I3D '09, 49--56. Google ScholarDigital Library
- Kry, P. G., James, D. L., and Pai, D. K. 2002. EigenSkin: Real time large deformation character skinning in hardware. In Proceedings of the 2002 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, ACM, New York, NY, USA, SCA '02, 153--159. Google ScholarDigital Library
- Kurihara, T., and Miyata, N. 2004. Modeling deformable human hands from medical images. In Proceedings of the 2004 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, SCA '04, 355--363. Google ScholarDigital Library
- Lawson, C. L., and Hanson, R. J. 1995. Solving least squares problems. Classics in applied mathematics. SIAM, Philadelphia, PA. SIAM : Society of industrial and applied mathematics.Google Scholar
- Le, B. H., and Deng, Z. 2012. Smooth skinning decomposition with rigid bones. ACM Trans. Graph. 31, 6 (Nov.), 199:1--199:10. Google ScholarDigital Library
- Le, B. H., and Deng, Z. 2014. Robust and accurate skeletal rigging from mesh sequences. ACM Trans. Graph. 33, 4 (July), 84:1--84:10. Google ScholarDigital Library
- Lewis, J. P., Cordner, M., and Fong, N. 2000. Pose space deformation: A unified approach to shape interpolation and skeleton-driven deformation. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, SIGGRAPH '00, 165--172. Google ScholarDigital Library
- Loper, M. M., and Black, M. J. 2014. OpenDR: An approximate differentiable renderer. In Computer Vision -- ECCV 2014, Springer, Heidelberg, D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, Eds., vol. 8695 of Lecture Notes in Computer Science, 154--169.Google Scholar
- Loper, M. M., Mahmood, N., and Black, M. J. 2014. MoSh: Motion and shape capture from sparse markers. ACM Trans. Graph., (Proc. SIGGRAPH Asia) 33, 6 (Nov.), 220:1--220:13. Google ScholarDigital Library
- Merry, B., Marais, P., and Gain, J. 2006. Animation space: A truly linear framework for character animation. ACM Trans. Graph. 25, 4 (Oct.), 1400--1423. Google ScholarDigital Library
- Miller, C., Arikan, O., and Fussell, D. 2010. Frankenrigs: Building character rigs from multiple sources. In Proceedings of the 2010 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, ACM, New York, NY, USA, I3D '10, 31--38. Google ScholarDigital Library
- Mohr, A., and Gleicher, M. 2003. Building efficient, accurate character skins from examples. ACM Trans. Graph. (Proc. SIGGRAPH), 562--568. Google ScholarDigital Library
- Nocedal, J., and Wright, S. J. 2006. Numerical Optimization, 2nd ed. Springer, New York.Google Scholar
- Pons-Moll, G., Romero, J., Mahmood, N., and Black, M. J. 2015. Dyna: A model of dynamic human shape in motion. ACM Transactions on Graphics, (Proc. SIGGRAPH) 34, 4 (July), 120:1--120:14. Google ScholarDigital Library
- Rhee, T., Lewis, J., and Neumann, U. 2006. Real-time weighted pose-space deformation on the GPU. EUROGRAPHICS 25, 3.Google ScholarCross Ref
- Robinette, K., Blackwell, S., Daanen, H., Boehmer, M., Fleming, S., Brill, T., Hoeferlin, D., and Burnsides, D. 2002. Civilian American and European Surface Anthropometry Resource (CAESAR) final report. Tech. Rep. AFRL-HE-WP-TR-2002-0169, US Air Force Research Laboratory.Google Scholar
- Rouet, C., and Lewis, J., 1999. Method and apparatus for creating lifelike digital representations of computer animated objects by providing corrective enveloping, Mar. 16. US Patent 5,883,638.Google Scholar
- Schaefer, S., and Yuksel, C. 2007. Example-based skeleton extraction. In Proceedings of the Fifth Eurographics Symposium on Geometry Processing, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, SGP '07, 153--162. Google ScholarDigital Library
- Seo, H., Cordier, F., and Magnenat-Thalmann, N. 2003. Synthesizing animatable body models with parameterized shape modifications. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, SCA '03, 120--125. Google ScholarDigital Library
- Tsoli, A., Mahmood, N., and Black, M. J. 2014. Breathing life into shape: Capturing, modeling and animating 3D human breathing. ACM Trans. Graph., (Proc. SIGGRAPH) 33, 4 (July), 52:1--52:11. Google ScholarDigital Library
- Wang, X. C., and Phillips, C. 2002. Multi-weight enveloping: Least-squares approximation techniques for skin animation. In Proceedings of the 2002 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, ACM, New York, NY, USA, SCA '02, 129--138. Google ScholarDigital Library
- Wang, R. Y., Pulli, K., and Popović, J. 2007. Real-time enveloping with rotational regression. ACM Trans. Graph. (Proc. SIGGRAPH) 26, 3 (July). Google ScholarDigital Library
- Weber, O., Sorkine, O., Lipman, Y., and Gotsman, C. 2007. Context-aware skeletal shape deformation. Computer Graphics Forum 26, 3 (Sept.), 265--274.Google ScholarCross Ref
Index Terms
- SMPL: a skinned multi-person linear model
Recommendations
SMPL: A Skinned Multi-Person Linear Model
Seminal Graphics Papers: Pushing the Boundaries, Volume 2We present a learned model of human body shape and posedependent shape variation that is more accurate than previous models and is compatible with existing graphics pipelines. Our Skinned Multi-Person Linear model (SMPL) is a skinned vertexbased model ...
Learning Nonlinear Soft-Tissue Dynamics for Interactive Avatars
We present a novel method to enrich existing vertex-based human body models by adding soft-tissue dynamics. Our model learns to predict per-vertex 3D offsets, referred to as dynamic blendshapes, that reproduce nonlinear mesh deformation effects as a ...
RigMesh: automatic rigging for part-based shape modeling and deformation
The creation of a 3D model is only the first stage of the 3D character animation pipeline. Once a model has been created, and before it can be animated, it must be rigged. Manual rigging is laborious, and automatic rigging approaches are far from real-...
Comments