Skip to main content
Top
Published in: Aesthetic Plastic Surgery 4/2014

01-08-2014 | Innovative Techniques

3D Landmarking in Multiexpression Face Analysis: A Preliminary Study on Eyebrows and Mouth

Authors: Enrico Vezzetti, Federica Marcolin

Published in: Aesthetic Plastic Surgery | Issue 4/2014

Login to get access

Abstract

The application of three-dimensional (3D) facial analysis and landmarking algorithms in the field of maxillofacial surgery and other medical applications, such as diagnosis of diseases by facial anomalies and dysmorphism, has gained a lot of attention. In a previous work, we used a geometric approach to automatically extract some 3D facial key points, called landmarks, working in the differential geometry domain, through the coefficients of fundamental forms, principal curvatures, mean and Gaussian curvatures, derivatives, shape and curvedness indexes, and tangent map. In this article we describe the extension of our previous landmarking algorithm, which is now able to extract eyebrows and mouth landmarks using both old and new meshes. The algorithm has been tested on our face database and on the public Bosphorus 3D database. We chose to work on the mouth and eyebrows as a separate study because of the role that these parts play in facial expressions. In fact, since the mouth is the part of the face that moves the most and affects mainly facial expressions, extracting mouth landmarks from various facial poses means that the newly developed algorithm is pose-independent.

No Level Assigned

This journal requires that authors assign a level of evidence to each submission to which Evidence-Based Medicine rankings are applicable. This excludes Review Articles, Book Reviews, and manuscripts that concern Basic Science, Animal Studies, Cadaver Studies, and Experimental Studies. For a full description of these Evidence-Based Medicine ratings, please refer to the Table of Contents or the online Instructions to Authors http://​www.​springer.​com/​00266.
Appendix
Available only for authorised users
Literature
1.
go back to reference Alker M, Frantz S, Rohr K, Stiehl HS (2001) Improving the robustness in extracting 3D point landmarks from 3D medical images using parametric deformable models. In: Niessen WJ, Viergever MA (eds) Medical image computing and computer-assisted intervention—MICCAI 2001, 4th international conference, Utrecht, The Netherlands, October 14–17, 2001. Proceedings series: lecture notes in computer science, vol 2208. Springer, Berlin, pp 582–590 Alker M, Frantz S, Rohr K, Stiehl HS (2001) Improving the robustness in extracting 3D point landmarks from 3D medical images using parametric deformable models. In: Niessen WJ, Viergever MA (eds) Medical image computing and computer-assisted intervention—MICCAI 2001, 4th international conference, Utrecht, The Netherlands, October 14–17, 2001. Proceedings series: lecture notes in computer science, vol 2208. Springer, Berlin, pp 582–590
2.
go back to reference Alyüz N, Gökberk B, Dibeklioğlu H, Savran A, Salah AA, Akarun L, Sankur B (2008) 3D face recognition benchmarks on the Bosphorus database with focus on facial expressions. In: Schouten B, Juul NC, Drygajlo A, Tistarelli M (eds) Biometrics and identity management, first European workshop, BIOID 2008, Roskilde, Denmark, May 7–9, 2008, lecture notes in computer science/image processing, computer vision, pattern recognition, and graphics (book 5372). Springer, Berlin, pp 57–66 Alyüz N, Gökberk B, Dibeklioğlu H, Savran A, Salah AA, Akarun L, Sankur B (2008) 3D face recognition benchmarks on the Bosphorus database with focus on facial expressions. In: Schouten B, Juul NC, Drygajlo A, Tistarelli M (eds) Biometrics and identity management, first European workshop, BIOID 2008, Roskilde, Denmark, May 7–9, 2008, lecture notes in computer science/image processing, computer vision, pattern recognition, and graphics (book 5372). Springer, Berlin, pp 57–66
3.
go back to reference Calignano F, Vezzetti E (2010) Soft tissue diagnosis in maxillofacial surgery: a preliminary study on three-dimensional face geometrical features-based analysis. Aesthet Plast Surg 34(2):200–211CrossRef Calignano F, Vezzetti E (2010) Soft tissue diagnosis in maxillofacial surgery: a preliminary study on three-dimensional face geometrical features-based analysis. Aesthet Plast Surg 34(2):200–211CrossRef
4.
go back to reference Creusot C, Pears N, Austin J (2013) A machine-learning approach to keypoint detection and landmarking on 3D meshes. Int J Comput Vis 102(1–3):146–179CrossRef Creusot C, Pears N, Austin J (2013) A machine-learning approach to keypoint detection and landmarking on 3D meshes. Int J Comput Vis 102(1–3):146–179CrossRef
5.
go back to reference D’Hose J, Colineau J, Bichon C, Dorizzi B (2007) Precise localization of landmarks on 3D faces using Gabor wavelets. In: First IEEE international conference on biometrics: theory, applications, and systems. IEEE, New York, pp 1–6 D’Hose J, Colineau J, Bichon C, Dorizzi B (2007) Precise localization of landmarks on 3D faces using Gabor wavelets. In: First IEEE international conference on biometrics: theory, applications, and systems. IEEE, New York, pp 1–6
6.
go back to reference Dibeklioğlu H, Salah AA, Akarun L (2008) 3D facial landmarking under expression, pose, and occlusion variations. In: Second IEEE international conference on biometrics: theory, applications and systems. IEEE, New York, pp 1–6 Dibeklioğlu H, Salah AA, Akarun L (2008) 3D facial landmarking under expression, pose, and occlusion variations. In: Second IEEE international conference on biometrics: theory, applications and systems. IEEE, New York, pp 1–6
7.
go back to reference Ekman P (1970) Universal facial expressions of emotions. Calif Mental Health Res Dig 8(4):151–158 Ekman P (1970) Universal facial expressions of emotions. Calif Mental Health Res Dig 8(4):151–158
8.
go back to reference Ekman P, Keltner D (1997) Facial expressions of emotions. Lawrence Erlbaum Associates, Mahwah, NJ Ekman P, Keltner D (1997) Facial expressions of emotions. Lawrence Erlbaum Associates, Mahwah, NJ
9.
go back to reference Frantz, S.; Rohr, K.; Stiehl, H.-S.: Multi-Step Procedures for the Localization of 2D and 3D Point Landmarks and Automatic ROI Size Selection Proc. European Conf. on Computer Vision (ECCV'98), June 1998, Freiburg, Germany, Vol. I, Lecture Notes in Computer Science 1406, H. Burkhardt and B. Neumann (Eds.), Springer Berlin Heidelberg 1998, 687-703 Frantz, S.; Rohr, K.;  Stiehl, H.-S.: Multi-Step Procedures for the Localization of 2D and 3D Point Landmarks and Automatic ROI Size Selection Proc. European Conf. on Computer Vision (ECCV'98), June 1998, Freiburg, Germany, Vol. I, Lecture Notes in Computer Science 1406, H. Burkhardt and B. Neumann (Eds.), Springer Berlin Heidelberg 1998, 687-703
10.
go back to reference Frantz S, Rohr K, Stiehl HS (1999) Improving the detection performance in semi-automatic landmark extraction. In: Taylor C, Colchester A (eds) Medical image computing and computer-assisted intervention—MICCAI ’99, second international conference, Cambridge, UK, September 19–22, 1999, Proceedings series: lecture notes in computer science vol 1679. Springer, Berlin, pp 253–262 Frantz S, Rohr K, Stiehl HS (1999) Improving the detection performance in semi-automatic landmark extraction. In: Taylor C, Colchester A (eds) Medical image computing and computer-assisted intervention—MICCAI ’99, second international conference, Cambridge, UK, September 19–22, 1999, Proceedings series: lecture notes in computer science vol 1679. Springer, Berlin, pp 253–262
11.
go back to reference Frantz S, Rohr K, Stiehl HS (2000) Localization of 3D anatomical point landmarks in 3D tomographic images using deformable models. In: Delp SL, DiGoia AM, Jaramaz B (eds) Medical image computing and computer-assisted intervention—MICCAI 2000, third international conference Pittsburgh, PA, USA, October 11–14, 2000, Proceedings series: lecture notes in computer science vol 1935. Springer, Berlin, pp 492–501 Frantz S, Rohr K, Stiehl HS (2000) Localization of 3D anatomical point landmarks in 3D tomographic images using deformable models. In: Delp SL, DiGoia AM, Jaramaz B (eds) Medical image computing and computer-assisted intervention—MICCAI 2000, third international conference Pittsburgh, PA, USA, October 11–14, 2000, Proceedings series: lecture notes in computer science vol 1935. Springer, Berlin, pp 492–501
12.
go back to reference Frantz S, Rohr K, Stiehl HS (2005) Development and validation of a multi-step approach to improved detection of 3D point landmarks in tomographic images. Image Vision Comput 23(11):956–971CrossRef Frantz S, Rohr K, Stiehl HS (2005) Development and validation of a multi-step approach to improved detection of 3D point landmarks in tomographic images. Image Vision Comput 23(11):956–971CrossRef
13.
go back to reference Heckbert PS (1986) Survey of texture mapping. Comput Graphics Appl 6(11):56–67CrossRef Heckbert PS (1986) Survey of texture mapping. Comput Graphics Appl 6(11):56–67CrossRef
14.
go back to reference Koenderink JJ and van Doorn AJ (1992) Surface shape and curvature scales. Image Vision Comput 10(8):557–564 Koenderink JJ and van Doorn AJ (1992) Surface shape and curvature scales. Image Vision Comput 10(8):557–564
15.
go back to reference Perakis P, Palassis G, Theoharis T, Kakadiaris LA (2009) Automatic 3D facial region retrieval from multi-pose facial datasets. Eurographics 2009 workshop on 3D object retrieval, Munich, Germany, 29 March 2009, 30th annual conference of the European association for computer graphics Perakis P, Palassis G, Theoharis T, Kakadiaris LA (2009) Automatic 3D facial region retrieval from multi-pose facial datasets. Eurographics 2009 workshop on 3D object retrieval, Munich, Germany, 29 March 2009, 30th annual conference of the European association for computer graphics
16.
go back to reference Romero M, Pears N (2009) Landmark localization in 3D face data. In: 6th IEEE international conference on advanced video and signal based surveillance, Genova, Italy, 2–4 September 2009. IEEE, New York, pp 73–78 Romero M, Pears N (2009) Landmark localization in 3D face data. In: 6th IEEE international conference on advanced video and signal based surveillance, Genova, Italy, 2–4 September 2009. IEEE, New York, pp 73–78
17.
go back to reference Romero M, Pears N (2009) Point-pair descriptors for 3D facial landmark localization. IEEE 3rd international conference on biometrics: theory, applications, and systems. IEEE, New York, pp 1–6 Romero M, Pears N (2009) Point-pair descriptors for 3D facial landmark localization. IEEE 3rd international conference on biometrics: theory, applications, and systems. IEEE, New York, pp 1–6
18.
go back to reference Ruiz MC, Illingworth J (2008) Automatic landmarking of faces in 3D - ALF3D. In: 5th international conference on visual information engineering, Xi’an, China, July 29–August 1, 2008, IET Conference Publication 543. Curran Associates, Red Hook, NY, pp 41–46 Ruiz MC, Illingworth J (2008) Automatic landmarking of faces in 3D - ALF3D. In: 5th international conference on visual information engineering, Xi’an, China, July 29–August 1, 2008, IET Conference Publication 543. Curran Associates, Red Hook, NY, pp 41–46
19.
go back to reference Salah AA, Akarun L (2006) 3D facial feature localization for registration. In: Gunsel B, Jain AK, Tekalp AM, Sankur B (eds) Multimedia content representation, classification and security, international workshop, MRCS 2006, Istanbul, Turkey, September 11–13, 2006, Proceedings series: lecture notes in computer science, vol 4105. Springer, Berlin, pp 338–345 Salah AA, Akarun L (2006) 3D facial feature localization for registration. In: Gunsel B, Jain AK, Tekalp AM, Sankur B (eds) Multimedia content representation, classification and security, international workshop, MRCS 2006, Istanbul, Turkey, September 11–13, 2006, Proceedings series: lecture notes in computer science, vol 4105. Springer, Berlin, pp 338–345
20.
go back to reference Salah AA, Akarun L (2006) Gabor factor analysis for 2D + 3D facial landmark localization. In: IEEE 14th signal processing and communications applications. IEEE, New York, pp 1–4 Salah AA, Akarun L (2006) Gabor factor analysis for 2D + 3D facial landmark localization. In: IEEE 14th signal processing and communications applications. IEEE, New York, pp 1–4
21.
go back to reference Salah AA, Çinar H, Akarun L, Sankur B (2007) Robust facial landmarking for registration. Ann Telecommun 62(1–2):83–108 Salah AA, Çinar H, Akarun L, Sankur B (2007) Robust facial landmarking for registration. Ann Telecommun 62(1–2):83–108
22.
go back to reference Sang-Jun P, Dong-Won S (2008) 3D face recognition based on feature detection using active shape models. International conference on control, automation and systems, Seoul, Korea, 14–17 October 2008, pp 1881–1886 Sang-Jun P, Dong-Won S (2008) 3D face recognition based on feature detection using active shape models. International conference on control, automation and systems, Seoul, Korea, 14–17 October 2008, pp 1881–1886
23.
go back to reference Vezzetti E, Marcolin F, Stola V (2013) 3D human face soft tissues landmarking method: an advanced approach. Comput Ind 64(9):1326–1354CrossRef Vezzetti E, Marcolin F, Stola V (2013) 3D human face soft tissues landmarking method: an advanced approach. Comput Ind 64(9):1326–1354CrossRef
24.
go back to reference Wörz S, Rohr K (2005) Localization of anatomical point landmarks in 3D medical images by fitting 3D parametric intensity models. Med Image Anal 10(1):41–58CrossRef Wörz S, Rohr K (2005) Localization of anatomical point landmarks in 3D medical images by fitting 3D parametric intensity models. Med Image Anal 10(1):41–58CrossRef
25.
go back to reference Zhang X, Wang Y, Pan G (2013) 3D facial landmark localization via a local surface descriptor HoSNI. In: Intelligent science and intelligent data engineering. Springer, Berlin, pp 313–321 Zhang X, Wang Y, Pan G (2013) 3D facial landmark localization via a local surface descriptor HoSNI. In: Intelligent science and intelligent data engineering. Springer, Berlin, pp 313–321
Metadata
Title
3D Landmarking in Multiexpression Face Analysis: A Preliminary Study on Eyebrows and Mouth
Authors
Enrico Vezzetti
Federica Marcolin
Publication date
01-08-2014
Publisher
Springer US
Published in
Aesthetic Plastic Surgery / Issue 4/2014
Print ISSN: 0364-216X
Electronic ISSN: 1432-5241
DOI
https://doi.org/10.1007/s00266-014-0334-2

Other articles of this Issue 4/2014

Aesthetic Plastic Surgery 4/2014 Go to the issue