Skip to main content
Top
Published in: Journal of NeuroEngineering and Rehabilitation 1/2018

Open Access 01-12-2018 | Research

Decoding the grasping intention from electromyography during reaching motions

Authors: Iason Batzianoulis, Nili E. Krausz, Ann M. Simon, Levi Hargrove, Aude Billard

Published in: Journal of NeuroEngineering and Rehabilitation | Issue 1/2018

Login to get access

Abstract

Background

Active upper-limb prostheses are used to restore important hand functionalities, such as grasping. In conventional approaches, a pattern recognition system is trained over a number of static grasping gestures. However, training a classifier in a static position results in lower classification accuracy when performing dynamic motions, such as reach-to-grasp. We propose an electromyography-based learning approach that decodes the grasping intention during the reaching motion, leading to a faster and more natural response of the prosthesis.

Methods and Results

Eight able-bodied subjects and four individuals with transradial amputation gave informed consent and participated in our study. All the subjects performed reach-to-grasp motions for five grasp types, while the elecromyographic (EMG) activity and the extension of the arm were recorded. We separated the reach-to-grasp motion into three phases, with respect to the extension of the arm. A multivariate analysis of variance (MANOVA) on the muscular activity revealed significant differences among the motion phases. Additionally, we examined the classification performance on these phases. We compared the performance of three different pattern recognition methods; Linear Discriminant Analysis (LDA), Support Vector Machines (SVM) with linear and non-linear kernels, and an Echo State Network (ESN) approach. Our off-line analysis shows that it is possible to have high classification performance above 80% before the end of the motion when with three-grasp types. An on-line evaluation with an upper-limb prosthesis shows that the inclusion of the reaching motion in the training of the classifier importantly improves classification accuracy and enables the detection of grasp intention early in the reaching motion.

Conclusions

This method offers a more natural and intuitive control of prosthetic devices, as it will enable controlling grasp closure in synergy with the reaching motion. This work contributes to the decrease of delays between the user’s intention and the device response and improves the coordination of the device with the motion of the arm.
Appendix
Available only for authorised users
Literature
1.
go back to reference Freeland AE, Psonak R. Traumatic below-elbow amputations. Orthopedics. 2007. Freeland AE, Psonak R. Traumatic below-elbow amputations. Orthopedics. 2007.
2.
go back to reference Raichle KA, Hanley MA, Molton I, Kadel NJ, Campbell K, Phelps E, Ehde D, Smith DG. Prosthesis use in persons with lower- and upper-limb amputation. J Rehabil Res Dev. 2008. Raichle KA, Hanley MA, Molton I, Kadel NJ, Campbell K, Phelps E, Ehde D, Smith DG. Prosthesis use in persons with lower- and upper-limb amputation. J Rehabil Res Dev. 2008.
3.
go back to reference Need-Directed Design of Prostheses and Enabling Resources in Amputation, Prosthesis Use, and Phantom Limb Pain: An Interdisciplinary Perspective. New York: Springer; 2010. Need-Directed Design of Prostheses and Enabling Resources in Amputation, Prosthesis Use, and Phantom Limb Pain: An Interdisciplinary Perspective. New York: Springer; 2010.
4.
go back to reference Oltstlie k, Lesjo IM, Franklin RJ, Garfelt B, Skjeldal OH, Magnus P. Prosthesis rejection in acquired major upper-limb amputees: a population-based survey. Disabil Rehabil. 2012. Oltstlie k, Lesjo IM, Franklin RJ, Garfelt B, Skjeldal OH, Magnus P. Prosthesis rejection in acquired major upper-limb amputees: a population-based survey. Disabil Rehabil. 2012.
5.
go back to reference Cordella F, Ciancio AL, Sacchetti R, Davalli A, Cutti AG, Guglielmelli E, Zollo L. Literature review on needs of upper limb prosthesis users. Front Neurosci. 2016. Cordella F, Ciancio AL, Sacchetti R, Davalli A, Cutti AG, Guglielmelli E, Zollo L. Literature review on needs of upper limb prosthesis users. Front Neurosci. 2016.
6.
go back to reference Novak D, Riener R. A survey of sensor fusion methods in wearable robotics. Robot Auton Syst. 2014. Novak D, Riener R. A survey of sensor fusion methods in wearable robotics. Robot Auton Syst. 2014.
7.
go back to reference Earley EJ, Hargrove LJ, Kuiken TA. Dual window pattern recognition classifier for improved partial-hand prosthesis control. Front Neurosci. 2016. Earley EJ, Hargrove LJ, Kuiken TA. Dual window pattern recognition classifier for improved partial-hand prosthesis control. Front Neurosci. 2016.
8.
go back to reference Miller LA, Stubblefield KA, Lipschutz RD, Lock BA, Kuiken TA. Improved myoelectric prosthesis control using targeted reinnervation surgery: A case series. IEEE Trans Neural Syst Rehabil Eng. 2008. Miller LA, Stubblefield KA, Lipschutz RD, Lock BA, Kuiken TA. Improved myoelectric prosthesis control using targeted reinnervation surgery: A case series. IEEE Trans Neural Syst Rehabil Eng. 2008.
9.
go back to reference Kuiken TA, Li BA, Guanglin amd Lock, Lipschutz RD, Miller LA, A SK, Englehart K. Targeted muscle reinnervation for real-time myoelectric control of multifunction artificial arms. JAMA. 2009. Kuiken TA, Li BA, Guanglin amd Lock, Lipschutz RD, Miller LA, A SK, Englehart K. Targeted muscle reinnervation for real-time myoelectric control of multifunction artificial arms. JAMA. 2009.
10.
go back to reference Urbanchek MG, Wei B, Baghmanli Z, Sugg K, Cederna PS. Long-term stability of regenerative peripheral nerve interfaces. Plast Reocnstractive Surg. 2011. Urbanchek MG, Wei B, Baghmanli Z, Sugg K, Cederna PS. Long-term stability of regenerative peripheral nerve interfaces. Plast Reocnstractive Surg. 2011.
11.
go back to reference Fougner AL, Stavdahl O, Kyberd PJ. System training and assessment in simultaneous proportional myoelectric prosthesis control. J Neuroengineering Rehabil. 2014. Fougner AL, Stavdahl O, Kyberd PJ. System training and assessment in simultaneous proportional myoelectric prosthesis control. J Neuroengineering Rehabil. 2014.
12.
go back to reference Young AJ, Smith LH, Rouse EJ, Hargrove LJ. Classification of simultaneous movements using surface emg pattern recognition. IEEE Trans Biomed Eng. 2013. Young AJ, Smith LH, Rouse EJ, Hargrove LJ. Classification of simultaneous movements using surface emg pattern recognition. IEEE Trans Biomed Eng. 2013.
13.
go back to reference Smith LH, Kuiken TA, Hargrove LJ. Evaluation of linear regression simultaneous myoelectric control using intramuscular emg. IEEE Trans Biomed Eng. 2016. Smith LH, Kuiken TA, Hargrove LJ. Evaluation of linear regression simultaneous myoelectric control using intramuscular emg. IEEE Trans Biomed Eng. 2016.
14.
go back to reference Gonzalez-Vargas J, Dosen S, Amsuess S, Yu W, Farina D. Human-machine interface for the control of multi-function systems based on electrocutaneous menu: Application to multi-grasp prosthetic hands. PLoS ONE. 2015. Gonzalez-Vargas J, Dosen S, Amsuess S, Yu W, Farina D. Human-machine interface for the control of multi-function systems based on electrocutaneous menu: Application to multi-grasp prosthetic hands. PLoS ONE. 2015.
15.
go back to reference Li G, Schultz AE, Kuiken TA. Quantifying pattern recognition-based myoelectric control of multifunctional transradial prostheses. IEEE Trans Neural Syst Rehabil Eng. 2010. Li G, Schultz AE, Kuiken TA. Quantifying pattern recognition-based myoelectric control of multifunctional transradial prostheses. IEEE Trans Neural Syst Rehabil Eng. 2010.
16.
go back to reference Naik GR, Al-Timemy AH, Nguyen HT. Transradial amputee gesture classification using an optimal number of semg sensors: An approach using ica clustering. IEEE Trans Neural Syst Rehabil Eng. 2016. Naik GR, Al-Timemy AH, Nguyen HT. Transradial amputee gesture classification using an optimal number of semg sensors: An approach using ica clustering. IEEE Trans Neural Syst Rehabil Eng. 2016.
17.
go back to reference Khushaba RN, Kodagoda S, Takruri M, Dissanayake G. Toward improved control of prosthetic fingers using surface electromyogram (emg) signals. Expert Syst Appl. 2012. Khushaba RN, Kodagoda S, Takruri M, Dissanayake G. Toward improved control of prosthetic fingers using surface electromyogram (emg) signals. Expert Syst Appl. 2012.
18.
go back to reference Scheme E, Biron K, Englehart K. Improving myoelectric pattern recognition positional robustness using advanced training protocols. In: 33rd Annual International Conference of the IEEE EMBS. Boston: 2011. Scheme E, Biron K, Englehart K. Improving myoelectric pattern recognition positional robustness using advanced training protocols. In: 33rd Annual International Conference of the IEEE EMBS. Boston: 2011.
19.
go back to reference Batzianoulis I, El-Khoury S, Pirondini E, Coscia M, Micera S, Billard A. Emg-based decoding of grasp gestures in reaching-to-grasping motions. Robot Auton Syst. 2017. Batzianoulis I, El-Khoury S, Pirondini E, Coscia M, Micera S, Billard A. Emg-based decoding of grasp gestures in reaching-to-grasping motions. Robot Auton Syst. 2017.
20.
go back to reference Martelloni C, Carpaneto J, Micera S. Characterization of emg patterns from proximal arm muscles during object- and orientation-specific grasps. IEEE Trans Biomed Eng. 2009. Martelloni C, Carpaneto J, Micera S. Characterization of emg patterns from proximal arm muscles during object- and orientation-specific grasps. IEEE Trans Biomed Eng. 2009.
21.
go back to reference Rand MK, Shimansky YP, Stelmach ABMI, Stelmach HE. Quantitative model of transport- aperture coordination during reach-to-grasp movements. Exp Brain Res. 2008. Rand MK, Shimansky YP, Stelmach ABMI, Stelmach HE. Quantitative model of transport- aperture coordination during reach-to-grasp movements. Exp Brain Res. 2008.
22.
go back to reference Wang J, Stelmach G. E. Coordination among the body segments during reach-to-grasp action involving the trunk. Exp Brain Res. 1998. Wang J, Stelmach G. E. Coordination among the body segments during reach-to-grasp action involving the trunk. Exp Brain Res. 1998.
23.
go back to reference Jeannerod M. The timing of natural prehension movements. J Motor Behav. 1984. Jeannerod M. The timing of natural prehension movements. J Motor Behav. 1984.
24.
go back to reference Haggard P, Wing A. Coordinated responses following mechanical perturbation of the arm during prehension. Exp Brain Res. 1995. Haggard P, Wing A. Coordinated responses following mechanical perturbation of the arm during prehension. Exp Brain Res. 1995.
25.
go back to reference Ghazaei G, Alameer A, Degenaar P, Morgan G, Nazarpour K. Deep learning-based artificial vision for grasp classification in myoelectric hands. J Neural Eng. 2017. Ghazaei G, Alameer A, Degenaar P, Morgan G, Nazarpour K. Deep learning-based artificial vision for grasp classification in myoelectric hands. J Neural Eng. 2017.
26.
go back to reference Amsuess S, Vujaklija I, Goebel P, Roche AD, Graimann B, Aszmann OC, Dario F. Context-dependent upper limb prosthesis control for natural and robust use. IEEE Trans Neural Syst Rehabil Eng. 2016. Amsuess S, Vujaklija I, Goebel P, Roche AD, Graimann B, Aszmann OC, Dario F. Context-dependent upper limb prosthesis control for natural and robust use. IEEE Trans Neural Syst Rehabil Eng. 2016.
27.
go back to reference Farrell TR, Weir RF. The optimal controller delay for myoelectric prostheses. IEEE Trans Neural Syst Rehabil Eng. 2007. Farrell TR, Weir RF. The optimal controller delay for myoelectric prostheses. IEEE Trans Neural Syst Rehabil Eng. 2007.
28.
go back to reference Santello M, Soechting JF. Gradual molding of the hand to object contours. J Neurophys. 1998. Santello M, Soechting JF. Gradual molding of the hand to object contours. J Neurophys. 1998.
29.
go back to reference Smith LH, L J Hargrove TAK B A Lock. Determining the optimal window length for pattern recognition-based myoelectric control: Balancing the competing effects of classification error and controller delay. IEEE Trans Neural Syst Rehabil Eng. 2011. Smith LH, L J Hargrove TAK B A Lock. Determining the optimal window length for pattern recognition-based myoelectric control: Balancing the competing effects of classification error and controller delay. IEEE Trans Neural Syst Rehabil Eng. 2011.
30.
go back to reference Englehart K, Hudgins B. A robust, real-time control scheme for multifunction myoelectric control. IEEE Trans Biomed Eng. 2003. Englehart K, Hudgins B. A robust, real-time control scheme for multifunction myoelectric control. IEEE Trans Biomed Eng. 2003.
31.
go back to reference Daley H, Englehart K, Hargrove L, Kuruganti U. High density electromyography data of normally limbed and transradial amputee subjects for multifunction prosthetic control. J Electromyogr Kinesiol. 2012. Daley H, Englehart K, Hargrove L, Kuruganti U. High density electromyography data of normally limbed and transradial amputee subjects for multifunction prosthetic control. J Electromyogr Kinesiol. 2012.
32.
go back to reference Jaeger H. The “echo state” approach to analyzing and training recurrent neural networks. Technical report, German National Research Institute for Computer Science. 2001. Jaeger H. The “echo state” approach to analyzing and training recurrent neural networks. Technical report, German National Research Institute for Computer Science. 2001.
33.
go back to reference Li D, Han M, Wang J. Chaotic time series prediction based on a novel robust echo state network. IEEE Trans Neural Netw Learn Syst. 2012. Li D, Han M, Wang J. Chaotic time series prediction based on a novel robust echo state network. IEEE Trans Neural Netw Learn Syst. 2012.
34.
go back to reference Xu M, Han M. Adaptive elastic echo state network for multivariate time series prediction. IEEE Trans Cybern; 46(10):2173–2183. Xu M, Han M. Adaptive elastic echo state network for multivariate time series prediction. IEEE Trans Cybern; 46(10):2173–2183.
35.
go back to reference Lenzi T, Lipsey J, Sensinger JW. The ric arm- a small anthropomorphic transhumeral prosthesis. IEEE Trans Mechatron. 2016. Lenzi T, Lipsey J, Sensinger JW. The ric arm- a small anthropomorphic transhumeral prosthesis. IEEE Trans Mechatron. 2016.
36.
go back to reference He J, Zhang D, Jiang N, Sheng X, Farina D, Zhu X. User adaptation in long-term, open-loop myoelectric training: implications for emg pattern recognition in prosthesis control. J Neural Eng. 2015. He J, Zhang D, Jiang N, Sheng X, Farina D, Zhu X. User adaptation in long-term, open-loop myoelectric training: implications for emg pattern recognition in prosthesis control. J Neural Eng. 2015.
37.
go back to reference Peerdeman B, Boere D, Witteveen H, Veld RH, Hermens H, Stramigioli S, Rietman H, Veltink P, Misra S. Myoelectric forearm prostheses: State of the art from a user-centered perspective. J Rehabil Res Dev. 2011. Peerdeman B, Boere D, Witteveen H, Veld RH, Hermens H, Stramigioli S, Rietman H, Veltink P, Misra S. Myoelectric forearm prostheses: State of the art from a user-centered perspective. J Rehabil Res Dev. 2011.
38.
go back to reference Y Geng YW O W Samuel, Li G. Improving the robustness of real-time myoelectric pattern recognition against arm position changes in transradial amputees. BioMed Res Int. 2017. Y Geng YW O W Samuel, Li G. Improving the robustness of real-time myoelectric pattern recognition against arm position changes in transradial amputees. BioMed Res Int. 2017.
39.
go back to reference J Liu XS, D Zhang, Zhu X. Quantification and solutions of arm movements effect on semg pattern recognition. Biomed Signal Process Control. 2014. J Liu XS, D Zhang, Zhu X. Quantification and solutions of arm movements effect on semg pattern recognition. Biomed Signal Process Control. 2014.
40.
go back to reference Yang D, Jiang L, Osborn L, Gu Y, Liu H. Dynamic training protocol improves the robustness of pr-based myoelectric control. Biomed Signal Process Control. 2017. Yang D, Jiang L, Osborn L, Gu Y, Liu H. Dynamic training protocol improves the robustness of pr-based myoelectric control. Biomed Signal Process Control. 2017.
41.
go back to reference Wing AM, Turton A. Grasp size and accuracy of approach in reaching. J Mot Behav. 1986. Wing AM, Turton A. Grasp size and accuracy of approach in reaching. J Mot Behav. 1986.
42.
go back to reference Jeannerod M, Paulignan PWY. Grasping an object: one movement, several components. In: Novartis Foundation Symposium. New Jersey: Wiley. Jeannerod M, Paulignan PWY. Grasping an object: one movement, several components. In: Novartis Foundation Symposium. New Jersey: Wiley.
43.
go back to reference T Supuk TBTKodek. Estimation of hand preshaping during human grasping. Med Eng Phys. 2005. T Supuk TBTKodek. Estimation of hand preshaping during human grasping. Med Eng Phys. 2005.
44.
go back to reference Hargrove LJ, Turner K, Kuiken TA, Miller LA. Myoelectric pattern recognition outperforms direct control for transhumeral amputees with targeted muscle reinnervation: A randomized clinical trial. Sci Rep. 2017. Hargrove LJ, Turner K, Kuiken TA, Miller LA. Myoelectric pattern recognition outperforms direct control for transhumeral amputees with targeted muscle reinnervation: A randomized clinical trial. Sci Rep. 2017.
45.
go back to reference Gentilucci M, Benuzzi F, Bertolani L, Daprati E, Gangitano M. Language and motor control. Exp Brain Res. 2000. Gentilucci M, Benuzzi F, Bertolani L, Daprati E, Gangitano M. Language and motor control. Exp Brain Res. 2000.
46.
go back to reference Bongersa RM, Zaala FT, Jeannerod M. Hand aperture patterns in prehension. Hum Mov Sci. 2012. Bongersa RM, Zaala FT, Jeannerod M. Hand aperture patterns in prehension. Hum Mov Sci. 2012.
47.
go back to reference Feix T, J Romero H-BS, Dollar AM, Kragic D. The grasp taxonomy of human grasp types. IEEE Trans Hum Mach Syst. 2015. Feix T, J Romero H-BS, Dollar AM, Kragic D. The grasp taxonomy of human grasp types. IEEE Trans Hum Mach Syst. 2015.
Metadata
Title
Decoding the grasping intention from electromyography during reaching motions
Authors
Iason Batzianoulis
Nili E. Krausz
Ann M. Simon
Levi Hargrove
Aude Billard
Publication date
01-12-2018
Publisher
BioMed Central
Published in
Journal of NeuroEngineering and Rehabilitation / Issue 1/2018
Electronic ISSN: 1743-0003
DOI
https://doi.org/10.1186/s12984-018-0396-5

Other articles of this Issue 1/2018

Journal of NeuroEngineering and Rehabilitation 1/2018 Go to the issue