Skip to main content
Top
Published in: Journal of NeuroEngineering and Rehabilitation 1/2019

Open Access 01-12-2019 | Research

Improving bimanual interaction with a prosthesis using semi-autonomous control

Authors: Robin Volkmar, Strahinja Dosen, Jose Gonzalez-Vargas, Marcus Baum, Marko Markovic

Published in: Journal of NeuroEngineering and Rehabilitation | Issue 1/2019

Login to get access

Abstract

Background

The loss of a hand is a traumatic experience that substantially compromises an individual’s capability to interact with his environment. The myoelectric prostheses are state-of-the-art (SoA) functional replacements for the lost limbs. Their overall mechanical design and dexterity have improved over the last few decades, but the users have not been able to fully exploit these advances because of the lack of effective and intuitive control. Bimanual tasks are particularly challenging for an amputee since prosthesis control needs to be coordinated with the movement of the sound limb. So far, the bimanual activities have been often neglected by the prosthetic research community.

Methods

We present a novel method to prosthesis control, which uses a semi-autonomous approach in order to simplify bimanual interactions. The approach supplements the commercial SoA two-channel myoelectric control with two additional sensors. Two inertial measurement units were attached to the prosthesis and the sound hand to detect the movement of both limbs. Once a bimanual interaction is detected, the system mimics the coordination strategies of able-bodied subjects to automatically adjust the prosthesis wrist rotation (pronation, supination) and grip type (lateral, palmar) to assist the sound hand during a bimanual task. The system has been evaluated in eight able-bodied subjects performing functional uni- and bi-manual tasks using the novel method and SoA two-channel myocontrol. The outcome measures were time to accomplish the task, semi-autonomous system misclassification rate, subjective rating of intuitiveness, and perceived workload (NASA TLX).

Results

The results demonstrated that the novel control interface substantially outperformed the SoA myoelectric control. While using the semi-autonomous control the time to accomplish the task and the perceived workload decreased for 25 and 27%, respectively, while the subjects rated the system as more intuitive then SoA myocontrol.

Conclusions

The novel system uses minimal additional hardware (two inertial sensors) and simple processing and it is therefore convenient for practical implementation. By using the proposed control scheme, the prosthesis assists the user’s sound hand in performing bimanual interactions while decreasing cognitive burden.
Literature
1.
go back to reference Ziegler-Graham K, MacKenzie EJ, Ephraim PL, Travison TG, Brookmeyer R. Estimating the prevalence of limb loss in the United States: 2005 to 2050. Arch Phys Med Rehabil. 2008;89(3):422–9.CrossRef Ziegler-Graham K, MacKenzie EJ, Ephraim PL, Travison TG, Brookmeyer R. Estimating the prevalence of limb loss in the United States: 2005 to 2050. Arch Phys Med Rehabil. 2008;89(3):422–9.CrossRef
2.
go back to reference Belter JT, Segil JL, Dollar AM, Weir RF. Mechanical design and performance specifications of anthropomorphic prosthetic hands: a review. J Rehabil Res Dev. 2013;50(5):599–618.CrossRef Belter JT, Segil JL, Dollar AM, Weir RF. Mechanical design and performance specifications of anthropomorphic prosthetic hands: a review. J Rehabil Res Dev. 2013;50(5):599–618.CrossRef
3.
go back to reference A. E. Kobrinskiy, “Bioelectrical Control of Prosthetic Devices,” Her. Acad. Sci., no. 30, pp. 58–61, 1960. A. E. Kobrinskiy, “Bioelectrical Control of Prosthetic Devices,” Her. Acad. Sci., no. 30, pp. 58–61, 1960.
4.
go back to reference R. Tomovic, G. Bekey, and W. Karplus, “A strategy for grasp synthesis with multifingered robot hands,” in Proc. IEEE Int Conf on Robotics and Automation, 1987, vol. 4, pp. 83–89. R. Tomovic, G. Bekey, and W. Karplus, “A strategy for grasp synthesis with multifingered robot hands,” in Proc. IEEE Int Conf on Robotics and Automation, 1987, vol. 4, pp. 83–89.
5.
go back to reference Kyberd PJ, et al. MARCUS: a two degree of freedom hand prosthesis with hierarchical grip control. IEEE Trans Rehabil Eng. 1995;3(1):70–6.CrossRef Kyberd PJ, et al. MARCUS: a two degree of freedom hand prosthesis with hierarchical grip control. IEEE Trans Rehabil Eng. 1995;3(1):70–6.CrossRef
6.
go back to reference Ning J, Dosen S, Muller K-R, Farina D. Myoelectric control of artificial limbs—is there a need to change focus? [in the spotlight]. IEEE Signal Process Mag. 2012;29(5):152–0.CrossRef Ning J, Dosen S, Muller K-R, Farina D. Myoelectric control of artificial limbs—is there a need to change focus? [in the spotlight]. IEEE Signal Process Mag. 2012;29(5):152–0.CrossRef
7.
go back to reference Jiang N, Farina D. Myoelectric control of upper limb prosthesis: current status, challenges and recent advances. Front Neuroeng. 2014;7(4):7–9. Jiang N, Farina D. Myoelectric control of upper limb prosthesis: current status, challenges and recent advances. Front Neuroeng. 2014;7(4):7–9.
8.
go back to reference I. Vujaklija, D. Farina, and O. Aszmann, “New developments in prosthetic arm systems,” Orthop. Res. Rev., vol. Volume 8, pp. 31–39, 2016. I. Vujaklija, D. Farina, and O. Aszmann, “New developments in prosthetic arm systems,” Orthop. Res. Rev., vol. Volume 8, pp. 31–39, 2016.
9.
go back to reference Schweitzer W, Thali MJ, Egger D. Case-study of a user-driven prosthetic arm design: bionic hand versus customized body-powered technology in a highly demanding work environment. J Neuroeng Rehabil. 2018;15(1):1–27.CrossRef Schweitzer W, Thali MJ, Egger D. Case-study of a user-driven prosthetic arm design: bionic hand versus customized body-powered technology in a highly demanding work environment. J Neuroeng Rehabil. 2018;15(1):1–27.CrossRef
10.
go back to reference A. D. Roche, B. Lakey, I. Mendez, I. Vujaklija, D. Farina, and O. C. Aszmann, “Clinical Perspectives in Upper Limb Prostheses: An Update,” Curr. Surg. Reports, vol. 7, no. 3, p. 5, 2019. A. D. Roche, B. Lakey, I. Mendez, I. Vujaklija, D. Farina, and O. C. Aszmann, “Clinical Perspectives in Upper Limb Prostheses: An Update,” Curr. Surg. Reports, vol. 7, no. 3, p. 5, 2019.
11.
go back to reference M. Asghari Oskoei and H. Hu, “Myoelectric control systems-A survey,” Biomedical Signal Processing and Control, vol. 2, no. 4. ELSEVIER SCI LTD, pp. 275–294, Oct-2007. M. Asghari Oskoei and H. Hu, “Myoelectric control systems-A survey,” Biomedical Signal Processing and Control, vol. 2, no. 4. ELSEVIER SCI LTD, pp. 275–294, Oct-2007.
12.
go back to reference A. D. Roche et al., “A structured rehabilitation protocol for improved multifunctional prosthetic control: a case study.,” J. Vis. Exp., no. 105, p. e52968, 2015. A. D. Roche et al., “A structured rehabilitation protocol for improved multifunctional prosthetic control: a case study.,” J. Vis. Exp., no. 105, p. e52968, 2015.
13.
go back to reference Novak D, Riener R. A survey of sensor fusion methods in wearable robotics. Rob Auton Syst. 2014;73:155–70.CrossRef Novak D, Riener R. A survey of sensor fusion methods in wearable robotics. Rob Auton Syst. 2014;73:155–70.CrossRef
14.
go back to reference A. Krasoulis, I. Kyranou, M. S. Erden, K. Nazarpour, and S. Vijayakumar, “Improved prosthetic hand control with concurrent use of myoelectric and inertial measurements,” J. Neuroeng. Rehabil., vol. 14, no. 1, p. 71, 2017. A. Krasoulis, I. Kyranou, M. S. Erden, K. Nazarpour, and S. Vijayakumar, “Improved prosthetic hand control with concurrent use of myoelectric and inertial measurements,” J. Neuroeng. Rehabil., vol. 14, no. 1, p. 71, 2017.
15.
go back to reference Fougner A, Scheme E, Chan ADC, Englehart K, Stavdahl Ø. Resolving the limb position effect in myoelectric pattern recognition. IEEE Trans Neural Syst Rehabil Eng. 2011;19(6):644–51.CrossRef Fougner A, Scheme E, Chan ADC, Englehart K, Stavdahl Ø. Resolving the limb position effect in myoelectric pattern recognition. IEEE Trans Neural Syst Rehabil Eng. 2011;19(6):644–51.CrossRef
16.
go back to reference Y. Geng, P. Zhou, and G. Li, “Toward attenuating the impact of arm positions on electromyography pattern-recognition based motion classification in transradial amputees,” J. Neuroeng. Rehabil., vol. 9, no. 1, p. 74, 2012. Y. Geng, P. Zhou, and G. Li, “Toward attenuating the impact of arm positions on electromyography pattern-recognition based motion classification in transradial amputees,” J. Neuroeng. Rehabil., vol. 9, no. 1, p. 74, 2012.
17.
go back to reference Bennett DA, Goldfarb M. IMU-based wrist rotation control of a transradial myoelectric prosthesis. IEEE Trans. Neural Syst. Rehabil. Eng. 2018;26(2):419–27.CrossRef Bennett DA, Goldfarb M. IMU-based wrist rotation control of a transradial myoelectric prosthesis. IEEE Trans. Neural Syst. Rehabil. Eng. 2018;26(2):419–27.CrossRef
18.
go back to reference G. K. Patel, J. M. Hahne, C. Castellini, D. Farina, and S. Dosen, “Context-dependent adaptation improves robustness of myoelectric control for upper-limb prostheses,” J. Neural Eng., vol. 14, no. 5, p. 056016, 2017. G. K. Patel, J. M. Hahne, C. Castellini, D. Farina, and S. Dosen, “Context-dependent adaptation improves robustness of myoelectric control for upper-limb prostheses,” J. Neural Eng., vol. 14, no. 5, p. 056016, 2017.
19.
go back to reference M. Markovic, S. Dosen, C. Cipriani, D. Popovic, and D. Farina, “Stereovision and augmented reality for closed-loop control of grasping in hand prostheses,” J. Neural Eng., vol. 11, no. 4, 2014. M. Markovic, S. Dosen, C. Cipriani, D. Popovic, and D. Farina, “Stereovision and augmented reality for closed-loop control of grasping in hand prostheses,” J. Neural Eng., vol. 11, no. 4, 2014.
20.
go back to reference Došen S, Popović DB. Transradial prosthesis: artificial vision for control of prehension. Artif Organs. 2011;35(1):37–48.CrossRef Došen S, Popović DB. Transradial prosthesis: artificial vision for control of prehension. Artif Organs. 2011;35(1):37–48.CrossRef
21.
go back to reference G. Ghazaei, A. Alameer, P. Degenaar, G. Morgan, and K. Nazarpour, “Deep learning-based artificial vision for grasp classification in myoelectric hands,” J. Neural Eng., vol. 14, no. 3, p. 036025, 2017. G. Ghazaei, A. Alameer, P. Degenaar, G. Morgan, and K. Nazarpour, “Deep learning-based artificial vision for grasp classification in myoelectric hands,” J. Neural Eng., vol. 14, no. 3, p. 036025, 2017.
22.
go back to reference M. Markovic, S. Dosen, D. Popovic, B. Graimann, and D. Farina, “Sensor fusion and computer vision for context-aware control of a multi degree-of-freedom prosthesis.,” J. Neural Eng., vol. 12, no. 6, p. 066022, Nov. 2015. M. Markovic, S. Dosen, D. Popovic, B. Graimann, and D. Farina, “Sensor fusion and computer vision for context-aware control of a multi degree-of-freedom prosthesis.,” J. Neural Eng., vol. 12, no. 6, p. 066022, Nov. 2015.
23.
go back to reference Jones LE, Davidson JH. Save that arm: a study of problems in the remaining arm of unilateral upper limb amputees. Prosthetics Orthot Int. 1999;23(1):55–8. Jones LE, Davidson JH. Save that arm: a study of problems in the remaining arm of unilateral upper limb amputees. Prosthetics Orthot Int. 1999;23(1):55–8.
24.
go back to reference Gambrell CR. Overuse syndrome and the unilateral upper limb amputee: consequences and prevention. J Prosthetics Orthot. 2008;20(3):126–32.CrossRef Gambrell CR. Overuse syndrome and the unilateral upper limb amputee: consequences and prevention. J Prosthetics Orthot. 2008;20(3):126–32.CrossRef
25.
go back to reference Strazzulla I, Nowak M, Controzzi M, Cipriani C, Castellini C. Online bimanual manipulation using surface electromyography and incremental learning. IEEE Trans. Neural Syst. Rehabil. Eng. 2016;4320(c):1–1. Strazzulla I, Nowak M, Controzzi M, Cipriani C, Castellini C. Online bimanual manipulation using surface electromyography and incremental learning. IEEE Trans. Neural Syst. Rehabil. Eng. 2016;4320(c):1–1.
26.
go back to reference Swinnen SP, Wenderoth N. Two hands, one brain: cognitive neuroscience of bimanual skill. Trends Cogn Sci. 2004;8(1):18–25.CrossRef Swinnen SP, Wenderoth N. Two hands, one brain: cognitive neuroscience of bimanual skill. Trends Cogn Sci. 2004;8(1):18–25.CrossRef
27.
go back to reference Mason AH, Bruyn JL. Manual asymmetries in bimanual prehension tasks: manipulation of object size and object distance. Hum Mov Sci. 2009;28(1):48–73.CrossRef Mason AH, Bruyn JL. Manual asymmetries in bimanual prehension tasks: manipulation of object size and object distance. Hum Mov Sci. 2009;28(1):48–73.CrossRef
28.
go back to reference M. Markovic et al., “The clinical relevance of advanced artificial feedback in the control of a multi-functional myoelectric prosthesis,” J. Neuroeng. Rehabil., vol. 15, no. 1, 2018. M. Markovic et al., “The clinical relevance of advanced artificial feedback in the control of a multi-functional myoelectric prosthesis,” J. Neuroeng. Rehabil., vol. 15, no. 1, 2018.
31.
go back to reference Luchetti M, Cutti AG, Verni G, Sacchetti R, Rossi N. Impact of Michelangelo prosthetic hand: findings from a crossover longitudinal study. J Rehabil Res Dev. 2015;52(5):605–18.CrossRef Luchetti M, Cutti AG, Verni G, Sacchetti R, Rossi N. Impact of Michelangelo prosthetic hand: findings from a crossover longitudinal study. J Rehabil Res Dev. 2015;52(5):605–18.CrossRef
32.
go back to reference Parker P, Englehart K, Hudgins B. Myoelectric signal processing for control of powered limb prostheses. J Electromyogr Kinesiol. 2006;16(6):541–8.CrossRef Parker P, Englehart K, Hudgins B. Myoelectric signal processing for control of powered limb prostheses. J Electromyogr Kinesiol. 2006;16(6):541–8.CrossRef
34.
go back to reference Hart SG, Staveland LE. Development of NASA-TLX (task load index): results of empirical and theoretical research. Adv Psychol. 1988;52(C):139–83.CrossRef Hart SG, Staveland LE. Development of NASA-TLX (task load index): results of empirical and theoretical research. Adv Psychol. 1988;52(C):139–83.CrossRef
35.
go back to reference J. M. Hahne, M. A. Schweisfurth, M. Koppe, and D. Farina, “Simultaneous control of multiple functions of bionic hand prostheses: Performance and robustness in end users,” Sci. Robot., vol. 3, no. 19, Jun. 2018. J. M. Hahne, M. A. Schweisfurth, M. Koppe, and D. Farina, “Simultaneous control of multiple functions of bionic hand prostheses: Performance and robustness in end users,” Sci. Robot., vol. 3, no. 19, Jun. 2018.
36.
go back to reference Chadwell A, et al. Upper limb activity in myoelectric prosthesis users is biased towards the intact limb and appears unrelated to goal-directed task performance. Sci Rep. 2018;8(1):1–12.CrossRef Chadwell A, et al. Upper limb activity in myoelectric prosthesis users is biased towards the intact limb and appears unrelated to goal-directed task performance. Sci Rep. 2018;8(1):1–12.CrossRef
37.
go back to reference S. Wang et al., “Evaluation of Performance-Based Outcome Measures for the Upper Limb: A Comprehensive Narrative Review,” PM R, vol. 10, no. 9, pp. 951–962.e3, 2018. S. Wang et al., “Evaluation of Performance-Based Outcome Measures for the Upper Limb: A Comprehensive Narrative Review,” PM R, vol. 10, no. 9, pp. 951–962.e3, 2018.
38.
go back to reference I. Vujaklija et al., “Translating research on myoelectric control into clinics-are the performance assessment methods adequate?,” Front. Neurorobot., vol. 11, no. FEB, pp. 1–7, 2017. I. Vujaklija et al., “Translating research on myoelectric control into clinics-are the performance assessment methods adequate?,” Front. Neurorobot., vol. 11, no. FEB, pp. 1–7, 2017.
39.
go back to reference Bayat A, Pomplun M, Tran DA. A study on human activity recognition using accelerometer data from smartphones. Procedia Comput Sci. 2014;34:450–7.CrossRef Bayat A, Pomplun M, Tran DA. A study on human activity recognition using accelerometer data from smartphones. Procedia Comput Sci. 2014;34:450–7.CrossRef
Metadata
Title
Improving bimanual interaction with a prosthesis using semi-autonomous control
Authors
Robin Volkmar
Strahinja Dosen
Jose Gonzalez-Vargas
Marcus Baum
Marko Markovic
Publication date
01-12-2019
Publisher
BioMed Central
Published in
Journal of NeuroEngineering and Rehabilitation / Issue 1/2019
Electronic ISSN: 1743-0003
DOI
https://doi.org/10.1186/s12984-019-0617-6

Other articles of this Issue 1/2019

Journal of NeuroEngineering and Rehabilitation 1/2019 Go to the issue