Top

Journal of NeuroEngineering and Rehabilitation

Published in:

Open Access 01-12-2020 | Review

Relying on more sense for enhancing lower limb prostheses control: a review

Authors: Michael Tschiedel, Michael Friedrich Russold, Eugenijus Kaniusas

Published in: Journal of NeuroEngineering and Rehabilitation | Issue 1/2020

Abstract

Modern lower limb prostheses have the capability to replace missing body parts and improve the patients’ quality of life. However, missing environmental information often makes a seamless adaptation to transitions between different forms of locomotion challenging. The aim of this review is to identify the progress made in this area over the last decade, addressing two main questions: which types of novel sensors for environmental awareness are used in lower limb prostheses, and how do they enhance device control towards more comfort and safety. A literature search was conducted on two Internet databases, PubMed and IEEE Xplore. Based on the criteria for inclusion and exclusion, 32 papers were selected for the review analysis, 18 of those are related to explicit environmental sensing and 14 to implicit environmental sensing. Characteristics were discussed with a focus on update rate and resolution as well as on computing power and energy consumption. Our analysis identified numerous state-of-the-art sensors, some of which are able to “look through” clothing or cosmetic covers. Five control categories were identified, how “next generation prostheses” could be extended. There is a clear tendency towards more upcoming object or terrain prediction concepts using all types of distance and depth-based sensors. Other advanced strategies, such as bilateral gait segmentation from unilateral sensors, could also play an important role in movement-dependent control applications. The studies demonstrated promising accuracy in well-controlled laboratory settings, but it is unclear how the systems will perform in real-world environments, both indoors and outdoors. At the moment the main limitation proves to be the necessity of having an unobstructed field of view.

Ziegler-Graham K, Mackenzie EJ, Ephraim PL, Travison TG, Brookmeyer R. Estimating the Prevalence of Limb Loss in the United States 2005 to 2050. Arch Phys Med Rehabil. 2008; 89(3):422–9.PubMed

Sinha R, van den Heuvel WJ, Arokiasamy P. Factors affecting quality of life in lower limb amputees. Prosthetics Orthot Int. 2011; 35(1):90–96. https://doi.org/10.1177/0309364610397087.

Winter DA. The biomechanics and motor control of human gait: normal, elderly and pathological, 2nd ed. Waterloo: Ont: Waterloo Biomechanics; 1991.

Pieringer DS, Grimmer M, Russold MF, Riener R. Review of the actuators of active knee prostheses and their target design outputs for activities of daily living. In: International Conference on Rehabilitation Robotics (ICORR). IEEE: 2017. p. 1246–1253. https://doi.org/10.1109/ICORR.2017.8009420.

Fluit R, Prinsen E, Wang S, Kooij HVD. A comparison of control strategies in commercial and research knee prostheses. IEEE Trans Biomed Eng. 2019; 67(1):277–90. https://doi.org/10.1109/TBME.2019.2912466.PubMed

Tucker M, Olivier J, Pagel A, Bleuler H, Bouri M, Lambercy O, et al. Control strategies for active lower extremity prosthetics and orthotics: a review. J Neuroengineering Rehabil. 2015; 12(1):1.

Varol HA, Sup F, Goldfarb M. Multiclass Real-Time Intent Recognition of a Powered Lower Limb Prosthesis. IEEE Trans Biomed Eng. 2010; 57(3):542–51.PubMed

Webster J, Watson RT. Analyzing the Past to Prepare for the Future: Writing a Literature Review. MIS Q. 2002; 26(2):xiii–xxiii.

Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. J Clin Epidemiol. 2009; 62(10):1006–12.PubMed

10.

Vallery H, Burgkart R, Hartmann C, Mitternacht J, Riener R, Buss M. Complementary limb motion estimation for the control of active knee prostheses. Biomed Tech Biomed Eng. 2011; 56(1):45–51. https://doi.org/10.1515/BMT.2010.057.

11.

Bernal-Torres M, Medellín-Castillo H, González A. Development of an Active Biomimetic-Controlled Transfemoral Knee Prosthesis. 2016. https://doi.org/10.1115/IMECE2016-67211.

12.

Bernal-Torres MG, Medellín-Castillo HI, undefined Juan C Arellano-González. Design and Control of a New Biomimetic Transfemoral Knee Prosthesis Using an Echo-Control Scheme. J Healthc Eng. 2018; 2018. https://doi.org/10.1155/2018/8783642.

13.

Su B, Wang J, Liu S, Sheng M, Jiang J, Xiang K. A CNN-Based Method for Intent Recognition Using Inertial Measurement Units and Intelligent Lower Limb Prosthesis. IEEE Trans Neural Syst and Rehabil Eng. 2019; 27(5):1032–42. https://doi.org/10.1109/TNSRE.2019.2909585.

14.

Goodfellow I, Bengio Y, Courville A. Deep learning. In: Adaptive computation and machine learning. Cambridge, Massachusetts London: The MIT Press: 2016.

15.

Ambrozic L, Gorsic M, Slajpah S, Kamnik R, Munih M. Wearable sensory system for robotic prosthesis. Internation J Mech Control JoMaC. 2014; 15(1):53–59.

16.

Ambrozic L, Gorsic M, Geeroms J, Flynn L, Lova RM, Kamnik R, et al. CYBERLEGs: A User-Oriented Robotic Transfemoral Prosthesis with Whole-Body Awareness Control. IEEE Robot Autom Mag. 2014; 21(4):82–93.

17.

Goršič M, Kamnik R, Ambrožič L, Vitiello N, Lefeber D, Pasquini G, et al. Online phase detection using wearable sensors for walking with a robotic prosthesis. Sensors (Basel). 2014; 14(2):2776–94.

18.

Parri A, Martini E, Geeroms J, Flynn L, Pasquini G, Crea S, et al. Whole Body Awareness for Controlling a Robotic Transfemoral Prosthesis. Front Neurorobotics. 2017;11(25). https://doi.org/10.3389/fnbot.2017.00025.

19.

Hu B, Rouse E, Hargrove L. Fusion of Bilateral Lower-Limb Neuromechanical Signals Improves Prediction of Locomotor Activities. Front Robot AI. 2018; 5:78.

20.

Hu B, Rouse E, Hargrove L. Using bilateral lower limb kinematic and myoelectric signals to predict locomotor activities: A pilot study. In: 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER). IEEE: 2017. p. 98–101. https://doi.org/10.1109/NER.2017.8008301.

21.

Hu B, Rouse E, Hargrove L. Benchmark Datasets for Bilateral Lower-Limb Neuromechanical Signals from Wearable Sensors during Unassisted Locomotion in Able-Bodied Individuals. Front Robot AI. 2018; 5:14.

22.

Krausz NE, Hu BH, Hargrove LJ. Subject- and Environment-Based Sensor Variability for Wearable Lower-Limb Assistive Devices. Sensors. 2019; 19:4887.

23.

Hu B, Krausz NE, Hargrove LJ. A novel method for bilateral gait segmentation using a single thigh-mounted depth sensor and IMU. In: 2018 7th IEEE International Conference on Biomedical Robotics and Biomechatronics (Biorob). IEEE: 2018. p. 807–12. https://doi.org/10.1109/BIOROB.2018.8487806.

24.

Fischler MA, Bolles RC. Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. Commun ACM. 1981; 24(6):381–95. https://doi.org/10.1145/358669.358692.

25.

Zhang F, Yan T, Meng MQ. Gait Phase Recognition Based on A Wearable Depth Camera*. In: 2018 IEEE International Conference on Information and Automation (ICIA). IEEE: 2018. p. 756–60. https://doi.org/10.1109/ICInfA.2018.8812542.

26.

Zheng E, Manca S, Yan T, Parri A, Vitiello N, Wang Q. Gait Phase Estimation Based on Noncontact Capacitive Sensing and Adaptive Oscillators. IEEE Trans Biomed Eng. 2017; 64(10):2419–30.PubMed

27.

Scandaroli GG, Borges GA, Ishihara JY, Terra MH, Rocha AFD, de Oliveira Nascimento FA. Estimation of foot orientation with respect to ground for an above knee robotic prosthesis. In: 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems. St. Louis: IEEE: 2009. p. 1112–7. https://doi.org/10.1109/IROS.2009.5354820.

28.

Ishikawa T, Murakami T. Real-time foot clearance and environment estimation based on foot-mounted wearable sensors. In: IECON 2018-44th Annual Conference of the IEEE Industrial Electronics Society. IEEE: 2018. p. 5475–5480. https://doi.org/10.1109/IECON.2018.8592894.

29.

Kleiner B, Cesmeci D. D8.4 - Foresighted Control of Active Foot Prostheses. In: SENSOR+TEST Conferences 2011 Nürnberg. vol. Proceedings SENSOR 2011 of D8 - Medical III: 2011. p. 669 –72. https://doi.org/10.5162/sensor11/d8.4.

30.

Zhang F, Fang Z, Liu M, Huang H. Preliminary design of a terrain recognition system. In: 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. vol 2011. Boston: IEEE: 2011. p. 5452–5. https://doi.org/10.1109/IEMBS.2011.6091391.

31.

Zhang X, Wang D, Yang Q, Huang H. An automatic and user-driven training method for locomotion mode recognition for artificial leg control. In: 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. vol 2012. San Diego: IEEE: 2012. p. 6116–9. https://doi.org/10.1109/EMBC.2012.6347389.

32.

Wang D, Du L, Huang H. Terrain recognition improves the performance of neural-machine interface for locomotion mode recognition. In: 2013 International Conference on Computing, Networking and Communications (ICNC). San Diego: IEEE: 2013. p. 87–91. https://doi.org/10.1109/ICCNC.2013.6504059.

33.

Liu M, Wang D, Huang H. Development of an Environment-Aware Locomotion Mode Recognition System for Powered Lower Limb Prostheses. IEEE Trans Neural Syst Rehabil Eng. 2016; 24(4):434–43.PubMed

34.

Zhang F, Liu M, Harper S, Lee M, Huang H. Engineering platform and experimental protocol for design and evaluation of a neurally-controlled powered transfemoral prosthesis. J Vis Exp. 2014;89(e51059). https://doi.org/10.3791/51059.

35.

Huang H, Zhang F, Hargrove LJ, Dou Z, Rogers DR, Englehart KB. Continuous Locomotion-Mode Identification for Prosthetic Legs Based on Neuromuscular-Mechanical Fusion. IEEE Trans Biomed Eng. 2011; 58(10):2867–75.PubMedPubMedCentral

36.

Carvalho S, Figueiredo J, Santos CP. Environment-aware locomotion mode transition prediction system. In: 2019 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC). IEEE: 2019. p. 1–6. https://doi.org/10.1109/ICARSC.2019.8733658.

37.

Sahoo S, Maheshwari M, Pratihar DK, Mukhopadhyay S. A Geometry Recognition-Based Strategy for Locomotion Transitions’ Early Prediction of Prosthetic Devices. IEEE Trans Instrum Meas. 2019; 69(4):1259–1267. https://doi.org/10.1109/TIM.2019.2909246.

38.

Varol HA, Massalin Y. A feasibility study of depth image based intent recognition for lower limb prostheses. In: 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). Orlando: IEEE: 2016. p. 5055–8. https://doi.org/10.1109/EMBC.2016.7591863.

39.

Massalin Y, Abdrakhmanova M, Varol HA. User-Independent Intent Recognition for Lower Limb Prostheses Using Depth Sensing. IEEE Trans Biomed Eng. 2018; 65(8):1759–70.PubMed

40.

Laschowski B, McNally W, Wong A, McPhee J. Preliminary design of an environment recognition system for controlling robotic lower-limb prostheses and exoskeletons. In: 2019 IEEE 16th International Conference on Rehabilitation Robotics (ICORR). IEEE: 2019. p. 868–73. https://doi.org/doi:10.1109/ICORR.2019.8779540.

41.

Yan T, Sun Y, Liu T, Cheung C, Meng MQ. A locomotion recognition system using depth images. In: 2018 IEEE International Conference on Robotics and Automation (ICRA). IEEE: 2018. p. 6766–6772. https://doi.org/10.1109/ICRA.2018.8460514.

42.

Duda R, Hart P. Use of the Hough transformation to detect lines and curves in pictures. Commun ACM. 1972; 15(1):11–15.

43.

Diaz JP, da Silva RL, Zhong B, Huang HH, Lobaton E. Visual terrain identification and surface inclination estimation for improving human locomotion with a lower-limb prosthetic. In: 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). Honolulu: IEEE: 2018. p. 1817–20. https://doi.org/10.1109/EMBC.2018.8512614.

44.

Csurka G, Dance CR, Fan L, Willamowski J, Bray C. Visual categorization with bags of keypoints. In: In Workshop on Statistical Learning in Computer Vision, ECCV. Prague: CTU Prague: 2004. p. 1–22.

45.

Krausz NE, Lenzi T, Hargrove LJ. Depth Sensing for Improved Control of Lower Limb Prostheses. IEEE Trans Biomed Eng. 2015; 62(11):2576–87.PubMed

46.

Kleiner B, Ziegenspeck N, Stolyarov R, Herr H, Schneider U, Verl A. A radar-based terrain mapping approach for stair detection towards enhanced prosthetic foot control. In: 2018 7th IEEE International Conference on Biomedical Robotics and Biomechatronics (Biorob). Enschede: IEEE: 2018. p. 105–10. https://doi.org/10.1109/BIOROB.2018.8487722.

47.

Zhang K, Xiong C, Zhang W, Liu H, Lai D, Rong Y, et al. Environmental Features Recognition for Lower Limb Prostheses Toward Predictive Walking. IEEE Trans Neural Syst Rehabil Eng. 2019; 27(3):465–76.PubMed

48.

Zhang K, Zhang W, Xiao W, Liu H, Silva CWD, Fu C. Sequential Decision Fusion for Environmental Classification in Assistive Walking. IEEE Neural Syst Rehabil Eng. 2019; 27(9):1780–90. https://doi.org/10.1109/TNSRE.2019.2935765.

49.

Perry J, Burnfield J. Gait Analysis: Normal and Pathological Function, 2nd ed. NJ, USA: Slack Incorporated: Thorofare; 2010.

50.

Dadashi F, Mariani B, Rochat S, Büla CJ, Santos-Eggimann B, Aminian K. Gait and foot clearance parameters obtained using shoe-worn inertial sensors in a large-population sample of older adults. Sensors (Basel). 2013; 14(1):443–57.

51.

Genium. Set-up Guide. https://shop.ottobock.us/media/pdf/647G868-EN-02-1210w.pdf. Accessed 9 Sept 2019.

52.

Han J, Shao L, Xu D, Shotton J. Enhanced Computer Vision With Microsoft Kinect Sensor: A Review. IEEE Trans Cybern. 2013; 43(5):1318–34.PubMed

53.

Scudellari M. Self-driving wheelchairs debut in hospitals and airports [News]. IEEE Spectrum. 2017; 54(10):14–14. https://doi.org/10.1109/MSPEC.2017.8048827.

54.

CES 2020 Press Release: new pmd 3D VGA ToF modul. https://www.pmdtec.com/html/pdf/press_release/PR20200107_CES_VGA_module.pdf. Accessed 18 Mar 2020.

55.

Richards MA. Principles of Modern Radar: 1 Basic Principles. Edison, NJ: Scitech Publ.; 2010.

56.

Lien J, Gillian N, Karagozler ME, Amihood P, Schwesig C, Olson E, et al. Soli: ubiquitous gesture sensing with millimeter wave Radar. ACM Trans Graph. 2016; 35(4):1–19. https://doi.org/10.1145/2897824.2925953.

57.

Parajuli N, Sreenivasan N, Bifulco P, Cesarelli M, Savino S, Niola V, et al. Real-time EMG based pattern recognition control for hand prostheses: a review on existing methods, challenges and future implementation. Sensors. 2019; 19(20):4596.

58.

Moloney D, Barry B, Richmond R, Connor F, Brick C, Donohoe D. Myriad 2: Eye of the computational vision storm. In: 2014 IEEE Hot Chips 26 Symposium (HCS). Cupertino: IEEE: 2014. p. 1–18. https://doi.org/10.1109/HOTCHIPS.2014.7478823.

59.

SoftKinetic DS325 Datasheet. https://www.sony-depthsensing.com/Portals/0/Download/WEB_20120907_SK_DS325_Datasheet_V2.1.pdf. Accessed 26 Sept 2019.

60.

CamBoard pico flexx Datasheet. https://pmdtec.com/picofamily/assets/datasheet/Data-sheet-PMD_RD_Brief_CB_pico_flexx_V0201.pdf. Accessed 26 Sept 2019.

61.

Zech C, Hulsmann A, Schlechtweg M, Reinold S, Giers C, Kleiner B, et al. A compact W-band LFMCW radar module with high accuracy and integrated signal processing. In: 2015 European Microwave Conference (EuMC). Paris: IEEE: 2015. p. 554–7. https://doi.org/10.1109/EuMC.2015.7345823.

Title: Relying on more sense for enhancing lower limb prostheses control: a review
Authors: Michael Tschiedel
Michael Friedrich Russold
Eugenijus Kaniusas
Publication date: 01-12-2020
Publisher: BioMed Central
Published in: Journal of NeuroEngineering and Rehabilitation / Issue 1/2020
Electronic ISSN: 1743-0003
DOI: https://doi.org/10.1186/s12984-020-00726-x

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Relying on more sense for enhancing lower limb prostheses control: a review

Abstract

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 1/2020

Simulation of human gait with body weight support: benchmarking models and unloading strategies

Dynamic body-weight support to boost rehabilitation outcomes in patients with non-traumatic spinal cord injury: an observational study

Role of data measurement characteristics in the accurate detection of Parkinson’s disease symptoms using wearable sensors

Supporting front crawl swimming in paraplegics using electrical stimulation: a feasibility study

Fall inducing movable platform (FIMP) for overground trips and slips

Time evolution of frontal plane dynamic balance during locomotor transitions of altered anticipation and complexity