Skip to main content
Top
Published in: Journal of Medical Systems 12/2018

Open Access 01-12-2018 | Patient Facing Systems

Patient-Tailored Augmented Reality Games for Assessing Upper Extremity Motor Impairments in Parkinson’s Disease and Stroke

Authors: Paulina J. M. Bank, Marina A. Cidota, P. (Elma) W. Ouwehand, Stephan G. Lukosch

Published in: Journal of Medical Systems | Issue 12/2018

Login to get access

Abstract

In clinical practice, upper extremity motor impairments are commonly assessed with disease-specific, subjectively scored and low-resolution rating scales that often do not consider the variations in tasks and environment that are essential aspects of daily life. Augmented reality (AR) systems with contactless tracking of the hand and upper body offer opportunities for objective quantification of motor (dys)function in a challenging, engaging and patient-tailored environment. In this study, we explore the potential of AR for evaluating 1) speed and goal-directedness of movements within the individually determined interaction space, 2) adaptation of hand opening to objects of different sizes, and 3) obstacle avoidance in healthy individuals (N = 10) and two highly prevalent neurological conditions (N = 10 patients with Parkinson’s Disease and N = 10 stroke patients). We successfully implemented three AR games to evaluate these key aspects of motor function. As expected, PD patients moved slower than controls and needed more time for task completion. No differences were observed between stroke patients and controls, perhaps because motor impairments in this patient group were relatively mild. Importantly, usability of our AR system was good and considerably improved compared to our previous study due to more natural and patient-tailored interaction. Although our findings testify to the potential of AR for assessing motor impairments in patients with neurological conditions and provide starting points for further improvement, there are still many steps to be taken towards application in clinical practice.
Appendix
Available only for authorised users
Literature
1.
go back to reference Hondori, H.M., Khademi, M., Dodakian, L., Cramer, S.C., and Lopes, C.V., A spatial augmented reality rehab system for post-stroke hand rehabilitation. In: Medicine Meets Virtual Reality, pp 279–285, 2013. Hondori, H.M., Khademi, M., Dodakian, L., Cramer, S.C., and Lopes, C.V., A spatial augmented reality rehab system for post-stroke hand rehabilitation. In: Medicine Meets Virtual Reality, pp 279–285, 2013.
2.
go back to reference Taske, A., Oppermann, L., Niemann, K., and Wilken, R., Design and Evaluation of a Stroke Rehabilitation Program. In: Virtuelle und Erweiterte Realität-12. Workshop der GI-Fachgruppe VR/AR. Shaker Verlag, pp 34–45, 2015. Taske, A., Oppermann, L., Niemann, K., and Wilken, R., Design and Evaluation of a Stroke Rehabilitation Program. In: Virtuelle und Erweiterte Realität-12. Workshop der GI-Fachgruppe VR/AR. Shaker Verlag, pp 34–45, 2015.
3.
go back to reference Burke, J. W., McNeill, M., Charles, D. K., Morrow, P. J., Crosbie, J. H., and McDonough, S. M., Optimising engagement for stroke rehabilitation using serious games. Vis. Comput. 25(12):1085, 2009.CrossRef Burke, J. W., McNeill, M., Charles, D. K., Morrow, P. J., Crosbie, J. H., and McDonough, S. M., Optimising engagement for stroke rehabilitation using serious games. Vis. Comput. 25(12):1085, 2009.CrossRef
4.
go back to reference Luo, X., Kenyon, R.V., Kline, T., Waldinger, H.C., and Kamper, D.G., An augmented reality training environment for post-stroke finger extension rehabilitation. In: 9th International Conference on Rehabilitation Robotics (ICORR). IEEE, pp 329–332, 2005. Luo, X., Kenyon, R.V., Kline, T., Waldinger, H.C., and Kamper, D.G., An augmented reality training environment for post-stroke finger extension rehabilitation. In: 9th International Conference on Rehabilitation Robotics (ICORR). IEEE, pp 329–332, 2005.
5.
go back to reference Shen, Y., Ong, S., and Nee, A., Hand rehabilitation based on augmented reality. In: Proceedings of the 3rd International Convention on Rehabilitation Engineering & Assistive Technology. ACM, p 23, 2009. Shen, Y., Ong, S., and Nee, A., Hand rehabilitation based on augmented reality. In: Proceedings of the 3rd International Convention on Rehabilitation Engineering & Assistive Technology. ACM, p 23, 2009.
6.
go back to reference Alamri, A., Cha, J., and El Saddik, A., AR-REHAB: An augmented reality framework for poststroke-patient rehabilitation. IEEE Trans. Instrum. Meas. 59(10):2554–2563, 2010.CrossRef Alamri, A., Cha, J., and El Saddik, A., AR-REHAB: An augmented reality framework for poststroke-patient rehabilitation. IEEE Trans. Instrum. Meas. 59(10):2554–2563, 2010.CrossRef
7.
go back to reference Liu, J., Mei, J., Zhang, X., Lu, X., and Huang, J., Augmented reality-based training system for hand rehabilitation. Multimedia Tools and Applications 76(13):14847–14867, 2017.CrossRef Liu, J., Mei, J., Zhang, X., Lu, X., and Huang, J., Augmented reality-based training system for hand rehabilitation. Multimedia Tools and Applications 76(13):14847–14867, 2017.CrossRef
8.
go back to reference Khademi, M., Hondori, H.M., Lopes, C.V., Dodakian, L., and Cramer, S.C., Haptic augmented reality to monitor human arm's stiffness in rehabilitation. In: 2012 IEEE EMBS Conference on Biomedical Engineering and Sciences (IECBES). IEEE, pp 892–895, 2012. Khademi, M., Hondori, H.M., Lopes, C.V., Dodakian, L., and Cramer, S.C., Haptic augmented reality to monitor human arm's stiffness in rehabilitation. In: 2012 IEEE EMBS Conference on Biomedical Engineering and Sciences (IECBES). IEEE, pp 892–895, 2012.
9.
go back to reference Khademi, M., Mousavi Hondori, H., McKenzie, A., Dodakian, L., Lopes, C.V., and Cramer, S.C., Comparing direct and indirect interaction in stroke rehabilitation. In: CHI'14 Extended Abstracts on Human Factors in Computing Systems. ACM, pp 1639–1644, 2014. Khademi, M., Mousavi Hondori, H., McKenzie, A., Dodakian, L., Lopes, C.V., and Cramer, S.C., Comparing direct and indirect interaction in stroke rehabilitation. In: CHI'14 Extended Abstracts on Human Factors in Computing Systems. ACM, pp 1639–1644, 2014.
10.
go back to reference Mousavi Hondori, H., Khademi, M., Dodakian, L., McKenzie, A., Lopes, C. V., and Cramer, S. C., Choice of human–computer interaction mode in stroke rehabilitation. Neurorehabil. Neural Repair 30(3):258–265, 2016.CrossRef Mousavi Hondori, H., Khademi, M., Dodakian, L., McKenzie, A., Lopes, C. V., and Cramer, S. C., Choice of human–computer interaction mode in stroke rehabilitation. Neurorehabil. Neural Repair 30(3):258–265, 2016.CrossRef
11.
go back to reference Sousa, M., Vieira, J., Medeiros, D., Arsenio, A., and Jorge, J., SleeveAR: Augmented Reality for Rehabilitation using Realtime Feedback. In: Proceedings of the 21st International Conference on Intelligent User Interfaces. ACM, pp 175–185, 2016. Sousa, M., Vieira, J., Medeiros, D., Arsenio, A., and Jorge, J., SleeveAR: Augmented Reality for Rehabilitation using Realtime Feedback. In: Proceedings of the 21st International Conference on Intelligent User Interfaces. ACM, pp 175–185, 2016.
12.
go back to reference Da Gama, A. E. F., Chaves, T. M., Figueiredo, L. S., Baltar, A., Meng, M., Navab, N., Teichrieb, V., and Fallavollita, P., MirrARbilitation: A clinically-related gesture recognition interactive tool for an AR rehabilitation system. Comput. Methods Prog. Biomed. 135:105–114, 2016.CrossRef Da Gama, A. E. F., Chaves, T. M., Figueiredo, L. S., Baltar, A., Meng, M., Navab, N., Teichrieb, V., and Fallavollita, P., MirrARbilitation: A clinically-related gesture recognition interactive tool for an AR rehabilitation system. Comput. Methods Prog. Biomed. 135:105–114, 2016.CrossRef
13.
go back to reference Regenbrecht, H., McGregor, G., Ott, C., Hoermann, S., Schubert, T., Hale, L., Hoermann, J., Dixon, B., and Franz, E., Out of reach?—A novel AR interface approach for motor rehabilitation. In: 10th IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, pp 219–228, 2011. Regenbrecht, H., McGregor, G., Ott, C., Hoermann, S., Schubert, T., Hale, L., Hoermann, J., Dixon, B., and Franz, E., Out of reach?—A novel AR interface approach for motor rehabilitation. In: 10th IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, pp 219–228, 2011.
14.
go back to reference Khademi, M., Mousavi Hondori, H., McKenzie, A., Dodakian, L., Lopes, C.V., and Cramer, S.C., Free-hand interaction with leap motion controller for stroke rehabilitation. In: CHI'14 Extended Abstracts on Human Factors in Computing Systems. ACM, pp 1663–1668, 2014. Khademi, M., Mousavi Hondori, H., McKenzie, A., Dodakian, L., Lopes, C.V., and Cramer, S.C., Free-hand interaction with leap motion controller for stroke rehabilitation. In: CHI'14 Extended Abstracts on Human Factors in Computing Systems. ACM, pp 1663–1668, 2014.
15.
go back to reference Cidota, M.A., Bank, P.J.M., Ouwehand, P.W., and Lukosch, S.G., Assessing Upper Extremity Motor Dysfunction Using an Augmented Reality Game. In: 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, pp 144–154, 2017. Cidota, M.A., Bank, P.J.M., Ouwehand, P.W., and Lukosch, S.G., Assessing Upper Extremity Motor Dysfunction Using an Augmented Reality Game. In: 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, pp 144–154, 2017.
16.
go back to reference Cidota, M.A., Lukosch, S.G., Bank, P.J.M., and Ouwehand, P.W., Towards Engaging Upper Extremity Motor Dysfunction Assessment Using Augmented Reality Games. In: 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct). IEEE, pp 275–278, 2017. Cidota, M.A., Lukosch, S.G., Bank, P.J.M., and Ouwehand, P.W., Towards Engaging Upper Extremity Motor Dysfunction Assessment Using Augmented Reality Games. In: 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct). IEEE, pp 275–278, 2017.
17.
go back to reference Hoehn, M. M., and Yahr, M. D., Parkinsonism: onset, progression, and mortality. Neurology 50(2):318–318, 1998.CrossRef Hoehn, M. M., and Yahr, M. D., Parkinsonism: onset, progression, and mortality. Neurology 50(2):318–318, 1998.CrossRef
18.
go back to reference Marinus, J., Visser, M., Stiggelbout, A. M., Rabey, J. M., Martínez-Martín, P., Bonuccelli, U., Kraus, P. H., and van Hilten, J. J., A short scale for the assessment of motor impairments and disabilities in Parkinson’s disease: the SPES/SCOPA. J. Neurol. Neurosurg. Psychiatry 75(3):388–395, 2004.CrossRef Marinus, J., Visser, M., Stiggelbout, A. M., Rabey, J. M., Martínez-Martín, P., Bonuccelli, U., Kraus, P. H., and van Hilten, J. J., A short scale for the assessment of motor impairments and disabilities in Parkinson’s disease: the SPES/SCOPA. J. Neurol. Neurosurg. Psychiatry 75(3):388–395, 2004.CrossRef
19.
go back to reference Van Swieten, J., Koudstaal, P., Visser, M., Schouten, H., and van Gijn, J., Interobserver agreement for the assessment of handicap in stroke patients. Stroke 19(5):604–607, 1988.CrossRef Van Swieten, J., Koudstaal, P., Visser, M., Schouten, H., and van Gijn, J., Interobserver agreement for the assessment of handicap in stroke patients. Stroke 19(5):604–607, 1988.CrossRef
20.
go back to reference Fugl-Meyer, A. R., Jääskö, L., Leyman, I., Olsson, S., and Steglind, S., The post-stroke hemiplegic patient. 1. a method for evaluation of physical performance. Scand. J. Rehabil. Med. 7(1):13–31, 1975.PubMed Fugl-Meyer, A. R., Jääskö, L., Leyman, I., Olsson, S., and Steglind, S., The post-stroke hemiplegic patient. 1. a method for evaluation of physical performance. Scand. J. Rehabil. Med. 7(1):13–31, 1975.PubMed
21.
go back to reference Hart, S.G., and Staveland, L.E., Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In: Advances in psychology, vol 52. Elsevier, pp 139–183, 1988. Hart, S.G., and Staveland, L.E., Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In: Advances in psychology, vol 52. Elsevier, pp 139–183, 1988.
22.
go back to reference IJsselsteijn, W., van den Hoogen, W., Klimmt, C., De Kort, Y., Lindley, C., Mathiak, K., Poels, K., Ravaja, N., Turpeinen, M., Vorderer, P., Measuring the experience of digital game enjoyment. In: Proceedings of Measuring Behavior, 2008. Noldus Information Tecnology Wageningen, Netherlands, pp 88–89, 2008. IJsselsteijn, W., van den Hoogen, W., Klimmt, C., De Kort, Y., Lindley, C., Mathiak, K., Poels, K., Ravaja, N., Turpeinen, M., Vorderer, P., Measuring the experience of digital game enjoyment. In: Proceedings of Measuring Behavior, 2008. Noldus Information Tecnology Wageningen, Netherlands, pp 88–89, 2008.
23.
go back to reference IJsselsteijn, W. A., de Kort, Y. A. W., and Poels, K., The Game Experience Questionnaire. Eindhoven: Technische Universiteit Eindhoven, 2013. IJsselsteijn, W. A., de Kort, Y. A. W., and Poels, K., The Game Experience Questionnaire. Eindhoven: Technische Universiteit Eindhoven, 2013.
24.
go back to reference Brooke, J., SUS-A quick and dirty usability scale. Usability Evaluation in Industry 189(194):4–7, 1996. Brooke, J., SUS-A quick and dirty usability scale. Usability Evaluation in Industry 189(194):4–7, 1996.
25.
go back to reference Gandy, M., Catrambone, R., MacIntyre, B., Alvarez, C., Eiriksdottir, E., Hilimire, M., Davidson, B., and McLaughlin, A.C., Experiences with an AR evaluation test bed: Presence, performance, and physiological measurement. In: 2010 9th IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, pp 127–136, 2010. Gandy, M., Catrambone, R., MacIntyre, B., Alvarez, C., Eiriksdottir, E., Hilimire, M., Davidson, B., and McLaughlin, A.C., Experiences with an AR evaluation test bed: Presence, performance, and physiological measurement. In: 2010 9th IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, pp 127–136, 2010.
26.
go back to reference Cidota, M. A., Lukosch, S. G., Dezentje, P., Bank, P. J.M., Lukosch, H. K., and Clifford, R. M., Serious Gaming in Augmented Reality using HMDs for Assessment of Upper Extremity Motor Dysfunctions. I-Com 15(2):155–169, 2016. Cidota, M. A., Lukosch, S. G., Dezentje, P., Bank, P. J.M., Lukosch, H. K., and Clifford, R. M., Serious Gaming in Augmented Reality using HMDs for Assessment of Upper Extremity Motor Dysfunctions. I-Com 15(2):155–169, 2016.
27.
go back to reference Field, A., Discovering statistics using IBM SPSS statistics. London, UK: SAGE Publications, 2013. Field, A., Discovering statistics using IBM SPSS statistics. London, UK: SAGE Publications, 2013.
28.
go back to reference Marsden, C., Slowness of movement in Parkinson's disease. Movement Disorders: Official Journal of the Movement Disorder Society 4(S1):S26–S37, 1989.CrossRef Marsden, C., Slowness of movement in Parkinson's disease. Movement Disorders: Official Journal of the Movement Disorder Society 4(S1):S26–S37, 1989.CrossRef
29.
go back to reference Bootsma, R. J., Marteniuk, R. G., MacKenzie, C. L., and Zaal, F. T., The speed-accuracy trade-off in manual prehension: effects of movement amplitude, object size and object width on kinematic characteristics. Exp. Brain Res. 98(3):535–541, 1994.CrossRef Bootsma, R. J., Marteniuk, R. G., MacKenzie, C. L., and Zaal, F. T., The speed-accuracy trade-off in manual prehension: effects of movement amplitude, object size and object width on kinematic characteristics. Exp. Brain Res. 98(3):535–541, 1994.CrossRef
30.
go back to reference Richard, P., Birebent, G., Coiffet, P., Burdea, G., Gomez, D., and Langrana, N., Effect of frame rate and force feedback on virtual object manipulation. Presence: Teleoperators & Virtual Environments 5(1):95–108, 1996.CrossRef Richard, P., Birebent, G., Coiffet, P., Burdea, G., Gomez, D., and Langrana, N., Effect of frame rate and force feedback on virtual object manipulation. Presence: Teleoperators & Virtual Environments 5(1):95–108, 1996.CrossRef
31.
go back to reference Levin, M. F., Magdalon, E. C., Michaelsen, S. M., and Quevedo, A. A., Quality of grasping and the role of haptics in a 3-D immersive virtual reality environment in individuals with stroke. IEEE Transactions on Neural Systems and Rehabilitation Engineering 23(6):1047–1055, 2015.CrossRef Levin, M. F., Magdalon, E. C., Michaelsen, S. M., and Quevedo, A. A., Quality of grasping and the role of haptics in a 3-D immersive virtual reality environment in individuals with stroke. IEEE Transactions on Neural Systems and Rehabilitation Engineering 23(6):1047–1055, 2015.CrossRef
32.
go back to reference Magdalon, E. C., Michaelsen, S. M., Quevedo, A. A., and Levin, M. F., Comparison of grasping movements made by healthy subjects in a 3-dimensional immersive virtual versus physical environment. Acta Psychol. 138(1):126–134, 2011.CrossRef Magdalon, E. C., Michaelsen, S. M., Quevedo, A. A., and Levin, M. F., Comparison of grasping movements made by healthy subjects in a 3-dimensional immersive virtual versus physical environment. Acta Psychol. 138(1):126–134, 2011.CrossRef
Metadata
Title
Patient-Tailored Augmented Reality Games for Assessing Upper Extremity Motor Impairments in Parkinson’s Disease and Stroke
Authors
Paulina J. M. Bank
Marina A. Cidota
P. (Elma) W. Ouwehand
Stephan G. Lukosch
Publication date
01-12-2018
Publisher
Springer US
Published in
Journal of Medical Systems / Issue 12/2018
Print ISSN: 0148-5598
Electronic ISSN: 1573-689X
DOI
https://doi.org/10.1007/s10916-018-1100-9

Other articles of this Issue 12/2018

Journal of Medical Systems 12/2018 Go to the issue