Top

Published in:

01-12-2020 | Multiple Sclerosis | Research Article

Machine learning analysis of motor evoked potential time series to predict disability progression in multiple sclerosis

Authors: Jan Yperman, Thijs Becker, Dirk Valkenborg, Veronica Popescu, Niels Hellings, Bart Van Wijmeersch, Liesbet M. Peeters

Published in: BMC Neurology | Issue 1/2020

Abstract

Background

Evoked potentials (EPs) are a measure of the conductivity of the central nervous system. They are used to monitor disease progression of multiple sclerosis patients. Previous studies only extracted a few variables from the EPs, which are often further condensed into a single variable: the EP score. We perform a machine learning analysis of motor EP that uses the whole time series, instead of a few variables, to predict disability progression after two years. Obtaining realistic performance estimates of this task has been difficult because of small data set sizes. We recently extracted a dataset of EPs from the Rehabiliation & MS Center in Overpelt, Belgium. Our data set is large enough to obtain, for the first time, a performance estimate on an independent test set containing different patients.

Methods

We extracted a large number of time series features from the motor EPs with the highly comparative time series analysis software package. Mutual information with the target and the Boruta method are used to find features which contain information not included in the features studied in the literature. We use random forests (RF) and logistic regression (LR) classifiers to predict disability progression after two years. Statistical significance of the performance increase when adding extra features is checked.

Results

Including extra time series features in motor EPs leads to a statistically significant improvement compared to using only the known features, although the effect is limited in magnitude (ΔAUC = 0.02 for RF and ΔAUC = 0.05 for LR). RF with extra time series features obtains the best performance (AUC = 0.75±0.07 (mean and standard deviation)), which is good considering the limited number of biomarkers in the model. RF (a nonlinear classifier) outperforms LR (a linear classifier).

Conclusions

Using machine learning methods on EPs shows promising predictive performance. Using additional EP time series features beyond those already in use leads to a modest increase in performance. Larger datasets, preferably multi-center, are needed for further research. Given a large enough dataset, these models may be used to support clinicians in their decision making process regarding future treatment.

Available only for authorised users

Sospedra M, Martin R. Immunology of multiple sclerosis. Ann Rev Immunol. 2004; 23(1):683–747. https://doi.org/10.1146/annurev.immunol.23.021704.115707.CrossRef

Montalban X, Gold R, Thompson AJ, Otero-Romero S, Amato MP, Chandraratna D, Clanet M, Comi G, Derfuss T, Fazekas F, et al.Ectrims/ean guideline on the pharmacological treatment of people with multiple sclerosis. Mult Scler J. 2018; 24(2):96–120.CrossRef

Tilling K, Lawton M, Robertson N, Tremlett H, Zhu F, Harding K, Oger J, Ben-Shlomo Y. Modelling disease progression in relapsing-remitting onset multiple sclerosis using multilevel models applied to longitudinal data from two natural history cohorts and one treated cohort. Health Technol Assess. 2016; 20(81):1–48. https://doi.org/10.3310/hta20810.CrossRefPubMedPubMedCentral

Kurtzke JF. Rating neurologic impairment in multiple sclerosis: an expanded disability status scale (edss). Neurology. 1983; 33(11):1444–52.CrossRefPubMed

Gajofatto A, Calabrese M, Benedetti MD, Monaco S. Clinical, mri, and csf markers of disability progression in multiple sclerosis. Dis Mark. 2013; 35(6):13. https://doi.org/10.1155/2013/484959.

Tintoré M, Rovira A, Río J, Tur C, Pelayo R, Nos C, Téllez N, Perkal H, Comabella M, Sastre-Garriga J, Montalban X. Do oligoclonal bands add information to mri in first attacks of multiple sclerosis?Neurology. 2008; 70(13 Part 2):1079. https://doi.org/10.1212/01.wnl.0000280576.73609.c6.CrossRefPubMed

Pelayo R, Montalban X, Minoves T, Moncho D, Rio J, Nos C, Tur C, Castillo J, Horga A, Comabella M, Perkal H, Rovira A, Tintoré M. Do multimodal evoked potentials add information to mri in clinically isolated syndromes?Mult Scler J. 2009; 16(1):55–61. https://doi.org/10.1177/1352458509352666.CrossRef

Martinelli V, Dalla Costa G, Messina MJ, Di Maggio G, Sangalli F, Moiola L, Rodegher M, Colombo B, Furlan R, Leocani L, Falini A, Comi G. Multiple biomarkers improve the prediction of multiple sclerosis in clinically isolated syndromes. Acta Neurol Scand. 2017; 136(5):454–61. https://doi.org/10.1111/ane.12761.CrossRefPubMed

Leocani L, Guerrieri S, Comi G. Visual evoked potentials as a biomarker in multiple sclerosis and associated optic neuritis. J Neuro-Ophthalmol. 2018; 38(3):350–7. https://doi.org/10.1097/wno.0000000000000704.CrossRef

10.

Nuwer MR, Packwood JW, Myers LW, Ellison GW. Evoked potentials predict the clinical changes in a multiple sclerosis drug study. Neurology. 1987; 37(11):1754. https://doi.org/10.1212/WNL.37.11.1754.CrossRefPubMed

11.

O’Connor P, Marchetti P, Lee L, Perera M. Evoked potential abnormality scores are a useful measure of disease burden in relapsing–remitting multiple sclerosis. Ann Neurol. 1998; 44(3):404–7. https://doi.org/10.1002/ana.410440320.CrossRefPubMed

12.

Fuhr P, Kappos L. Evoked potentials for evaluation of multiple sclerosis. Clin Neurophysiol. 2001; 112(12):2185–9. https://doi.org/10.1016/S1388-2457(01)00687-3.CrossRefPubMed

13.

Fuhr P, Borggrefe-Chappuis A, Schindler C, Kappos L. Visual and motor evoked potentials in the course of multiple sclerosis. Brain. 2001; 124(11):2162–8. https://doi.org/10.1093/brain/124.11.2162.CrossRefPubMed

14.

Kallmann BA, Fackelmann S, Toyka KV, Rieckmann P, Reiners K. Early abnormalities of evoked potentials and future disability in patients with multiple sclerosis. Mult Scler J. 2006; 12(1):58–65. https://doi.org/10.1191/135248506ms1244oa.CrossRef

15.

Leocani L, Rovaris M, Boneschi FM, Medaglini S, Rossi P, Martinelli V, Amadio S, Comi G. Multimodal evoked potentials to assess the evolution of multiple sclerosis: a longitudinal study. J Neurol Neurosurg Psychiatry. 2006; 77(9):1030. https://doi.org/10.1136/jnnp.2005.086280.CrossRefPubMedPubMedCentral

16.

Jung P, Beyerle A, Ziemann U. Multimodal evoked potentials measure and predict disability progression in early relapsing–remitting multiple sclerosis. Mult Scler J. 2008; 14(4):553–6. https://doi.org/10.1177/1352458507085758.CrossRef

17.

Bejarano B, Bianco M, Gonzalez-Moron D, Sepulcre J, Goñi J, Arcocha J, Soto O, Del Carro U, Comi G, Leocani L, et al.Computational classifiers for predicting the short-term course of multiple sclerosis. BMC Neurol. 2011; 11(1):67.CrossRefPubMedPubMedCentral

18.

Invernizzi P, Bertolasi L, Bianchi MR, Turatti M, Gajofatto A, Benedetti MD. Prognostic value of multimodal evoked potentials in multiple sclerosis: the ep score. J Neurol. 2011; 258(11):1933–9. https://doi.org/10.1007/s00415-011-6033-x.CrossRefPubMed

19.

Schlaeger R, D’Souza M, Schindler C, Grize L, Kappos L, Fuhr P. Combined evoked potentials as markers and predictors of disability in early multiple sclerosis. Clin Neurophysiol. 2012; 123(2):406–10. https://doi.org/10.1016/j.clinph.2011.06.021.CrossRefPubMed

20.

Schlaeger R, D’Souza M, Schindler C, Grize L, Dellas S, Radue EW, Kappos L, Fuhr P. Prediction of long-term disability in multiple sclerosis. Mult Scler J. 2011; 18(1):31–8. https://doi.org/10.1177/1352458511416836.CrossRef

21.

Schlaeger R, D’Souza M, Schindler C, Grize L, Kappos L, Fuhr P. Prediction of ms disability by multimodal evoked potentials: Investigation during relapse or in the relapse-free interval?Clin Neurophysiol. 2014; 125(9):1889–92. https://doi.org/10.1016/j.clinph.2013.12.117.CrossRefPubMed

22.

Schlaeger R, D’Souza M, Schindler C, Grize L, Kappos L, Fuhr P. Electrophysiological markers and predictors of the disease course in primary progressive multiple sclerosis. Mult Scler J. 2013; 20(1):51–6. https://doi.org/10.1177/1352458513490543.CrossRef

23.

Schlaeger R, Hardmeier M, D’Souza M, Grize L, Schindler C, Kappos L, Fuhr P. Monitoring multiple sclerosis by multimodal evoked potentials: Numerically versus ordinally scaled scoring systems. Clin Neurophysiol. 2016; 127(3):1864–71. https://doi.org/10.1016/j.clinph.2015.11.041.CrossRefPubMed

24.

Giffroy X, Maes N, Albert A, Maquet P, Crielaard J-M, Dive D. Multimodal evoked potentials for functional quantification and prognosis in multiple sclerosis. BMC Neurol. 2016; 16:83–3. https://doi.org/10.1186/s12883-016-0608-1.CrossRefPubMedPubMedCentral

25.

Hardmeier M, Hatz F, Naegelin Y, Hight D, Schindler C, Kappos L, Seeck M, Michel CM, Fuhr P. Improved characterization of visual evoked potentials in multiple sclerosis by topographic analysis. Brain Topogr. 2014; 27(2):318–27. https://doi.org/10.1007/s10548-013-0318-6.CrossRefPubMed

26.

Giffroy X, Maes N, Albert A, Maquet P, Crielaard JM, Dive D. Do evoked potentials contribute to the functional follow-up and clinical prognosis of multiple sclerosis?Acta Neurol Belg. 2017; 117(1):53–9. https://doi.org/10.1007/s13760-016-0650-1.CrossRefPubMed

27.

Schlaeger R, Schindler C, Grize L, Dellas S, Radue EW, Kappos L, Fuhr P. Combined visual and motor evoked potentials predict multiple sclerosis disability after 20 years. Mult Scler J. 2014; 20(10):1348–54. https://doi.org/10.1177/1352458514525867.CrossRef

28.

Margaritella N, Mendozzi L, Garegnani M, Colicino E, Gilardi E, DeLeonardis L, Tronci F, Pugnetti L. Sensory evoked potentials to predict short-term progression of disability in multiple sclerosis. Neurol Sci. 2012; 33(4):887–92. https://doi.org/10.1007/s10072-011-0862-3.CrossRefPubMed

29.

Margaritella N, Mendozzi L, Tronci F, Colicino E, Garegnani M, Nemni R, Gilardi E, Pugnetti L. The evoked potentials score improves the identification of benign ms without cognitive impairment. Eur J Neurol. 2013; 20(10):1423–5. https://doi.org/10.1111/ene.12071.CrossRefPubMed

30.

Ramanathan S, Lenton K, Burke T, Gomes L, Storchenegger K, Yiannikas C, Vucic S. The utility of multimodal evoked potentials in multiple sclerosis prognostication. J Clin Neurosci. 2013; 20(11):1576–81. https://doi.org/10.1016/j.jocn.2013.01.020.CrossRefPubMed

31.

Canham LJW, Kane N, Oware A, Walsh P, Blake K, Inglis K, Homewood J, Witherick J, Faulkner H, White P, Lewis A, Furse-Roberts C, Cottrell DA. Multimodal neurophysiological evaluation of primary progressive multiple sclerosis – an increasingly valid biomarker, with limits. Mult Scler Relat Disord. 2015; 4(6):607–13. https://doi.org/10.1016/j.msard.2015.07.009.CrossRefPubMed

32.

London F, El Sankari S, van Pesch V. Early disturbances in multimodal evoked potentials as a prognostic factor for long-term disability in relapsing-remitting multiple sclerosis patients. Clin Neurophysiol. 2017; 128(4):561–9. https://doi.org/10.1016/j.clinph.2016.12.029.CrossRefPubMed

33.

Comi G, Leocani L, Medaglini S, Locatelli T, Martinelli V, Santuccio G, Rossi P. Measuring evoked responses in multiple sclerosis. Mult Scler J. 1999; 5(4):263–7. https://doi.org/10.1177/135245859900500412.CrossRef

34.

Hardmeier M, Leocani L, Fuhr P. A new role for evoked potentials in ms? repurposing evoked potentials as biomarkers for clinical trials in ms. Mult Scler J. 2017; 23(10):1309–19. https://doi.org/10.1177/1352458517707265.CrossRef

35.

Fernández O, Fernández V. Evoked potentials are of little use in the diagnosis or monitoring of ms: No. Mult Scler J. 2013; 19(14):1822–3.CrossRef

36.

McGuigan C. Evoked potentials are of little use in the diagnosis or monitoring of ms: Yes. Mult Scler J. 2013; 19(14):1820–1.CrossRef

37.

Hutchinson M. Evoked potentials are of little use in the diagnosis or monitoring of ms: Commentary. Mult Scler J. 2013; 19(14):1824–5.CrossRef

38.

Walsh P, Kane N, Butler S. The clinical role of evoked potentials. J Neurol Neurosurg Psychiatr. 2005; 76(suppl 2):16–22. https://doi.org/10.1136/jnnp.2005.068130.

39.

Neter J, Kutner MH, Nachtsheim CJ, Wasserman W. Applied Linear Statistical Models vol. 4: Irwin Chicago; 1996.

40.

Spelman T, Jokubaitis V, Kalincik T, Butzkueven H, Grammond P, Hupperts R, Oreja-Guevara C, Boz C, Pucci E, Bergamaschi R, Lechner-Scott J, Alroughani R, Van Pesch V, Iuliano G, Fernandez-Bolaños R, Ramo C, Terzi M, Slee M, Spitaleri D, Verheul F, Cristiano E, Sánchez-Menoyo JL, Fiol M, Gray O, Cutter G, Cabrera-Gomez JA, Barnett M, Horakova D, Havrdova E, Trojano M, Izquierdo G, Prat A, Girard M, Duquette P, Lugaresi A, Grand’Maison F. Defining reliable disability outcomes in multiple sclerosis. Brain. 2015; 138(11):3287–98. https://doi.org/10.1093/brain/awv258. http://oup.prod.sis.lan/brain/article-pdf/138/11/3287/13798678/awv258.pdf.CrossRefPubMed

41.

Livingston SC, Ingersoll CD. Intra-rater reliability of a transcranial magnetic stimulation technique to obtain motor evoked potentials. Int J Neurosci. 2008; 118(2):239–56. https://doi.org/10.1080/00207450701668020. http://arxiv.org/abs/https://doi.org/10.1080/00207450701668020.CrossRefPubMed

42.

Cacchio A, Paoloni M, Cimini N, Mangone M, Liris G, Aloisi P, Santilli V, Marrelli A. Reliability of TMS-related measures of tibialis anterior muscle in patients with chronic stroke and healthy subjects. J Neurol Sci. 2011; 303(1):90–4. https://doi.org/10.1016/j.jns.2011.01.004. Accessed 25 Sept 2019.CrossRefPubMed

43.

Hoonhorst MH, Kollen BJ, Van Den Berg PS, Emmelot CH, Kwakkel G. How reproducible are transcranial magnetic stimulation–induced meps in subacute stroke?J Clin Neurophysiol. 2014; 31(6):556–62.CrossRefPubMed

44.

Hardmeier M, Jacques F, Albrecht P, Bousleiman H, Schindler C, Leocani L, Fuhr P. Multicentre assessment of motor and sensory evoked potentials in multiple sclerosis: reliability and implications for clinical trials. Mult Scler J Exp Transl Clin. 2019; 5(2):2055217319844796. https://doi.org/10.1177/2055217319844796. http://arxiv.org/abs/https://doi.org/10.1177/2055217319844796.PubMedPubMedCentral

45.

Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E. Scikit-learn: Machine learning in Python. J Mach Learn Res. 2011; 12:2825–30.

46.

Kursa M, Rudnicki W. Feature selection with the boruta package. J Stat Softw Artic. 2010; 36(11):1–13. https://doi.org/10.18637/jss.v036.i11.

47.

Fulcher BD, Little MA, Jones NS. Highly comparative time-series analysis: the empirical structure of time series and their methods. J R Soc Interface. 2013; 10(83):20130048.CrossRefPubMedPubMedCentral

48.

Fulcher BD, Jones NS. A computational framework for automated time-series phenotyping using massive feature extraction. Cell Syst. 2017; 5(5):527–5313. https://doi.org/10.1016/j.cels.2017.10.001.CrossRefPubMed

49.

Lines J, Taylor S, Bagnall A. Time series classification with hive-cote: The hierarchical vote collective of transformation-based ensembles. ACM Trans Knowl Discov Data. 2018; 12(5):52–15235. https://doi.org/10.1145/3182382.CrossRef

50.

Aerts HJ, Velazquez ER, Leijenaar RT, Parmar C, Grossmann P, Carvalho S, Bussink J, Monshouwer R, Haibe-Kains B, Rietveld D, et al.Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun. 2014; 5:4006.CrossRefPubMed

51.

Degenhardt F, Seifert S, Szymczak S. Evaluation of variable selection methods for random forests and omics data sets. Brief Bioinf. 2017. https://doi.org/10.1093/bib/bbx124. http://oup.prod.sis.lan/bib/advance-article-pdf/doi/10.1093/bib/bbx124/21301018/bbx124.pdf.

52.

Hastie T, Tibshirani R, Friedman J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition, Springer Series in Statistics: Springer; 2009. https://books.google.be/books?id=tVIjmNS3Ob8C.

53.

Mukherjee S, Tamayo P, Rogers S, Rifkin R, Engle A, Campbell C, Mesirov J. Estimating dataset size requirements for classifying dna microarray data. J Comput Biol. 2003; 10:119–42. https://doi.org/10.1089/106652703321825928.CrossRefPubMed

54.

Cho J, Lee K, Shin E, Choy G, Do S. How much data is needed to train a medical image deep learning system to achieve necessary high accuracy?arXiv preprint. 2015. arXiv:1511.06348.

55.

Zhu X, Vondrick C, Fowlkes CC, Ramanan D. Do we need more training data?Int J Comput Vis. 2016; 119(1):76–92.CrossRef

56.

Sun C, Shrivastava A, Singh S, Gupta A. Revisiting unreasonable effectiveness of data in deep learning era. In: Proceedings of the IEEE International Conference on Computer Vision: 2017. p. 843–52. https://doi.org/10.1109/iccv.2017.97.

57.

DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988; 44(3):837–45.CrossRefPubMed

58.

De Brouwer E, Peeters L, Becker T, Altintas A, Soysal A, Van Wijmeersch B, Boz C, Oreja-Guevara C, Gobbi C, Solaro C, et al.Introducing machine learning for full ms patient trajectories improves predictions for disability score progression. Mult Scler J. 2019; 25:63–5.CrossRef

59.

Lipton ZC, Kale D, Wetzel R. Directly modeling missing data in sequences with rnns: Improved classification of clinical time series In: Doshi-Velez F, Fackler J, Kale D, Wallace B, Wiens J, editors. Proceedings of the 1st Machine Learning for Healthcare Conference, Proceedings of Machine Learning Research, vol. 56. Children’s Hospital LA, Los Angeles: PMLR: 2016. p. 253–70. http://proceedings.mlr.press/v56/Lipton16.html.

60.

Che Z, Purushotham S, Cho K, Sontag D, Liu Y. Recurrent neural networks for multivariate time series with missing values. Sci Rep. 2018; 8(1):6085. https://doi.org/10.1038/s41598-018-24271-9.CrossRefPubMedPubMedCentral

61.

Trojano M, Tintore M, Montalban X, Hillert J, Kalincik T, Iaffaldano P, Spelman T, Sormani MP, Butzkueven H. Treatment decisions in multiple sclerosis - insights from real-world observational studies. Nat Rev Neurol. 2017; 13(2):105–18. https://doi.org/10.1038/nrneurol.2016.188.CrossRefPubMed

62.

Kalincik T, Brown JWL, Robertson N, Willis M, Scolding N, Rice CM, Wilkins A, Pearson O, Ziemssen T, Hutchinson M, McGuigan C, Jokubaitis V, Spelman T, Horakova D, Havrdova E, Trojano M, Izquierdo G, Lugaresi A, Prat A, Girard M, Duquette P, Grammond P, Alroughani R, Pucci E, Sola P, Hupperts R, Lechner-Scott J, Terzi M, Van Pesch V, Rozsa C, Grand’Maison F, Boz C, Granella F, Slee M, Spitaleri D, Olascoaga J, Bergamaschi R, Verheul F, Vucic S, McCombe P, Hodgkinson S, Sanchez-Menoyo JL, Ampapa R, Simo M, Csepany T, Ramo C, Cristiano E, Barnett M, Butzkueven H, Coles A, Group MSS. Treatment effectiveness of alemtuzumab compared with natalizumab, fingolimod, and interferon beta in relapsing-remitting multiple sclerosis: a cohort study. Lancet Neurol. 2017; 16(4):271–81. https://doi.org/10.1016/S1474-4422(17)30007-8.CrossRefPubMed

63.

Gafson A, Craner MJ, Matthews PM. Personalised medicine for multiple sclerosis care. Mult Scler. 2017; 23(3):362–9. https://doi.org/10.1177/1352458516672017.CrossRefPubMed

64.

Hardmeier M, Jacques F, Albrecht P, Bousleiman H, Schindler C, Leocani L, Fuhr P. F107. sensory and motor evoked potentials in a multicenter setting: Estimation of detectable group differences at varying sample sizes. Clin Neurophysiol. 2018; 129:106–7. https://doi.org/10.1016/j.clinph.2018.04.270.CrossRef

65.

Hardmeier M, Jacques F, Albrecht P, Bousleiman H, Schindler C, Leocani L, Fuhr P. T85. sensory and motor evoked potentials in a multicenter setting: Definition of significant change in repeated measurements in healthy subjects on individual level. Clin Neurophysiol. 2018; 129:34–5. https://doi.org/10.1016/j.clinph.2018.04.086.CrossRef

Title: Machine learning analysis of motor evoked potential time series to predict disability progression in multiple sclerosis
Authors: Jan Yperman
Thijs Becker
Dirk Valkenborg
Veronica Popescu
Niels Hellings
Bart Van Wijmeersch
Liesbet M. Peeters
Publication date: 01-12-2020
Publisher: BioMed Central
Keywords: Multiple Sclerosis
Motor Evoked Potential
Evoked Potential
Motor Evoked Potential
Evoked Potential
Published in: BMC Neurology / Issue 1/2020
Electronic ISSN: 1471-2377
DOI: https://doi.org/10.1186/s12883-020-01672-w

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Machine learning analysis of motor evoked potential time series to predict disability progression in multiple sclerosis

Abstract

Background

Methods

Results

Conclusions

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2020

“The strategies are the same, the problems may be different”: a qualitative study exploring the experiences of healthcare and service providers with medication therapy management for individuals with spinal cord injury/dysfunction

Different subtypes of collateral vessels in hemorrhagic moyamoya disease with p.R4810K variant

Non breathing-related sleep fragmentation and imaging markers in patients with atherosclerotic cerebral small vessel disease (CSVD): a cross-sectional case-control study

Cervical myelopathy due to neurovascular compression syndrome caused by persistent first intersegmental artery: a case report

Two years’ experience of implementing a comprehensive telemedical stroke network comprising in mainly rural region: the Transregional Network for Stroke Intervention with Telemedicine (TRANSIT-Stroke)

Early diagnostic value of serum procalcitonin levels for catheter-related blood stream infection in first-ever acute ischemic stroke patients