Top

BMC Medical Research Methodology

Published in:

Open Access 01-12-2022 | Coronavirus | Research article

Detecting the patient’s need for help with machine learning based on expressions

Author: Lauri Lahti

Published in: BMC Medical Research Methodology | Issue 1/2022

Abstract

Background

Developing machine learning models to support health analytics requires increased understanding about statistical properties of self-rated expression statements used in health-related communication and decision making. To address this, our current research analyzes self-rated expression statements concerning the coronavirus COVID-19 epidemic and with a new methodology identifies how statistically significant differences between groups of respondents can be linked to machine learning results.

Methods

A quantitative cross-sectional study gathering the “need for help” ratings for twenty health-related expression statements concerning the coronavirus epidemic on an 11-point Likert scale, and nine answers about the person’s health and wellbeing, sex and age. The study involved online respondents between 30 May and 3 August 2020 recruited from Finnish patient and disabled people’s organizations, other health-related organizations and professionals, and educational institutions (n = 673). We propose and experimentally motivate a new methodology of influence analysis concerning machine learning to be applied for evaluating how machine learning results depend on and are influenced by various properties of the data which are identified with traditional statistical methods.

Results

We found statistically significant Kendall rank-correlations and high cosine similarity values between various health-related expression statement pairs concerning the “need for help” ratings and a background question pair. With tests of Wilcoxon rank-sum, Kruskal-Wallis and one-way analysis of variance (ANOVA) between groups we identified statistically significant rating differences for several health-related expression statements in respect to groupings based on the answer values of background questions, such as the ratings of suspecting to have the coronavirus infection and having it depending on the estimated health condition, quality of life and sex. Our new methodology enabled us to identify how statistically significant rating differences were linked to machine learning results thus helping to develop better human-understandable machine learning models.

Conclusions

The self-rated “need for help” concerning health-related expression statements differs statistically significantly depending on the person’s background information, such as his/her estimated health condition, quality of life and sex. With our new methodology statistically significant rating differences can be linked to machine learning results thus enabling to develop better machine learning to identify, interpret and address the patient’s needs for well-personalized care.

Available only for authorised users

Gallagher JE, Wilkie AA, Cordner A, Hudgens E, Ghio A, Birch R, Wade T. (2016). Factors associated with self-reported health: implications for screening level community-based health and environmental studies. BMC Public Health, 16, 640 (2016). https://doi.org/10.1186/s12889-016-3321-5.

Wu S, Wang R, Zhao Y, Ma X, Wu M, Yan X, He J. The relationship between self-rated health and objective health status: a population-based study. BMC Public Health. 2013;13:320. https://doi.org/10.1186/1471-2458-13-320.CrossRefPubMedPubMedCentral

Cullati S, Bochatay N, Rossier C, Guessous I, Burton-Jeangros C, Courvoisier DS. Does the single-item self-rated health measure the same thing across different wordings? Construct validity study. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment Care Rehabilitation. 2020;29(9):2593–604. https://doi.org/10.1007/s11136-020-02533-2.CrossRef

Garbarski D, Schaeffer NC, Dykema J. The effect of response option order on self-rated health: a replication study. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment Care Rehabilitation. 2016;25(8):2117–21. https://doi.org/10.1007/s11136-016-1249-y.CrossRef

Joffer J, Jerdén L, Öhman A, Flacking R. Exploring self-rated health among adolescents: a think-aloud study. BMC Public Health. 2016;16:156. https://doi.org/10.1186/s12889-016-2837-z.CrossRefPubMedPubMedCentral

Borraccino A, Pera R, Lemma P. “What being healthy means to me”: aqualitative analysis uncovering the core categories of adolescents’ perception of health. PLOS One. 2019;14(6):e0218727. https://doi.org/10.1371/journal.pone.0218727.

Tucker CA, Escorpizo R, Cieza A, Lai JS, Stucki G, Ustun TB, Kostanjsek N, Cella D, Forrest CB. Mapping the content of the Patient-Reported Outcomes Measurement Information System (PROMIS®) using the International Classification of Functioning, Health and Disability. Quality Life Res Int J Quality Life Aspects Treatment Care Rehabil. 2014;23(9):2431–8. https://doi.org/10.1007/s11136-014-0691-y.CrossRef

Anttila H, Kokko K, Hiekkala S, Weckström P, Paltamaa J. (2017). Asiakaslähtöinen Toimintakykyni-sovellus. Kehittäminen ja käytettävyystutkimus. Kansaneläkelaitos (Kela), Työpapereita 119, ISSN: 2323-9239. http://hdl.handle.net/10138/187061.

Jacobson RP, Kang D, Houck J. Can Patient-Reported Outcomes Measurement Information System® (PROMIS) measures accurately enhance understanding of acceptable symptoms and functioning in primary care? Journal of Patient-Reported Outcomes. 2020;4(1):39. https://doi.org/10.1186/s41687-020-00206-9.CrossRefPubMedPubMedCentral

10.

Schalet BD, Revicki DA, Cook KF, Krishnan E, Fries JF, Cella D. Establishing a common metric for physical function: linking the HAQ-DI and SF-36 PF subscale to PROMIS Physical Function. J Gen Intern Med. 2015;30(10):1517–23. https://doi.org/10.1007/s11606-015-3360-0.CrossRefPubMedPubMedCentral

11.

Deo RC. Machine learning in medicine. Circulation. 2015;132(20):1920–30. https://doi.org/10.1161/CIRCULATIONAHA.115.001593.CrossRefPubMedPubMedCentral

12.

Zhao X, Miao C, Xing Z. Identifying cognitive distortion by convolutional neural network based text classification. Int J Information Technol. 2017;23(No. 1): pp. 1-12 (2017), Singapore Computer Society, LILY Research Centre, Nanyang Technological University, Singapore. http://www.intjit.org/cms/journal/volume/23/1/231_4.pdf.

13.

Gehrmann S, Dernoncourt F, Li Y, Carlson E, Wu J, Welt J, Foote J Jr, Moseley E, Grant D, Tyler P, Celi L. Comparing deep learning and concept extraction based methods for patient phenotyping from clinical narratives. PLOS One. 2018;13(2):e0192360. https://doi.org/10.1371/journal.pone.0192360.CrossRefPubMedPubMedCentral

14.

Rojas-Barahona L, Tseng B, Dai Y, Mansfield C, Ramadan O, Ultes S, Crawford M, Gašić M. (2018). Deep learning for language understanding of mental health concepts derived from cognitive behavioural therapy. Proceedings of the Ninth International Workshop on Health Text Mining and Information Analysis, 2018. DOI:https://doi.org/10.18653/v1/w18-5606.

15.

Shickel B, Siegel S, Heesacker M, Benton S, Rashidi P. Automatic detection and classification of cognitive distortions in mental health text. Research Article . 2019 manuscript published on 23 September 2019 at https://arxiv.org/abs/1909.07502.

16.

Abd-Alrazaq A, Safi Z, Alajlani M, Warren J, Househ M, Denecke K. Technical metrics used to evaluate health care chatbots: scoping review. Journal of Medical Internet Research. 2020;22(6):e18301. https://doi.org/10.2196/18301.CrossRefPubMedPubMedCentral

17.

Laranjo L, Dunn A, Tong H, Kocaballi A, Chen J, Bashir R, Surian D, Gallego B, Magrabi F, Lau A, Coiera E. Conversational agents in healthcare: a systematic review. J Am Med Inform Assoc. 2018;25(9):1248–58. https://doi.org/10.1093/jamia/ocy072.CrossRefPubMedPubMedCentral

18.

Knäuper B, Carrière K, Chamandy M, Xu Z, Schwarz N, Rosen NO. How aging affects self-reports. Eur J Ageing. 2016;13(2):185–93. https://doi.org/10.1007/s10433-016-0369-0.CrossRefPubMedPubMedCentral

19.

Sinclair S, Jaggi P, Hack TF, Russell L, McClement S, Cuthbertson L, Selman L, Leget C. (2020). Initial validation of a patient-reported measure of compassion: determining the content validity and clinical sensibility among patients living with a life-limiting and incurable illness. The Patient - Patient-Centered Outcomes Research, volume 13, pages 327–337. https://doi.org/10.1007/s40271-020-00409-8.

20.

Sinclair S, Jaggi P, Hack TF, McClement SE, Cuthbertson LA. A practical guide for item generation in measure development: insights from the development of a patient-reported experience measure of compassion. J Nurs Meas. 2020 28(1), DOI:https://doi.org/10.1891/JNM-D-19-00020.

21.

Lahti L. (2017). Interpretation of health-related expressions and dialogues: enabling personalized care with contextual measuring and machine learning. International Journal of New Technology and Research (IJNTR), volume 3, issue 11, November 2017, pages 171-179, ISSN 2454-4116. https://www.ijntr.org/download_data/IJNTR03110081.pdf and http://urn.fi/URN:NBN:fi:aalto-201712298340.

22.

Lahti L. Supporting care by interpretation of expressions about patient experience with machine learning. Int J New Technol Research (IJNTR). 2018: 4,issue 12:27-34. ISSN 2454-4116. DOI: https://doi.org/10.31871/IJNTR.4.12.16. https://www.ijntr.org/download_data/IJNTR04120016.pdf and http://urn.fi/URN:NBN:fi:aalto-201812015463.

23.

Bradley MM, Lang PJ. (1999). Affective norms for English words (ANEW): Instruction manual and affective ratings. Technical Report C-1, The Center for Research in Psychophysiology, University of Florida. http://www.uvm.edu/pdodds/teaching/courses/2009-08UVM-300/docs/others/everything/bradley1999a.pdf.

24.

Warriner A, Kuperman V, Brysbaert M. Norms of valence, arousal, and dominance for 13,915 English lemmas. Behavior Research Methods. 2013; 45(4), 1191-1207. http://link.springer.com/article/10.3758/s13428-012-0314-x.

25.

Mauss IB, Robinson MD. Measures of emotion: a review. Cogn Emot. 2009;23(2):209–37. https://doi.org/10.1080/02699930802204677.CrossRefPubMedPubMedCentral

26.

Berna C, Vincent K, Moore J, Tracey I, Goodwin G, Holmes E. Presence of mental imagery associated with chronic pelvic pain: a pilot study. Pain Med. 2011;12:issue 7, pages 1086–93. https://doi.org/10.1111/j.1526-4637.2011.01152.x.

27.

Hughes M, Li I, Kotoulas S, Suzumura T. (2017). Medical text classification using convolutional neural networks. Studies in Health Technology and Informatics, volume 235, pages 246-250. PMID: 28423791.

28.

Yao L, Mao C, Luo Y. Clinical text classification with rule-based features and knowledge-guided convolutional neural networks. BMC Med Informatics Decision Making. 2019;19:71. https://doi.org/10.1186/s12911-019-0781-4.CrossRef

29.

Qing L, Linhong W, Xuehai D. A novel neural network-based method for medical text classification. Future Internet. 2019;11(12):255. DOI:https://doi.org/10.3390/fi11120255.CrossRef

30.

Bhandare A, Bhide M, Gokhale P, Chandavarkar R. (2016). Applications of convolutional neural networks. International Journal of Computer Science and Information Technologies (IJCSIT), vol. 7(5) (2016), 2206-2215. https://ijcsit.com/docs/Volume%207/vol7issue5/ijcsit20160705014.pdf.

31.

Jelodar H, Wang Y, Orji R, Huang S. Deep sentiment classification and topic discovery on novel coronavirus or COVID-19 online discussions: NLP using LSTM recurrent neural network approach. IEEE J Biomed Health Informatics. 2020;24(10):2733–42. https://doi.org/10.1109/JBHI.2020.3001216.CrossRef

32.

Koskinen S, Lundqvist A, Ristiluoma N, editors (2012). Health, functional capacity and welfare in Finland in 2011. National Institute for Health and Welfare in Finland (THL [as of 2019 the new official English name is: Finnish Institute for Health and Welfare]), Report 68/2012. 290 pages. Helsinki, Finland, 2012. ISBN 978-952-245-768-4 (printed), ISBN 978-952-245-769-1 (online publication, http://urn.fi/URN:ISBN:978-952-245-769-1). A health questionnaire appendix “Terveys 2011, tutkimus suomalaisten terveydestä ja toimintakyvystä, terveyskysely”, form T4095, https://thl.fi/documents/10531/2797097/T4095_terveyskysely.pdf.

33.

Lahti L. (2020). Interpretation of the patient’s need for help can be supported with machine learning. In Mansnérus, Juli, Lahti, Raimo, & Blick, Amanda, editors, Personalized medicine: legal and ethical challenges. Faculty of Law, University of Helsinki, Finland, Forum Iuris Series, Helsinki, 2020. ISBN 978-951-51-6940-2 (printed), ISBN 978-951-51-5021-9 (pdf), ISSN 2670-1219. DOI: https://doi.org/10.31885/9789515150219. https://doi.org/10.31885/9789515169419.

34.

Lahti L. (2021). Development of care decision-making model. A manuscript completed and self-archived on 14 July 2021 in the open-access Aaltodoc publication archive at https://aaltodoc.aalto.fi/handle/123456789/108753 and http://urn.fi/URN:NBN:fi:aalto-202107148007.

35.

de Bruin A, Picavet H, Nossikov A, editors (1996). Health interview surveys: towards international harmonization of methods and instruments. World Health Organization (WHO). WHO regional publications, European series, no. 58. ISBN 92-890-1322-2. https://www.euro.who.int/__data/assets/pdf_file/0017/111149/E72841.pdf.

36.

Nosikov A, Gudex C, editors. EUROHIS: Developing common instruments for health surveys. World Health Organization (WHO) and IOS Press, Amsterdam, the Netherlands. 2003. https://www.euro.who.int/__data/assets/pdf_file/0015/101193/WA9502003EU.pdf.

37.

Aalto A, Korpilahti U, Sainio P, Malmivaara A, Koskinen S, Saarni S, Valkeinen H, Luoma M. (2013). Aikuisten geneeriset elämänlaatumittarit terveys- ja hyvinvointitutkimuksessa sekä terveys- ja kuntoutuspalvelujen vaikutusten arvioinnissa. Terveyden ja hyvinvoinnin laitos (THL). TOIMIA-suositus, TOIMIA-verkosto. http://urn.fi/URN:NBN:fi-fe2016092224121; https://www.terveysportti.fi/dtk/tmi/tms00040.

38.

Akoglu H. User’s guide to correlation coefficients. Turkish Journal of Emergency Medicine. 2018;18(3):91–3. https://doi.org/10.1016/j.tjem.2018.08.001.CrossRefPubMedPubMedCentral

39.

TensorFlow image classification tutorial. (2020). TensorFlow image classification tutorial with Python language scripts. https://www.tensorflow.org/tutorials/images/classification; https://github.com/tensorflow/docs/blob/master/site/en/tutorials/images/classification.ipynb; https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv2D.

40.

Lahti L, Tenhunen H, Nieminen M. (2018). How patients talk about care? Identifying patient experience expressions from online discussions. Proc. Medical Informatics Europe (MIE 2018), 24-26 April 2018, Gothenburg, Sweden (eds. Ugon, A., Karlsson, D., Klein, G., & Moen, A.), 116-120. European Federation for Medical Informatics (EFMI). ISBN 978-1-61499-851-8 (print) and ISBN 978-1-61499-852-5 (online). http://ebooks.iospress.nl/volumearticle/48765 and http://urn.fi/URN:NBN:fi:aalto-201812106386.

41.

Finnish Institute for Health and Welfare. (2020). Web pages offering guidelines about coronavirus (COVID-19) disease. When to seek admission for care and guidance concerning the symptoms and care. Published in Finnish. Terveyden ja hyvinvoinnin laitos (THL). https://thl.fi/fi/web/infektiotaudit-ja-rokotukset/taudit-ja-torjunta/taudit-ja-taudinaiheuttajat-a-o/koronavirus-covid-19/koronavirustauti-milloin-on-hakeuduttava-hoitoon; https://thl.fi/fi/web/infektiotaudit-ja-rokotukset/ajankohtaista/ajankohtaista-koronaviruksesta-covid-19/oireet-ja-hoito-koronavirus/koronaviruksen-hoito-ja-ohjeet-sairastuneelle;https://thl.fi/fi/web/infektiotaudit-ja-rokotukset/ajankohtaista/ajankohtaista-koronaviruksesta-covid-19/oireet-ja-hoito-koronavirus.

42.

World Health Organization. Coronavirus. A web page offering guidelines about coronavirus (COVID-19) disease. World Health Organization (WHO). 2020https://www.who.int/health-topics/coronavirus.

43.

Boateng GO, Neilands TB, Frongillo EA, Melgar-Quiñonez HR, Young SL. Best practices for developing and validating scales for health, social, and behavioral research: a primer. Frontiers in Public Health. 2018;6:149. https://doi.org/10.3389/fpubh.2018.00149.CrossRefPubMedPubMedCentral

44.

Kulikowski K. Measurement of work engagement with single-item measure. Polish Psychological Bulletin. 2018;49(4):406–15. DOI:https://doi.org/10.24425/119509.CrossRef

45.

Siegel J, van Dolen W. Child helplines: exploring determinants and boundary conditions of volunteer encounter satisfaction. J Serv Mark. 2020;34(5):589–600. https://doi.org/10.1108/JSM-05-2019-0200.CrossRef

46.

Tolvanen E, Koskela TH, Kosunen E. Comparison of the Patient Enablement Instrument (PEI) with two single-item measures among Finnish health care centre patients. BMC Health Serv Res. 2019;19(1):376. https://doi.org/10.1186/s12913-019-4182-2.CrossRefPubMedPubMedCentral

47.

Sullivan M, Bishop S, Pivik J. The pain catastrophizing scale: development and validation. Psychological Assessment. 1995;7(4):524–53.CrossRef

48.

Ganis G, Thompson WL, Kosslyn SM. Brain areas underlying visual mental imagery and visual perception: an fMRI study. Cogn Brain Res. 2004;20(2):226–41. DOI:https://doi.org/10.1016/j.cogbrainres.2004.02.012.CrossRef

49.

McNorgan C. (2012). A meta-analytic review of multisensory imagery identifies the neural correlates of modality-specific and modality-general imagery. Frontiers in Human Neuroscience, 6, article 285. DOI:https://doi.org/10.3389/fnhum.2012.00285.

50.

Pearson D, Deeprose C, Wallace-Hadrill S, Heyes SB, Holmes E. Assessing mental imagery in clinical psychology: a review of imagery measures and a guiding framework. Clin Psychol Rev. 2013;33(1):1–23. https://doi.org/10.1016/j.cpr.2012.09.001.CrossRefPubMedPubMedCentral

51.

Cui X, Jeter CB, Yang D, Montague PR, Eagleman DM. Vividness of mental imagery: individual variability can be measured objectively. Vision Res. 2007;47(4):474–8. DOI:https://doi.org/10.1016/j.visres.2006.11.013.CrossRefPubMedPubMedCentral

52.

Herholz SC, Halpern AR, Zatorre RJ. Neuronal correlates of perception, imagery, and memory for familiar tunes. J Cogn Neurosci. 2012;24(6):1382–97. DOI:https://doi.org/10.1162/jocn_a_00216.CrossRefPubMed

53.

Belardinelli M, Palmiero M, Sestieri C, Nardo D, Di Matteo R, Londei A, D’Ausilio A, Ferretti A, Del Gratta C, Romani G. An fMRI investigation on image generation in different sensory modalities: The influence of vividness. Acta Physiol (Oxf). 2009;132:190–200. DOI:https://doi.org/10.1016/j.actpsy.2009.06.009.CrossRef

54.

Carroll JS. The effect of imagining an event on expectations for the event: an interpretation in terms of the availability heuristic. J Exp Soc Psychol. 1978;14(1):88–96.CrossRef

55.

Sherman SJ, Cialdini RB, Schwartzman DF, Reynolds KD. Imagining can heighten or lower the perceived likelihood of contracting a disease: the mediating effect of ease of imagery. Pers Soc Psychol Bull. 1985;11(1):118–27.CrossRef

56.

Lombrozo T. Simplicity and probability in causal explanation. Cogn Psychol. 2007;55:232–57.CrossRef

Title: Detecting the patient’s need for help with machine learning based on expressions
Author: Lauri Lahti
Publication date: 01-12-2022
Publisher: BioMed Central
Keywords: Coronavirus
Care
Published in: BMC Medical Research Methodology / Issue 1/2022
Electronic ISSN: 1471-2288
DOI: https://doi.org/10.1186/s12874-021-01502-8

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Detecting the patient’s need for help with machine learning based on expressions

Abstract

Background

Methods

Results

Conclusions

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2022

Assessing transferability in systematic reviews of health economic evaluations – a review of methodological guidance

K-means for shared frailty models

Adopting AMSTAR 2 critical appraisal tool for systematic reviews: speed of the tool uptake and barriers for its adoption

Harmonized procedure coding system for surgical procedures and analysis of surgical site infections (SSI) of five European countries

Going virtual during the COVID-19 pandemic: adaptation of a mixed-methods dietary behavior study within a community-based participatory research study of African-American adults at risk for cardiovascular disease

Conclusiveness, linguistic characteristics and readability of Cochrane plain language summaries of intervention reviews: a cross-sectional study