Top

BMC Psychiatry

Published in:

Open Access 01-12-2021 | Bulimia | Research article

Measuring diagnostic heterogeneity using text-mining of the lived experiences of patients

Authors: Chandril Chandan Ghosh, Duncan McVicar, Gavin Davidson, Ciaran Shannon

Published in: BMC Psychiatry | Issue 1/2021

Abstract

Background

The diagnostic system is fundamental to any health discipline, including mental health, as it defines mental illness and helps inform possible treatment and prognosis. Thus, the procedure to estimate the reliability of such a system is of utmost importance. The current ways of measuring the reliability of the diagnostic system have limitations. In this study, we propose an alternative approach for verifying and measuring the reliability of the existing system.

Methods

We perform Jaccard’s similarity index analysis between first person accounts of patients with the same disorder (in this case Major Depressive Disorder) and between those who received a diagnosis of a different disorder (in this case Bulimia Nervosa) to demonstrate that narratives, when suitably processed, are a rich source of data for this purpose. We then analyse 228 narratives of lived experiences from patients with mental disorders, using Python code script, to demonstrate that patients with the same diagnosis have very different illness experiences.

Results

The results demonstrate that narratives are a statistically viable data resource which can distinguish between patients who receive different diagnostic labels. However, the similarity coefficients between 99.98% of narrative pairs, including for those with similar diagnoses, are low (< 0.3), indicating diagnostic Heterogeneity.

Conclusions

The current study proposes an alternative approach to measuring diagnostic Heterogeneity of the categorical taxonomic systems (e.g. the Diagnostic and Statistical Manual, DSM). In doing so, we demonstrate the high Heterogeneity and limited reliability of the existing system using patients’ written narratives of their illness experiences as the only data source. Potential applications of these outputs are discussed in the context of healthcare management and mental health research.

One example of within-disorder heterogeneity is the almost 24,000 possible symptom combinations for panic disorder in DSM-5 [20]. Likewise, the DSM diagnosis of major depression is made when a patient has any 5 out of 9 symptoms. Thus, a patient who presents with psychomotor retardation, hypersomnia and gaining weight is scored the same as a patient who is agitated, sleeping poorly and is losing weight [27].

An example of co-morbidity across diagnostic groups is that a presentation of 270 million combinations of symptoms would meet the criteria for both Post-Traumatic Stress Disorder and Major Depressive Disorder [48]. A response to the apparent lack of homogeneity in DSM is an acceptance that an individual patient may simultaneously develop more than one “co-morbid” disorder [21]. However, the extent of co-occurring diagnoses found raises concern about “the validity of the diagnostic categories themselves – do these disorders constitute distinct clinical entities?” ([29] ,p.380).

Allsopp K, Read J, Corcoran R, Kinderman P. Heterogeneity in psychiatric diagnostic classification. Psychiatry Res. 2019;279:15–22.PubMedCrossRef

American Psychiatric Association. Diagnostic and statistical manual of mental disorders. Washington, DC: Author; 1952.

American Psychiatric Association (APA). Diagnostic and Statistical Manual of Mental Disorders. 5th ed. Washington, DC: American Psychiatric Association; 2013.CrossRef

Anderson C. Presenting and evaluating qualitative research. Am J Pharm Educ. 2010;74(8):141.PubMedPubMedCentralCrossRef

Andrews G, Slade T, Issakidis C. Deconstructing current comorbidity: data from the Australian National Survey of Mental Health and Well-Being. Br J Psychiatry. 2002;181(4):306–14.PubMedCrossRef

Bethlehem J. Selection bias in web surveys. Int Stat Rev. 2010;78(2):161–88.CrossRef

Bijl R, Ravelli A, van Zessen G. Prevalence of psychiatric disorder in the general population: results of the Netherlands Mental Health Survey and Incidence Study (NEMESIS). Soc Psychiatry Psychiatr Epidemiol. 1998;33(12):587–95.PubMedCrossRef

Blashfield RK. The classification of psychopathology: Neo-Kraepelinian and quantitative approaches. New York: Plenum; 1984.CrossRef

Blumenthal-Barby J, Krieger H. Cognitive biases and heuristics in medical decision making. Med Decis Mak. 2014;35(4):539–57.CrossRef

10.

Borsboom D, Cramer A. Network analysis: an integrative approach to the structure of psychopathology. Annu Rev Clin Psychol. 2013;9(1):91–121.PubMedCrossRef

11.

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.CrossRef

12.

Brown T, Campbell L, Lehman C, Grisham J, Mancill R. Current and lifetime comorbidity of the DSM-IV anxiety and mood disorders in a large clinical sample. J Abnorm Psychol. 2001;110(4):585–99.PubMedCrossRef

13.

Bulik C, Sullivan P, Fear J, Joyce P. Eating disorders and antecedent anxiety disorders: a controlled study. Acta Psychiatr Scand. 1997;96(2):101–7.PubMedCrossRef

14.

Byers A, Vittinghoff E, Lui L, Hoang T, Blazer D, Covinsky K, et al. Twenty-year depressive trajectories among older women. Arch Gen Psychiatry. 2012;69(10):1073.PubMedPubMedCentralCrossRef

15.

Cheng P, Ramos R, Bitsch J, Jonas S, Ix T, See P, Wehrle K. Psychologist in a pocket: lexicon development and content validation of a mobile-based app for depression screening. JMIR Mhealth Uhealth. 2016;4(3):e88 https://doi.org/10.2196/mhealth.5284.PubMedPubMedCentralCrossRef

16.

Detmer D, Fryback D, Gassner K. Heuristics and biases in medical decision-making. Acad Med. 1978;53(8):682–3.CrossRef

17.

Fried E. Problematic assumptions have slowed down depression research: why symptoms, not syndromes are the way forward. Front Psychol. 2015;6. https://doi.org/10.3389/fpsyg.2015.00309.

18.

Fried E. The 52 symptoms of major depression: lack of content overlap among seven common depression scales. J Affect Disord. 2017;208:191–7.PubMedCrossRef

19.

Fried E, Nesse R. Depression sum-scores don’t add up: why analyzing specific depression symptoms is essential. BMC Med. 2015;13(1). https://doi.org/10.1186/s12916-015-0325-4.

20.

Galatzer-Levy I, Bryant R. 636,120 ways to have posttraumatic stress disorder. Perspect Psychol Sci. 2013;8(6):651–62. https://doi.org/10.1177/1745691613504115.CrossRefPubMed

21.

Goldberg D. The heterogeneity of “major depression”. World Psychiatry. 2011;10(3):226–8 6.

22.

Grant B, Stinson F, Dawson D, Chou S, Dufour M, Compton W, et al. Prevalence and co-occurrence of substance use disorders and IndependentMood and anxiety disorders. Arch Gen Psychiatry. 2004;61(8):807.PubMedCrossRef

23.

Hussain A, Oestreicher J. Clinical decision-making: heuristics and cognitive biases for the ophthalmologist. Surv Ophthalmol. 2018;63(1):119–24.PubMedCrossRef

24.

Kessler R, Chiu W, Demler O, Walters E. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62(6):617.PubMedPubMedCentralCrossRef

25.

Kotov R, Krueger R, Watson D, Achenbach T, Althoff R, Bagby R, et al. The hierarchical taxonomy of psychopathology (HiTOP): a dimensional alternative to traditional nosologies. J Abnorm Psychol. 2017;126(4):454–77.PubMedCrossRef

26.

Lilienfeld S, Treadway M. Clashing diagnostic approaches: DSM-ICD versus RDoC. Annu Rev Clin Psychol. 2016;12(1):435–63 https://doi.org/10.1146/annurev-clinpsy-021815-093122.PubMedPubMedCentralCrossRef

27.

Lux V, Kendler K. Deconstructing major depression: a validation study of the DSM-IV symptomatic criteria. Psychol Med. 2010;40(10):1679–90.PubMedPubMedCentralCrossRef

28.

Mehra B. Bias in qualitative research: voices from an online classroom. Qual Rep. 2002;7(1):1–19 Retrieved from http://nsuworks.nova.edu/tqr/vol7/iss1/2.

29.

Mineka S, Watson D, Clark L. Comorbidity of anxiety and unipolar mood disorders. Annu Rev Psychol. 1998;49(1):377–412. https://doi.org/10.1146/annurev.psych.49.1.377.CrossRefPubMed

30.

Mohammad S, Turney P. Crowdsourcing a word-emotion association lexicon. Comput Intell. 2012;29(3):436–65 https://doi.org/10.1111/j.1467-8640.2012.00460.x.CrossRef

31.

Nathan PE. DSM-IV: Empirical, accessible, not yet ideal. J Clin Psychol. 1994;50:103–10.PubMedCrossRef

32.

Nguyen T, Phung D, Dao B, Venkatesh S, Berk M. Affective and content analysis of online depression communities. IEEE Trans Affect Comput. 2014;5(3):217–26 https://doi.org/10.1109/taffc.2014.2315623.CrossRef

33.

Olbert C, Gala G, Tupler L. Quantifying heterogeneity attributable to polythetic diagnostic criteria: theoretical framework and empirical application. J Abnorm Psychol. 2014;123(2):452–62. https://doi.org/10.1037/a0036068.CrossRefPubMed

34.

Ormel J, Raven D, van Oort F, Hartman C, Reijneveld S, Veenstra R, et al. Mental health in Dutch adolescents: a TRAILS report on prevalence, severity, age of onset, continuity and comorbidity of DSM disorders. Psychol Med. 2015;45(2):345–60.PubMedCrossRef

35.

Parker G. Beyond major depression. Psychol Med. 2005;35(4):467–74.PubMedCrossRef

36.

Payton F, Kvasny L. Online HIV awareness and technology affordance benefits for black female collegians — maybe not: the case of stigma. J Am Med Inform Assoc. 2016;23(6):1121–6 https://doi.org/10.1093/jamia/ocw017.PubMedCrossRef

37.

Payton F, Yarger L, Pinter A. Text mining mental health reports for issues impacting today’s college students: qualitative study. JMIR Mental Health. 2018;5(4):e10032 https://doi.org/10.2196/10032.PubMedPubMedCentralCrossRef

38.

Plana-Ripoll O, Pedersen C, Holtz Y, Benros M, Dalsgaard S, de Jonge P, et al. Exploring comorbidity within mental disorders among a Danish National Population. JAMA Psychiatry. 2019;76(3):259.PubMedPubMedCentralCrossRef

39.

Regier D, Narrow W, Clarke D, Kraemer H, Kuramoto S, Kuhl E, et al. DSM-5 field trials in the United States and Canada, part II: test-retest reliability of selected categorical diagnoses. Am J Psychiatr. 2013;170(1):59–70.PubMedCrossRef

40.

Simchon A, Gilead M. A psychologically informed approach to CLPsych shared task 2018. In: Proceedings of the fifth workshop on computational linguistics and clinical psychology: from keyboard to clinic; 2018. https://doi.org/10.18653/v1/w18-0612.

41.

Tausczik Y, Pennebaker J. The psychological meaning of words: LIWC and computerized text analysis methods. J Lang Soc Psychol. 2009;29(1):24–54 https://doi.org/10.1177/0261927x09351676.CrossRef

42.

Teesson M, Slade T, Mills K. Comorbidity in Australia: findings of the 2007 National Survey of mental health and wellbeing. Aust N Z J Psychiatry. 2009;43(7):606–14.PubMedCrossRef

43.

Timimi S. No more psychiatric labels: why formal psychiatric diagnostic systems should be abolished. Int J Clin Health Psychol. 2014;14(3):208–15.CrossRef

44.

Wardenaar K, de Jonge P. Diagnostic heterogeneity in psychiatry: towards an empirical solution. BMC Med. 2013;11(1):1–3.CrossRef

45.

Widiger T, Clark L. Toward DSM-V and the classification of psychopathology. Psychol Bull. 2000;126(6):946–63.PubMedCrossRef

46.

Widiger T, Samuel D. Diagnostic categories or dimensions? A question for the diagnostic and statistical manual of mental disorders--fifth edition. J Abnorm Psychol. 2005;114(4):494–504.PubMedCrossRef

47.

World Health Organization (WHO). The ICD-10 classification of mental and behavioral disorders: Clinical descriptions and diagnostic guidelines. Geneva: Author; 1992.

48.

Young G, Lareau C, Pierre B. One quintillion ways to have PTSD comorbidity: recommendations for the disordered DSM-5. Psychol Inj Law. 2014;7(1):61–74. https://doi.org/10.1007/s12207-014-9186-y.CrossRef

Title: Measuring diagnostic heterogeneity using text-mining of the lived experiences of patients
Authors: Chandril Chandan Ghosh
Duncan McVicar
Gavin Davidson
Ciaran Shannon
Publication date: 01-12-2021
Publisher: BioMed Central
Keyword: Bulimia
Published in: BMC Psychiatry / Issue 1/2021
Electronic ISSN: 1471-244X
DOI: https://doi.org/10.1186/s12888-021-03044-1

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Measuring diagnostic heterogeneity using text-mining of the lived experiences of patients

Abstract

Background

Methods

Results

Conclusions

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2021

Effect of medication therapy combined with transcranial direct current stimulation on depression and response inhibition of patients with bipolar disorder type I: a clinical trial

Sex differences between serum total bilirubin levels and cognition in patients with schizophrenia

Benign ethnic neutropenia: an analysis of prevalence, timing and identification accuracy in two large inner-city NHS hospitals

Counselling and psychotherapy service use in Chinese sexual minority populations: a nationwide survey

Understanding the psycho-social context for a new early intervention for resistance to change that aims to strike a beneficial balance between structure and flexibility

Quality of diabetes care in patients with schizophrenia: a case-control study in Qatar