Top

BMC Medical Research Methodology

Published in:

Open Access 01-12-2015 | Research article

Validity of single item responses to short message service texts to monitor depression: an mHealth sub-study of the UK ACUDep trial

Authors: Ada Keding, Jan R. Böhnke, Tim J. Croudace, Stewart J. Richmond, Hugh MacPherson

Published in: BMC Medical Research Methodology | Issue 1/2015

Abstract

Background

An increasing number of research designs are using text messaging (SMS) as a means of self-reported symptom and outcome monitoring in a variety of long-term health conditions, including severity ratings of depressed mood. The validity of such a single item SMS score to measure latent depression is not currently known and is vital if SMS data are to inform clinical evaluation in the future.

Methods

A sub-set of depressed participants in the UK ACUDep trial submitted a single SMS text score (R-SMS-DS) between 1 and 9 on how depressed they felt around the same time as completing the PHQ-9 depression questionnaire on paper at 3 months follow-up of the trial. Exploratory categorical data factor analysis (EFA) was used to ascertain the alignment of R-SMS-DS scores with the factor structure of the PHQ-9. Any response bias with regard to age or gender was assessed by differential item functioning (DIF) analysis.

Results

Depression scores based on the PHQ-9 and R-SMS-DS at 3 months were available for 337 participants (74 % female; mean age: 42 years, SD = 11.1), 213 of which completed the two outcomes within 6 days of each other. R-SMS-DS scores aligned with the underlying latent depression of the PHQ-9 (factor loading of 0.656) and in particular its affective rather than somatic dimension. The R-SMS-DS score was most strongly correlated with depressed mood (r = 0.607), feeling bad about oneself (r = 0.588) and anhedonia (r = 0.573). R-SMS-DS responses were invariant with respect to gender (p = 0.302). However, there was some evidence for age related response bias (p = 0.031), with older participants being more likely to endorse lower R-SMS-DS scores than younger ones.

Conclusions

The R-SMS-DS used in the ACUDep trial was found to be a valid measure of latent affective depression with no gender related response bias. This text message item may therefore represent a useful assessment and monitoring tool meriting evaluation in further research. For future study designs we recommend the collection of outcome data by new health technologies in combination with gold standard instruments to ensure concurrent validity.

Murray CJ, Vos T, Lozano R, Naghavi M, Flaxman AD, Michaud C, et al. Disability-adjusted life years (DALYs) for 291 diseases and injuries in 21 regions, 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012;380(9859):2197–223. doi:10.1016/s0140-6736(12)61689-4.PubMedCrossRef

Whiteford HA, Degenhardt L, Rehm J, Baxter AJ, Ferrari AJ, Erskine HE, et al. Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. Lancet. 2013;382(9904):1575–86. doi:10.1016/s0140-6736(13)61611-6.PubMedCrossRef

Price M, Yuen EK, Goetter EM, Herbert JD, Forman EM, Acierno R, et al. mHealth: A Mechanism to Deliver More Accessible, More Effective Mental Health Care. Clin Psychol Psychother. 2014;21(5):427–36. doi:10.1002/cpp.1855.PubMedCrossRef

Patrick K, Griswold WG, Raab F, Intille SS. Health and the mobile phone. Am J Prev Med. 2008;35(2):177–81. doi:10.1016/j.amepre.2008.05.001.PubMedPubMedCentralCrossRef

Benhamou PY, Melki V, Boizel R, Perreal F, Quesada JL, Bessieres-Lacombe S, et al. One-year efficacy and safety of Web-based follow-up using cellular phone in type 1 diabetic patients under insulin pump therapy: the PumpNet study. Diabetes Metabol. 2007;33(3):220–6. doi:10.1016/j.diabet.2007.01.002.CrossRef

Kim SI, Kim HS. Effectiveness of mobile and internet intervention in patients with obese type 2 diabetes. Int J Med Informat. 2008;77(6):399–404. doi:10.1016/j.ijmedinf.2007.07.006.CrossRef

Newton KH, Wiltshire EJ, Elley CR. Pedometers and text messaging to increase physical activity: randomized controlled trial of adolescents with type 1 diabetes. Diabetes Care. 2009;32(5):813–5. doi:10.2337/dc08-1974.PubMedPubMedCentralCrossRef

Anhoj J, Moldrup C. Feasibility of collecting diary data from asthma patients through mobile phones and SMS (short message service): response rate analysis and focus group evaluation from a pilot study. J Med Internet Res. 2004;6(4), e42. doi:10.2196/jmir.6.4.e42.PubMedPubMedCentralCrossRef

Ryan D, Cobern W, Wheeler J, Price D, Tarassenko L. Mobile phone technology in the management of asthma. J Telemed Telecare. 2005;11 Suppl 1:43–6. doi:10.1258/1357633054461714.PubMedCrossRef

10.

Ostojic V, Cvoriscec B, Ostojic SB, Reznikoff D, Stipic-Markovic A, Tudjman Z. Improving asthma control through telemedicine: a study of short-message service. Telemed J e Health. 2005;11(1):28–35. doi:10.1089/tmj.2005.11.28.PubMedCrossRef

11.

Johansen B, Wedderkopp N. Comparison between data obtained through real-time data capture by SMS and a retrospective telephone interview. Chiropract Osteopathy. 2010;18:10. doi:10.1186/1746-1340-18-10.CrossRef

12.

Kent P, Kongsted A. Identifying clinical course patterns in SMS data using cluster analysis. Chiropract Manual Ther. 2012;20(1):20. doi:10.1186/2045-709x-20-20.CrossRef

13.

Macedo LG, Maher CG, Latimer J, McAuley JH. Feasibility of using short message service to collect pain outcomes in a low back pain clinical trial. Spine. 2012;37(13):1151–5. doi:10.1097/BRS.0b013e3182422df0.PubMedCrossRef

14.

Brabyn S, Adamson J, MacPherson H, Tilbrook H, Torgerson DJ. Short message service text messaging was feasible as a tool for data collection in a trial of treatment for irritable bowel syndrome. J Clin Epidemiol. 2014;67(9):993–1000. doi:10.1016/j.jclinepi.2014.05.004.PubMedCrossRef

15.

Haapala I, Barengo NC, Biggs S, Surakka L, Manninen P. Weight loss by mobile phone: a 1-year effectiveness study. Public Health Nutr. 2009;12(12):2382–91. doi:10.1017/s1368980009005230.PubMedCrossRef

16.

Patrick K, Raab F, Adams MA, Dillon L, Zabinski M, Rock CL, et al. A text message-based intervention for weight loss: randomized controlled trial. J Med Internet Res. 2009;11(1), e1. doi:10.2196/jmir.1100.PubMedPubMedCentralCrossRef

17.

Rodgers A, Corbett T, Bramley D, Riddell T, Wills M, Lin RB, et al. Do u smoke after txt? Results of a randomised trial of smoking cessation using mobile phone text messaging. Tobac Contr. 2005;14(4):255–61. doi:10.1136/tc.2005.011577.CrossRef

18.

Whitford HM, Donnan PT, Symon AG, Kellett G, Monteith-Hodge E, Rauchhaus P, et al. Evaluating the reliability, validity, acceptability, and practicality of SMS text messaging as a tool to collect research data: results from the Feeding Your Baby project. J Am Med Informat Assoc. 2012;19(5):744–9. doi:10.1136/amiajnl-2011-000785.CrossRef

19.

Christie A, Dagfinrud H, Dale O, Schulz T, Hagen KB. Collection of patient-reported outcomes;--text messages on mobile phones provide valid scores and high response rates. BMC Med Res Meth. 2014;14:52. doi:10.1186/1471-2288-14-52.CrossRef

20.

Moore PJ, Little MA, McSharry PE, Geddes JR, Goodwin GM. Forecasting depression in bipolar disorder. IEEE Trans Biomed Eng. 2012;59(10):2801–7. doi:10.1109/tbme.2012.2210715.PubMedCrossRef

21.

Ainsworth J, Palmier-Claus JE, Machin M, Barrowclough C, Dunn G, Rogers A, et al. A comparison of two delivery modalities of a mobile phone-based assessment for serious mental illness: native smartphone application vs text-messaging only implementations. J Med Internet Res. 2013;15(4), e60. doi:10.2196/jmir.2328.PubMedPubMedCentralCrossRef

22.

Granholm E, Ben-Zeev D, Link PC, Bradshaw KR, Holden JL. Mobile Assessment and Treatment for Schizophrenia (MATS): a pilot trial of an interactive text-messaging intervention for medication adherence, socialization, and auditory hallucinations. Schizophr Bull. 2012;38(3):414–25. doi:10.1093/schbul/sbr155.PubMedCrossRef

23.

Spaniel F, Vohlidka P, Kozeny J, Novak T, Hrdlicka J, Motlova L, et al. The Information Technology Aided Relapse Prevention Programme in Schizophrenia: an extension of a mirror-design follow-up. Int J Clin Pract. 2008;62(12):1943–6. doi:10.1111/j.1742-1241.2008.01903.x.PubMedPubMedCentralCrossRef

24.

Dunstan DA, Tooth SM. Using technology to improve patient assessment and outcome evaluation. Rural Rem Health. 2012;12:2048.

25.

Aguilera A, Munoz RF. Text messaging as an adjunct to CBT in low-income populations: a usability and feasibility pilot study. Prof Psychol Res Pract. 2011;42(6):472–8. doi:10.1037/a0025499.CrossRef

26.

Aguilera A, Berridge C. Qualitative feedback from a text messaging intervention for depression: benefits, drawbacks, and cultural differences. JMIR MHealth UHealth. 2014;2(4), e46. doi:10.2196/mhealth.3660.PubMedPubMedCentralCrossRef

27.

Aguilera A, Schueller SM, Leykin Y. Daily mood ratings via text message as a proxy for clinic based depression assessment. J Affect Disord. 2015;175:471–4. doi:10.1016/j.jad.2015.01.033.PubMedPubMedCentralCrossRef

28.

Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16(9):606–13.PubMedPubMedCentralCrossRef

29.

MacPherson H, Richmond S, Bland M, Brealey S, Gabe R, Hopton A, et al. Acupuncture and counselling for depression in primary care: a randomised controlled trial. PLoS Med. 2013;10(9):e1001518. doi:10.1371/journal.pmed.1001518.PubMedPubMedCentralCrossRef

30.

Richmond SJ, Keding A, Hover M, Gabe R, Cross B, Torgerson D, et al. Feasibility, acceptability and validity of SMS text messaging for measuring change in depression during a randomised controlled trial. BMC Psychiatr. 2015;15:68. doi:10.1186/s12888-015-0456-3.CrossRef

31.

Whooley MA, Avins AL, Miranda J, Browner WS. Case-finding instruments for depression. Two questions are as good as many. J Gen Intern Med. 1997;12(7):439–45.PubMedPubMedCentralCrossRef

32.

Zimmerman M, Ruggero CJ, Chelminski I, Young D, Posternak MA, Friedman M, et al. Developing brief scales for use in clinical practice: the reliability and validity of single-item self-report measures of depression symptom severity, psychosocial impairment due to depression, and quality of life. J Clin Psychiatr. 2006;67(10):1536–41.CrossRef

33.

Bech P. Depressed mood as a core symptom of depression. Mediographia. 2008;30(1):9–11.

34.

Löwe B, Kroenke K, Grafe K. Detecting and monitoring depression with a two-item questionnaire (PHQ-2). J Psychosom Res. 2005;58(2):163–71. doi:10.1016/j.jpsychores.2004.09.006.PubMedCrossRef

35.

Böhnke JR, Lutz W. Using item and test information to optimize targeted assessments of psychological distress. Assessment. 2014;21(6):679–93. doi:10.1177/1073191114529152.PubMedCrossRef

36.

Huang FY, Chung H, Kroenke K, Delucchi KL, Spitzer RL. Using the Patient Health Questionnaire-9 to measure depression among racially and ethnically diverse primary care patients. J Gen Intern Med. 2006;21(6):547–52. doi:10.1111/j.1525-1497.2006.00409.x.PubMedPubMedCentralCrossRef

37.

Cameron IM, Crawford JR, Lawton K, Reid IC. Psychometric comparison of PHQ-9 and HADS for measuring depression severity in primary care. Br J Gen Pract. 2008;58(546):32–6. doi:10.3399/bjgp08X263794.PubMedPubMedCentralCrossRef

38.

Yu X, Tam WW, Wong PT, Lam TH, Stewart SM. The Patient Health Questionnaire-9 for measuring depressive symptoms among the general population in Hong Kong. Compr Psychiatr. 2012;53(1):95–102. doi:10.1016/j.comppsych.2010.11.002.CrossRef

39.

Böhnke JR, Lutz W, Delgadillo J. Negative affectivity as a transdiagnostic factor in patients with common mental disorders. J Affect Disord. 2014;166:270–8. doi:10.1016/j.jad.2014.05.023.PubMedCrossRef

40.

Krause JS, Reed KS, McArdle JJ. Factor structure and predictive validity of somatic and nonsomatic symptoms from the patient health questionnaire-9: a longitudinal study after spinal cord injury. Arch Phys Med Rehabil. 2010;91(8):1218–24. doi:10.1016/j.apmr.2010.04.015.PubMedCrossRef

41.

Elhai JD, Contractor AA, Tamburrino M, Fine TH, Prescott MR, Shirley E, et al. The factor structure of major depression symptoms: a test of four competing models using the Patient Health Questionnaire-9. Psychiatr Res. 2012;199(3):169–73. doi:10.1016/j.psychres.2012.05.018.CrossRef

42.

Chilcot J, Rayner L, Lee W, Price A, Goodwin L, Monroe B, et al. The factor structure of the PHQ-9 in palliative care. J Psychosom Res. 2013;75(1):60–4. doi:10.1016/j.jpsychores.2012.12.012.PubMedCrossRef

43.

Petersen JJ, Paulitsch MA, Hartig J, Mergenthal K, Gerlach FM, Gensichen J. Factor structure and measurement invariance of the Patient Health Questionnaire-9 for female and male primary care patients with major depression in Germany. J Affect Disord. 2015;170:138–42. doi:10.1016/j.jad.2014.08.053.PubMedCrossRef

44.

Piccinelli M, Wilkinson G. Gender differences in depression. Critical review. Br J Psychiatr. 2000;177:486–92.CrossRef

45.

Whiteford HA, Ferrari AJ, Degenhardt L, Feigin V, Vos T. The global burden of mental, neurological and substance use disorders: an analysis from the global burden of disease study 2010. PloS One. 2015;10(2), e0116820. doi:10.1371/journal.pone.0116820.PubMedPubMedCentralCrossRef

46.

Kuehner C. Gender differences in unipolar depression: an update of epidemiological findings and possible explanations. Acta Psychiatr Scand. 2003;108(3):163–74.PubMedCrossRef

47.

Blazer DG. Depression in late life: review and commentary. J Gerontol Biol Med Sci. 2003;58(3):249–65.CrossRef

48.

Möller-Leimkühler AM. Barriers to help-seeking by men: a review of sociocultural and clinical literature with particular reference to depression. J Affect Disord. 2002;71(1–3):1–9.PubMedCrossRef

49.

Richards D. Prevalence and clinical course of depression: a review. Clin Psychol Rev. 2011;31(7):1117–25. doi:10.1016/j.cpr.2011.07.004.PubMedCrossRef

50.

Böhnke JR, Croudace TJ. Factors of psychological distress: clinical value, measurement substance, and methodological artefacts. Soc Psychiatr Psychiatr Epidemiol. 2015. doi:10.1007/s00127-015-1022-5.

51.

Beck AT, Steer RA, Brown GK. Manual for the Beck Depression Inventory-II. San Antonio, TX: Psychological Corporation; 1996.

52.

Wirth RJ, Edwards MC. Item factor analysis: current approaches and future directions. Psychol Meth. 2007;12(1):58–79. doi:10.1037/1082-989x.12.1.58.CrossRef

53.

American Psychiatric Association. Diagnostic and statistical manual of mental disorders: DSM-IV. 4th ed. Washington, DC: American Psychiatric Association; 1994.

54.

Lorenzo-Seva U, Ferrando PJ. FACTOR: a computer program to fit the exploratory factor analysis model. Behav Res Meth. 2006;38(1):88–91.CrossRef

55.

Buja A, Eyuboglu N. Remarks on Parallel Analysis. Multivariate Behav Res. 1992;27(4):509–40. doi:10.1207/s15327906mbr2704_2.PubMedCrossRef

56.

Timmerman ME, Lorenzo-Seva U. Dimensionality assessment of ordered polytomous items with parallel analysis. Psychol Meth. 2011;16(2):209–20. doi:10.1037/a0023353.CrossRef

57.

Gaskin CJ, Happell B. On exploratory factor analysis: a review of recent evidence, an assessment of current practice, and recommendations for future use. Int J Nurs Stud. 2014;51(3):511–21. doi:10.1016/j.ijnurstu.2013.10.005.PubMedCrossRef

58.

Crane PK, Gibbons LE, Ocepek-Welikson K, Cook K, Cella D, Narasimhalu K, et al. A comparison of three sets of criteria for determining the presence of differential item functioning using ordinal logistic regression. Qual Life Res. 2007;16 Suppl 1:69–84. doi:10.1007/s11136-007-9185-5.PubMedCrossRef

59.

Scott NW, Fayers PM, Aaronson NK, Bottomley A, de Graeff A, Groenvold M, et al. Differential item functioning (DIF) analyses of health-related quality of life instruments using logistic regression. Health Qualf Life Outcome. 2010;8:81. doi:10.1186/1477-7525-8-81.

60.

StataCorp. Stata Statistical Software: Release 12. College Station, TX: StataCorp LP; 2009.

61.

Krause JS, Bombardier C, Carter RE. Assessment of depressive symptoms during inpatient rehabilitation for spinal cord injury: Is there an underlying somatic factor when using the PHQ? Rehabil Psychol. 2008;53(4):513–20. doi:10.1037/a0013354.CrossRef

62.

Richardson EJ, Richards JS. Factor structure of the PHQ-9 screen for depression across time since injury among persons with spinal cord injury. Rehabil Psychol. 2008;53(2):243–9. doi:10.1037/0090-5550.53.2.243.CrossRef

63.

Silverstein B. Gender differences in the prevalence of somatic versus pure depression: a replication. Am J Psychiatr. 2002;159(6):1051–2.PubMedCrossRef

64.

Kessler RC, Birnbaum H, Bromet E, Hwang I, Sampson N, Shahly V. Age differences in major depression: results from the National Comorbidity Survey Replication (NCS-R). Psychol Med. 2010;40(2):225–37. doi:10.1017/s0033291709990213.PubMedCrossRef

65.

Cameron IM, Crawford JR, Lawton K, Reid IC. Differential item functioning of the HADS and PHQ-9: an investigation of age, gender and educational background in a clinical UK primary care sample. J Affect Disord. 2013;147(1–3):262–8. doi:10.1016/j.jad.2012.11.015.PubMedCrossRef

66.

Chen H, Cohen P, Chen S. How big is a big odds ratio? Interpreting the magnitudes of odds ratios in epidemiological studies. Comm Stat Simulat Comput. 2010;39(4):860–4. doi:10.1080/03610911003650383.CrossRef

67.

Gelin MN, Zumbo BD. Differential item functioning results may change depending on how an item is scored: an illustration with the center for epidemiologic studies depression scale. Educ Psychol Meas. 2003;63(1):65–74. doi:10.1177/0013164402239317.CrossRef

Title: Validity of single item responses to short message service texts to monitor depression: an mHealth sub-study of the UK ACUDep trial
Authors: Ada Keding
Jan R. Böhnke
Tim J. Croudace
Stewart J. Richmond
Hugh MacPherson
Publication date: 01-12-2015
Publisher: BioMed Central
Published in: BMC Medical Research Methodology / Issue 1/2015
Electronic ISSN: 1471-2288
DOI: https://doi.org/10.1186/s12874-015-0054-6

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Validity of single item responses to short message service texts to monitor depression: an mHealth sub-study of the UK ACUDep trial

Abstract

Background

Methods

Results

Conclusions

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2015

Determine the therapeutic role of radiotherapy in administrative data: a data mining approach

Early results from a multi-component French public-private partnership initiative to improve participation in clinical research – CeNGEPS: a prospective before-after study

Sample size calculations for skewed distributions

Erratum to: “Comparison of intervention effects in split-mouth and parallel-arm randomized controlled trials: a meta-epidemiological study”

Barriers and facilitators to patient recruitment to a cluster randomized controlled trial in primary care: lessons for future trials

Comparing denominator degrees of freedom approximations for the generalized linear mixed model in analyzing binary outcome in small sample cluster-randomized trials