Top

Journal of Public Health

Published in:

01-02-2019 | Original Article

Comparing single-level and multilevel regression analysis for risk adjustment of treatment outcomes in common mental health disorders

Authors: Lisanne Warmerdam, Edwin de Beurs, Marko Barendregt, Jos Twisk

Published in: Journal of Public Health | Issue 1/2019

Abstract

Aim

The aim of this paper is to compare single-level and multilevel regression analysis to obtain risk-adjusted outcomes from mental health care providers.

Subject and methods

The study population consisted of adult patients receiving treatment for common mental health disorders. The outcome was self-reported symptom level at post-test. Risk adjustment models were developed using single- and multilevel regression analysis. In the multilevel approach, a random intercept for each provider was included. The intraclass correlation coefficient was used to estimate the proportion of variability in treatment outcome between providers. Spearman correlation coefficient of ranks was used to compare results between the two approaches.

Results

The effects of most casemix variables on outcomes were similar for the two models. The ranking of providers in both methods was also quite similar (ρ = .99). The multilevel model estimated that 5.4% of total variability in adjusted post-test scores was explained by the provider factor.

Conclusions

The findings of risk adjustment of mental health outcomes are quite robust for the use of single-level or multilevel regression analysis in the current study. However, given the small but significant amount of variation in outcomes that is attributable to providers, the multilevel approach is recommended for dealing with outcomes when patients are clustered within providers.

American Psychiatric Association (1994) Diagnostic and statistical manual of mental disorders, 4th edition, (DSM-IV). American Psychiatric Association, Washington DC

Arling G, Lewis T, Kane RL, Mueller C, Flood S (2007) Improving quality assessment through multilevel modeling: the case of nursing home compare. Health Serv Res 43:1177–1199. https://doi.org/10.1111/j.1475-6773.2006.00647.x CrossRef

Arrindell WA, Ettema JHM (2003) SCL-90: Herziene handleiding bij een multidimensionale psychopathologie indicator. Swets & Zeitlinger, Lisse

Barkham M, Gilbert N, Connell J, Marshall C, Twigg E (2005) Suitability and utility of the CORE-OM and CORE-A for assessing severity of presenting problems in psychological therapy services based in primary and secondary care settings. Br J Psychiatry 186:239–246. https://doi.org/10.1192/bjp.186.3.239 CrossRefPubMed

Burgess JF, Christiansen CL, Michalak SE, Morris CN (2000) Medical profiling: improving standards and risk adjustment using hierarchical models. J Health Econ 19:291–309. https://doi.org/10.1016/S0167-6296(99)00034-X CrossRefPubMed

de Beurs E (2008) Brief symptom inventory handleiding. PITS B.V, Leiden

de Beurs E (2010) De genormaliseerde T-score, Een “euro” voor testuitslagen. MGv 65:685–696

de Jong K, Nugter MA, Polak MG, Wagenborg JEA, Spinhoven P, Heiser WA (2007) The outcome questionnaire (OQ-45) in a Dutch population: a cross-cultural validation. Clin Psychol Psychother 14:288–301. https://doi.org/10.1002/cpp.529 CrossRef

Derogatis LR (1975a) The symptom checklist-90-R. Clinical Psychometric Research, Inc, Baltimore

Derogatis LR (1975b) The brief symptom inventory. Clinical Psychometric Research, Inc, Baltimore

Goldstein H, Healy MJR (1995) The graphical presentation of a collection of means. J R Stat Soc Ser A Stat Soc 158:175–177. https://doi.org/10.2307/2983411 CrossRef

Goldstein H, Spiegelhalter DJ (1996) League tables and their limitations: statistical issues in comparisons of institutional performance. J R Stat Soc Ser A Stat Soc 159:385–443. https://doi.org/10.2307/2983325 CrossRef

Hall RC (1995) Global assessment of functioning: a modified scale. Psychosomatics 36:267–275. https://doi.org/10.1016/S0033-3182(95)71666-8 CrossRefPubMed

Hermann RC, Rollins CK, Chan JA (2007) Risk-adjusting outcomes of mental health and substance-related care: a review of the literature. Harv Rev Psychiatry 15:52–69. https://doi.org/10.1080/10673220701307596 CrossRefPubMed

Iezzoni LE (2013) Risk adjustment for measuring healthcare outcomes, 4rd edn. Health Administration, Chicago

Israels A (2013) Methods of standardisation. Statistics Netherlands: The Hague/Heerlen. https://www.cbs.nl/nr/rdonlyres/0579a307-01ef-43ca-8e29-c45b46334903/0/2013x3702.pdf

Lange A, Appelo M (2007) Korte Klachtenlijst, handleiding. Bohn Stafleu van Loghum, Houten

Lambert MJ, Gregersen AT, Burlingame GM (2004) The outcome questionnaire 45. In: Maruish M (ed) The use of psychological testing for treatment planning and outcomes assessment: instruments for adults. Lawrence Erlbaum Associates, Mahwah, pp 191–234

Lovaglio PG (2012) Benchmarking strategies for measuring the quality of healthcare: problems and prospects. Sci World J 2012:1–13. https://doi.org/10.1100/2012/606154 CrossRef

Lovibond SH, Lovibond PF (1995) Manual for the depression anxiety stress scales Second edition. Psychology Foundation, Sydney

Moran V, Jacobs R (2015) Comparing the performance of English mental health providers in achieving patient outcomes. Soc Sci Med 140:127–135. https://doi.org/10.1016/j.socscimed.2015.07.009 CrossRefPubMed

Nuttall D, Parkin D, Devlin N (2013) Inter-provider comparison of patient-reported outcomes: developing an adjustment to account for differences in patient case mix. Health Econ 24:41–54. https://doi.org/10.1002/hec.2999 CrossRefPubMed

Rosen AK, Chatterjee S, Glickman ME, Spiro A, Seal P, Eisen SV (2009) Improving risk adjustment of self-reported mental health outcomes. J Behav Health Ser Res 37:292–306. https://doi.org/10.1007/s11414-009-9196-9 CrossRef

Snijders TAB, Bosker RJ (1999) Multilevel analysis: an introduction to basic and advanced multilevel modeling. Sage Publishers, London

Warmerdam L, Barendregt M, de Beurs E (2017) Risk adjustment of self-reported clinical outcomes in Dutch mental health care. J Public Health 25:311–319. https://doi.org/10.1007/s10389-017-0785-4 CrossRef

Title: Comparing single-level and multilevel regression analysis for risk adjustment of treatment outcomes in common mental health disorders
Authors: Lisanne Warmerdam
Edwin de Beurs
Marko Barendregt
Jos Twisk
Publication date: 01-02-2019
Publisher: Springer Berlin Heidelberg
Published in: Journal of Public Health / Issue 1/2019
Print ISSN: 2198-1833
Electronic ISSN: 1613-2238
DOI: https://doi.org/10.1007/s10389-018-0921-9

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Comparing single-level and multilevel regression analysis for risk adjustment of treatment outcomes in common mental health disorders

Abstract

Aim

Subject and methods

Results

Conclusions

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Aim

Subject and methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2019

Implementing a regression discontinuity design to explore the heterogeneous effects of obesity on labour income: the case of Mexico

The role of individual characteristics and municipalities in social inequalities in perceived health (Italy, 2010–2012): a multilevel study

Correction to: Female sexual dysfunction in Egyptian women with anxiety: prevalence and patterns

Tracking the efficacy of the test and treat model of HIV prevention in India using National Family Health Surveys (2005–16)

Lifestyle habits and well-being among primary health physicians in western Saudi Arabia

Correction to: Impact of educational intervention on osteoporosis knowledge among university students in Beni-Suef, Egypt