Skip to main content
Top
Published in: Social Psychiatry and Psychiatric Epidemiology 8/2007

01-08-2007 | ORIGINAL PAPER

Assessing the content of mental health services: a review of measures

Authors: Brynmor Lloyd-Evans, Sonia Johnson, Mike Slade

Published in: Social Psychiatry and Psychiatric Epidemiology | Issue 8/2007

Login to get access

Abstract

Background

Measurement of service content is necessary to understand what services actually provide and explain variation in service outcomes. There is no consensus about how to measure content of care in mental health services.

Method

Content of care measures for use in mental health services were identified through a search of electronic databases, hand searching of references from selected studies and consultation with experts in the field. Measures are presented in an organising methodological framework. Studies which introduced or cited the measures were read and investigations of empirical associations between content of care and outcomes were identified.

Results

Twenty five measures of content of care were identified, which used three different data collection methods and five information sources. Seven of these measures have been used to identify links between content of care and outcomes, most commonly in Assertive Community Treatment settings.

Discussion

Measures have been developed which can provide information about service content. However, there is a need for measures to demonstrate more clearly a theoretical or empirical basis, robust psychometric properties and feasibility in a range of service settings. Further comparison of the feasibility and reliability of different measurement methods is needed. Contradictory findings of associations between service content and outcomes may reflect measures’ uncertain reliability, or that crucial process variables are not being measured.

Conclusion

Measures providing a greater depth of information about the nature of interventions are needed. In the absence of a gold standard content of care measure, a multi-methods approach should be adopted.
Literature
1.
go back to reference Abramowitz S, Obten N, Cohen H (1998) Measuring HIV/AIDS Case Management. Soc Work Healthcare 27(3):1–28CrossRef Abramowitz S, Obten N, Cohen H (1998) Measuring HIV/AIDS Case Management. Soc Work Healthcare 27(3):1–28CrossRef
2.
go back to reference Beecham J, Knapp M (1992) Costing psychiatric interventions. In: Thornicroft G, Brewin CWJ (eds) Measuring mental health needs, Gaskell, pp 163–183 Beecham J, Knapp M (1992) Costing psychiatric interventions. In: Thornicroft G, Brewin CWJ (eds) Measuring mental health needs, Gaskell, pp 163–183
3.
go back to reference Bond G, Salyers M (2004) Prediction of outcome from the Dartmouth Assertive Community Treatment fidelity Scale. CNS-Spectrums 9(12):937–942PubMed Bond G, Salyers M (2004) Prediction of outcome from the Dartmouth Assertive Community Treatment fidelity Scale. CNS-Spectrums 9(12):937–942PubMed
4.
go back to reference Bowers L, Brennan G, Flood C, Lipang M, Oladapo P (2006) Preliminary outcomes of a trial to reduce conflict and containment on acute psychiatric wards: City Nurses. J Psychiatr Mental Health Nursing 13:165–172CrossRef Bowers L, Brennan G, Flood C, Lipang M, Oladapo P (2006) Preliminary outcomes of a trial to reduce conflict and containment on acute psychiatric wards: City Nurses. J Psychiatr Mental Health Nursing 13:165–172CrossRef
5.
go back to reference Bowie P, Mountain G (1993) Using direct observation to record the behaviour of long-stay patients with dementia. Int J Geriatr Psychiatry 8:857–864CrossRef Bowie P, Mountain G (1993) Using direct observation to record the behaviour of long-stay patients with dementia. Int J Geriatr Psychiatry 8:857–864CrossRef
6.
go back to reference Brekke J (1987) The model-guided method for monitoring program implementation. Eval Rev 11(3):281–299 Brekke J (1987) The model-guided method for monitoring program implementation. Eval Rev 11(3):281–299
7.
go back to reference Brekke J, Long J (1997) The impact of service characteristics on functional outcomes from Community Support Programs for persons with schizophrenia: a growth curve analysis. J Consult Clin Psychol 65(3):464–475PubMedCrossRef Brekke J, Long J (1997) The impact of service characteristics on functional outcomes from Community Support Programs for persons with schizophrenia: a growth curve analysis. J Consult Clin Psychol 65(3):464–475PubMedCrossRef
9.
go back to reference Brekke J, Ansel J, Long J, Slade E, Weinstein M (1999) Intensity and continuity of services and functional outcomes in the rehabilitation of persons with schizophrenia. Psychiatr Services 50(2):248–256 Brekke J, Ansel J, Long J, Slade E, Weinstein M (1999) Intensity and continuity of services and functional outcomes in the rehabilitation of persons with schizophrenia. Psychiatr Services 50(2):248–256
9.
go back to reference Brugha T, Glover G (1998) Process and health outcomes: need for clarity in systematic reviews of case management for severe mental disorders. Health Trends 30(3):76–79 Brugha T, Glover G (1998) Process and health outcomes: need for clarity in systematic reviews of case management for severe mental disorders. Health Trends 30(3):76–79
10.
go back to reference Brugha TS, Lindsay F (1996) Quality of mental health service care: the forgotten pathway from process to outcome. Social Psychiatry Psychiatr Epidemiol 31(2):89–98CrossRef Brugha TS, Lindsay F (1996) Quality of mental health service care: the forgotten pathway from process to outcome. Social Psychiatry Psychiatr Epidemiol 31(2):89–98CrossRef
11.
go back to reference Bryman A (2004) Social research methods, 2nd edn. Oxford University Press Bryman A (2004) Social research methods, 2nd edn. Oxford University Press
12.
go back to reference Burns T, Fiander M, Kent A, Ukoumunne OC, Byford S, Fahy T, Kumar KR (2000) Effects of case-load size on the process of care of patients with severe psychotic illness. Report from the UK700 trial. Br J Psychiatry 177:427–433PubMed Burns T, Fiander M, Kent A, Ukoumunne OC, Byford S, Fahy T, Kumar KR (2000) Effects of case-load size on the process of care of patients with severe psychotic illness. Report from the UK700 trial. Br J Psychiatry 177:427–433PubMed
13.
go back to reference Burns T, Priebe S (1996) Mental health care systems and their characteristics: A proposal. Acta Psychiatr Scand 94(6):381–385PubMed Burns T, Priebe S (1996) Mental health care systems and their characteristics: A proposal. Acta Psychiatr Scand 94(6):381–385PubMed
14.
go back to reference Dean R, Proudfoot R (1993) The Quality of Interactions Schedule (QUIS): Development, Reliability and Use in the Evaluation of Two Domus Units. Int J Geriatr Psychiatry 8:819–826CrossRef Dean R, Proudfoot R (1993) The Quality of Interactions Schedule (QUIS): Development, Reliability and Use in the Evaluation of Two Domus Units. Int J Geriatr Psychiatry 8:819–826CrossRef
15.
go back to reference De Jong A (2000) Development of the International Classification of Mental Health Care (ICMHC). Acta Psychiatr Scand 102(Suppl 405):8–13CrossRef De Jong A (2000) Development of the International Classification of Mental Health Care (ICMHC). Acta Psychiatr Scand 102(Suppl 405):8–13CrossRef
16.
go back to reference De Jong A, Giel R, Tentlom G (1991) International classification of mental health care: a tool for classifying services providing mental health care Part 1. Dept Social Psychiatry, University of Groningen and World Health organisation Regional office for Europe, Groningen De Jong A, Giel R, Tentlom G (1991) International classification of mental health care: a tool for classifying services providing mental health care Part 1. Dept Social Psychiatry, University of Groningen and World Health organisation Regional office for Europe, Groningen
17.
go back to reference Dietzen L, Bond G (1993) Relationship between case manager contact and outcome for frequently hospitalised psychiatric clients. Hospital Community Psychiatry 44(9):839–843PubMed Dietzen L, Bond G (1993) Relationship between case manager contact and outcome for frequently hospitalised psychiatric clients. Hospital Community Psychiatry 44(9):839–843PubMed
18.
go back to reference Donabedian A (1980) Explorations in Quality Assessment and Monitoring, Volume 1: the Definition of Quality and Approaches to its Assessment Health Administration Press Donabedian A (1980) Explorations in Quality Assessment and Monitoring, Volume 1: the Definition of Quality and Approaches to its Assessment Health Administration Press
19.
go back to reference Falloon I, Fadden G (1995) Integrated mental health care: a comprehensive community-based approach Cambridge University Press Falloon I, Fadden G (1995) Integrated mental health care: a comprehensive community-based approach Cambridge University Press
20.
go back to reference Fiander M, Burns T (2000) A Delphi approach to describing service models of community mental health practice. Psychiatr Serv 51(5):656–658PubMedCrossRef Fiander M, Burns T (2000) A Delphi approach to describing service models of community mental health practice. Psychiatr Serv 51(5):656–658PubMedCrossRef
21.
go back to reference Fisher G, Landis D, Clark D (1988) Case management service provision and client change. Commun Mental Health J 24(2):134–142CrossRef Fisher G, Landis D, Clark D (1988) Case management service provision and client change. Commun Mental Health J 24(2):134–142CrossRef
22.
go back to reference Glick I, Burti L, Suzuki K, Sacks M (1991) Effectiveness in psychiatric care: 1. A cross-national study of the process of treatment and outcomes of major depressive disorder. J Nerv Mental Disease 179(2):55–63CrossRef Glick I, Burti L, Suzuki K, Sacks M (1991) Effectiveness in psychiatric care: 1. A cross-national study of the process of treatment and outcomes of major depressive disorder. J Nerv Mental Disease 179(2):55–63CrossRef
23.
go back to reference Hansson KS, Allebeck P, Malm U (2001) Event recording in psychiatric care: Developing an instrument and 1-year results. J Nordic Psychiatry 55(1):25–31CrossRef Hansson KS, Allebeck P, Malm U (2001) Event recording in psychiatric care: Developing an instrument and 1-year results. J Nordic Psychiatry 55(1):25–31CrossRef
24.
go back to reference Higgins R, Hurst K, Wistow G (1999) Nursing acute psychiatric patients: a quantitative and qualitative study. J Advanced Nursing 29(1):52–63CrossRef Higgins R, Hurst K, Wistow G (1999) Nursing acute psychiatric patients: a quantitative and qualitative study. J Advanced Nursing 29(1):52–63CrossRef
25.
go back to reference Johnson S, Salvador-Carulla L (1998) Description and classification of mental health services: a European perspective. European Psychiatry 13:333–341CrossRefPubMed Johnson S, Salvador-Carulla L (1998) Description and classification of mental health services: a European perspective. European Psychiatry 13:333–341CrossRefPubMed
26.
go back to reference Johnson S, Kuhlman R (2000) The EPCAT Group, “The European Service Mapping Schedule (ESMS): development of an instrument for the description and classification of mental health services”. Acta Psychiatr Scand 102:14–23CrossRef Johnson S, Kuhlman R (2000) The EPCAT Group, “The European Service Mapping Schedule (ESMS): development of an instrument for the description and classification of mental health services”. Acta Psychiatr Scand 102:14–23CrossRef
27.
go back to reference Kitwood T (1997) Evaluating dementia care: the DCM method, 7th edn. Universwity of Bradford Kitwood T (1997) Evaluating dementia care: the DCM method, 7th edn. Universwity of Bradford
28.
go back to reference Kovess V, Lafleche M (1988) How do teams practice community psychiatry?”. Canada’s Mental Health 36(2):1–28 Kovess V, Lafleche M (1988) How do teams practice community psychiatry?”. Canada’s Mental Health 36(2):1–28
29.
go back to reference McGrew J, Bond G, Dietzen L, Salyers M (1994) Measuring the fidelity of implementation of a mental health program model. J Consult Clin Psychol 62:670–678PubMedCrossRef McGrew J, Bond G, Dietzen L, Salyers M (1994) Measuring the fidelity of implementation of a mental health program model. J Consult Clin Psychol 62:670–678PubMedCrossRef
30.
go back to reference McHugo G, Drake R, Teague G, Xie H (1999) Fidelity to assertive community treatment and client outcomes in the new hampshire dual disorders study. Psychiatr Service 50(6):818–824 McHugo G, Drake R, Teague G, Xie H (1999) Fidelity to assertive community treatment and client outcomes in the new hampshire dual disorders study. Psychiatr Service 50(6):818–824
31.
go back to reference Mechanic D (1996) Emerging issues in international mental health services research. Psychiatr Services 47(4):371–375 Mechanic D (1996) Emerging issues in international mental health services research. Psychiatr Services 47(4):371–375
32.
go back to reference Morley S, Snaith P (1992) Principles of psychological assessment. In: Freeman C, Tyrer P (eds) Research methods in psychiatry, 2nd edn. Gaskell, pp 135–152 Morley S, Snaith P (1992) Principles of psychological assessment. In: Freeman C, Tyrer P (eds) Research methods in psychiatry, 2nd edn. Gaskell, pp 135–152
33.
go back to reference Morse G, Calsyn R, Klinkenberg W, Helminiak T, Wolff N, Drake R, Yonker R, Lama G, Lemming M, McCudden S (2006) Treating homeless clients with severe mental illness and substance use disorders: costs and outcomes. Commun Mental Health J 42(4):377–404CrossRef Morse G, Calsyn R, Klinkenberg W, Helminiak T, Wolff N, Drake R, Yonker R, Lama G, Lemming M, McCudden S (2006) Treating homeless clients with severe mental illness and substance use disorders: costs and outcomes. Commun Mental Health J 42(4):377–404CrossRef
34.
go back to reference Olusina AK, Ohaeri JU, Olatawura MO (2003) The quality of interactions between staff and psychiatric inpatients in a Nigerian general hospital. Psychopathology 36(5):269–275PubMedCrossRef Olusina AK, Ohaeri JU, Olatawura MO (2003) The quality of interactions between staff and psychiatric inpatients in a Nigerian general hospital. Psychopathology 36(5):269–275PubMedCrossRef
35.
go back to reference Patmore C, Weaver T (1989) A measure of care. Health Serv J 99(5142):330–331PubMed Patmore C, Weaver T (1989) A measure of care. Health Serv J 99(5142):330–331PubMed
36.
go back to reference Paul G (1987) Rational operations in residential treatment settings through ongoing assessment of staff and client functioning. In: Peterson D, Fishman D (eds) Assessment for decision, Rutgers University Press, pp 145–203 Paul G (1987) Rational operations in residential treatment settings through ongoing assessment of staff and client functioning. In: Peterson D, Fishman D (eds) Assessment for decision, Rutgers University Press, pp 145–203
37.
go back to reference Pawson R, Tilley N (1997) Realistic Evaluation. Sage Pawson R, Tilley N (1997) Realistic Evaluation. Sage
38.
go back to reference Popkin M, Callies A, Lurie N, Harman J, Stoner T, Manning W (1997) An instrument to evaluate the process of care in ambulatory settings. Psychiatr Service 48(4):524–527 Popkin M, Callies A, Lurie N, Harman J, Stoner T, Manning W (1997) An instrument to evaluate the process of care in ambulatory settings. Psychiatr Service 48(4):524–527
39.
go back to reference Quirk A, Lelliot P (2001) What do we know about life on acute psychiatric wards in the UK? A review of the research evidence. Soc Sci Med 53(12):1565–1574PubMedCrossRef Quirk A, Lelliot P (2001) What do we know about life on acute psychiatric wards in the UK? A review of the research evidence. Soc Sci Med 53(12):1565–1574PubMedCrossRef
40.
go back to reference Resnick S, Neale M, Rosenheck R (2003) Impact of Public Support Payments, intensive pychiatric community care and program fidelity on employment outcomes for people with severe mental illness. J Nerv Mental Disease 191(3):139–144CrossRef Resnick S, Neale M, Rosenheck R (2003) Impact of Public Support Payments, intensive pychiatric community care and program fidelity on employment outcomes for people with severe mental illness. J Nerv Mental Disease 191(3):139–144CrossRef
41.
go back to reference Rossberg J, Friis S (2004) Patients’ and staff’s perceptions of the psychiatric ward environment. Psychiatr Service 55(7):798–803CrossRef Rossberg J, Friis S (2004) Patients’ and staff’s perceptions of the psychiatric ward environment. Psychiatr Service 55(7):798–803CrossRef
42.
go back to reference Rossi P, Freeman H (1989) Evaluation: a systematic approach, 4th edn. Sage Rossi P, Freeman H (1989) Evaluation: a systematic approach, 4th edn. Sage
43.
go back to reference Rossi P, Freeman H, Lipsey G (1999) Evaluation: a systematic approach, 6th edn. Sage Rossi P, Freeman H, Lipsey G (1999) Evaluation: a systematic approach, 6th edn. Sage
44.
go back to reference Salize HJ, Kustner BM, Torres-Gonzalez F, Reinhard I, Estevez JFJ, Rossler W (1999) Needs for care and effectiveness of mental health care provision for schizophrenic patients in two European regions: a comparison between Granada (Spain) and Mannheim (Germany). Acta Psychiatr Scand 100(5):328–334PubMedCrossRef Salize HJ, Kustner BM, Torres-Gonzalez F, Reinhard I, Estevez JFJ, Rossler W (1999) Needs for care and effectiveness of mental health care provision for schizophrenic patients in two European regions: a comparison between Granada (Spain) and Mannheim (Germany). Acta Psychiatr Scand 100(5):328–334PubMedCrossRef
45.
go back to reference Shepherd G, Richardson R (1979) Organization and interaction in psychiatric day centers. Psychol Med 9:573–579PubMedCrossRef Shepherd G, Richardson R (1979) Organization and interaction in psychiatric day centers. Psychol Med 9:573–579PubMedCrossRef
46.
go back to reference Slade M, Phelan M, Thornicroft G (1998) A comparison of needs assessed by staff and by and epidemiologically representative sample of patients with psychosis. Psychol Med 28(3):543–550PubMedCrossRef Slade M, Phelan M, Thornicroft G (1998) A comparison of needs assessed by staff and by and epidemiologically representative sample of patients with psychosis. Psychol Med 28(3):543–550PubMedCrossRef
47.
go back to reference Slade M, Thornicroft G, Loftus S, Phelan M, Wykes T (1999) The camberwell assessment of need Gaskell, London Slade M, Thornicroft G, Loftus S, Phelan M, Wykes T (1999) The camberwell assessment of need Gaskell, London
48.
go back to reference Teague G, Bond G, Drake R (1998) Programme fidelity in ACT: development and use of a measure. Am J Orthopsychiatry 68:216–232PubMedCrossRef Teague G, Bond G, Drake R (1998) Programme fidelity in ACT: development and use of a measure. Am J Orthopsychiatry 68:216–232PubMedCrossRef
49.
go back to reference Thornicroft G, Tansella M (1999) The mental health matrix: a Manual to Improve Services Cambridge University Press Thornicroft G, Tansella M (1999) The mental health matrix: a Manual to Improve Services Cambridge University Press
50.
go back to reference Thornton A, Hatton A, Tatham A (2004) Dementia care mapping reconsidered: exploring the reliability and validity of the observational tool. Int J Geriatr Psychiatry 19:718–726PubMedCrossRef Thornton A, Hatton A, Tatham A (2004) Dementia care mapping reconsidered: exploring the reliability and validity of the observational tool. Int J Geriatr Psychiatry 19:718–726PubMedCrossRef
51.
go back to reference Tugwell P (1979) A methodological perspective on process measures of the quality of medical care. Clin Investigat Med 2:113–121 Tugwell P (1979) A methodological perspective on process measures of the quality of medical care. Clin Investigat Med 2:113–121
52.
go back to reference Tyson G, Lambert G, Beattie L (1995) The quality of psychiatric nurses’ interactions with patients: an observational study. Int J Nursing Studies 32(1):49–58CrossRef Tyson G, Lambert G, Beattie L (1995) The quality of psychiatric nurses’ interactions with patients: an observational study. Int J Nursing Studies 32(1):49–58CrossRef
53.
go back to reference Whitley R, Crawford M (2005) Qualitative research in psychiatry. Can J Psychiatry 50(2):108–114PubMed Whitley R, Crawford M (2005) Qualitative research in psychiatry. Can J Psychiatry 50(2):108–114PubMed
54.
go back to reference Wing J, Brown G (1970) Institutionalism and Schizophrenia Cambridge University Press Wing J, Brown G (1970) Institutionalism and Schizophrenia Cambridge University Press
55.
go back to reference Wright R, Sklebar T, Heiman J (1987) Patterns of case management activity in an intensive community support programme. Commun Mental Health J 23(1):53–59CrossRef Wright R, Sklebar T, Heiman J (1987) Patterns of case management activity in an intensive community support programme. Commun Mental Health J 23(1):53–59CrossRef
56.
go back to reference Young A, Sullivan G, Burnam A, Brook R (1998) Measuring the quality of outpatient treatment for schizophrenia. Arch Gen Psychiatry 55:611–617PubMedCrossRef Young A, Sullivan G, Burnam A, Brook R (1998) Measuring the quality of outpatient treatment for schizophrenia. Arch Gen Psychiatry 55:611–617PubMedCrossRef
Metadata
Title
Assessing the content of mental health services: a review of measures
Authors
Brynmor Lloyd-Evans
Sonia Johnson
Mike Slade
Publication date
01-08-2007
Publisher
D. Steinkopff-Verlag
Published in
Social Psychiatry and Psychiatric Epidemiology / Issue 8/2007
Print ISSN: 0933-7954
Electronic ISSN: 1433-9285
DOI
https://doi.org/10.1007/s00127-007-0216-x

Other articles of this Issue 8/2007

Social Psychiatry and Psychiatric Epidemiology 8/2007 Go to the issue