Top

Published in:

01-11-2016 | Original Paper

Using Routine Outcome Measures to Provide Feedback at the Service Agency Level

Authors: David Roe, Liron Lapid, Vered Baloush-Kleinman, Paula Garber-Epstein, Miriam Isolde Gornemann, Marc Gelkopf

Published in: Community Mental Health Journal | Issue 8/2016

Abstract

The use of routine outcome measurement (ROM) to assess service effectiveness has been on the rise in mental health settings. However, there is a scarcity of information on the use of ROM data to provide feedback to teams of service providers. In this paper we review the existing literature to identify the principles that can guide the use of ROM data as feedback with the aim to improve quality of service provision in mental health settings. We present a pilot trial of 12 agencies participating in group feedback sessions. The guiding principles and core processes, the procedure and implementation in a pilot trial, lessons learned and future directions are discussed. Based on this experience we conclude that using ROM to implement group feedback among mental health stakeholders is feasible and can generate discussions and directions for improvement.

Barkham, M., Stiles, W. B., Connell, J., & Mellor-Clark, J. (2012). Psychological treatment outcomes in routine NHS services: what do we mean by treatment effectiveness? Psychology Psychotherapy, 85, 1–16.CrossRefPubMed

Batty, M. J., Moldavsky, M., Foroushani, P. S., Pass, S., Marriott, M., Sayal, K., & Hollis, C. (2013). Implementing routine outcome measures in child and adolescent mental health services: from present to future practice. Child and Adolescent Mental Health, 18, 82–87.CrossRef

Bickman, L., Kelley, S. D., Breda, C., de Andrade, A. R., & Riemer, M. (2011). Effects of routine feedback to clinicians on mental health outcomes of youths: results of a randomized trial. Psychiatric Services, 62, 1423–1429.CrossRefPubMed

Black, J., Lewis, T., McIntosh, P., Callaly, T., Coombs, T., Hunter, A., & Moore, L. (2009). It’s not that bad: the views of consumers and carers about routine outcome measurement in mental health. Australian Health Review, 33, 93–99.CrossRefPubMed

Boswell, J. F., Kraus, D. R., Miller, S. D., & Lambert, M. J. (2013). Implementing routine outcome monitoring in clinical practice: benefits, challenges, and solutions. Psychotherapy Research, 26, 1–14.

Burgess, P., Coombs, T., Clarke, A., Dickson, R., & Pirkis, J. (2012). “Achievements in mental health outcome measurement in Australia: reflections on progress made by the Australian Mental Health Outcomes and Classification Network (AMHOCN). International Journal of Mental Health Systems, 6, 4–14.CrossRefPubMedPubMedCentral

Carlier, I. V. E., Meuldijk, D., Van Vliet, I. M., Van Fenema, E., Van der Wee, N. J. A., & Zitman, F. M. D. (2012). Routine outcome monitoring and feedback on physical or mental health status: evidence and theory. Journal of Evaluation in Clinical Practice, 18, 104–110.CrossRefPubMed

Clark, D. M. (2011). Implementing NICE guidelines for the psychological treatment of depression and anxiety disorders: the IAPT experience. International Review of Psychiatry, 23, 318–327.CrossRefPubMedPubMedCentral

Coombs, T., Stapley, K., & Pirkis, J. (2011). The multiple uses of routine mental health outcome measures in Australia and New Zealand: experiences from the field. Australasian Psychiatry: Bulletin of Royal Australian and New Zealand College of Psychiatrists, 19, 247–253.CrossRef

Corrigan, P. W., Salzer, M., Ralph, R. O., Sangster, Y., & Keck, L. (2004). Examining the factor structure of the recovery assessment scale. Schizophrenia Bulletin, 30, 1035–1041.CrossRefPubMed

De Beurs, E., den Hollander-Gijsman, M. E., van Rood, Y. R., van der Wee, N. J., Giltay, E. J., van Noorden, M. S., …, Zitman, F. G. (2011). Routine outcome monitoring in the Netherlands: practical experiences with a web-based strategy for the assessment of treatment outcome in clinical practice. Clinical Psychology and Psychotherapy, 18, 1–12.CrossRefPubMed

De Jong, K., van Sluis, P., Nugter, M. A., Heiser, W. J., & Spinhoven, P. (2012). Understanding the differential impact of outcome monitoring: therapist variables that moderate feedback effects in a randomized clinical trial. Psychotherapy Research, 22, 464–474.CrossRefPubMed

Dickens, G., Weleminsky, J., Onifade, Y., & Sugarman, P. (2012). Star: validating user recovery. The Psychiatrist Online, 36, 45–50.CrossRef

Gabaldón Poc, O., Ruiz-Iriondo, M., & Bobowik, M. (2013). Multi-informant perception of quality of life and adaptation in chronic schizophrenia. The Spanish Journal of Psychology, 16, 1–8.CrossRef

Gabelica, C., Van den Bossche, P., Segers, M., & Gijselaers, W. (2012). Feedback, a powerful lever in teams: a review. Educational Research Review, 7, 123–144.CrossRef

Goodman, S. H., Sewell, D. R., Cooley, E. L., & Leavitt, N. (1993). Assessing levels of adaptive functioning: the Role Functioning Scale. Community Mental Health Journal, 29, 119–131.CrossRefPubMed

Guy, W. (1976). ECDEU assessment manual for psychopharmacology (Revised Ed.) (pp. 217–222). Rockville, MD: National Institute of Mental Health: US Department of Health Education and Welfare

Gyani, A., Shafran, R., Layard, R., & Clark, D. M. (2013). Enhancing recovery rates: Lessons from year one of IAPT. Behaviour Research and Therapy, 51, 597–606.CrossRefPubMedPubMedCentral

Harmon, C., Lambert, M. J., Smart, D. M., Hawkins, E. J., Nielsen, S. L., Slade, K., & Lutz, W. (2005). Improving outcomes for poorly responding clients: the use of clinical support tools and feedback to clients. Journal of Clinical Psychology, 61, 175–185.CrossRefPubMed

Haro, J. M., Kamath, S. A., Ochoa, S., Novick, D., Rele, K., Fargas, A., …, Jones, P. B. (2003). The Clinical Global Impression—Schizophrenia scale: a simple instrument to measure the diversity of symptoms present in schizophrenia. Acta Psychiatrica Scandinavica, 107, 16–23.CrossRef

Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77, 81–112.CrossRef

Higa-McMillan, C. K., Powell, C. K. K., Daleiden, E. L., & Mueller, C. W. (2011). Pursuing an evidence-based culture through contextualized feedback: aligning youth outcomes and practices. Professional Psychology: Research and Practice, 42, 137–144.CrossRef

Hunter, R., Cameron, R., & Norrie, J. (2009). Using patient-reported outcomes in schizophrenia: the Scottish Schizophrenia Outcomes Study. Psychiatric Services, 60, 240–245.CrossRefPubMed

Hysong, S. J. (2009). Meta-analysis: audit and feedback features impact effectiveness on care quality. Medical Care, 47, 356–363.CrossRefPubMedPubMedCentral

Jacobs, R. (2009). Investigating patient outcome measures in mental health. CHE Research Paper 48. Centre for Health Economics, University of York, York YO10 5DD, UK.

Knaup, C., Koesters, M., Schoefer, D., Becket, T., & Puschner, B. (2009). Effect of feedback of treatment outcome in specialist mental healthcare: meta-analysis. The British Journal of Psychiatry, 195, 15–22.CrossRefPubMed

Kopala, L., Smith, G., Malla, A., Williams, R., Love, L., Talling, D., & Balshaw, R. (2006). Resource utilization in a Canadian national study of people with schizophrenia and related psychotic disorders. Acta Psychiatrica Scandinavica, 113, 29–39.CrossRef

Lambert, M., Hansen, N. B., & Finch, A. E. (2001). Patient-focused research: using patient outcome data to enhanced treatment effects. Journal of Consulting and Clinical Psychology, 69, 159–172.CrossRefPubMed

Lambert, M. J. (2010). Prevention of treatment failure: the use of measuring, monitoring, and feedback in clinical practice. Washington: APA.CrossRef

Lambert, M. J., & Shimokawa, K. (2011). Collecting client feedback. Psychotherapy, 48, 72–79.CrossRefPubMed

Lasalvia, A., Ruggeri, M., Mazzi, A., & Dall Agnola, R. B. (2000). The perception of needs for care in staff and patients in community-based mental health services. The South-Verona Outcome Project 3. Acta Psychiatrica Scandinavica, 102, 366–375.CrossRefPubMed

Lehman, A. F., Fischer, E. P., Postrado, L., Delahanty, J., Johnstone, B. M., Russo, P. A., & Crown, W. H. (2003). The Schizophrenia Care and Assessment Program Health Questionnaire (SCAP-HQ). Schizophrenia Bulletin, 29, 247–256.CrossRefPubMed

Malla, A., Williams, R., Kopala, L., Smith, G., Talling, D., & Balshaw, R. (2006). Outcome on quality of life in a Canadian national sample of patients with schizophrenia and related psychotic disorders. Acta Psychiatrica Scandinavica, 430, 22–28.CrossRef

McInnes, B. (2006). Management at a crossroads: the service management challenge of implementing routine evaluation and performance management in psychological therapy and counselling services. European Journal of Psychotherapy and Counselling, 8, 163–176.CrossRef

McKay, R., Coombs, T., & Duerden, D. (2014). The art and science of using routine outcome measurement in mental health benchmarking. Australasian Psychiatry, 22, 13–18.CrossRefPubMed

McKay, R., Coombs, T., & Pirkis, J. (2012). A framework for exploring the potential of routine outcome measurement to improve mental health care. Australasian Psychiatry: Bulletin of Royal Australian and New Zealand College of Psychiatrists, 20, 127–133.CrossRef

Musiat, P., Hoffmann, L., & Schmidt, U. (2012). Personalized computarized feedback in E-mental health. Journal of Mental Health, 21, 346–354.CrossRefPubMed

Parabiaghi, A., Rapisarda, F., D’Avanzo, B., Arcadio Erlicher, A., Lora, A., & Barbato, A. (2011). Measuring clinical change in routine mental health care: differences between first time and longer term service users. Australian and New Zealand Journal of Psychiatry, 45, 558–568.CrossRefPubMed

Priebe, S., Huxley, P., Knight, S., & Evans, S. (1999). Application and results of the Manchester short assessment of quality of life (MANSA). International Journal of Social Psychiatry, 45, 7–12.CrossRefPubMed

Roe, D., Drake, R. E., & Slade, M. (2015a). Routine outcome monitoring: an international endeavor. International Review of Psychiatry, 27, 257–260.CrossRefPubMed

Roe, D., & Gelkopf, M. (2012). Report of the Israeli national outcome measurement pilot project of community rehabilitation services. Haifa: The Center for Community Mental Health Research, Practice, and Policy, University of Haifa.

Roe, D., Gelkopf, M., Gornemann, M. I., Baloush-Kleinman, V., & Shadmi, E. (2015b). Implementing routine outcome measurement in psychiatric rehabilitation services in Israel. International Review of Psychiatry, 27, 345–353.CrossRefPubMed

Roe, D., Mashiach-Eizenberg, M., & Lysaker, P. H. (2011). The relation between objective and subjective domains of recovery among persons with schizophrenia-related disorders. Schizophrenia Research, 131, 133–138.CrossRefPubMed

Roe, D., Werbeloff, N., & Gelkopf, M. (2010). Do persons with severe mental illness who consume the psychiatric rehabilitation basket of services in Israel have better outcomes than those who do not. Israel Journal of Psychiatry and Related Sciences, 47, 166–170.PubMed

Schulte-van Maaren, Y. W., Carlier, I. V., Giltay, E. J., van Noorden, M. S., de Waal, M. W., van der Wee, N. J., & Zitman, F. G. (2013). Reference values for mental health assessment instruments: objectives and methods of the Leiden Routine Outcome Monitoring Study. Journal of Evaluation and Clinical Practice, 19, 342–350.CrossRef

Sheehan, D. V. (1983). The anxiety disease. New York: Charles Scribner and Sons.

Sheehan, K. H., & Sheehan, D. V. (2008). Assessing treatment effects in clinical trials with the discan metric of the Sheehan Disability Scale. International Clinical Psychopharmacology, 23, 70–83.CrossRefPubMed

Shimokawa, K., M. J. Lambert and D. W. Smart (2010). Enhancing treatment outcome of patients at risk of treatment failure: meta-analytic and mega-analytic review of a psychotherapy quality assurance system. Journal of Consulting Clinical Psychology, 78, 298–311.CrossRefPubMed

Thornicroft, G. & Tansella, M. (Eds.) (2010). Mental health outcome measures. London: RCPsych Publications.

Trauer, T. (Ed.). (2010). Outcome measurement in mental health: theory and practice. Cambridge: Cambridge University Press.CrossRef

Unsworth, G., Cowie, H., & Green, A. (2012). Therapists’ and clients’ perceptions of routine outcome measurement in the NHS: a qualitative study. Counselling and Psychotherapy Research: Linking research with practice, 12, 71–80.CrossRef

Veerbeek, M. A., Oude Voshaar, R. C., & Pot, A. M. (2014). Effectiveness and predictors of outcome in routine out-patient mental health care for older adults. International Psychogeriatrics, 23, 1–10.CrossRef

Whipple, J. L., & Lambert, M. J. (2011). Outcome measures for practice. Annual Review Clinical Psychology, 7, 87–111.CrossRef

Wing, J. K., Beevor, A. S., Curtis, R. H., Park, S. B., Hadden, S., & Burns, A. (1998). Health of the Nation Outcome Scales (HoNOS). Research and development: British Journal of Psychiatry, 172, 11–18.PubMed

Wolpert, M., Deighton, J., De Francesco, D., Martin, P., Fonagy, P., & Ford, T. (2014). From ‘reckless’ to ‘mindful’ in the use of outcome data to inform service-level performance management: perspectives from child mental health. British Medical Journal Quality and Safety, 23, 272–276.CrossRef

Wolpert, M., Ford, T., Trustam, E., Law, D., Deighton, J., Flannery, H., & Fugard, A. (2012). Patient-reported outcomes in child and adolescent mental health services (CAMHS): use of idiographic and standardized measures. Journal of Mental Health, 12, 165–173.CrossRef

Title: Using Routine Outcome Measures to Provide Feedback at the Service Agency Level
Authors: David Roe
Liron Lapid
Vered Baloush-Kleinman
Paula Garber-Epstein
Miriam Isolde Gornemann
Marc Gelkopf
Publication date: 01-11-2016
Publisher: Springer US
Published in: Community Mental Health Journal / Issue 8/2016
Print ISSN: 0010-3853
Electronic ISSN: 1573-2789
DOI: https://doi.org/10.1007/s10597-016-0039-x

At a glance: The STEP trials

Springer Medicine

Using Routine Outcome Measures to Provide Feedback at the Service Agency Level

Abstract

At a glance: The STEP trials

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 8/2016

Predictors of Clinical Outcome of Assertive Community Treatment (ACT) in a Rural Area in Denmark: An Observational Study with a Two-Year Follow-Up

The Relationship Between Background Characteristics and Death Anxiety in Times of War: A Comparison Between Three Generations Arab and Jewish Families in Israel

Ameliorating Psychosocial Risk Among Mothers with Intellectual Impairment

“Sometimes What They Think is Helpful is Not Really Helpful”: Understanding Engagement in the Program of Assertive Community Treatment (PACT)

Comorbid Moderate–Severe Depressive Symptoms and their Association with Quality of Life in Chinese Patients with Schizophrenia Treated in Primary Care

Forensic Assertive Community Treatment: A Review of the Literature