Top

Administration and Policy in Mental Health and Mental Health Services Research

Published in:

Open Access 01-11-2019 | Original Article

Attitudes Towards the Use of Routine Outcome Monitoring of Psychological Therapies Among Mental Health Providers: The EBPAS–ROM

Authors: Marte Rye, Kamilla Rognmo, Gregory A. Aarons, Ingunn Skre

Published in: Administration and Policy in Mental Health and Mental Health Services Research | Issue 6/2019

Abstract

Implementation of routine outcome monitoring (ROM) in mental health care is progressing slowly. Knowledge about factors influencing ROM implementation, including health providers’ attitudes towards ROM, is necessary. Based on a survey of 662 psychologists and nurses, this article describes (1) the development of a short instrument measuring provider attitudes towards ROM, derived from the Evidence-based Practice Attitude Scale (EBPAS), and (2) how attitudinal domains relate to clinicians’ current use of standardized instruments for treatment evaluation. The EBPAS–ROM showed concurrent validity in predicting aspects important for the implementation of ROM, including perceived limitations and the value of organizational support.

Available only for authorised users

Aarons, G. A. (2004). Mental health provider attitudes toward adoption of evidence-based practice: The Evidence-Based Practice Attitude Scale (EBPAS). Mental Health Services Research, 6(2), 61–74.CrossRefPubMedPubMedCentral

Aarons, G. A., Cafri, G., Lugo, L., & Sawitzky, A. (2012). Expanding the domains of attitudes towards evidence-based practice: The evidence based practice attitude scale-50. Administration and Policy in Mental Health, 39(5), 331–340. https://doi.org/10.1007/s10488-010-0302-3.CrossRefPubMed

Aarons, G. A., Hurlbert, M., & Horowitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health, 38, 4–23. https://doi.org/10.1007/s10488-010-0327-7.CrossRefPubMed

Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179–211. https://doi.org/10.1016/0749-5978(91)90020-t.CrossRef

Amble, I., Gude, T., Stubdal, S., Andersen, B. J., & Wampold, B. E. (2015). The effect of implementing the Outcome Questionnaire-45.2 feedback system in Norway: A multisite randomized clinical trial in a naturalistic setting. Psychotherapy Research, 25(6), 669–677. https://doi.org/10.1080/10503307.2014.928756.CrossRefPubMed

Bickman, L. (2008). A measurement feedback system (MFS) is necessary to improve mental health outcomes. Journal of the American Academy of Child and Adolescent Psychiatry, 47(10), 1114–1119. https://doi.org/10.1097/chi.0b013e3181825af8.CrossRefPubMed

Bickman, L., Douglas, S. R., De Andrade, A. R., Tomlinson, M., Gleacher, A., Olin, S., et al. (2016a). Implementing a measurement feedback system: A tale of two sites. Administration and Policy in Mental Health, 43(3), 410–425. https://doi.org/10.1007/s10488-015-0647-8.CrossRefPubMedPubMedCentral

Bickman, L., Kelley, S. D., Breda, C., de Andrade, A. R., & Riemer, M. (2011). Effects of routine feedback to clinicians on mental health outcomes of youths: Results of a randomized trial. Psychiatric Services, 62(12), 1423–1429. https://doi.org/10.1176/appi.ps.002052011.CrossRefPubMed

Bickman, L., Lyon, A. R., & Wolpert, M. (2016b). Achieving precision mental health through effective assessment, monitoring, and feedback processes: Introduction to the special issue. Administration and Policy in Mental Health, 43(3), 271–276. https://doi.org/10.1007/s10488-016-0718-5.CrossRefPubMedPubMedCentral

Boswell, J. F., Kraus, D. R., Miller, S. D., & Lambert, M. J. (2015). Implementing routine outcome monitoring in clinical practice: Benefits, challenges, and solutions. Psychotherapy Research, 25(1), 6–19. https://doi.org/10.1080/10503307.2013.817696.CrossRefPubMed

Boyce, M. B., Browne, J. P., & Greenhalgh, J. (2014). The experiences of professionals with using information from patient-reported outcome measures to improve the quality of healthcare: A systematic review of qualitative research. BMJ Quality and Safety, 23(6), 508–518. https://doi.org/10.1136/bmjqs-2013-002524.CrossRefPubMed

Brattland, H., Koksvik, J. M., Burkeland, O., Grawe, R. W., Klockner, C., Linaker, O. M., …, Iversen, V. C. (2018). The effects of routine outcome monitoring (ROM) on therapy outcomes in the course of an implementation process. A randomized clinical trial. Journal of Counseling Psychology. https://doi.org/10.1037/cou0000286.CrossRefPubMed

Carlier, I. V., Meuldijk, D., Van Vliet, I. M., Van Fenema, E., Van der Wee, N. J., & Zitman, F. G. (2012). Routine outcome monitoring and feedback on physical or mental health status: Evidence and theory. Journal of Evaluation in Clinical Practice, 18(1), 104–110. https://doi.org/10.1111/j.1365-2753.2010.01543.x.CrossRefPubMed

Chaudoir, S. R., Dugan, A. G., & Barr, C. H. (2013). Measuring factors affecting implementation of health innovations: A systematic review of structural, organizational, provider, patient, and innovation level measures. Implementation Science, 8, 22. https://doi.org/10.1186/1748-5908-8-22.CrossRefPubMed

Chor, K. H. B., Wisdom, J. P., Olin, S. C. S., Hoagwood, K. E., & Horwitz, S. M. (2015). Measures for predictors of innovation adoption. Administration and Policy in Mental Health and Mental Health Services Research, 42(5), 545–573. https://doi.org/10.1007/s10488-014-0551-7.CrossRefPubMed

Chung, G. H., Choi, J. N., & Du, J. (2017). Tired of innovations? Learned helplessness and fatigue in the context of continuous streams of innovation implementation. Journal of Organizational Behavior, 38(7), 1130–1148. https://doi.org/10.1002/job.2191.CrossRef

De Jong, K., Timman, R., Hakkaart-Van Roijen, L., Vermeulen, P., Kooiman, K., Passchier, J., et al. (2014). The effect of outcome monitoring feedback to clinicians and patients in short and long-term psychotherapy: A randomized controlled trial. Psychotherapy Research, 24(6), 629–639. https://doi.org/10.1080/10503307.2013.871079.CrossRefPubMed

Eccles, M. P., Hrisos, S., Francis, J., Kaner, E. F., Dickinson, H. O., Beyer, F., et al. (2006). Do self-reported intentions predict clinicians’ behaviour: A systematic review. Implementation Science. https://doi.org/10.1186/1748-5908-1-28.CrossRefPubMedPubMedCentral

Edbrooke-Childs, J., Wolpert, M., & Deighton, J. (2016). Using patient reported outcome measures to improve service effectiveness (UPROMISE): Training clinicians to use outcome measures in child mental health. Administration and Policy in Mental Health and Mental Health Services Research, 43(3), 302–308. https://doi.org/10.1007/s10488-014-0600-2.CrossRefPubMed

Garland, A. F., Kruse, M., & Aarons, G. A. (2003). Clinicians and outcome measurement: What’s the use? Journal of Behavioral Health Services and Research, 30(4), 393–405. https://doi.org/10.1007/bf02287427.CrossRefPubMed

Glasgow, R. E., & Riley, W. T. (2013). Pragmatic measures: What they are and why we need them. American Journal of Preventive Medicine, 45(2), 237–243. https://doi.org/10.1016/j.amepre.2013.03.010.CrossRefPubMed

Gleacher, A. A., Olin, S. S., Nadeem, E., Pollock, M., Ringle, V., Bickman, L., et al. (2016). Implementing a measurement feedback system in community mental health clinics: A case study of multilevel barriers and facilitators. Administration and Policy in Mental Health, 43(3), 426–440. https://doi.org/10.1007/s10488-015-0642-0.CrossRefPubMedPubMedCentral

Goldman, H. H., & Seybolt, D. C. (2015). What can we learn from the ongoing challenge to implement routine outcome measures? International Review of Psychiatry, 27(4), 261–263. https://doi.org/10.3109/09540261.2015.1069513.CrossRefPubMed

Hall, C. L., Moldavsky, M., Taylor, J., Sayal, K., Marriott, M., Batty, M. J., et al. (2014). Implementation of routine outcome measurement in child and adolescent mental health services in the United Kingdom: A critical perspective. European Child and Adolescent Psychiatry, 23(4), 239–242. https://doi.org/10.1007/s00787-013-0454-2.CrossRefPubMed

Hatfield, D. R., & Ogles, B. M. (2004). The use of outcome measures by psychologists in clinical practice. Professional Psychology Research and Practice, 35(5), 485–491. https://doi.org/10.1037/0735-7028.35.5.485.CrossRef

Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling—A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118.CrossRef

Ionita, G., & Fitzpatrick, M. (2014). Bringing science to clinical practice: A Canadian survey of psychological practice and usage of progress monitoring measures. Canadian Psychology, 55(3), 187–196. https://doi.org/10.1037/a0037355.CrossRef

Ionita, G., Fitzpatrick, M., Tomaro, J., Chen, V. V., & Overington, L. (2016). Challenges of using progress monitoring measures: Insights from practicing clinicians. Journal of Counseling Psychology, 63(2), 173–182. https://doi.org/10.1037/cou0000122.CrossRefPubMed

Kaiser, T., Schmutzhart, L., & Laireiter, A. R. (2018). Attitudes of Austrian psychotherapists towards process and outcome monitoring. Administration and Policy in Mental Health, 45(5), 765–779. https://doi.org/10.1007/s10488-018-0862-1.CrossRefPubMedPubMedCentral

Kendrick, T., El-Gohary, M., Stuart, B., Gilbody, S., Churchill, R., Aiken, L., …, Moore, M. (2016). Routine use of patient reported outcome measures (PROMs) for improving treatment of common mental health disorders in adults. The Cochrane Database of Systematic Reviews, 7, CD011119. https://doi.org/10.1002/14651858.cd011119.pub2.

Lambert, M. (2007). Presidential address: What we have learned from a decade of research aimed at improving psychotherapy outcome in routine care. Psychotherapy Research, 17(1), 1–14. https://doi.org/10.1080/10503300601032506.CrossRef

Lambert, M. J., & Harmon, K. L. (2018). The merits of implementing routine outcome monitoring in clinical practice. Clinical Psychology: Science and Practice. https://doi.org/10.1111/cpsp.12268.CrossRef

Lambert, M. J., Whipple, J. L., & Kleinstauber, M. (2018). Collecting and delivering progress feedback: A meta-analysis of routine outcome monitoring. Psychotherapy (Chic), 55(4), 520–537. https://doi.org/10.1037/pst0000167.CrossRef

Lambert, M. J., Whipple, J. L., Smart, D. W., Vermeersch, D. A., Nielsen, S. L., & Hawkins, E. J. (2001). The effects of providing therapists with feedback on patient progress during psychotherapy: Are outcomes enhanced? Psychotherapy Research, 11(1), 49–68. https://doi.org/10.1093/ptr/11.1.49.CrossRefPubMed

Lewis, C. C., Boyd, M., Puspitasari, A., Navarro, E., Howard, J., Kassab, H., …, Kroenke, K. (2019). Implementing measurement-based care in behavioral health: A review. JAMA Psychiatry, 76(3), 324–335. https://doi.org/10.1001/jamapsychiatry.2018.3329.CrossRefPubMed

Lewis, C. C., Stanick, C. F., Martinez, R. G., Weiner, B. J., Kim, M., Barwick, M., et al. (2015). The Society for Implementation Research Collaboration Instrument Review Project: A methodology to promote rigorous evaluation. Implementation Science. https://doi.org/10.1186/s13012-014-0193-x.CrossRefPubMed

Martinez, R. G., Lewis, C. C., & Weiner, B. J. (2014). Instrumentation issues in implementation science. Implementation Science. https://doi.org/10.1186/s13012-014-0118-8.CrossRefPubMed

Miller, S. D., Hubble, M. A., Chow, D., & Seidel, J. (2015). Beyond measures and monitoring: Realizing the potential of feedback-informed treatment. Psychotherapy, 52(4), 449–457. https://doi.org/10.1037/pst0000031.CrossRefPubMed

Moullin, J. C., Ehrhart, M. G., & Aarons, G. A. (2018). Development and testing of the Measure of Innovation-Specific Implementation Intentions (MISII) using Rasch measurement theory. Implementation Science. https://doi.org/10.1186/s13012-018-0782-1.CrossRefPubMed

Norman, S., Dean, S., Hansford, L., & Ford, T. (2014). Clinical practitioner’s attitudes towards the use of Routine Outcome Monitoring within Child and Adolescent Mental Health Services: A qualitative study of two Child and Adolescent Mental Health Services. Clinical Child Psychology and Psychiatry, 19(4), 576–595. https://doi.org/10.1177/1359104513492348.CrossRefPubMed

Okamura, K. H., Hee, P. J., Jackson, D., & Nakamura, B. J. (2018). Furthering our understanding of therapist knowledge and attitudinal measurement in youth community mental health. Administration and Policy in Mental Health, 45(5), 699–708. https://doi.org/10.1007/s10488-018-0854-1.CrossRefPubMed

Overington, L., Fitzpatrick, M., Hunsley, J., & Drapeau, M. (2015). Trainees’ experiences using progress monitoring measures. Training and Education in Professional Psychology, 9(3), 202–209. https://doi.org/10.1037/tep0000088.CrossRef

Persons, J. B., Koeman, K., Eidelman, P., & Thomas, C. (2016). Increasing psychotherapists’ adoption and implementation of the evidence-based practice of progress monitoring. Behaviour Research and Therapy, 76, 24–31. https://doi.org/10.1016/j.brat.2015.11.004.CrossRefPubMed

Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., et al. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38(2), 65–76. https://doi.org/10.1007/s10488-010-0319-7.CrossRefPubMed

Rogers, E. M. (2003). Diffusion of innovations. New York: Free Press.

Ross, D. F., Ionita, G., & Stirman, S. W. (2016). System-wide implementation of routine outcome monitoring and measurement feedback system in a National Network of Operational Stress Injury Clinics. Administration and Policy in Mental Health and Mental Health Services Research, 43(6), 927–944. https://doi.org/10.1007/s10488-016-0749-y.CrossRefPubMed

Rye, M., Friborg, O., & Skre, I. (2019). Attitudes of mental health providers towards adoption of evidence-based interventions: Relationship to workplace, staff roles and social and psychological factors at work. BMC Health Services Research. https://doi.org/10.1186/s12913-019-3933-4.CrossRefPubMedPubMedCentral

Rye, M., Torres, E. M., Friborg, O., Skre, I., & Aarons, G. A. (2017). The Evidence-based Practice Attitude Scale36 (EBPAS-36): A brief and pragmatic measure of attitudes to evidence-based practice validated in US and Norwegian samples. Implementation Science. https://doi.org/10.1186/s13012-017-0573-0.CrossRefPubMed

Scott, K., & Lewis, C. C. (2015). Using measurement-based care to enhance any treatment. Cognitive and Behavioral Practice, 22(1), 49–59. https://doi.org/10.1016/j.cbpra.2014.01.010.CrossRefPubMedPubMedCentral

Sharples, E., Qin, C., Goveas, V., Gondek, D., Deighton, J., Wolpert, M., et al. (2017). A qualitative exploration of attitudes towards the use of outcome measures in child and adolescent mental health services. Clinical Child Psychology and Psychiatry, 22(2), 219–228. https://doi.org/10.1177/1359104516652929.CrossRefPubMed

Shimokawa, K., Lambert, M. J., & Smart, D. W. (2010). Enhancing treatment outcome of patients at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy quality assurance system. Journal of Consulting and Clinical Psychology, 78(3), 298–311. https://doi.org/10.1037/a0019247.CrossRefPubMed

Simon, W., Lambert, M. J., Harris, M. W., Busath, G., & Vazquez, A. (2012). Providing patient progress information and clinical support tools to therapists: Effects on patients at risk of treatment failure. Psychotherapy Research, 22(6), 638–647. https://doi.org/10.1080/10503307.2012.698918.CrossRefPubMed

Stasiak, K., Parkin, A., Seymour, F., Lambie, I., Crengle, S., Pasene-Mizziebo, E., et al. (2013). Measuring outcome in child and adolescent mental health services: Consumers’ views of measures. Clinical Child Psychology and Psychiatry, 18(4), 519–535. https://doi.org/10.1177/1359104512460860.CrossRefPubMed

Statistics Norway. (2019, July 15). Specialist health service. Retrieved July 15, 2019 from https://www.ssb.no/en/statbank/table/09547.

The Joint Commission. (2018, July 15). New outcome measures standard. Retrieved July 15, 2019 from https://www.jointcommission.org/accreditation/bhc_new_outcome_measures_standard.aspx.

Van Horn, P. S., Green, K. E., & Martinussen, M. (2009). Survey response rates and survey administration in counseling and clinical psychology: A meta-analysis. Educational and Psychological Measurement, 69(3), 389–403. https://doi.org/10.1177/0013164408324462.CrossRef

Wampold, B. E. (2015). Routine outcome monitoring: Coming of age—With the usual developmental challenges. Psychotherapy (Chic), 52(4), 458–462. https://doi.org/10.1037/pst0000037.CrossRef

Wisdom, J. P., Chor, K. H. B., Hoagwood, K. E., & Horwitz, S. M. (2014). Innovation adoption: A review of theories and constructs. Administration and Policy in Mental Health and Mental Health Services Research, 41(4), 480–502. https://doi.org/10.1007/s10488-013-0486-4.CrossRefPubMed

Wolpert, M., Curtis-Tyler, K., & Edbrooke-Childs, J. (2016). A qualitative exploration of patient and clinician views on patient reported outcome measures in child mental health and diabetes services. Administration and Policy in Mental Health, 43(3), 309–315. https://doi.org/10.1007/s10488-014-0586-9.CrossRefPubMed

Youn, S. J., Kraus, D. R., & Castonguay, L. G. (2012). The treatment outcome package: Facilitating practice and clinically relevant research. Psychotherapy, 49(2), 115–122. https://doi.org/10.1037/a0027932.CrossRefPubMed

Title: Attitudes Towards the Use of Routine Outcome Monitoring of Psychological Therapies Among Mental Health Providers: The EBPAS–ROM
Authors: Marte Rye
Kamilla Rognmo
Gregory A. Aarons
Ingunn Skre
Publication date: 01-11-2019
Publisher: Springer US
Published in: Administration and Policy in Mental Health and Mental Health Services Research / Issue 6/2019
Print ISSN: 0894-587X
Electronic ISSN: 1573-3289
DOI: https://doi.org/10.1007/s10488-019-00968-5

Keynote webinar | Spotlight on sleep in brain health

Springer Medicine

Attitudes Towards the Use of Routine Outcome Monitoring of Psychological Therapies Among Mental Health Providers: The EBPAS–ROM

Abstract

Keynote webinar | Spotlight on sleep in brain health

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 6/2019

Developing the Behavioral Health Workforce: Lessons from the States

Practitioners' Use and Evaluation of Transdiagnostic Youth Psychotherapy Years After Training and Consultation Have Ended

Managing and Adapting Practice: Provider Perceptions of an Evidence-Informed Framework for Delivering Mental Health Services

The Conceptualization of Mental Health Service Quality Assessment: Consumer Perspective

Therapist Financial Strain and Turnover: Interactions with System-Level Implementation of Evidence-Based Practices

Application of Self-determination Theory to Illness Self-management Interventions: Identifying Mediators