Top

Administration and Policy in Mental Health and Mental Health Services Research

Published in:

01-09-2015 | Original Article

Clinicians’ Attitudes Towards Outcome and Process Monitoring: A Validation of the Outcome Measurement Questionnaire

Authors: Dave F. Smits, Laurence Claes, Nele Stinckens, Dirk J. M. Smits

Published in: Administration and Policy in Mental Health and Mental Health Services Research | Issue 5/2015

Abstract

Valid and reliable instruments to measure monitoring attitudes of clinicians are scarce. The influence of sociodemographics and professional characteristics on monitoring attitudes is largely unknown. First, we investigated the factor structure and reliability of the Outcome Measurement Questionnaire among a sample of Flemish mental health professionals (n = 170). Next, we examined the relationship between clinicians’ sociodemographic and professional characteristics and monitoring attitudes. Construct validity was determined using a confirmatory factor analysis. Internal consistency was ascertained using Cronbach’s alpha. Mean level differences in monitoring attitudes related to clinicians’ gender, work setting, level of education and psychotherapeutic training, were investigated using ANOVAs. The relationships between clinicians’ age, clinical experience and attitudes were calculated using the Pearson correlation coefficient. A model with one general factor and a method factor referring to reverse-worded items best fitted our data. Internal consistency was good. Clinicians with psychotherapeutic training reported more favorable monitoring attitudes than those without such training. Compared to clinicians working in subsidized outpatient services, private practitioners and clinicians from inpatient mental health clinics had more positive attitudes. Results highlight the need for sustained and targeted training, with particular focus on transforming measurement data into meaningful clinical support tools.

The first and fourth author are experienced trainers and implementation coaches of monitoring systems in multiple Flemish mental health centres.

In Flanders, both bachelor and master’s graduates can become psychotherapists. Prior to a 4 year psychotherapy training course, bachelor graduates need to attend a bridging program. Without attending this bridging program, a 4-year counseling course leads to a Counselor certificate. Both groups are subsequently referred to as a group with ‘psychotherapeutic training’.

Allen, A., Montgomery, M., Tubman, J., Frazier, L., & Escovar, L. (2003). The effects of assessment feedback on rapport-building and self-enhancement processes. Journal of Mental Health Counseling, 25, 165–181.CrossRef

Bentler, P. M. (2007). On tests and indices for evaluating structural models. Personality and Individual Differences, 42, 825–829.CrossRef

Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley.CrossRef

Bowman, J., Lannin, N., Cook, C., & McCluskey, A. (2009). Development and psychometric testing of the clinician readiness for measuring outcomes scale. Journal of Evaluation in Clinical Practice, 15, 76–84.CrossRefPubMed

Callaly, T., & Hallebone, E. L. (2001). Introducing the routine use of outcomes measurement to mental health services. Australian Health Review, 24, 43–50.CrossRefPubMed

Close-Goedjen, J. L., & Saunders, S. M. (2002). The effect of technical support on clinician attitudes toward an outcome assessment instrument. Journal of Behavioral Health Service and Research, 29, 99–108.CrossRef

Cronbach, L. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334.CrossRef

De Jong, K., Van Sluis, P., Nugter, M. A., Heiser, W. J., & Spinhoven, P. (2012). Understanding the differential impact of outcome monitoring: therapist variables that moderate feedback effects in a randomized clinical trial. Psychotherapy Research, 22, 464–474.CrossRefPubMed

Dekker, J. (2011). Monitoring als therapeutisch instrument. Een exploratieve studie naar intermediërende processen, geïnitieerd door systematische evaluatie en feedback in psychotherapie. Unpublished Master’s thesis, KU Leuven, Leuven.

Flora, D. B., & Curran, P. J. (2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods, 9, 466–491.PubMedCentralCrossRefPubMed

Guthrie, D., McIntosh, M., Callaly, T., Trauer, T., & Coombs, T. (2008). Consumer attitudes towards the use of routine outcome measures in a public mental health service: A consumer-driven study. International Journal of Mental Health Nursing, 17, 92–97.CrossRefPubMed

Hilsenroth, M. J., Peters, E. J., & Ackerman, S. J. (2004). The development of therapeutic alliance during psychological assessment: Patient and therapist perspectives across treatment. Journal of Personality Assessment, 83, 332–344.CrossRefPubMed

Howard, K. I., Moras, K., Brill, P. L., Martinovich, Z., & Lutz, W. (1996). Evaluation of psychotherapy: Efficacy, effectiveness, and patient progress. American Psychologist, 51, 1059–1064.CrossRefPubMed

Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.CrossRef

Huffman, L. C., Martin, J., Botcheva, L., Williams, S. E., & Dyer-Friedman, J. P. (2004). Practitioners’ attitudes toward the use of treatment progress and outcomes data in child mental health services. Evaluation and the Health Professions, 27, 165–188.CrossRefPubMed

Jöreskog, K., & Sörbom, D. (2012). LISREL 8.8 (computer program). Lincolnwood: Scientific Software International.

Kline, R. B. (2005). Principles and practices of structural equation modeling. New York: Guilford.

Knaup, C., Koesters, M., Schoefer, D., Becker, T., & Puschner, B. (2009). Effect of feedback of treatment outcome in specialist mental healthcare: meta-analysis. The British Journal of Psychiatry, 195, 15–22.CrossRefPubMed

Lambert, M. J. (2010a). Yes, it is time for clinicians to routinely track treatment outcome. In B. L. Duncan, S.D. Miller, B. E. Wampold, & M. A. Hubble (Eds.), The heart and soul of change. Delivering what works (pp. 239–268). Washington, DC: American Psychological Association.

Lambert, M. J. (2010b). Prevention of treatment failure: The use of measuring, monitoring, and feedback in clinical practice. Washington, DC: American Psychological Association.

Lambert, M. J., Hansen, N. B., & Finch, A. E. (2001). Patient-focused research: using patient outcome data to enhance treatment effects. Journal of Consulting and Clinical Psychology, 69, 159.CrossRefPubMed

Lei, P.-W. (2009). Evaluating estimation methods for ordinal data in structural equation modeling. Quality & Quantity, 43, 495–507.CrossRef

Podsakoff, P. M., MacKensie, S. B., Lee, J.-Y., & Posdakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88, 879–903.CrossRefPubMed

Prochaska, J. M., Prochaska, J. O., & Levesque, D. A. (2001). A transtheortical approach to changing organizations. Administration and Policy in Mental Health, 28, 247–261.CrossRefPubMed

Sapyta, J., Riemer, M., & Bickman, L. (2005). Feedback to clinicians: Theory, research, and practice. Journal of Clinical Psychology, 61, 145–153.CrossRefPubMed

Schweizer, K. (2010). Some guidelines concerning the modeling of traits and abilities in test construction. European Journal of Psychological Assessment, 26, 1–2.CrossRef

Shimokawa, K., Lambert, M. J., & Smart, D. W. (2010). Enhancing treatment outcome of patients at risk of treatment failure: meta-analytic and Mega-analytic review of a psychotherapy quality assurance system. Journal of Consulting and Clinical Psychology, 78, 298.CrossRefPubMed

Trauer, T., Gill, L., Pedwell, G., & Slattery, P. (2006). Routine outcome measurement in public mental health—what do clinicians think?. Australian Health Review, 30, 144–147.CrossRefPubMed

Trauer, T., Pedwell, G., & Gill, L. (2009). The effect of guidance in the use of routine outcome measures in clinical meetings. Australian Health Review, 33, 144–151. Retrieved from http://search.proquest.com/docview/231737482?accountid=17215.

Walter, G., Cleary, M., & Rey, J. M. (1998). Attitudes of mental health personnel towards rating outcome. Journal of Quality in Clinical Practice, 18, 109–115.PubMed

Willis, A., Deane, F. P., & Coombs, T. (2009). Improving clinicians’ attitudes toward providing feedback on routine outcome assessments. International Journal of Mental Health Nursing, 18, 211–215.CrossRefPubMed

World Health Organization. (n.d.). Process of translation and adaptation of instruments. In World Health Organization, Programmes and Projects. Retrieved January 22, 2011, from http://www.who.int/substance_abuse/research_tools/translation/en/.

Title: Clinicians’ Attitudes Towards Outcome and Process Monitoring: A Validation of the Outcome Measurement Questionnaire
Authors: Dave F. Smits
Laurence Claes
Nele Stinckens
Dirk J. M. Smits
Publication date: 01-09-2015
Publisher: Springer US
Published in: Administration and Policy in Mental Health and Mental Health Services Research / Issue 5/2015
Print ISSN: 0894-587X
Electronic ISSN: 1573-3289
DOI: https://doi.org/10.1007/s10488-014-0597-6

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Clinicians’ Attitudes Towards Outcome and Process Monitoring: A Validation of the Outcome Measurement Questionnaire

Abstract

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 5/2015

Integrating Science and Engineering to Implement Evidence-Based Practices in Health Care Settings

Measures for Predictors of Innovation Adoption

Long-Term Clinical Outcomes from a Randomized Controlled Trial of Two Implementation Strategies to Promote Collaborative Care Attendance in Community Practices

Sharing State Mental Health Data for Research: Building Toward Ongoing Learning in Mental Health Care Systems

Purposeful Sampling for Qualitative Data Collection and Analysis in Mixed Method Implementation Research

Blending Qualitative and Computational Linguistics Methods for Fidelity Assessment: Experience with the Familias Unidas Preventive Intervention