Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
BMC Psychiatry
Published in: BMC Psychiatry 1/2016

Open Access 01-12-2016 | Study protocol

A randomized trial to identify accurate and cost-effective fidelity measurement methods for cognitive-behavioral therapy: project FACTS study protocol

Authors: Rinad S. Beidas, Johanna Catherine Maclean, Jessica Fishman, Shannon Dorsey, Sonja K. Schoenwald, David S. Mandell, Judy A. Shea, Bryce D. McLeod, Michael T. French, Aaron Hogue, Danielle R. Adams, Adina Lieberman, Emily M. Becker-Haimes, Steven C. Marcus

Published in: BMC Psychiatry | Issue 1/2016

Login to get access

Abstract

Background

This randomized trial will compare three methods of assessing fidelity to cognitive-behavioral therapy (CBT) for youth to identify the most accurate and cost-effective method. The three methods include self-report (i.e., therapist completes a self-report measure on the CBT interventions used in session while circumventing some of the typical barriers to self-report), chart-stimulated recall (i.e., therapist reports on the CBT interventions used in session via an interview with a trained rater, and with the chart to assist him/her) and behavioral rehearsal (i.e., therapist demonstrates the CBT interventions used in session via a role-play with a trained rater). Direct observation will be used as the gold-standard comparison for each of the three methods.

Methods/design

This trial will recruit 135 therapists in approximately 12 community agencies in the City of Philadelphia. Therapists will be randomized to one of the three conditions. Each therapist will provide data from three unique sessions, for a total of 405 sessions. All sessions will be audio-recorded and coded using the Therapy Process Observational Coding System for Child Psychotherapy-Revised Strategies scale. This will enable comparison of each measurement approach to direct observation of therapist session behavior to determine which most accurately assesses fidelity. Cost data associated with each method will be gathered. To gather stakeholder perspectives of each measurement method, we will use purposive sampling to recruit 12 therapists from each condition (total of 36 therapists) and 12 supervisors to participate in semi-structured qualitative interviews.

Discussion

Results will provide needed information on how to accurately and cost-effectively measure therapist fidelity to CBT for youth, as well as important information about stakeholder perspectives with regard to each measurement method. Findings will inform fidelity measurement practices in future implementation studies as well as in clinical practice.

Trial registration

NCT02820623, June 3rd, 2016.
Footnotes
1
For the purposes of this study, the trained individual refers to research assistants who will conduct chart-stimulated recall and behavioral rehearsal.
 
2
We elected not to include session 1 because it is usually oriented toward assessment and rapport building, rather than active intervention.
 
3
Note, the TPOCS-RS does not currently contain a competence rating. We will develop this as part of this project.
 
Literature
1.
IOM (Institute of Medicine). Psychosocial interventions for mental and substance use disorders: A framework for establishing evidence-based standards. Washington: The National Academies Press; 2015.
2.
Perepletchikova F, Treat TA, Kazdin AE. Treatment integrity in psychotherapy research: Analysis of the studies and examination of the associated factors. J Consult Clin Psych. 2007;75:829–41.CrossRef
3.
McLeod BD, Southam-Gerow MA, Tully CB, Rodriguez A, Smith MM. Making a case for treatment integrity as a psychosocial treatment quality indicator for youth mental health care. Clin Psychol-Sci Pr. 2013;20:14–32.CrossRef
4.
Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2011;38:4–23.CrossRefPubMed
5.
Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M. Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38:65–76.CrossRefPubMed
6.
Southam-Gerow MA, McLeod BD. Advances in applying treatment integrity research for dissemination and implementation science: Introduction to special issue. Clin Psychol-Sci Pr. 2013;20:1–13.CrossRef
7.
Schoenwald SK, Garland AF, Chapman JE, Frazier SL, Sheidow AJ, Southam-Gerow MA. Toward the effective and efficient measurement of implementation fidelity. Adm Policy Ment Health. 2011;38:32–43.CrossRefPubMedPubMedCentral
8.
Sherill J. Pragmatic Strategies for Assessing Psychotherapy Quality in Practice. Washington: National Institutes of Mental Health; 2015.
9.
Schoenwald SK, Garland AF. A review of treatment adherence measurement methods. Psychol Assess. 2013;25:146–56.CrossRefPubMed
10.
Beidas RS, Cross W, Dorsey S. Show me, don’t tell me: Behavioral rehearsal as a training and analogue fidelity tool. Cogn Behav Pract. 2014;21:1–11.CrossRefPubMed
11.
Hurlburt MS, Garland AF, Nguyen K, Brookman-Frazee L. Child and family therapy process: Concordance of therapist and observational perspectives. Adm Policy Ment Health. 2010;37:230–44.CrossRefPubMed
12.
Cunnington JP, Hanna E, Turnhbull J, Kaigas TB, Norman GR. Defensible assessment of the competency of the practicing physician. Acad Med. 1997;72:9–12.CrossRefPubMed
13.
Goulet F, Jacques A, Gagnon R, Racette P, Sieber W. Assessment of family physicians’ performance using patient charts: Interrater reliability and concordance with chart-stimulated recall interview. Eval Health Prof. 2007;30:376–92.CrossRefPubMed
14.
Miller PA, Nayer M, Eva KW. Psychometric properties of a peer-assessment program to assess continuing competence in physical therapy. Phys Ther. 2010;90:1026–38.CrossRefPubMed
15.
Salvatori P, Simonavicius N, Moore J, Rimmer G, Patterson M. Meeting the challenge of assessing clinical competence of occupational therapists within a program management environment. Can J Occup Ther. 2008;75:51–60.CrossRefPubMed
16.
Jennett P, Affleck L. Chart audit and chart stimulated recall as methods of needs assessment in continuing professional health education. J Contin Educ Health Prof. 1998;18:163–71.CrossRef
17.
Chambless DL, Ollendick TH. Empirically supported psychological interventions: Controversies and evidence. Annu Rev Psychol. 2001;52:685–716.CrossRefPubMed
18.
Guerra CE, Jacobs SE, Holmes JH, Shea JA. Are physicians discussing prostate cancer screening with their patients and why or why not? A pilot study. J Gen Intern Med. 2007;22:901–7.CrossRefPubMedPubMedCentral
19.
McLeod BD, Weisz JR. The Therapy Process Observational Coding System for Child Psychotherapy-Strategies scale. J Clin Child Adolesc. 2010;39:436–43.CrossRef
20.
Weersing VR, Weisz JR, Donenberg GR. Development of the Therapy Procedures Checklist: A therapist-report measure of technique use in child and adolescent treatment. J Clin Child Adolesc. 2002;31:168–80.CrossRef
21.
22.
Carroll KM, Nich C, Sifry RL, Nuro KF, Frankforter TL, Ball SA, Fenton L, Rounsaville BJ. A general system for evaluating therapist adherence and competence in psychotherapy research in the addictions. Drug Alcohol Depend. 2000;57:225–38.CrossRefPubMed
23.
Stirman SW, Spokas M, Creed TA, Farabaugh DT, Bhar SS, Brown GK, Perivoliotis D, Grant PM, Beck AT. Training and consultation in evidence-based psychosocial treatments in public mental health settings: The ACCESS model. Prof Psychol Res Pr. 2010;41:48–56.CrossRefPubMedPubMedCentral
24.
Cohen J, Mannarino AP. Disseminating and implementing trauma-focused CBT in community settings. Trauma Violence Abuse. 2008;9:214–26.CrossRefPubMed
25.
Beidas RS, Aarons G, Barg F, Evans A, Hadley T, Hoagwood K, Marcus S, Schoenwald S, Walsh L, Mandell DS. Policy to implementation: Evidence-based practice in community mental health—study protocol. Implement Sci. 2013;8:38.CrossRefPubMedPubMedCentral
26.
Beidas R, Marcus S, Aarons GA, Hoagwood K, Schoenwald S, Evans A, Hurford M, Hadley T, Barg F, Walsh L, et al. Predictors of community therapists’ use of therapy techniques in a large public mental health system. JAMA Pediatr. in press.
27.
Peris T, Compton S, Kendall PC, Birmaher B, Sherill J, March J, Gosch E, Ginsburg G, Rynn M, McCracken J, et al. Trajectories of change in youth anxiety during cognitive behavior therapy. J Consult Clin Psych. in press.
28.
Rice ME, Harris GT. Comparing effect sizes in follow-up studies: ROC Area, Cohen’s d, and r. Law Human Behav. 2005;29:615–20.CrossRef
29.
Drummond M, Sculpher M. Common methodological flaws in economic evaluations. Med Care. 2005;43:5–14.CrossRefPubMed
30.
French MT, McGeary KA. Estimating the economic cost of substance abuse treatment. Health Econ. 1997;6:539–44.CrossRefPubMed
31.
French MT, Roebuck MC, Dennis ML, Diamond G, Godley SH, Tims F, Webb C, Herrell JM. The economic cost of outpatient marijuana treatment for adolescents: Findings from a multi-site field experiment. Addiction. 2002;97:84–97.CrossRefPubMed
32.
French MT, Roebuck MC, Dennis ML, Godley SH, Liddle HA, Tims FM. Outpatient marijuana treatment for adolescents: Economic evaluation of a multisite field experiment. Eval Rev. 2003;27:421–59.CrossRefPubMed
33.
French MT, Roebuck MC, McLellan AT. Cost estimation when time and resources are limited: The Brief DATCAP. J Subst Abuse Treat. 2004;27:187–93.CrossRefPubMed
34.
French MT, Salome HJ, Sindelar JL, McLellan AT. Benefit-cost analysis of addiction treatment: Methodological guidelines and empirical application using the DATCAP and ASI. Health Serv Res. 2002;37:433–55.CrossRefPubMedPubMedCentral
35.
McCollister KE, French MT, Sheidow AJ, Henggeler SW, Halliday-Boykins CA. Estimating the differential costs of criminal activity for juvenile drug court participants: Challenges and recommendations. J Behav Health Ser R. 2009;36:111–26.CrossRef
36.
Roebuck MC, French MT, McLellan AT. DATStats: results from 85 studies using the Drug Abuse Treatment Cost Analysis Program. J Subst Abuse Treat. 2003;25:51–7.CrossRefPubMed
37.
Salome HJ, French MT. Using cost and financing instruments for economic evaluation of substance abuse treatment services. Recent Devel Alcohol. 2001;15:253–69.
38.
Salome HJ, French MT, Miller M, McLellan AT. Estimating the client costs of addiction treatment: First findings from the client drug abuse treatment cost analysis program (Client DATCAP). Drug Alcohol Depend. 2003;71:195–206.CrossRefPubMed
39.
Sindelar JL, Jofre-Bonet M, French MT, McLellan AT. Cost-effectiveness analysis of addiction treatment: Paradoxes of multiple outcomes. Drug Alcohol Depend. 2004;73:41–50.CrossRefPubMed
40.
41.
Glasgow RE, Riley WT. Pragmatic measures: What they are and why we need them. Am J Prev Med. 2013;45:237–43.CrossRefPubMed
42.
Accurso EC, Taylor RM, Garland AF. Evidence-based practices addressed in community-based children’s mental health clinical supervision. Train Educ Prof Psychol. 2011;5:88–96.PubMedPubMedCentral
43.
Schoenwald SK, Chapman JE, Kelleher K, Hoagwood KE, Landsverk J, Stevens J, Glisson C, Rolls-Reutz J, Health RNoYM. A survey of the infrastructure for children’s mental health services: Implications for the implementation of empirically supported treatments (ESTs). Adm Policy Ment Health. 2008;35:84–97.CrossRefPubMed
44.
Chambers DA, Azrin ST. Research and services partnerships: A fundamental component of dissemination and implementation research. Psychiatr Serv. 2013;64:509–11.CrossRefPubMed
45.
46.
Cialdini R. Influence: Science and Practice. Boston: Allyn & Bacon; 2001.
47.
Fishbein M, Cappella JN. The role of theory in developing effective health communications. J Commun. 2006;56:S1–S17.CrossRef
48.
Fishbein M, Ajzen I. Predicting and Changing Behavior: The Reasoned Action Approach. New York: Psychology Press; 2010.
49.
Guest G, Bunce A, Johnson L. How many interviews are enough? An experiment with data saturation and variability. Field Method. 2006;18:59–82.CrossRef
50.
Beatty P, Willis G. Research synthesis: The practice of cognitive interviewing. Public Opin Q. 2007;71:287–311.CrossRef
51.
Middlestadt S. Beliefs underlying eating better and moving more: Lessons learned from comparative salient belief elicitations. Ann Am Acad Polit SS. 2012;640:81–100.CrossRef
Metadata
Title
A randomized trial to identify accurate and cost-effective fidelity measurement methods for cognitive-behavioral therapy: project FACTS study protocol
Authors
Rinad S. Beidas
Johanna Catherine Maclean
Jessica Fishman
Shannon Dorsey
Sonja K. Schoenwald
David S. Mandell
Judy A. Shea
Bryce D. McLeod
Michael T. French
Aaron Hogue
Danielle R. Adams
Adina Lieberman
Emily M. Becker-Haimes
Steven C. Marcus
Publication date
01-12-2016
Publisher
BioMed Central
Published in
BMC Psychiatry / Issue 1/2016
Electronic ISSN: 1471-244X
DOI
https://doi.org/10.1186/s12888-016-1034-z

Other articles of this Issue 1/2016

BMC Psychiatry 1/2016 Go to the issue

Research article

Quality of life as a vulnerability and recovery factor in eating disorders: a community-based study

Research article

Pathway to mental health recovery: a qualitative and quantitative study on the needs of Chinese psychiatric inpatients

Research article

Determinants of help-seeking behavior in depression: a cross-sectional study

Research article

Sociodemographic, clinical, and psychosocial factors associated with depression among type 2 diabetic outpatients in Black Lion General Specialized Hospital, Addis Ababa, Ethiopia: a cross-sectional study

Research article

Benzodiazepine prescription for patients in treatment for drug use disorders: a nationwide cohort study in Denmark, 2000–2010

Research article

Resilience in rural left-behind middle school students in Yunyang county of the Three Gorges area in China: a prospective cohort study