Top

Published in:

01-09-2016 | Original Research

Development and validation of the ASPIRE-VA coaching fidelity checklist (ACFC): a tool to help ensure delivery of high-quality weight management interventions

Authors: Laura J. Damschroder, MS, MPH, David E. Goodrich, EdD, Hyungjin Myra Kim, Sc.D, Robert Holleman, MPH, Leah Gillon, MSW, Susan Kirsh, MD, MPH, Caroline R. Richardson, MD, Lesley D. Lutes, PhD

Published in: Translational Behavioral Medicine | Issue 3/2016

Abstract

Practical and valid instruments are needed to assess fidelity of coaching for weight loss. The purpose of this study was to develop and validate the ASPIRE Coaching Fidelity Checklist (ACFC). Classical test theory guided ACFC development. Principal component analyses were used to determine item groupings. Psychometric properties, internal consistency, and inter-rater reliability were evaluated for each subscale. Criterion validity was tested by predicting weight loss as a function of coaching fidelity. The final 19-item ACFC consists of two domains (session process and session structure) and five subscales (sets goals and monitor progress, assess and personalize self-regulatory content, manages the session, creates a supportive and empathetic climate, and stays on track). Four of five subscales showed high internal consistency (Cronbach alphas > 0.70) for group-based coaching; only two of five subscales had high internal reliability for phone-based coaching. All five sub-scales were positively and significantly associated with weight loss for group- but not for phone-based coaching. The ACFC is a reliable and valid instrument that can be used to assess fidelity and guide skill-building for weight management interventionists.

Available only for authorised users

Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008; 41(3-4): 327-350. doi:10.1007/s10464-008-9165-0.CrossRefPubMed

Dane AV, Schneider BH. Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clin Psychol Rev. 1998; 18(1): 23-45.CrossRefPubMed

Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci IS. 2013; 8(1): 117. doi:10.1186/1748-5908-8-117.CrossRefPubMed

Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011; 38(2): 65-76. doi:10.1007/s10488-010-0319-7.CrossRef

Bellg AJ, Borrelli B, Resnick B, et al. Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH Behavior Change Consortium. Health Psychol. 2004; 23(5): 443-451.CrossRefPubMed

Borrelli B. The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. J Public Health Dent. 2011; 71(s1): S52-S63. doi:10.1111/j.1752-7325.2011.00233.x.CrossRefPubMedCentral

Faulkner MS. Intervention fidelity: ensuring application to practice for youth and families. J Spec Pediatr Nurs JSPN. 2012; 17(1): 33-40. doi:10.1111/j.1744-6155.2011.00305.x.CrossRefPubMed

Bopp M, Saunders RP, Lattimore D. The tug-of-war: fidelity versus adaptation throughout the health promotion program life cycle. J Prim Prev. 2013; 34(3): 193-207. doi:10.1007/s10935-013-0299-y.CrossRefPubMed

Borrelli B, Sepinwall D, Ernst D, et al. A new tool to assess treatment fidelity and evaluation of treatment fidelity across 10 years of health behavior research. J Consult Clin Psychol. 2005; 73(5): 852-860. doi:10.1037/0022-006x.73.5.852.CrossRefPubMed

10.

Hardeman W, Michie S, Fanshawe T, et al. Fidelity of delivery of a physical activity intervention: predictors and consequences. Psychol Health. 2007; 23(1): 11-24.CrossRef

11.

Abraham C, Michie S. A taxonomy of behavior change techniques used in interventions. Health Psychol. 2008; 27(3): 379-387. doi:10.1037/0278-6133.27.3.379.CrossRefPubMed

12.

Damschroder LJ, Hagedorn HJ. A guiding framework and approach for implementation research in substance use disorders treatment. Psychol Addict Behav J Soc Psychol Addict Beh. 2011; 25(2). doi:10.1037/a0022284.

13.

Cohen DJ, Crabtree BF, Etz RS, et al. Fidelity versus flexibility: translating evidence-based research into practice. Am J Prev Med. 2008; 35(5 Suppl): S381-S389. doi:10.1016/j.amepre.2008.08.005.CrossRefPubMed

14.

Wyatt Kaminski J, Valle L, Filene J, et al. A meta-analytic review of components associated with parent training program effectiveness. J Abnorm Child Psychol. 2008; 36(4): 567-589. doi:10.1007/s10802-007-9201-9.CrossRef

15.

Leblanc ES, O'Connor E, Whitlock EP, et al. Effectiveness of primary care-relevant treatments for obesity in adults: a systematic evidence review for the U.S. Preventive Services Task Force. Ann Intern Med. 2011; 155(7): 434-447. doi:10.7326/0003-4819-155-7-201110040-00006.CrossRefPubMed

16.

Lutes LD, Winett RA, Barger SD, et al. Small changes in nutrition and physical activity promote weight loss and maintenance: 3-month evidence from the ASPIRE Randomized Trial. nn. Behav Med. 2008; 35(3): 351-357.CrossRef

17.

Lutes LD, Daiss SR, Barger SD, et al. Small changes approach promotes initial and continued weight loss with a phone-based follow-up: nine-month outcomes from ASPIRES II. Am J Health Promot. 2012; 26(4): 235-238. doi:10.4278/ajhp.090706-QUAN-216.CrossRefPubMed

18.

Damschroder LJ, Lutes LD, Kirsh S, et al. Small-changes obesity treatment among veterans: 12-month outcomes. Am J Prev Med. 2014; 47(5): 541-553. doi:10.1016/j.amepre.2014.06.016.CrossRefPubMed

19.

Gearing RE, El-Bassel N, Ghesquiere A, et al. Major ingredients of fidelity: a review and scientific guide to improving quality of intervention research implementation. Clin Psychol Rev. 2011; 31(1): 79-88. doi:10.1016/j.cpr.2010.09.007.CrossRefPubMed

20.

Carroll C, Patterson M, Wood S, et al. A conceptual framework for implementation fidelity. Implement Sci IS. 2007; 2(1): 40.CrossRefPubMed

21.

Nelson MC, Cordray DS, Hulleman CS, et al. A procedure for assessing intervention fidelity in experiments testing educational and behavioral interventions. J Behav Health Serv Res. 2012; 39(4): 374-396. doi:10.1007/s11414-012-9295-x.CrossRefPubMed

22.

Lutes LD, Dinatale E, Goodrich DE, et al. A randomized trial of a small changes approach for weight loss in veterans: design, rationale, and baseline characteristics of the ASPIRE-VA trial. Contemp Clin Trials. 2013; 34(1): 161-172. doi:10.1016/j.cct.2012.09.007.CrossRefPubMed

23.

Kinsinger LS, Jones KR, Kahwati L, et al. Design and dissemination of the MOVE! Weight-management program for veterans. Prev Chronic Dis. 2009; 6(3): A98.PubMedPubMedCentral

24.

Hill JO. Can a small-changes approach help address the obesity epidemic? A report of the Joint Task Force of the American Society for Nutrition, Institute of Food Technologists, and International Food Information Council. Am J Clin Nutr. 2009; 89(2): 477-484.CrossRefPubMed

25.

Hill JO, Peters JC, Wyatt HR. Using the energy gap to address obesity: a commentary. J Am Diet Assoc. 2009; 109(11): 1848-1853. doi:10.1016/j.jada.2009.08.007.CrossRefPubMedPubMedCentral

26.

Schoenwald SK, Garland AF, Chapman JE, et al. Toward the effective and efficient measurement of implementation fidelity. Adm Policy Ment Health. 2011; 38(1): 32-43. doi:10.1007/s10488-010-0321-0.CrossRefPubMedPubMedCentral

27.

Colquhoun H, Leeman J, Michie S, et al. Towards a common terminology: a simplified framework of interventions to promote and integrate evidence into health practices, systems, and policies. Implement Sci. 2014; 9(1): 51.CrossRefPubMedPubMedCentral

28.

Resnicow K, McMaster F. Motivational Interviewing: moving from why to how with autonomy support. Int Behav Nutr Phys Act. 2012; 9: 19. doi:10.1186/1479-5868-9-19.CrossRef

29.

Moyers TB, Miller WR, Hendrickson SM. How does motivational interviewing work? Therapist interpersonal skill predicts client involvement within motivational interviewing sessions. J Consult Clin Psychol. 2005; 73(4): 590-598. doi:10.1037/0022-006X.73.4.590.CrossRefPubMed

30.

Resnicow K, McMaster F, Rollnick S. Action reflections: a client-centered technique to bridge the WHY-HOW transition in Motivational Interviewing. Behav Cogn Psychother. 2012; 40(4): 474-480. doi:10.1017/S1352465812000124.CrossRefPubMed

31.

Bauer MS, McBride L. Structured group psychotherapy for bipolar disorder: the life goals program. 2nd ed. New York: Springer Publishing Co.; 2003.

32.

Bauer MS, McBride L, Williford WO, et al. Collaborative care for bipolar disorder: part II. Impact on clinical outcome, function, and costs. Psychiatr Serv. 2006; 57(7): 937-945. doi:10.1176/appi.ps.57.7.937.CrossRefPubMed

33.

Kilbourne AM, Goodrich DE, Lai Z, et al. Life goals collaborative care for patients with bipolar disorder and cardiovascular disease risk. Psychiatr Serv. 2012; 63(12): 1234-1238. doi:10.1176/appi.ps.201100528.CrossRefPubMedPubMedCentral

34.

Kilbourne AM, Goodrich DE, Lai Z, et al. Randomized controlled trial to assess reduction of cardiovascular disease risk in patients with bipolar disorder: the self-management addressing heart risk trial (SMAHRT). J Clin Psychiatry. 2013; 74(7): e655-e662. doi:10.4088/JCP.12m08082.CrossRefPubMedPubMedCentral

35.

Kilbourne AM, Post EP, Nossek A, et al. Improving medical and psychiatric outcomes among individuals with bipolar disorder: a randomized controlled trial. Psychiatr Serv. 2008; 59(7): 760-768. doi:10.1176/appi.ps.59.7.760.CrossRefPubMed

36.

Waxmonsky J, Kilbourne AM, Goodrich DE, et al. Enhanced fidelity to treatment for bipolar disorder: results from a randomized controlled implementation trial. Psychiatr Serv. 2014; 65(1): 81-90. doi:10.1176/appi.ps.201300039.CrossRefPubMedPubMedCentral

37.

Desu MM, Raghavarao D. Sample size methodology. New York: Academic Press; 1990.

38.

Machin D, Campbell M, Fayers P, et al. Sample size tables for clinical studies. 2nd ed. Malden: Blackwell Science; 1997.

39.

Hahn GJ, Meeker WQ. Statistical Intervals. New York, NY: Wiley.

40.

Power Analysis & Sample Size (PASS) Software. Kaysville, UT: NCSS; 2005.

41.

Piette JD, Richardson C, Himle J, et al. A randomized trial of telephonic counseling plus walking for depressed diabetes patients. Med Care. 2011; 49(7): 641-648. doi:10.1097/MLR.0b013e318215d0c9.CrossRefPubMedPubMedCentral

42.

Young J, Beck A. Cognitive therapy scale rating manual. Philadelphia: University of Pennsylvania Psychotherapy Research Unit; 1980.

43.

Viera AJ, Garrett JM. Understanding interobserver agreement: the kappa statistic. Fam Med. 2005; 37(5): 360-363.PubMed

44.

StataCorp. Intercooled stata 13.1. 11th ed. College Station: StataCorp LP; 2013.

45.

Beehler GP, Funderburk JS, Possemato K, et al. Psychometric assessment of the primary care behavioral health provider adherence questionnaire (PPAQ). Transl Behav Med. 2013; 3(4): 379-391. doi:10.1007/s13142-013-0216-1.CrossRefPubMedPubMedCentral

46.

Mars T, Ellard D, Carnes D, Homer K, Underwood M, Taylor SJC. Fidelity in complex behaviour change interventions: a standardised approach to evaluate intervention integrity. BMJ Open. 2013; 3(11).

47.

Yamada J, Stevens B, Sidani S, et al. Content validity of a process evaluation checklist to measure intervention implementation fidelity of the EPIC intervention. Worldviews Evid-Based Nurs. 2010; 7(3): 158-164. doi:10.1111/j.1741-6787.2010.00182.x.CrossRefPubMed

48.

Breitenstein SM, Fogg L, Garvey C, et al. Measuring implementation fidelity in a community-based parenting intervention. Nurs Res. 2010; 59(3): 158-165. doi:10.1097/NNR.0b013e3181dbb2e2.CrossRefPubMedPubMedCentral

49.

Breitenstein SM, Gross D, Garvey CA, et al. Implementation fidelity in community-based interventions. Res Nurs Health. 2010; 33(2): 164-173. doi:10.1002/nur.20373.PubMedPubMedCentral

50.

Goode AD, Reeves MM, Eakin EG. Telephone-delivered interventions for physical activity and dietary behavior change: an updated systematic review. Am J Prev Med. 2012; 42(1): 81-88. doi:10.1016/j.amepre.2011.08.025.CrossRefPubMed

51.

Mowbray C, Holter M, Teague G, et al. Fidelity criteria: development, measurement, and validation. Am J Eval. 2003; 24: 315-340.CrossRef

52.

Schinckus L, Van den Broucke S, Housiaux M, et al. Assessment of implementation fidelity in diabetes self-management education programs: a systematic review. Patient Educ Couns. 2014; 96(1): 13-21. doi:10.1016/j.pec.2014.04.002.CrossRefPubMed

53.

Swanson E, Wanzek J, Haring C, Ciullo S, McCulley L. Intervention fidelity in special and general education research journals. J Spec Ed. 2011; 0022466911419516. doi:10.1177/0022466911419516.

Title: Development and validation of the ASPIRE-VA coaching fidelity checklist (ACFC): a tool to help ensure delivery of high-quality weight management interventions
Authors: Laura J. Damschroder, MS, MPH
David E. Goodrich, EdD
Hyungjin Myra Kim, Sc.D
Robert Holleman, MPH
Leah Gillon, MSW
Susan Kirsh, MD, MPH
Caroline R. Richardson, MD
Lesley D. Lutes, PhD
Publication date: 01-09-2016
Publisher: Springer US
Published in: Translational Behavioral Medicine / Issue 3/2016
Print ISSN: 1869-6716
Electronic ISSN: 1613-9860
DOI: https://doi.org/10.1007/s13142-015-0336-x

Keynote webinar | Spotlight on sleep in brain health

Springer Medicine

Development and validation of the ASPIRE-VA coaching fidelity checklist (ACFC): a tool to help ensure delivery of high-quality weight management interventions

Abstract

Keynote webinar | Spotlight on sleep in brain health

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 3/2016

Do Arabic weight-loss apps adhere to evidence-informed practices?

The weight loss blogosphere: an online survey of weight loss bloggers

The E-Coach technology-assisted care transition system: a pragmatic randomized trial

The effect of a locally adapted, secondary stroke risk factor self-management program on medication adherence among veterans with stroke/TIA

Society of Behavioral Medicine’s (SBM) position on emerging policy issues regarding electronic nicotine delivery systems (ENDS): A need for regulation

Erratum to: The effect of a locally adapted, secondary stroke risk factor self-management program on medication adherence among veterans with stroke