Top

BMC Health Services Research

Published in:

Open Access 01-12-2018 | Research article

Qualitative approaches to use of the RE-AIM framework: rationale and methods

Authors: Jodi Summers Holtrop, Borsika A. Rabin, Russell E. Glasgow

Published in: BMC Health Services Research | Issue 1/2018

Abstract

Background

There have been over 430 publications using the RE-AIM model for planning and evaluation of health programs and policies, as well as numerous applications of the model in grant proposals and national programs. Full use of the model includes use of qualitative methods to understand why and how results were obtained on different RE-AIM dimensions, however, recent reviews have revealed that qualitative methods have been used infrequently. Having quantitative and qualitative methods and results iteratively inform each other should enhance understanding and lessons learned.

Methods

Because there have been few published examples of qualitative approaches and methods using RE-AIM for planning or assessment and no guidance on how qualitative approaches can inform these processes, we provide guidance on qualitative methods to address the RE-AIM model and its various dimensions. The intended audience is researchers interested in applying RE-AIM or similar implementation models, but the methods discussed should also be relevant to those in community or clinical settings.

Results

We present directions for, examples of, and guidance on how qualitative methods can be used to address each of the five RE-AIM dimensions. Formative qualitative methods can be helpful in planning interventions and designing for dissemination. Summative qualitative methods are useful when used in an iterative, mixed methods approach for understanding how and why different patterns of results occur.

Conclusions

In summary, qualitative and mixed methods approaches to RE-AIM help understand complex situations and results, why and how outcomes were obtained, and contextual factors not easily assessed using quantitative measures.

Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322–7. PMID: 10474547CrossRefPubMedPubMedCentral

Klesges LM, Estabrooks PA, Dzewaltowski DA, et al. Beginning with the application in mind: designing and planning health behavior change interventions to enhance dissemination. Ann Beh Med. 2005;29(Suppl):66–75. PMID: 15921491CrossRef

Gaglio B, Shoup JA, Glasgow RE. The RE-AIM framework: a systematic review of use over time. Am J Public Health. 2013;103(6):e38–46. https://doi.org/10.2105/AJPH.2013.301299. PMID: 23597377CrossRefPubMedPubMedCentral

RE-AIM framework. http:/ww.re-aim.org.

Prescription for Health. http://www.prescriptionforhealth.org/.

Active for Life Program. http://www.activeforlife.info/.

Move Program. https://www.move.va.gov/.

Vinson CA, Stamatakis KA, Kerner JF. Dissemination and implementation research in community and public health settings. In: Brownson RC, Colditz GA, Proctor EK, editors. Dissemination and implementation research in health: translating science to practice. 2nd ed. New York, NY: Oxford University Press; 2017. p. 355–70.

Shoup JA, Gaglio B, Varda D, Glasgow RE. Network analysis of RE-AIM framework: chronology of the field and the connectivity of its contributors. Transl Beh Med. 2015;5(2):216–32. https://doi.org/10.1007/s13142-014-0300-1. PMID: 26029284CrossRef

10.

Kessler RS, Purcell EP, Glasgow RE, et al. What does it mean to "employ" the RE-AIM model? Eval Health Prof. 2013;36(1):44–66. https://doi.org/10.1177/0163278712446066. PMID: 22615498CrossRefPubMed

11.

Glasgow RE. What does it mean to be pragmatic? Pragmatic methods, measures, and models to facilitate research translation. Health Educ Behav. 2013;40(3):257–65. https://doi.org/10.1177/1090198113486805. PMID: 23709579CrossRefPubMed

12.

Harden SM, Gaglio B, Shoup JA, et al. Fidelity to and comparative results across behavioral interventions evaluated through the RE-AIM framework: a systematic review. Syst Rev. 2015;4:155. https://doi.org/10.1186/s13643-015-0141-0. PMID: 26547687CrossRefPubMedPubMedCentral

13.

Crabtree BF, Miller WL. Doing qualitative research. 2nd ed. Thousand Oaks, CA: Sage Publications; 1999.

14.

Green J, Thorogood N. Qualitative methods for health research. Thousand Oaks, CA: Sage Publications; 2014.

15.

Creswell J. Qualitative inquiry and research design: choosing among five approaches. 3rd ed. Thousand Oaks, CA: Sage Publications; 2013.

16.

Silverman D. Qualitative research. 3rd ed. Thousand Oaks, CA: Sage Publications; 2010.

17.

Glasgow RE, Estabrooks PE. Pragmatic Applications of RE-AIM for Health Care Initiatives in Community and Clinical Settings. Prev Chronic Dis. 2018;15:E02. https://doi.org/10.5888/pcd15.170271.

18.

Potworowski G, Green LA. Cognitive task analysis: methods to improve patient-centered medical home models by understanding and leveraging its knowledge work. Rockville, MD: Agency for Healthcare Research and Quality; 2013.

19.

Crandall B, Klein G, Hoffman RR. Working minds: a practitioner's guide to cognitive task analysis. Cambridge, MA: MIT Press; 2006.

20.

Ritzwoller DP, Glasgow RE, Sukhanova AY, et al. Economic analyses of the be fit be well program: a weight loss program for community health centers. J Gen Intern Med. 2013;28(12):1581–8. PMID: 23733374CrossRefPubMedPubMedCentral

21.

Ritzwoller DP, Sukhanova A, Gaglio B, et al. Costing behavioral interventions: a practical guide to enhance translation. Ann Behav Med. 2009;37(2):218–27. https://doi.org/10.1007/s12160-009-9088-5. PMID: 19291342CrossRefPubMed

22.

Bellg AJ, Borrelli B, Resnick B, et al. Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH behavior change consortium. Health Psychol. 2004;23(5):443–51. PMID: 15367063CrossRefPubMed

23.

Allen J, Linnan LA, Emmons KM. Fidelity and its relationship to implementation effectiveness, adaptations, and dissemination. In: Brownson R, Colditz GA, Proctor EK, editors. Dissemination and implementation research in health. Oxford: New York, NY; 2012.

24.

Chambers DA, Norton WE. The Adaptome: advancing the science of intervention adaptation Am J Prev Med 2016;51(4 Suppl 2):S124–S131. doi: https://doi.org/10.1016/j.amepre.2016.05.011. PMID: 27371105.

25.

Stirman SW, Miller CJ, Toder K, et al. Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implement Sci. 2013;8:65. https://doi.org/10.1186/1748-5908-8-65. PMID: 23758995CrossRefPubMedPubMedCentral

26.

Hall T, Holtrop JS, Dickinson LM, et al. Understanding adaptations to patient-centered medical home activities: the PCMH adaptations model. Trans Beh Med Submitted for publication. 2017. https://doi.org/10.1007/s13142-017-0511-3. [Epub ahead of print]

27.

Castro FG, Barrera M Jr, Martinez CR Jr. The cultural adaptation of prevention interventions: resolving tensions between fidelity and fit. Prev Sci. 2004;5(1):41–5. PMID: 15058911CrossRefPubMed

28.

Bauman AE, Cabassa LJ, Wiltsey SS. Adaptation in dissemination and implementation science. In: Brownson RC, Colditz G, Proctor EK, editors. Dissemination and implementation research in health. New York: Oxford University Press; 2017.

29.

Rhodes W, Ritzwoller DP, Glasgow RE. Stakeholder perspectives on costs and resource expenditures: addressing economic issues most relevant to patients, providers and clinics. 2018. In Press.

30.

Loudon K, Zwarenstein M, Sullivan F, et al. Making clinical trials more relevant: improving and validating the PRECIS tool for matching trial design decisions to trial purpose. Trials. 2013;14:115. https://doi.org/10.1186/1745-6215-14-115. PMID: 23782862CrossRefPubMedPubMedCentral

31.

Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10:53. https://doi.org/10.1186/s13012-015-0242. PMID: 25895742CrossRefPubMedPubMedCentral

32.

Damschroder LJ, Aron DC, Keith RE, et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50. https://doi.org/10.1186/1748-5908-4-50. PMID: 19664226CrossRefPubMedPubMedCentral

33.

CFIR Technical Assistance Website. http://cfirguide.org/

34.

Feldstein A, Glasgow RE. A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice. Jt Comm J Qual Patient Saf. 2008;34(4):228–43. PMID: 18468362CrossRefPubMed

35.

Tomoaia-Cotisel A, Scammon DL, Waitzman NJ, et al. Context matters: the experience of 14 research teams in systematically reporting contextual factors important for practice change. Ann Fam Med. 2013;11(Suppl 1):S115–23. https://doi.org/10.1370/afm.1549. PMID: 23690380CrossRefPubMedPubMedCentral

36.

Normalization process theory on-line users’ manual, toolkit and NoMAD instrument. http://normalizationprocess.org/.

37.

May CR, Mair F, Finch T, et al. Development of a theory of implementation and integration: normalization process theory. Implement Sci. 2009;4:29. https://doi.org/10.1186/1748-5908-4-29. PMID: 19460163CrossRefPubMedPubMedCentral

38.

Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013;8:117. https://doi.org/10.1186/1748-5908-8-117. PMID: 24088228CrossRefPubMedPubMedCentral

39.

Tabak R, Khoong EC, Chambers DA, et al. Bridging research and practice: models for dissemination and implementation research. Am J Prev Med. 2012;43(3):337–50. https://doi.org/10.1016/j.amepre.2012.05.024. PMID: 22898128CrossRefPubMedPubMedCentral

40.

Dissemination and implementation research models. http://www.dissemination-implementation.org/.

41.

Wagner EH, Austin BT, Von Korff M. Organizing care for patients with chronic illness. Milbank Q. 1996;74(4):511–44. PMID: 8941260CrossRefPubMed

42.

Donabedian A. Evaluating the quality of medical care. Millbank Q. 1966;44:Suppl 166–206. PMID: 16279964.

43.

Bodenheimer T, Ghorob A, Willard-Grace R, et al. The 10 building blocks of high-performing primary care. Ann Fam Med. 2014;12(2):166–71. https://doi.org/10.1370/afm.1616. PMID: 24615313CrossRefPubMedPubMedCentral

44.

Planning and Evaluation Questions for Initiatives Intended to Produce Public Health Impact. Available from http://re-aim.org/resources-and-tools/self-rating-quiz/. Accessed January 7, 2018.

45.

Ory M, Altpeter M, Belza B, et al. Perceived utility of the RE-AIM framework for health promotion/disease prevention initiatives for older adults: a case study from the U.S. evidence-based disease prevention initiative. Front Public Health. 2015;2:143. https://doi.org/10.3389/fpubh.2014.00143. PMID: 25964897PubMedPubMedCentral

46.

Voorberg WH, VJJM B, Tummers LG. A systematic review of co-creation and co-production: embarking on the social innovation journey. Public Manage Rev. 2014:1333–57. https://doi.org/10.1080/14719037.2014.930505.

47.

Wozniak L, Soprovich A, Rees S, et al. Contextualizing the effectiveness of a collaborative care model for primary care patients with diabetes and depression (teamcare): a qualitative assessment using RE-AIM. Can J Diabetes. 2015;39(Suppl 3):S83–91. https://doi.org/10.1016/j.jcjd.2015.05.004. PMID: 26227866CrossRefPubMed

48.

Minkler M, Salvatore AL. Participatory approaches for study design and analysis in dissemination and implementation research. In: Brownson R, Colditz G, Proctor E, editors. Dissemination and implementation research in health: translating science into practice. New York, NY: Oxford University Press; 2012.

49.

Flanagan J. The critical incident technique. Psychol Bull. 1954;51(4):327–58.CrossRefPubMed

50.

QUALRIS workgroup: qualitative research in implementation science. Unpublished report, 2017. https://researchtoreality.cancer.gov/learning-communities/qualris.

51.

Hamilton AB. Qualitative methods in rapid turn-around Health Serv Res Dec 11, 2013. https://www.hsrd.research.va.gov/for_researchers/cyber_seminars/archives/780-notes.pdf.

52.

Creswell JW, Klassen AC, Plano C, et al. Best practices for mixed methods research in the health sciences. Bethesda, MD: Office of the Behavioral and Social Sciences Research, National Institutes of Health; 2011.CrossRef

53.

Curry LA, Krumholz HM, O’Cathain A, et al. Mixed methods in biomedical and health services research. Circ Cardiovasc Qual Outcomes. 2013;6(1):119–23. https://doi.org/10.1161/CIRCOUTCOMES.112.967885. PMID: 23322807CrossRefPubMedPubMedCentral

54.

Plano Clark VL. The adoption and practice of mixed methods: U.S. trends in federally funded health-related research. Qual Inq. 2010;16:428–40.CrossRef

55.

Miller WL, Crabtree BF, Harrison MI, et al. Integrating mixed methods in health services and delivery system research. Health Serv Res. 2013;48(6 Pt 2):2125–33. https://doi.org/10.1111/1475-6773.12123. PMID: 24279834CrossRefPubMedPubMedCentral

56.

Palinkas LA, Cooper BR. Mixed methods evaluation in dissemination and implementation science. In: Brownson RC, Colditz G, Proctor EK, editors. Dissemination and implementation research in health. New York: Oxford University Press; 2017.

57.

Jr BM, Castro FG, Strycker LA, et al. Cultural adaptations of behavioral health interventions: a progress report. J Consult Clin Psychol. 2013;81(2):196–205. https://doi.org/10.1037/a0027085. PMID: 22289132CrossRef

58.

Estabrooks P, You W, Hedrick V, et al. A pragmatic examination of active and passive recruitment methods to improve the reach of community lifestyle programs: the talking health trial. Int J Behav Nutr Phys Act. 2017;14(1):7. https://doi.org/10.1186/s12966-017-0462-6. PMID: 28103935CrossRefPubMedPubMedCentral

Title: Qualitative approaches to use of the RE-AIM framework: rationale and methods
Authors: Jodi Summers Holtrop
Borsika A. Rabin
Russell E. Glasgow
Publication date: 01-12-2018
Publisher: BioMed Central
Published in: BMC Health Services Research / Issue 1/2018
Electronic ISSN: 1472-6963
DOI: https://doi.org/10.1186/s12913-018-2938-8

Keynote webinar | Spotlight on sleep in brain health

Springer Medicine

Qualitative approaches to use of the RE-AIM framework: rationale and methods

Abstract

Background

Methods

Results

Conclusions

Keynote webinar | Spotlight on sleep in brain health

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2018

Unraveling the drivers of regional variation in healthcare spending by analyzing prevalent chronic diseases

Vendor of choice and the effectiveness of policies to promote health information exchange

Correction to: The impact of patient advisors on healthcare outcomes: a systematic review

The impact of health literacy on shared decision making before elective surgery: a propensity matched case control analysis

Exploring the networking behaviors of hospital organizations

Results from the CLUES study: a cluster randomized trial for the evaluation of cardiovascular guideline implementation in primary care in Spain