Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
Implementation Science
Published in: Implementation Science 1/2012

Open Access 01-12-2012 | Debate

Methods and metrics challenges of delivery-system research

Authors: Jeffrey A Alexander, Larry R Hearld

Published in: Implementation Science | Issue 1/2012

Login to get access

Abstract

Background

Many delivery-system interventions are fundamentally about change in social systems (both planned and unplanned). This systems perspective raises a number of methodological challenges for studying the effects of delivery-system change--particularly for answering questions related to whether the change will work under different conditions and how the change is integrated (or not) into the operating context of the delivery system.

Methods

The purpose of this paper is to describe the methodological and measurement challenges posed by five key issues in delivery-system research: (1) modeling intervention context; (2) measuring readiness for change; (3) assessing intervention fidelity and sustainability; (4) assessing complex, multicomponent interventions; and (5) incorporating time in delivery-system models to discuss recommendations for addressing these issues. For each issue, we provide recommendations for how research may be designed and implemented to overcome these challenges.

Results and conclusions

We suggest that a more refined understanding of the mechanisms underlying delivery-system interventions (treatment theory) and the ways in which outcomes for different classes of individuals change over time are fundamental starting points for capturing the heterogeneity in samples of individuals exposed to delivery-system interventions. To support the research recommendations outlined in this paper and to advance understanding of the "why" and "how" questions of delivery-system change and their effects, funding agencies should consider supporting studies with larger organizational sample sizes; longer duration; and nontraditional, mixed-methods designs.
A version of this paper was prepared under contract with the Agency for Healthcare Research and Quality (AHRQ), US Department of Health and Human Services for presentation and discussion at a meeting on "The Challenge and Promise of Delivery System Research," held in Sterling, VA, on February 16-17, 2011. The opinions in the paper are those of the author and do not represent the views or recommendations of AHRQ or the US Department of Health and Human Services.1
Appendix
Available only for authorised users
Literature
1.
Harrison MI, Henriksen K, Hughes RG: Improving the Health Care Work Environment: Implications for Research, Practice, and Policy. JtComm J Qual Patient Saf. 2007, 33 (Supplement 1): 81-84.
2.
Institute of Medicine: To err is human: Building a safer health care system. 2000, Washington, DC: National Academies Press
3.
Institute of Medicine: Crossing the Quality Chasm: A New Health System for the 21st Century. 2001, Washington, DC: National Academy Press
4.
Pawson R, Tilley N: Realistic Evaluation. 1997, London: Sage Publications, Ltd
5.
6.
Johnson JK, Miller SH, Horowitz SD: Systems-Based Practice: Improving the Safety and Quality of Patient Care by Recognizing and Improving the Systems in Which We Work. Advances in Patient Safety: New Directions and Alternative Approaches. Vol 2: Culture and Redesign. Edited by: Henriksesn K, Battles JB, Keyes MA, Grady ML. 2008, Rockville, MD: Agency for Healthcare Research and Quality, Accessed December 15, 2011, [http://​www.​ahrq.​gov/​downloads/​pub/​advances2/​vol2/​Advances-Johnson_​90.​pdf]
7.
8.
9.
10.
11.
12.
Fuller BE, Rieckmann T, Nunes EV, Miller M, Arfken C, Edmundson E, McCarty D: Organizational readiness for change and opinions toward treatment innovations. J Subst Abuse Treat. 2007, 33 (2): 183-192. 10.1016/j.jsat.2006.12.026.CrossRefPubMedPubMedCentral
13.
Orwin RG: Assessing program fidelity in substance abuse health services research. Addiction. 2000, 95 (11): 309-327. 10.1080/09652140020004250.CrossRef
14.
English M, Nzinga J, Mbindyo P, Ayieko P, Irimu G, Mbaabu L: Explaining the effects of a multifaceted intervention to improve inpatient care in rural Kenyan hospitals - interpretation based on retrospective examination of data from participant observation, quantitative and qualitative studies. Implementation Science. 2011, 6: 1-10.1186/1748-5908-6-1. [http://​www.​implementationsc​ience.​com/​content/​6/​1/​124]CrossRef
15.
Brekke JS, Long JD, Nesbitt N, Sobel E: The impact of service characteristics on functional outcomes from community support programs for persons with schizophrenia: A growth curve analysis. J Consult ClinPsychol. 1997, 65 (3): 464-475.CrossRef
16.
Griffin M: Specifying organizational contexts: Systematic links between contexts and processes in organizational behavior. J Organ Behav. 2007, 28: 859-863. 10.1002/job.489.CrossRef
17.
Johns G: The essential impact of context on organizational behavior. AcadManag Rev. 2006, 31 (2): 386-408.
18.
Nutting PA, Miller WL, Crabtree BF, Jaen CR, Stewart EE, Stange KC: Initial lessons from the first national demonstration project on practice transformation to a patient-centered medical home. Ann Fam Med. 2009, 7 (3): 254-260. 10.1370/afm.1002.CrossRefPubMedPubMedCentral
19.
Rousseau DM, Fried Y: Location, location, location: Contextualizing organizational research. J Organ Behav. 2001, 22 (1): 1-13. 10.1002/job.78.CrossRef
20.
Hackman JR: Learning more by crossing levels: Evidence from airplanes, hospitals, and orchestras. J Organ Behav. 2003, 24: 905-922. 10.1002/job.226.CrossRef
21.
Raudenbush SW, Bryk AS: Hierarchical Linear Models: Applications and Data Analysis Methods. 2002, Thousand Oaks, CA: Sage
22.
Hedeker D, Gibbons RD, Flay BR: Random-effects regression models for clustered data with an example from smoking prevention research. J Consult ClinPsychol. 1994, 62 (4): 757-765.CrossRef
23.
Goldstein H: Multilevel Statistical Models. 1995, New York, NY: Halstead Press, 2
24.
Weiner BJ, Amick H, Lee SD: Conceptualization and measurement of organizational readiness for change: A review of the literature in health services research and other fields. Med Care Res Rev. 2008, 65 (4): 379-463. 10.1177/1077558708317802.CrossRefPubMed
25.
Nutting PA, Crabtree BF, Miller WL, Stewart EE, Stange KC, Jaen CR: Journey to the patient-centered medical home: A qualitative analysis of the experiences of practices in thenational demonstration project. Ann Fam Med. 2010, 8 (Suppl_1): S45-56.CrossRefPubMedPubMedCentral
26.
Dumas JE, Lynch AM, Laughlin JE, Smith EP, Prinz RJ: Promoting implementation fidelity: Conceptual issues, methods, and preliminary results from the Early Alliance prevention trial. Am J Prev Med. 2001, 20 (Supplement 1): 38-47.CrossRefPubMed
27.
Mowbray CT, Holter MC, Teague GB, Bybee D: Fidelity criteria: Development, measurement, and validation. Am J Eval. 2003, 24 (3): 315-340.CrossRef
28.
Rossi PH, Lipsey MW, Freeman HE: Evaluation: A Systematic Approach. 2004, Thousand Oaks, CA: Sage, 7
29.
Benn J, Burnett S, Parand A, Pinto A, Iskander S, Vincent C: Studying large-scale programmes to improve patient safety in whole care systems: Challenges for research. SocSci Med. 2009, 69: 1767-1776. 10.1016/j.socscimed.2009.09.051.CrossRef
30.
Alexander JA, Hearld LR, Jiang J, Fraser I: Increasing the relevance of research to health care managers: Hospital CEO imperatives for improving quality and lowering costs. Health Care Manage Rev. 2007, 32 (2): 150-159.CrossRefPubMed
31.
Creswell JW: Research design: Qualitative, Quantitative, and Mixed Methods Approaches. 2003, Thousand Oaks, CA: Sage, 2
32.
Creswell JW, Goodchild LF, Turner PP: Integrated qualitative and quantitative research: Epistemology, history, and designs. Higher Education: Handbook of Theory and Research. Edited by: Smart JC. 1996, New York, NY: Agathon Press, 90-136.
33.
Advances in Mixed-Method Evaluation: The Challenges and Benefits of Integrating Diverse Paradigms. Edited by: Green JC, Caracelli VJ. 1997, San Francisco, CA: Jossey-Bass
34.
Teddlie C, Tashakkori A: Major issues and controversies in the use of mixed methods in the social and behavioral sciences. Handbook on Mixed Methods in the Behavioral and Social Sciences. Edited by: Tashakkori A, Teddlie C. 2003, Thousand Oaks, CA: Sage, 3-50.
35.
36.
Collins LM, Graham JW: The effect of timing and spacing of observations in longitudinal studies of tobacco and other drug use: Temporal design considerations. Drug Alcohol Depend. 2002, 68: S85-S96.CrossRefPubMed
37.
Diez-Rouz A: Bringing context back into epidemiology: Variables and fallacies in multilevel analysis. Am J Public Health. 1998, 88: 216-222. 10.2105/AJPH.88.2.216.CrossRef
38.
Raudenbush SW: Comparing personal trajectories and drawing causal inferences from longitudinal data. Annu Rev Psychol. 2001, 52: 501-525. 10.1146/annurev.psych.52.1.501.CrossRefPubMed
39.
Chen HT: Theory-Driven Evaluations. 1990, Newbury Park, CA: Sage
40.
Bickman L: Using program theory in evaluation. New Directions for Program Evaluation. Edited by: Bickman L. 1987, San Francisco, CA: Jossey-Bass, 33: 5-18.
41.
Muthen B, Muthen L: The development of heavy drinking and alcohol-related problems from ages 18 to 37 in a U.S. national sample. J Stud Alcohol Drugs. 2000, 61 (2): 290-300.CrossRef
Metadata
Title
Methods and metrics challenges of delivery-system research
Authors
Jeffrey A Alexander
Larry R Hearld
Publication date
01-12-2012
Publisher
BioMed Central
Published in
Implementation Science / Issue 1/2012
Electronic ISSN: 1748-5908
DOI
https://doi.org/10.1186/1748-5908-7-15

Other articles of this Issue 1/2012

Implementation Science 1/2012 Go to the issue

Research

Use of health systems and policy research evidence in the health policymaking in eastern Mediterranean countries: views and practices of researchers

Systematic review

A systematic review of hand hygiene improvement strategies: a behavioural approach

Study protocol

How do surgeons decide to refer patients for adjuvant cancer treatment? Protocol for a qualitative study

Study protocol

Determinants of successful clinical networks: the conceptual framework and study protocol

Study protocol

Clinical governance implementation in a selected teaching emergency department: a systems approach

Research

Learning curves, taking instructions, and patient safety: using a theoretical domains framework in an interview study to investigate prescribing errors among trainee doctors