Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
Implementation Science
Published in: Implementation Science 1/2010

Open Access 01-12-2010 | Methodology

Improving benchmarking by using an explicit framework for the development of composite indicators: an example using pediatric quality of care

Authors: Jochen Profit, Katri V Typpo, Sylvia J Hysong, LeChauncy D Woodard, Michael A Kallen, Laura A Petersen

Published in: Implementation Science | Issue 1/2010

Login to get access

Abstract

Background

The measurement of healthcare provider performance is becoming more widespread. Physicians have been guarded about performance measurement, in part because the methodology for comparative measurement of care quality is underdeveloped. Comprehensive quality improvement will require comprehensive measurement, implying the aggregation of multiple quality metrics into composite indicators.

Objective

To present a conceptual framework to develop comprehensive, robust, and transparent composite indicators of pediatric care quality, and to highlight aspects specific to quality measurement in children.

Methods

We reviewed the scientific literature on composite indicator development, health systems, and quality measurement in the pediatric healthcare setting. Frameworks were selected for explicitness and applicability to a hospital-based measurement system.

Results

We synthesized various frameworks into a comprehensive model for the development of composite indicators of quality of care. Among its key premises, the model proposes identifying structural, process, and outcome metrics for each of the Institute of Medicine's six domains of quality (safety, effectiveness, efficiency, patient-centeredness, timeliness, and equity) and presents a step-by-step framework for embedding the quality of care measurement model into composite indicator development.

Conclusions

The framework presented offers researchers an explicit path to composite indicator development. Without a scientifically robust and comprehensive approach to measurement of the quality of healthcare, performance measurement will ultimately fail to achieve its quality improvement goals.
Appendix
Available only for authorised users
Literature
1.
2.
Schoen C, Davis K, How SKH, Schoenbaum SC: U.S. health system performance: a national scorecard. Health Aff. 2006, w457-w475. 10.1377/hlthaff.25.w457.
3.
Lindenauer PK, Remus D, Roman S, Rothberg MB, Benjamin EM, Ma A, Bratzler DW: Public reporting and pay for performance in hospital quality improvement. N Engl J Med. 2007, 365: 486-496. 10.1056/NEJMsa064964.CrossRef
4.
Jencks SF, Huff ED, Cuerdon T: Change in the quality of care delivered to Medicare beneficiaries, 1998-1999 to 2000-2001. JAMA. 2003, 289: 305-312. 10.1001/jama.289.3.305.CrossRefPubMed
5.
Epstein AJ: Do cardiac surgery report cards reduce mortality? Assessing the evidence. Med Care Res Rev. 2006, 63: 403-426. 10.1177/1077558706288831.CrossRefPubMed
6.
Grossbart SR: What's the return? Assessing the effect of "pay-for-performance" initiatives on the quality of care delivery. Med Care Res Rev. 2006, 63: 29S-48. 10.1177/1077558705283643.CrossRefPubMed
7.
Petra E, Varughese P, Epifania L, Buneo L, Scarfone K: Use of quality index tracking to drive improvement in clinical outcomes. Nephrology News and Issues. 2006, 20: 67-83.PubMed
8.
9.
Australian Institute of Health and Welfare: Key national indicators of children's health, development and wellbeing: indicator framework for A picture of Australia's children 2009. Canberra. 2008
10.
11.
Petersen LA, Woodard LD, Urech T, Daw C, Sookanan S: Does pay-for-performance improve the quality of health care?. Ann Intern Med. 2006, 145: 265-272.CrossRefPubMed
12.
Profit J, Zupancic JA, Gould JB, Petersen LA: Implementing pay-for-performance in the neonatal intensive care unit. Pediatrics. 2007, 119: 975-982. 10.1542/peds.2006-1565.CrossRefPubMedPubMedCentral
13.
Profit J, Petersen LA: Pay for performance is growing up. Arch Pediatr Adolesc Med. 2007, 161: 713-714. 10.1001/archpedi.161.7.713.CrossRefPubMed
14.
Jacobs R, Goddard M, Smith PC: How robust are hospital ranks based on composite performance measures?. Med Care. 2005, 43: 1177-1184. 10.1097/01.mlr.0000185692.72905.4a.CrossRefPubMed
15.
Nardo M, Saisana M, Saltelli A, Tarantolo S, Hoffman A, Giovanini E: Handbook on constructing composite indicators: methodology and user guide. 2005, Paris, France: OECD Publishing, full_text.CrossRef
16.
Rosenthal GE: Weak associations between hospital mortality rates for individual diagnoses: implications for profiling hospital quality. Am J Public Health. 1997, 87: 429-433. 10.2105/AJPH.87.3.429.CrossRefPubMedPubMedCentral
17.
Wilson IB, Landon BE, Marsden PV, Hirschhorn LR, McInnes K, Ding L, Cleary PD: Correlations among measures of quality in HIV care in the United States: cross sectional study. BMJ. 2007, 335: 1085-1091. 10.1136/bmj.39364.520278.55.CrossRefPubMedPubMedCentral
18.
Schauffler HH, Mordavsky JK: Consumer reports in health care: do they make a difference?. Annual Review of Public Health. 2001, 22: 69-89. 10.1146/annurev.publhealth.22.1.69.CrossRefPubMed
19.
Williams SC, Koss RG, Morton DJ, Loeb JM: Performance of top-ranked heart care hospitals on evidence-based process measures. Circulation. 2006, 114: 558-564. 10.1161/CIRCULATIONAHA.105.600973.CrossRefPubMed
20.
21.
Asch SM, McGlynn EA, Hogan MM, Hayward RA, Shekelle P, Rubenstein L, Keesey J, Adams J, Kerr EA: Comparison of quality of care for patients in the Veterans Health Administration and patients in a national sample. Ann Intern Med. 2004, 141: 938-945.CrossRefPubMed
22.
Sexton JB, Helmreich RL, Neilands TB, Rowan K, Vella K, Boyden J, Roberts PR, Thomas EJ: The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res. 2006, 6: 44-10.1186/1472-6963-6-44.CrossRefPubMedPubMedCentral
23.
24.
Mattke S, Epstein AM, Leatherman S: The OECD Health Care Quality Indicators Project: history and background. Int J Qual Health Care. 2006, 18: 1S-4. 10.1093/intqhc/mzl019.CrossRef
25.
Brand DA, Saisana M, Rynn LA, Pennoni F, Lowenfels AB: Comparative analysis of alcohol control policies in 30 countries. PLoS Medicine. 2007, 4: e151-10.1371/journal.pmed.0040151.CrossRefPubMedPubMedCentral
26.
Arah OA, Westert GP, Hurst J, Klazinga NS: A conceptual framework for the OECD Health Care Quality Indicators Project. Int J Qual Health Care. 2006, 18: 5S-13. 10.1093/intqhc/mzl024.CrossRef
27.
Roberts MJ, Hsiao W, Berman P, Reich MR: Getting Health Reform Right: A Guide To Improving Performance And Equity. 2003, Oxford: Oxford University Press
28.
Institute of Medicine: Crossing The Quality Chasm: A New Health System For The 21st Century. 2001, Washington, DC: National Academy Press
29.
Donabedian A: Evaluating the quality of medical care. Milbank Mem Fund Q. 1966, 44: 166S-206. 10.2307/3348969.CrossRef
30.
Jha AK: Measuring hospital quality: what physicians do? How patients fare? Or both?. JAMA. 2006, 296: 95-97. 10.1001/jama.296.1.95.CrossRefPubMed
31.
32.
33.
34.
Brick JM, Kalton G: Handling missing data in survey research. Stat Methods Med Res. 1996, 5: 215-238. 10.1177/096228029600500302.CrossRefPubMed
35.
Duffy ME: Handling missing data: a commonly encountered problem in quantitative research. Clin Nurse Spec. 2006, 20: 273-276. 10.1097/00002800-200611000-00005.CrossRefPubMed
36.
Rubin DB: Multiple imputation after 18+ years. Journal of the American Statistical Association. 1996, 91: 473-489. 10.2307/2291635.CrossRef
37.
Rubin DB: Multiple Imputation for Nonresponse in Surveys. 1987, New York: John WileyCrossRef
38.
Bobko P, Roth PL, Buster MA: The usefulness of unit weights in creating composite scores: a literature review, application to content validity, and meta-analysis. Organizational Research Methods. 2007, 10: 689-709. 10.1177/1094428106294734.CrossRef
39.
Brand DA, Saisana M, Rynn LA, Pennoni F, Lowenfels AB: Comparative analysis of alcohol control policies in 30 countries. PLoS Med. 2007, 4: e151-10.1371/journal.pmed.0040151.CrossRefPubMedPubMedCentral
40.
Nolan TP, Berwick DMM: All-or-none measurement raises the bar on performance. JAMA. 2006, 295: 1168-1170. 10.1001/jama.295.10.1168.CrossRefPubMed
41.
Hayward RA: All-or-nothing treatment targets make bad performance measures. Am J Manag Care. 2007, 13: 126-128.PubMed
42.
Saisana M, Saltelli A, Tarantola S: Uncertainty and sensitivity analysis techniques as tools for the quality assessment of composite indicators. Journal of the Royal Statistical Society Series A. 2005, 168: 307-323.CrossRef
43.
Parry GJ, Gould CR, McCabe CJ, Tarnow-Mordi WO: Annual league tables of mortality in neonatal intensive care units: longitudinal study. International Neonatal Network and the Scottish Neonatal Consultants and Nurses Collaborative Study Group. BMJ. 1998, 316: 1931-1935.CrossRefPubMedPubMedCentral
44.
45.
Chien AT, Dudley RA: Pay-for-performance in pediatrics: proceed with caution. Pediatrics. 2007, 120: 186-188. 10.1542/peds.2007-1158.CrossRefPubMed
46.
Peerzada JM, Richardson DK, Burns JP: Delivery room decision-making at the threshold of viability. The Journal of Pediatrics. 2004, 145: 492-498. 10.1016/j.jpeds.2004.06.018.CrossRefPubMed
47.
Varni JW, Limbers CA, Burwinkle TM: Parent proxy-report of their children's health-related quality of life: an analysis of 13,878 parents' reliability and validity across age subgroups using the PedsQL 4.0 Generic Core Scales. Health Qual Life Outcomes. 2007, 5: 2-10.1186/1477-7525-5-2.CrossRefPubMedPubMedCentral
48.
Varni JW, Limbers CA, Burwinkle TM: How young can children reliably and validly self-report their health-related quality of life?: an analysis of 8,591 children across age subgroups with the PedsQL 4.0 Generic Core Scales. Health Qual Life Outcomes. 2007, 5: 1-10.1186/1477-7525-5-1.CrossRefPubMedPubMedCentral
49.
Chesney M, Lindeke L, Johnson L, Jukkala A, Lynch S, Disch J, Densford KJ: Comparison of child and parent satisfaction ratings of ambulatory pediatric subspecialty care. J Pediatr Health Care. 2005, 19: 221-229. 10.1016/j.pedhc.2005.02.003.CrossRefPubMed
50.
Gold MR, Siegel JE, Russell LB, Weinstein MC: Cost-Effectiveness In Health And Medicine. 1996, Oxford, England: Oxford University Press
51.
Streiner DL, Saigal S, Burrows E, Stoskopf B, Rosenbaum P: Attitudes of parents and health care professionals toward active treatment of extremely premature infants. Pediatrics. 2001, 108: 152-157. 10.1542/peds.108.1.152.CrossRefPubMed
52.
Saigal S, Rosenbaum P, Stoskopf B, Hoult L, Furlong W, Feeny D, Burrows E, Torrance G: Comprehensive assessment of the health status of extremely low birth weight children at eight years of age: comparison with a reference group. J Pediatr. 1994, 125: 411-7. 10.1016/S0022-3476(05)83288-3.CrossRefPubMed
53.
Saigal S, Feeny D, Rosenbaum P, Furlong W, Burrows E, Stoskopf B: Self-perceived health status and health-related quality of life of extremely low-birth-weight infants at adolescence. JAMA. 1996, 276: 453-9. 10.1001/jama.276.6.453.CrossRefPubMed
54.
Saigal S, Rosenbaum PL, Feeny D, Burrows E, Furlong W, Stoskopf BL, Hoult L: Parental perspectives of the health status and health-related quality of life of teen-aged children who were extremely low birth weight and term controls. Pediatrics. 2000, 105: 569-574. 10.1542/peds.105.3.569.CrossRefPubMed
55.
Saigal S, Stoskopf B, Boyle M, Paneth N, Pinelli J, Streiner D, Goddeeris J: Comparison of current health, functional limitations, and health care use of young adults who were born with extremely low birth weight and normal birth weight. Pediatrics. 2007, 119: e562-e573. 10.1542/peds.2006-2328.CrossRefPubMed
56.
Zwicker JG, Harris SR: Quality of life of formerly preterm and very low birth weight infants from preschool age to adulthood: a systematic review. Pediatrics. 2008, 121: e366-e376. 10.1542/peds.2007-0169.CrossRefPubMed
57.
Halfon N, Hochstein M: Life course health development: an integrated framework for developing health, policy, and research. Milbank Q. 2002, 80: 433-79. 10.1111/1468-0009.00019.CrossRefPubMedPubMedCentral
58.
McCormick MC, Brooks-Gunn J, Buka SL, Goldman J, Yu J, Salganik M, Scott DT, Bennett FC, Kay LL, Bernbaum JC: Early intervention in low birth weight premature infants: results at 18 years of age for the Infant Health and Development Program. Pediatrics. 2006, 117: 771-780. 10.1542/peds.2005-1316.CrossRefPubMed
59.
Casalino LP, Alexander GC, Jin L, Konetzka RT: General internists' views on pay-for-performance and public reporting of quality scores: a national survey. Health Aff. 2007, 26: 492-499. 10.1377/hlthaff.26.2.492.CrossRef
60.
Beckman H, Suchman AL, Curtin K, Greene RA: Physician reactions to quantitative individual performance reports. Am J Med Qual. 2006, 21: 192-199. 10.1177/1062860606287577.CrossRefPubMed
61.
Young GJ, Meterko M, White B, Bokhour BG, Sautter KM, Berlowitz D, Burgess JF: Physician attitudes toward pay-for-quality programs: perspectives from the front line. Med Care Res Rev. 2007, 64: 331-343. 10.1177/1077558707300091.CrossRefPubMed
Metadata
Title
Improving benchmarking by using an explicit framework for the development of composite indicators: an example using pediatric quality of care
Authors
Jochen Profit
Katri V Typpo
Sylvia J Hysong
LeChauncy D Woodard
Michael A Kallen
Laura A Petersen
Publication date
01-12-2010
Publisher
BioMed Central
Published in
Implementation Science / Issue 1/2010
Electronic ISSN: 1748-5908
DOI
https://doi.org/10.1186/1748-5908-5-13

Other articles of this Issue 1/2010

Implementation Science 1/2010 Go to the issue

Systematic Review

Knowledge translation to fitness trainers: A systematic review

Methodology

The Rx for Change database: a first-in-class tool for optimal prescribing and medicines use

Short report

Pilot study evaluating the effects of an intervention to enhance culturally appropriate hypertension education among healthcare providers in a primary care setting

Research article

General practitioners' perceptions of the effectiveness of medical interventions: an exploration of underlying constructs

Study protocol

The impact of social networks on knowledge transfer in long-term care facilities: Protocol for a study

Study protocol

Study protocol: Cost effectiveness of two strategies to implement the NVOG guidelines on hypertension in pregnancy: An innovative strategy including a computerised decision support system compared to a common strategy of professional audit and feedback, a randomized controlled trial