Skip to main content
Top
Published in: BMC Health Services Research 1/2012

Open Access 01-12-2012 | Research article

Clinicians’ and patients’ views of metrics of change derived from patient reported outcome measures (PROMs) for comparing providers’ performance of surgery

Authors: Zoe Hildon, Jenny Neuburger, Dominique Allwood, Jan van der Meulen, Nick Black

Published in: BMC Health Services Research | Issue 1/2012

Login to get access

Abstract

Background

Patient reported outcome measures (PROMs) are increasingly being used to compare the performance of health care providers. Our objectives were to determine the relative frequency of use of different metrics that can be derived from PROMs, explore clinicians’ and patients’ views of the options available, and make recommendations.

Methods

First a rapid review of the literature on metrics derived from two generic (EQ-5D and EQ-VAS) and three disease-specific (Oxford Hip Score; Oxford Knee Score; Aberdeen Varicose Vein Questionnaire) PROMs was conducted. Next, the findings of the literature review were mapped onto our typology of metrics to determine their relative frequency of use, Finally, seven group meetings with surgical clinicians (n = 107) and six focus groups with patients (n = 45) were held which were audio-taped, transcribed and analysed thematically.

Results

Only nine studies (9.3% of included papers) used metrics for comparing providers. These were derived from using either the follow-up PROM score (n = 3) or the change in score as an outcome (n = 5), both adjusted for pre-intervention score. There were no recorded uses of the proportion reaching a specified (‘good’) threshold and only two studies used the proportion reaching a minimally important difference (MID).
Surgical clinicians wanted multiple outcomes, with most support expressed for the mean change in score, perceiving it to be more interpretable; there was also some support for the MID. For patients it was apparent that rather than the science behind these measures, the most important aspects were the use of language that would make the metrics personally meaningful and linking the metric to a familiar scale.

Conclusions

For clinicians the recommended metrics are the mean change in score and the proportion achieving a MID, both adjusted for pre-intervention score. Both need to be clearly described and explained. For patients we recommend the proportion achieving a MID or proportion achieving a significant improvement in hip function, both adjusted for pre-intervention score.
Appendix
Available only for authorised users
Literature
2.
go back to reference Black N, Jenkinson C: How can patients' views of their care enhance quality improvement?. BMJ. 2009, 339: 202-6. Black N, Jenkinson C: How can patients' views of their care enhance quality improvement?. BMJ. 2009, 339: 202-6.
4.
go back to reference Robertson R, Thorlby R: Patient choice. The Kings Fund. 2008, 2008. Robertson R, Thorlby R: Patient choice. The Kings Fund. 2008, 2008.
6.
go back to reference Hildon Z, Allwood D, Black N: Impact of format and content of visual display of data on comprehension, choice and preference: a systematic review. Int J Qual Health Care. 2012, 24: 55-64. 10.1093/intqhc/mzr072.CrossRefPubMed Hildon Z, Allwood D, Black N: Impact of format and content of visual display of data on comprehension, choice and preference: a systematic review. Int J Qual Health Care. 2012, 24: 55-64. 10.1093/intqhc/mzr072.CrossRefPubMed
7.
go back to reference Hildon Z, Allwood D, Black N: Patients’ views of the format and content of statistical comparisons of healthcare performance. Patient Educ Couns. in press Hildon Z, Allwood D, Black N: Patients’ views of the format and content of statistical comparisons of healthcare performance. Patient Educ Couns. in press
8.
go back to reference Allwood D, Hildon Z, Black N: Clinicians’ views of formats of performance comparisons. J Eval Clin Pract. 2011, 10.1111/j.1365-2753.2011.01777.x. doi. Allwood D, Hildon Z, Black N: Clinicians’ views of formats of performance comparisons. J Eval Clin Pract. 2011, 10.1111/j.1365-2753.2011.01777.x. doi.
9.
go back to reference Brooks R, EuroQol Group: EuroQol: the current state of play. Health Policy. 1996, 37: 53-72. 10.1016/0168-8510(96)00822-6.CrossRefPubMed Brooks R, EuroQol Group: EuroQol: the current state of play. Health Policy. 1996, 37: 53-72. 10.1016/0168-8510(96)00822-6.CrossRefPubMed
10.
go back to reference Cheung K, Oemar M, Oppe M, Rabin R: EQ-5D User Guide: Basic Information on how to Use the EQ-5D. Version 2.0. 2009, EuroQoL Group, Rotterdam Cheung K, Oemar M, Oppe M, Rabin R: EQ-5D User Guide: Basic Information on how to Use the EQ-5D. Version 2.0. 2009, EuroQoL Group, Rotterdam
11.
go back to reference Euroqol – a new facility for the measurement of health-related quality of life. Health Policy. 1990, 16: 199-208. Euroqol – a new facility for the measurement of health-related quality of life. Health Policy. 1990, 16: 199-208.
12.
go back to reference Garratt AM, Macdonald LM, Ruta DA, Russell IT, Buckingham JK, Krukowski ZH: Towards measurement of outcome for patients with varicose veins. Qual Health Care. 1993, 2: 5-10. 10.1136/qshc.2.1.5.CrossRefPubMedPubMedCentral Garratt AM, Macdonald LM, Ruta DA, Russell IT, Buckingham JK, Krukowski ZH: Towards measurement of outcome for patients with varicose veins. Qual Health Care. 1993, 2: 5-10. 10.1136/qshc.2.1.5.CrossRefPubMedPubMedCentral
13.
go back to reference Dawson J, Fitzpatrick R, Carr A, Murray D: Questionnaire on the perceptions of patients about total hip replacement. J Bone Joint Surg Br. 1996, 78: 185-90.PubMed Dawson J, Fitzpatrick R, Carr A, Murray D: Questionnaire on the perceptions of patients about total hip replacement. J Bone Joint Surg Br. 1996, 78: 185-90.PubMed
14.
go back to reference Dawson J, Fitzpatrick R, Murray D, Carr A: Questionnaire on the perceptions of patients about total knee replacement. J Bone Joint Surg Br. 1998, 80-B: 63-9.CrossRef Dawson J, Fitzpatrick R, Murray D, Carr A: Questionnaire on the perceptions of patients about total knee replacement. J Bone Joint Surg Br. 1998, 80-B: 63-9.CrossRef
15.
go back to reference Petticrew M, Roberts H: Systematic reviews in the social sciences: A practical guide. 2005, Wiley-Blackwell, Oxford Petticrew M, Roberts H: Systematic reviews in the social sciences: A practical guide. 2005, Wiley-Blackwell, Oxford
16.
go back to reference Spiepelhalter D, Sherlaw-Johnson C, Bardsley M, Blut I, Wood C, Grigg OJR: Statistical methods for healthcare regulation: rating screening and surveillance. R J Statist Soc A. 2012, 175: 1-25. 10.1111/j.1467-985X.2011.01010.x.CrossRef Spiepelhalter D, Sherlaw-Johnson C, Bardsley M, Blut I, Wood C, Grigg OJR: Statistical methods for healthcare regulation: rating screening and surveillance. R J Statist Soc A. 2012, 175: 1-25. 10.1111/j.1467-985X.2011.01010.x.CrossRef
17.
go back to reference Laird N: Comparative analyses of pre-test:post-test research designs. Amer Statistician. 1983, 37: 329-330. Laird N: Comparative analyses of pre-test:post-test research designs. Amer Statistician. 1983, 37: 329-330.
18.
go back to reference Chard J, Kuczawski M, Black N, van der Meulen J: Outcomes of elective surgery undertaken in Independent Sector Treatment Centres and NHS providers in England: the Patient Outcomes in Surgery Audit. BMJ. 2011, 343: d6404-10.1136/bmj.d6404.CrossRefPubMedPubMedCentral Chard J, Kuczawski M, Black N, van der Meulen J: Outcomes of elective surgery undertaken in Independent Sector Treatment Centres and NHS providers in England: the Patient Outcomes in Surgery Audit. BMJ. 2011, 343: d6404-10.1136/bmj.d6404.CrossRefPubMedPubMedCentral
19.
go back to reference Shepperd S, Harwood D, Jenkinson C, Gray A, Vessey M, Morgan P: Randomised controlled trial comparing hospital at home care with inpatient hospital care. I: Three month follow up of health outcomes. BMJ. 1998, 316: 1786-1791. 10.1136/bmj.316.7147.1786.CrossRefPubMedPubMedCentral Shepperd S, Harwood D, Jenkinson C, Gray A, Vessey M, Morgan P: Randomised controlled trial comparing hospital at home care with inpatient hospital care. I: Three month follow up of health outcomes. BMJ. 1998, 316: 1786-1791. 10.1136/bmj.316.7147.1786.CrossRefPubMedPubMedCentral
20.
go back to reference Round A, Crabb T, Buckingham K, Mejzner R, Pearce V, Ayres R, Weeks C, Hamilton W: Six month outcomes after emergency admission of elderly patients to a community or a district general hospital. Family Practice. 2004, 21: 173-179. 10.1093/fampra/cmh212.CrossRefPubMed Round A, Crabb T, Buckingham K, Mejzner R, Pearce V, Ayres R, Weeks C, Hamilton W: Six month outcomes after emergency admission of elderly patients to a community or a district general hospital. Family Practice. 2004, 21: 173-179. 10.1093/fampra/cmh212.CrossRefPubMed
21.
go back to reference Campbell A: The evaluation of a model of primary mental health care in rural Tasmania. Aust J Rural Health. 2005, 13: 142-148. 10.1111/j.1440-1854.2005.00694.x.CrossRefPubMed Campbell A: The evaluation of a model of primary mental health care in rural Tasmania. Aust J Rural Health. 2005, 13: 142-148. 10.1111/j.1440-1854.2005.00694.x.CrossRefPubMed
22.
go back to reference Siggeirsdottir K, Olafsson O, Jonsson H, Iwarsson S, Gudnason V, Jonsson BY: Short hospital stay augmented with education and home-based rehabilitation improves function and quality of life after hip replacement: Randomized study of 50 patients with 6 months of follow-up. Acta Orthop. 2005, 76: 555-562. 10.1080/17453670510041565.CrossRefPubMed Siggeirsdottir K, Olafsson O, Jonsson H, Iwarsson S, Gudnason V, Jonsson BY: Short hospital stay augmented with education and home-based rehabilitation improves function and quality of life after hip replacement: Randomized study of 50 patients with 6 months of follow-up. Acta Orthop. 2005, 76: 555-562. 10.1080/17453670510041565.CrossRefPubMed
23.
go back to reference Ryan T, Enderby P, Rigby AS: A randomized controlled trial to evaluate intensity of community-based rehabilitation provision following stroke or hip fracture in old age: Results at 12-month follow-up. Int J Disabil Hum Dev. 2006, 5: 83-89.CrossRef Ryan T, Enderby P, Rigby AS: A randomized controlled trial to evaluate intensity of community-based rehabilitation provision following stroke or hip fracture in old age: Results at 12-month follow-up. Int J Disabil Hum Dev. 2006, 5: 83-89.CrossRef
25.
go back to reference Browne J, van der Meulen JH, Lewsey JD, Lamping DL, Black N: Mathematical coupling accounted for the association between baseline severity and minimally important difference value. J Clin Epidemiol. 2009, 63: 865-874.CrossRef Browne J, van der Meulen JH, Lewsey JD, Lamping DL, Black N: Mathematical coupling accounted for the association between baseline severity and minimally important difference value. J Clin Epidemiol. 2009, 63: 865-874.CrossRef
26.
go back to reference Judge A, Arden N, Cooper C, Dieppe P: A statistical tool for individual prediction for individual risk and benefit from knee replacement surgery. 2010, NHS Institute for Innovation and Improvement, London Judge A, Arden N, Cooper C, Dieppe P: A statistical tool for individual prediction for individual risk and benefit from knee replacement surgery. 2010, NHS Institute for Innovation and Improvement, London
27.
go back to reference van Eck JWM, van Hemel NM, van den BA, Taks W, Grobbee DE, Moons KGM: Predictors of improved quality of life 1 year after pacemaker implantation. Am Heart J. 2008, 156: 491-97. 10.1016/j.ahj.2008.04.029.CrossRefPubMed van Eck JWM, van Hemel NM, van den BA, Taks W, Grobbee DE, Moons KGM: Predictors of improved quality of life 1 year after pacemaker implantation. Am Heart J. 2008, 156: 491-97. 10.1016/j.ahj.2008.04.029.CrossRefPubMed
28.
go back to reference Walters SJ, Brazier JE: Comparison of the minimally important difference for two health state utility measures: EQ-5D and SF-6D. Qual Life Res. 2005, 14: 1523-32. 10.1007/s11136-004-7713-0.CrossRefPubMed Walters SJ, Brazier JE: Comparison of the minimally important difference for two health state utility measures: EQ-5D and SF-6D. Qual Life Res. 2005, 14: 1523-32. 10.1007/s11136-004-7713-0.CrossRefPubMed
29.
go back to reference Jaeschke R, Singer J, Guyatt GH: Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin Trials. 1989, 10: 407e15.CrossRef Jaeschke R, Singer J, Guyatt GH: Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin Trials. 1989, 10: 407e15.CrossRef
30.
go back to reference Kvien TK, Heiberg T, Hagen KB: Minimal clinically important improvement/difference (MCII/MCID) and patient acceptable symptom state (PASS): what do these concepts mean?. Ann Rheum Dis. 2007, 66 (Suppl-3): iii40-iii41.PubMedPubMedCentral Kvien TK, Heiberg T, Hagen KB: Minimal clinically important improvement/difference (MCII/MCID) and patient acceptable symptom state (PASS): what do these concepts mean?. Ann Rheum Dis. 2007, 66 (Suppl-3): iii40-iii41.PubMedPubMedCentral
31.
go back to reference van der Roer N, Ostelo RW, Bekkering GE, van Tulder MW, de Vet HC: Minimal clinically important change for pain intensity, functional status, and general health status in patients with nonspecific low back pain. Spine. 2006, 31: 578-82. 10.1097/01.brs.0000201293.57439.47.CrossRefPubMed van der Roer N, Ostelo RW, Bekkering GE, van Tulder MW, de Vet HC: Minimal clinically important change for pain intensity, functional status, and general health status in patients with nonspecific low back pain. Spine. 2006, 31: 578-82. 10.1097/01.brs.0000201293.57439.47.CrossRefPubMed
32.
go back to reference Norman G, Wyrwich KW, Sloan JA: Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med Care. 2003, 41: 582-92.PubMed Norman G, Wyrwich KW, Sloan JA: Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med Care. 2003, 41: 582-92.PubMed
33.
go back to reference Alonso J, Croudace T, Brown J, Gasquet I, Knapp M, Suarez D, Novick D: Health-related quality of life (HRQL) and continuous antipsychotic treatment: 3-year results from the schizophrenia health outcomes (SOHO) study. Value Health. 2009, 12: 536-543. 10.1111/j.1524-4733.2008.00495.x.CrossRefPubMed Alonso J, Croudace T, Brown J, Gasquet I, Knapp M, Suarez D, Novick D: Health-related quality of life (HRQL) and continuous antipsychotic treatment: 3-year results from the schizophrenia health outcomes (SOHO) study. Value Health. 2009, 12: 536-543. 10.1111/j.1524-4733.2008.00495.x.CrossRefPubMed
34.
go back to reference Anis AH, Nosyk B, Sun H, Guh DP, Bansback N, Li X, Barnett PG, Joyce V, Swanson KM, Kyriakides TC, Holodniy M, Cameron DW, Brown ST: Quality of life of patients with advanced HIV/AIDS: Measuring the impact of both AIDS-defining events and non-AIDS serious adverse events. J Acquir Immune Defic Syndr. 2009, 51: 631-639. 10.1097/QAI.0b013e3181a4f00d.CrossRefPubMed Anis AH, Nosyk B, Sun H, Guh DP, Bansback N, Li X, Barnett PG, Joyce V, Swanson KM, Kyriakides TC, Holodniy M, Cameron DW, Brown ST: Quality of life of patients with advanced HIV/AIDS: Measuring the impact of both AIDS-defining events and non-AIDS serious adverse events. J Acquir Immune Defic Syndr. 2009, 51: 631-639. 10.1097/QAI.0b013e3181a4f00d.CrossRefPubMed
35.
go back to reference Darlington A, Dippel DWJ, Ribbers GM, van Balen R, Passchier J, Busschbach JJ: Coping strategies as determinants of quality of life in stroke patients: A longitudinal study. Cerebrovasc Dis. 2007, 23: 401-407. 10.1159/000101463.CrossRefPubMed Darlington A, Dippel DWJ, Ribbers GM, van Balen R, Passchier J, Busschbach JJ: Coping strategies as determinants of quality of life in stroke patients: A longitudinal study. Cerebrovasc Dis. 2007, 23: 401-407. 10.1159/000101463.CrossRefPubMed
36.
go back to reference Hajat S, Fitzpatrick R, Morris R, Reeves B, Rigge M, Williams O, Murray D, Gregg P: Does waiting for total hip replacement matter? Prospective cohort study. J Health Serv Res Policy. 2002, 7: 19-25. 10.1258/1355819021927638.CrossRefPubMed Hajat S, Fitzpatrick R, Morris R, Reeves B, Rigge M, Williams O, Murray D, Gregg P: Does waiting for total hip replacement matter? Prospective cohort study. J Health Serv Res Policy. 2002, 7: 19-25. 10.1258/1355819021927638.CrossRefPubMed
37.
go back to reference Lacey EA, Walters SJ: Continuing inequality: Gender and social class influences on self perceived health after a heart attack. J Epidemiol Community Health. 2003, 57: 622-627. 10.1136/jech.57.8.622.CrossRefPubMedPubMedCentral Lacey EA, Walters SJ: Continuing inequality: Gender and social class influences on self perceived health after a heart attack. J Epidemiol Community Health. 2003, 57: 622-627. 10.1136/jech.57.8.622.CrossRefPubMedPubMedCentral
38.
go back to reference Schluessmann E, Diel P, Aghayev E, Zweig T, Moulin P, Roder C, SWISSspine Registry Group: SWISSspine: A nationwide registry for health technology assessment of lumbar disc prostheses. Eur Spine J. 2009, 18: 851-61. 10.1007/s00586-009-0934-8.CrossRefPubMedPubMedCentral Schluessmann E, Diel P, Aghayev E, Zweig T, Moulin P, Roder C, SWISSspine Registry Group: SWISSspine: A nationwide registry for health technology assessment of lumbar disc prostheses. Eur Spine J. 2009, 18: 851-61. 10.1007/s00586-009-0934-8.CrossRefPubMedPubMedCentral
39.
go back to reference Kalairajah Y, Azurza K, Hulme C, Molloy S, Drabu KJ: Health Outcome Measures in the evaluation of total hip Arthroplasties – a comparison between the Harris Hip score and the Oxford Hip Score. J Arthroplasty. 2005, 20: 1037-1041. 10.1016/j.arth.2005.04.017.CrossRefPubMed Kalairajah Y, Azurza K, Hulme C, Molloy S, Drabu KJ: Health Outcome Measures in the evaluation of total hip Arthroplasties – a comparison between the Harris Hip score and the Oxford Hip Score. J Arthroplasty. 2005, 20: 1037-1041. 10.1016/j.arth.2005.04.017.CrossRefPubMed
40.
go back to reference Streiner DL, Norman GR: Health Measurement Scales. A practical guide to their development and use. 2008, Oxford University Press, OxfordCrossRef Streiner DL, Norman GR: Health Measurement Scales. A practical guide to their development and use. 2008, Oxford University Press, OxfordCrossRef
Metadata
Title
Clinicians’ and patients’ views of metrics of change derived from patient reported outcome measures (PROMs) for comparing providers’ performance of surgery
Authors
Zoe Hildon
Jenny Neuburger
Dominique Allwood
Jan van der Meulen
Nick Black
Publication date
01-12-2012
Publisher
BioMed Central
Published in
BMC Health Services Research / Issue 1/2012
Electronic ISSN: 1472-6963
DOI
https://doi.org/10.1186/1472-6963-12-171

Other articles of this Issue 1/2012

BMC Health Services Research 1/2012 Go to the issue