Skip to main content
Top
Published in: Health and Quality of Life Outcomes 1/2013

Open Access 01-12-2013 | Review

Patient-reported outcomes in meta-analyses -Part 2: methods for improving interpretability for decision-makers

Authors: Bradley C Johnston, Donald L Patrick, Kristian Thorlund, Jason W Busse, Bruno R da Costa, Holger J Schünemann, Gordon H Guyatt

Published in: Health and Quality of Life Outcomes | Issue 1/2013

Login to get access

Abstract

Systematic reviews and meta-analyses of randomized trials that include patient-reported outcomes (PROs) often provide crucial information for patients, clinicians and policy-makers facing challenging health care decisions. Based on emerging methods, guidance on improving the interpretability of meta-analysis of patient-reported outcomes, typically continuous in nature, is likely to enhance decision-making. The objective of this paper is to summarize approaches to enhancing the interpretability of pooled estimates of PROs in meta-analyses. When differences in PROs between groups are statistically significant, decision-makers must be able to interpret the magnitude of effect. This is challenging when, as is often the case, clinical trial investigators use different measurement instruments for the same construct within and between individual randomized trials. For such cases, in addition to pooling results as a standardized mean difference, we recommend that systematic review authors use other methods to present results such as relative (relative risk, odds ratio) or absolute (risk difference) dichotomized treatment effects, complimented by presentation in either: natural units (e.g. overall depression reduced by 2.4 points when measured on a 50-point Hamilton Rating Scale for Depression); minimal important difference units (e.g. where 1.0 unit represents the smallest difference in depression that patients, on average, perceive as important the depression score was 0.38 (95% CI 0.30 to 0.47) units less than the control group); or a ratio of means (e.g. where the mean in the treatment group is divided by the mean in the control group, the ratio of means is 1.27, representing a 27% relative reduction in the mean depression score).
Literature
2.
go back to reference Johnston BC, Patrick DL, Busse JW, Schünemann HJ, Agarwal A, Guyatt GH: Patient-reported outcomes in meta-analyses–part 1: assessing risk of bias and combining outcomes. BMC Health Qual Life Outcomes 2013,11(1):109. 10.1186/1477-7525-11-109CrossRef Johnston BC, Patrick DL, Busse JW, Schünemann HJ, Agarwal A, Guyatt GH: Patient-reported outcomes in meta-analyses–part 1: assessing risk of bias and combining outcomes. BMC Health Qual Life Outcomes 2013,11(1):109. 10.1186/1477-7525-11-109CrossRef
3.
go back to reference Guyatt GH, Thorlund K, Oxman AD, Walter SD, Patrick D, Furukawa TA, Johnston BC, Karanikolas P, Vist G, Kunz R, Brozek J, Meerpohl J, Akl EA, Christensen R, Schünemann HJ: Preparing summary of findings tables: continuous outcomes. J Clin Epidemiol 2013,66(2):173–183. 10.1016/j.jclinepi.2012.08.001PubMedCrossRef Guyatt GH, Thorlund K, Oxman AD, Walter SD, Patrick D, Furukawa TA, Johnston BC, Karanikolas P, Vist G, Kunz R, Brozek J, Meerpohl J, Akl EA, Christensen R, Schünemann HJ: Preparing summary of findings tables: continuous outcomes. J Clin Epidemiol 2013,66(2):173–183. 10.1016/j.jclinepi.2012.08.001PubMedCrossRef
4.
go back to reference Guyatt GH, Oxman AD, Santesso N, Helfand M, Vist G, Kunz R, Brozek J, Norris S, Meerpohl J, Djulbegovic B, Alonso-Coello P, Post PN, Busse JW, Glasziou P, Christensen R, Schünemann HJ: GRADE guidelines: 12: preparing summary of findings tables-binary outcomes. J Clin Epidemiol 2013,66(2):158–172. 10.1016/j.jclinepi.2012.01.012PubMedCrossRef Guyatt GH, Oxman AD, Santesso N, Helfand M, Vist G, Kunz R, Brozek J, Norris S, Meerpohl J, Djulbegovic B, Alonso-Coello P, Post PN, Busse JW, Glasziou P, Christensen R, Schünemann HJ: GRADE guidelines: 12: preparing summary of findings tables-binary outcomes. J Clin Epidemiol 2013,66(2):158–172. 10.1016/j.jclinepi.2012.01.012PubMedCrossRef
5.
go back to reference Brozek JL, Guyatt GH, Schünemann HJ: How a well-grounded minimal important difference can enhance transparency of labelling claims and improve interpretation of a patient-reported outcome measure. BMC Health Qual Life Outcomes 2006, 27: 4–69. Brozek JL, Guyatt GH, Schünemann HJ: How a well-grounded minimal important difference can enhance transparency of labelling claims and improve interpretation of a patient-reported outcome measure. BMC Health Qual Life Outcomes 2006, 27: 4–69.
6.
go back to reference Schünemann HJ, Akl EA, Guyatt GH: Interpreting the results of patient-reported outcome measures in clinical trials: the clinician’s perspective. BMC Health Qual Life Outcomes 2006, 4: 62. 10.1186/1477-7525-4-62CrossRef Schünemann HJ, Akl EA, Guyatt GH: Interpreting the results of patient-reported outcome measures in clinical trials: the clinician’s perspective. BMC Health Qual Life Outcomes 2006, 4: 62. 10.1186/1477-7525-4-62CrossRef
7.
go back to reference Guyatt GH, Juniper EF, Walter SD, Griffith LE, Goldstein RS: Interpreting treatment effects in randomised trials. BMJ 1998,316(7132):690–693. 10.1136/bmj.316.7132.690PubMedCentralPubMedCrossRef Guyatt GH, Juniper EF, Walter SD, Griffith LE, Goldstein RS: Interpreting treatment effects in randomised trials. BMJ 1998,316(7132):690–693. 10.1136/bmj.316.7132.690PubMedCentralPubMedCrossRef
9.
go back to reference Wardlaw JM, Murray V, Berge E, Del Zoppo GJ: Thrombolysis for acute ischaemic stroke. Cochrane Database Syst Rev 2009, 4: CD000213.PubMed Wardlaw JM, Murray V, Berge E, Del Zoppo GJ: Thrombolysis for acute ischaemic stroke. Cochrane Database Syst Rev 2009, 4: CD000213.PubMed
10.
go back to reference Guyatt GH, Osoba D, Wu AW, Wyrwich KW, Norman GR, Clinical Significance Consensus Meeting Group: Methods to explain the clinical significance of health status measures. Mayo Clin Proc 2002, 77: 371–383. 10.4065/77.4.371PubMedCrossRef Guyatt GH, Osoba D, Wu AW, Wyrwich KW, Norman GR, Clinical Significance Consensus Meeting Group: Methods to explain the clinical significance of health status measures. Mayo Clin Proc 2002, 77: 371–383. 10.4065/77.4.371PubMedCrossRef
11.
go back to reference Schünemann HJ, Goldstein R, Mador MJ, McKim D, Stahl E, Puhan MA, Griffith LE, Grant B, Austin P, Collins R, Guyatt GH: A randomised trial to evaluate the self-administered standardised chronic respiratory questionnaire. Europ Respirat J 2005, 25: 31–40. 10.1183/09031936.04.00029704CrossRef Schünemann HJ, Goldstein R, Mador MJ, McKim D, Stahl E, Puhan MA, Griffith LE, Grant B, Austin P, Collins R, Guyatt GH: A randomised trial to evaluate the self-administered standardised chronic respiratory questionnaire. Europ Respirat J 2005, 25: 31–40. 10.1183/09031936.04.00029704CrossRef
12.
go back to reference Lacasse Y, Goldstein R, Lasserson TJ, Martin S: Pulmonary rehabilitation for chronic obstructive pulmonary disease. Cochrane Database Syst Rev 2006,18(4):CD003793. Lacasse Y, Goldstein R, Lasserson TJ, Martin S: Pulmonary rehabilitation for chronic obstructive pulmonary disease. Cochrane Database Syst Rev 2006,18(4):CD003793.
13.
go back to reference Deeks J, Higgins J, Altman D: Chapter 9–Analysing Data and Undertaking Meta-analyses: Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration; 2011. Available from: . [Accessed 3 Sept, 2013] http://www.cochrane-handbook.org Deeks J, Higgins J, Altman D: Chapter 9–Analysing Data and Undertaking Meta-analyses: Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration; 2011. Available from: . [Accessed 3 Sept, 2013] http://​www.​cochrane-handbook.​org
14.
go back to reference Johnston BC, Thorlund K, Schunemann HJ, Xie F, Murad MH, Montori VM, Guyatt GH: Improving the interpretation of quality of life evidence in meta-analysis: the application of minimally important difference units. BMC Health Qual Life Outcomes 2010, 8: 116. 10.1186/1477-7525-8-116CrossRef Johnston BC, Thorlund K, Schunemann HJ, Xie F, Murad MH, Montori VM, Guyatt GH: Improving the interpretation of quality of life evidence in meta-analysis: the application of minimally important difference units. BMC Health Qual Life Outcomes 2010, 8: 116. 10.1186/1477-7525-8-116CrossRef
15.
go back to reference Cohen J: Statistical Power Analysis in the Behavioral Sciences. 2nd edition. Hillsdale (NJ): Lawrence Erlbaum Associates, Inc; 1988. Cohen J: Statistical Power Analysis in the Behavioral Sciences. 2nd edition. Hillsdale (NJ): Lawrence Erlbaum Associates, Inc; 1988.
16.
go back to reference Norman GR, Sloan JA, Wyrwich KW: Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med Care 2003, 41: 582–592.PubMed Norman GR, Sloan JA, Wyrwich KW: Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med Care 2003, 41: 582–592.PubMed
17.
go back to reference Farivar SS, Liu H, Hays RD: Half standard deviation estimate of the minimally important difference in HRQOL scores. Expert Rev Pharmacoenconomics Outcomes Res. 2004,4(5):515–523. 10.1586/14737167.4.5.515CrossRef Farivar SS, Liu H, Hays RD: Half standard deviation estimate of the minimally important difference in HRQOL scores. Expert Rev Pharmacoenconomics Outcomes Res. 2004,4(5):515–523. 10.1586/14737167.4.5.515CrossRef
18.
go back to reference Johnston BC, Thorlund K, Da Costa BR, Furukawa TA, Guyatt GH: New methods can extend the use of minimal important difference units in meta-analyses of continuous outcome measures. J Clin Epidemiol 2012,65(8):817–826. 10.1016/j.jclinepi.2012.02.008PubMedCrossRef Johnston BC, Thorlund K, Da Costa BR, Furukawa TA, Guyatt GH: New methods can extend the use of minimal important difference units in meta-analyses of continuous outcome measures. J Clin Epidemiol 2012,65(8):817–826. 10.1016/j.jclinepi.2012.02.008PubMedCrossRef
19.
go back to reference Anzures-Cabrera J, Sarpatwari A, Higgins JP: Expressing findings from meta-analyses of continuous outcomes in terms of risks. Stat Med 2011,30(25):2967–2985. 10.1002/sim.4298PubMedCrossRef Anzures-Cabrera J, Sarpatwari A, Higgins JP: Expressing findings from meta-analyses of continuous outcomes in terms of risks. Stat Med 2011,30(25):2967–2985. 10.1002/sim.4298PubMedCrossRef
20.
go back to reference Friedrich JO, Adhikari NKJ, Beyene J: The ratio of means method as an alternative to mean differences for analyzing continuous outcome variables in meta-analysis: a simulation study. BMC Med Res Methodol 2008,8(32):1–15. Friedrich JO, Adhikari NKJ, Beyene J: The ratio of means method as an alternative to mean differences for analyzing continuous outcome variables in meta-analysis: a simulation study. BMC Med Res Methodol 2008,8(32):1–15.
22.
go back to reference Hasselblad V, Hedges LV: Meta-analysis of screening and diagnostic tests. Psychol Bull 1995, 117: 167–178.PubMedCrossRef Hasselblad V, Hedges LV: Meta-analysis of screening and diagnostic tests. Psychol Bull 1995, 117: 167–178.PubMedCrossRef
23.
go back to reference Da Costa BR, Rutjes AW, Johnston BC, Reichenbach S, Nüesch E, Tonia T, Gemperli A, Guyatt GH, Jüni P: Methods to convert continuous outcomes into odds ratios of treatment response and numbers needed to treat: meta-epidemiological study. Int J Epidemiol 2012,41(5):1445–1459. 10.1093/ije/dys124PubMedCrossRef Da Costa BR, Rutjes AW, Johnston BC, Reichenbach S, Nüesch E, Tonia T, Gemperli A, Guyatt GH, Jüni P: Methods to convert continuous outcomes into odds ratios of treatment response and numbers needed to treat: meta-epidemiological study. Int J Epidemiol 2012,41(5):1445–1459. 10.1093/ije/dys124PubMedCrossRef
24.
go back to reference Friedrich JO, Adhikari NKJ, Beyene J: Ratio of means for analyzing continuous outcomes in meta-analysis performed as well as mean difference methods. J Clin Epidemiol 2011, 64: 556–564. 10.1016/j.jclinepi.2010.09.016PubMedCrossRef Friedrich JO, Adhikari NKJ, Beyene J: Ratio of means for analyzing continuous outcomes in meta-analysis performed as well as mean difference methods. J Clin Epidemiol 2011, 64: 556–564. 10.1016/j.jclinepi.2010.09.016PubMedCrossRef
25.
go back to reference Thorlund K, Walter SD, Johnston BC, Furukawa TA, Guyatt GH: Pooling continuous outcomes in meta-analysis: a comprehensive review of methods for enhancing interpretability. Res Synth Meth 2011, 2: 188–203. 10.1002/jrsm.46CrossRef Thorlund K, Walter SD, Johnston BC, Furukawa TA, Guyatt GH: Pooling continuous outcomes in meta-analysis: a comprehensive review of methods for enhancing interpretability. Res Synth Meth 2011, 2: 188–203. 10.1002/jrsm.46CrossRef
26.
go back to reference Johnston BC, Bandayrel K, Friedrich JO, Akl EA, Da Costa BR, Neumann I, Adhikari NKJ, Alonso-Coello P, Crawford MW, Mustafa RA, Svendrovski A, Thabane L, Tikkinen KAO, Vandvik PO, Guyatt GH: Presentation of continuous outcomes in meta-analysis: a survey of clinicians’ understanding and preferences: 21st Cochrane Colloquium. Suppl 1–212 edition. Quebec City, Canada: Cochrane Database Syst Rev; 2013. 4.097 Johnston BC, Bandayrel K, Friedrich JO, Akl EA, Da Costa BR, Neumann I, Adhikari NKJ, Alonso-Coello P, Crawford MW, Mustafa RA, Svendrovski A, Thabane L, Tikkinen KAO, Vandvik PO, Guyatt GH: Presentation of continuous outcomes in meta-analysis: a survey of clinicians’ understanding and preferences: 21st Cochrane Colloquium. Suppl 1–212 edition. Quebec City, Canada: Cochrane Database Syst Rev; 2013. 4.097
27.
go back to reference Barbui C, Furukawa TA, Cipriani A: Effectiveness of paroxetine in the acute phase treatment of adults with major depression: a systematic re-examination of published and unpublished randomised data. CMAJ 2008,178(3):296–305.PubMedCentralPubMedCrossRef Barbui C, Furukawa TA, Cipriani A: Effectiveness of paroxetine in the acute phase treatment of adults with major depression: a systematic re-examination of published and unpublished randomised data. CMAJ 2008,178(3):296–305.PubMedCentralPubMedCrossRef
28.
go back to reference Furukawa TA: Assessment of mood: guides for clinicians. J Psychosom Res 2010,68(6):581–589. 10.1016/j.jpsychores.2009.05.003PubMedCrossRef Furukawa TA: Assessment of mood: guides for clinicians. J Psychosom Res 2010,68(6):581–589. 10.1016/j.jpsychores.2009.05.003PubMedCrossRef
29.
go back to reference Bandelow B, Baldwin DS, Dolberg OT, Andersen HF, Stein DJ: What is the threshold for symptomatic response and remission for major depressive disorder, panic disorder, social anxiety disorder, and generalized anxiety disorder? J Clin Psychiatry 2006,67(1):428–434. Bandelow B, Baldwin DS, Dolberg OT, Andersen HF, Stein DJ: What is the threshold for symptomatic response and remission for major depressive disorder, panic disorder, social anxiety disorder, and generalized anxiety disorder? J Clin Psychiatry 2006,67(1):428–434.
30.
go back to reference Furukawa TA, Akechi T, Azuma H, Okuyama T, Higuchi T: Evidence-based guidelines for interpretation of the Hamilton rating scale for depression. J Clin Psychopharmacol 2007,27(5):531–534. 10.1097/JCP.0b013e31814f30b1PubMedCrossRef Furukawa TA, Akechi T, Azuma H, Okuyama T, Higuchi T: Evidence-based guidelines for interpretation of the Hamilton rating scale for depression. J Clin Psychopharmacol 2007,27(5):531–534. 10.1097/JCP.0b013e31814f30b1PubMedCrossRef
31.
go back to reference Hamilton M: Development of a rating scale for primary depressive illness. Br J Soc Clin Psychol 1967, 6: 278–296. 10.1111/j.2044-8260.1967.tb00530.xPubMedCrossRef Hamilton M: Development of a rating scale for primary depressive illness. Br J Soc Clin Psychol 1967, 6: 278–296. 10.1111/j.2044-8260.1967.tb00530.xPubMedCrossRef
32.
go back to reference Suissa S: Binary methods for continuous outcomes: a paremetric alternative. J Clin Epidemiol 1991, 44: 241–248. 10.1016/0895-4356(91)90035-8PubMedCrossRef Suissa S: Binary methods for continuous outcomes: a paremetric alternative. J Clin Epidemiol 1991, 44: 241–248. 10.1016/0895-4356(91)90035-8PubMedCrossRef
33.
go back to reference Cox D, Snell E (Eds): Analysis of binary data. London: Chapman and Hall; 1989. Cox D, Snell E (Eds): Analysis of binary data. London: Chapman and Hall; 1989.
34.
go back to reference Akl EA, Oxman AD, Herrin J, Vist GE, Terrenato I, Sperati F, Costiniuk C, Blank D, Schünemann H: Using alternative statistical formats for presenting risks and risk reductions. Cochrane Database Syst Rev 2011, 3: CD006776.PubMed Akl EA, Oxman AD, Herrin J, Vist GE, Terrenato I, Sperati F, Costiniuk C, Blank D, Schünemann H: Using alternative statistical formats for presenting risks and risk reductions. Cochrane Database Syst Rev 2011, 3: CD006776.PubMed
35.
Metadata
Title
Patient-reported outcomes in meta-analyses -Part 2: methods for improving interpretability for decision-makers
Authors
Bradley C Johnston
Donald L Patrick
Kristian Thorlund
Jason W Busse
Bruno R da Costa
Holger J Schünemann
Gordon H Guyatt
Publication date
01-12-2013
Publisher
BioMed Central
Published in
Health and Quality of Life Outcomes / Issue 1/2013
Electronic ISSN: 1477-7525
DOI
https://doi.org/10.1186/1477-7525-11-211

Other articles of this Issue 1/2013

Health and Quality of Life Outcomes 1/2013 Go to the issue