Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
Health and Quality of Life Outcomes
Published in: Health and Quality of Life Outcomes 1/2019

Open Access 01-12-2019 | Research

The impact of patient-reported outcome (PRO) data from clinical trials: a systematic review and critical analysis

Authors: Samantha Cruz Rivera, Derek G. Kyte, Olalekan Lee Aiyegbusi, Anita L. Slade, Christel McMullan, Melanie J. Calvert

Published in: Health and Quality of Life Outcomes | Issue 1/2019

Login to get access

Abstract

Background

Patient-reported outcomes (PROs) are commonly collected in clinical trials and should provide impactful evidence on the effect of interventions on patient symptoms and quality of life. However, it is unclear how PRO impact is currently realised in practice. In addition, the different types of impact associated with PRO trial results, their barriers and facilitators, and appropriate impact metrics are not well defined. Therefore, our objectives were: i) to determine the range of potential impacts from PRO clinical trial data, ii) identify potential PRO impact metrics and iii) identify barriers/facilitators to maximising PRO impact; and iv) to examine real-world evidence of PRO trial data impact based on Research Excellence Framework (REF) impact case studies.

Methods

Two independent investigators searched MEDLINE, EMBASE, CINAHL+, HMIC databases from inception until December 2018. Articles were eligible if they discussed research impact in the context of PRO clinical trial data. In addition, the REF 2014 database was systematically searched. REF impact case studies were included if they incorporated PRO data in a clinical trial.

Results

Thirty-nine publications of eleven thousand four hundred eighty screened met the inclusion criteria. Nine types of PRO trial impact were identified; the most frequent of which centred around PRO data informing clinical decision-making. The included publications identified several barriers and facilitators around PRO trial design, conduct, analysis and report that can hinder or promote the impact of PRO trial data. Sixty-nine out of two hundred nine screened REF 2014 case studies were included. 12 (17%) REF case studies led to demonstrable impact including changes to international guidelines; national guidelines; influencing cost-effectiveness analysis; and influencing drug approvals.

Conclusions

PRO trial data may potentially lead to a range of benefits for patients and society, which can be measured through appropriate impact metrics. However, in practice there is relatively limited evidence demonstrating directly attributable and indirect real world PRO-related research impact. In part, this is due to the wider challenges of measuring the impact of research and PRO-specific issues around design, conduct, analysis and reporting. Adherence to guidelines and multi-stakeholder collaboration is essential to maximise the use of PRO trial data, facilitate impact and minimise research waste.

Trial registration

Systematic Review registration PROSPERO CRD42017067799.
Appendix
Available only for authorised users
Literature
1.
Mitchell K. How do Patient-Reported measures contribute to value in health care? vol. 2017: Institute of Healthcare Improvement; 2014.
2.
3.
Calvert M, Kyte D, Mercieca-Bebber R, et al. Guidelines for inclusion of patient-reported outcomes in clinical trial protocols: the spirit-pro extension. JAMA. 2018;319:483–94.CrossRefPubMed
4.
Vodicka E, Kim K, Devine E, Gnanasakthy A, Scoggins J, Patrick D. Inclusion of patient-reported outcome measures in registered clinical trials: evidence from ClinicalTrials.gov (2007–2013). Contemp Clin Trials. 2015;43:1–9.CrossRefPubMed
5.
Kyte D, Retzer A, Ahmed K, Keeley T, Armes J, Brown JM, Calman L, Gavin A, Glaser AW, Greenfield DM, et al. Systematic evaluation of Patient-Reported outcome protocol content and reporting in Cancer trials. J Natl Cancer Inst. 2019;111(11):1–9..
6.
Mercieca-Bebber R, King MT, Calvert MJ, Stockler MR, Friedlander M. The importance of patient-reported outcomes in clinical trials and strategies for future optimization. Patient Relat Outcome Meas. 2018;9:353–67.CrossRefPubMedPubMedCentral
7.
Calvert M, Blazeby J, Revicki D, Moher D, Brundage M. Reporting quality of life in clinical trials: a CONSORT extension. Lancet. 2011;378:1684–5.CrossRefPubMed
8.
9.
10.
Cruz Rivera S, Kyte DG, Aiyegbusi OL, Keeley TJ, Calvert MJ. Assessing the impact of healthcare research: a systematic review of methodological frameworks. PLoS Med. 2017;14:e1002370.CrossRefPubMedPubMedCentral
11.
12.
13.
Morton S. Progressing research impact assessment: A ‘contributions’ approach. Res Eval. 2015;24:405–19.CrossRef
14.
15.
Brueton VC, Vale CL, Choodari-Oskooei B, Jinks R, Tierney JF. Measuring the impact of methodological research: a framework and methods to identify evidence of impact. Trials. 2014;15:464 461p.CrossRefPubMedPubMedCentral
16.
Weiss AP. Measuring the impact of medical research: moving from outputs to outcomes. Am J Psychiatr. 2007;164:206–14.CrossRefPubMed
17.
Lavis J, Ross S, McLeod C, Gildiner A. Measuring the impact of health research. J Health Serv Res Policy. 2003;8:165–70.CrossRefPubMed
18.
Canavan J, Gillen A, Shaw A. Measuring research impact : developing practical and cost-effective approaches. Evid Policy. 2009;5:167–77.CrossRef
19.
Buykx P, Humphreys J, Wakerman J, Perkins D, Lyle D, McGrail M, Kinsman L. ‘Making evidence count’: a framework to monitor the impact of health services research. Aust J Rural Health. 2012;20:51–8.CrossRefPubMed
20.
Spaapen J, van Drooge L. Introducing ‘productive interactions’ in social impact assessment. Res Eval. 2011;20:211–8.CrossRef
21.
Guinea J, Sela E, Gómez-Núñez AJ, Mangwende T, Ambali A, Ngum N, Jaramillo H, Gallego JM, Patiño A, Latorre C, et al. Impact oriented monitoring: a new methodology for monitoring and evaluation of international public health research projects. Res Eval. 2015;24:131–45.CrossRef
22.
Buxton M, Hanney S. How can payback from health services research be assessed? J Health Serv Res Policy. 1996;1:35–43.CrossRefPubMed
23.
Meagher L, Lyall C, Nutley S. Flows of knowledge, expertise and influence: a method for assessing policy and practice impacts from social science research. Res Eval. 2008;17:163–73.CrossRef
24.
Graham KER, Chorzempa HL, Valentine PA, Magnan J. Evaluating health research impact: development and implementation of the Alberta innovates – health solutions impact framework. Res Eval. 2012;21:354–67.CrossRef
25.
Cohen G, Schroeder J, Newson R, King L, Rychetnik L, Milat AJ, Bauman AE, Redman S, Chapman S. Does health intervention research have real world policy and practice impacts: testing a new impact assessment tool. Health Res Policy Syst. 2014;13:3–3 1p.CrossRef
26.
27.
Kok MO, Schuit AJ. Contribution mapping: a method for mapping the contribution of research to enhance its impact. Health Res Policy Syst. 2012;10.
28.
Landry R, Amara N, Lamari M. Climbing the ladder of research utilization - evidence from social science research. Sci Commun. 2001;22:396–422.CrossRef
29.
30.
31.
32.
Donovan C. The Australian research quality framework: a live experiment in capturing the social, economic, environmental, and cultural returns of publicly funded research. N Dir Eval. 2008;118:47–60.CrossRef
33.
Jacob R, McGregor M. Assessing the impact of health technology assessment. Int J Technol Assess Health Care. 1997;13:68–80.CrossRefPubMed
34.
Reed M. What makes a 4* research impact case study for REF2021? In: Fast Track Impact, vol. 2018; 2017.
35.
Penfield T, Baker MJ, Scoble R, Wykes MC. Assessment, evaluations, and definitions of research impact: a review. Res Eval. 2014;23:21–32.CrossRef
36.
Moher D, Liberati A, Tetzlaff J, Altman DG, The PG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:e1000097.CrossRefPubMedPubMedCentral
37.
iProspect. iProspect Search Engine User Behavior Study: iProspectcom, Inc; 2006.
38.
Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15:1277–88.CrossRefPubMed
39.
Efficace F, Bottomley A, Osoba D, Gotay C, Flechtner H, D'Haese S, Zurlo A. Beyond the development of health-related quality-of-life (HRQOL) measures: a checklist for evaluating HRQOL outcomes in cancer clinical trials -- does HRQOL evaluation in prostate cancer research inform clinical decision making? J Clin Oncol. 2003;21:3502–11.CrossRefPubMed
40.
Bjordal K. Impact of quality of life measurement in daily clinical practice. Ann Oncol. 2004;15:iv279–82.CrossRefPubMed
41.
Avery K, Blazeby JM. Quality of life assessment in surgical oncology trials. World J Surg. 2006;30:1163–72.CrossRefPubMed
42.
Efficace F, Kemmler G, Vignetti M, Mandelli F, Molica S, Holzner B. Health-related quality of life assessment and reported outcomes in leukaemia randomised controlled trials - a systematic review to evaluate the added value in supporting clinical decision making. Eur J Cancer. 2008;44:1497–506.CrossRefPubMed
43.
Gujral S, Avery KNL, Blazeby JM. Quality of life after surgery for colorectal cancer: clinical implications of results from randomised trials. Support Care Cancer. 2008;16:127–32.CrossRefPubMed
44.
Parameswaran R, McNair A, Avery KN, Berrisford RG, Wajed SA, Sprangers MA, Blazeby JM. The role of health-related quality of life outcomes in clinical decision making in surgery for esophageal cancer: a systematic review. Ann Surg Oncol. 2008;15:2372–9.CrossRefPubMed
45.
Lemieux J, Goodwin PJ, Bordeleau LJ, Lauzier S, Theberge V. Quality-of-life measurement in randomized clinical trials in breast cancer: an updated systematic review (2001-2009). J Natl Cancer Inst. 2011;103:178–231.CrossRefPubMed
46.
Calvert M, Brundage M, Jacobsen PB, Schunemann HJ, Efficace F. The CONSORT Patient-Reported outcome (PRO) extension: implications for clinical trials and practice. Health Qual Life Outcomes. 2013;11:184.CrossRefPubMedPubMedCentral
47.
Jacobs M, Macefield RC, Blazeby JM, Korfage IJ, Henegouwen MIV, de Haes H, Smets EM, Sprangers MAG. Systematic review reveals limitations of studies evaluating health-related quality of life after potentially curative treatment for esophageal cancer. Qual Life Res. 2013;22:1787–803.CrossRefPubMed
48.
Dirven L, Taphoorn MJB, Reijneveld JC, Blazeby J, Jacobs M, Pusic A, La Sala E, Stupp R, Fayers P, Efficace F, Patient Reported O. The level of patient-reported outcome reporting in randomised controlled trials of brain tumour patients: a systematic review. Eur J Cancer. 2014;50:2432–48.CrossRefPubMed
49.
Efficace F, Feuerstein M, Fayers P, Cafaro V, Eastham J, Pusic A, Blazeby J. Patient-reported outcomes in randomised controlled trials of prostate cancer: methodological quality and impact on clinical decision making. Eur Urol. 2014;66:416–27.CrossRefPubMed
50.
Efficace F, Jacobs M, Pusic A, Greimel E, Piciocchi A, Kieffer JM, Gilbert A, Fayers P, Blazeby J, Registry EQoLGPROMOTIO-P. Patient-reported outcomes in randomised controlled trials of gynaecological cancers: investigating methodological quality and impact on clinical decision-making. Eur J Cancer. 2014;50:1925–41.CrossRefPubMed
51.
Rees JR, Whale K, Fish D, Fayers P, Cafaro V, Pusic A, Blazeby JM, Efficace F. Patient-reported outcomes in randomised controlled trials of colorectal cancer: an analysis determining the availability of robust data to inform clinical decision-making. J Cancer Res Clin Oncol. 2015;141:2181–92.CrossRefPubMedPubMedCentral
52.
Rouette J, Blazeby J, King M, Calvert M, Peng Y, Meyer RM, Ringash J, Walker M, Brundage MD. Integrating health-related quality of life findings from randomized clinical trials into practice: an international study of oncologists' perspectives. Qual Life Res. 2015;24:1317–25.CrossRefPubMed
53.
McNair AGK, Macefield RC, Blencowe NS, Brookes ST, Blazeby JM. ‘Trial exegesis’: methods for synthesizing clinical and Patient Reported outcome (PRO) data in trials to inform clinical practice. A systematic review. PLoS One. 2016;11:e0160998.CrossRefPubMedPubMedCentral
54.
Blazeby JM, Avery K, Sprangers M, Pikhart H, Fayers P, Donovan J. Health-related quality of life measurement in randomized clinical trials in surgical oncology. J Clin Oncol. 2006;24:3178–86.CrossRefPubMed
55.
Snyder C, Brundage M. Integrating patient-reported outcomes in healthcare policy, research and practice. Expert Rev Pharmacoecon Outcomes Res. 2010;10:351–3.CrossRefPubMed
56.
Calvert M, Brundage M, Jacobsen PB, Schünemann HJ, Efficace F. The CONSORT Patient-Reported outcome (PRO) extension: implications for clinical trials and practice. Health Qual Life Outcomes. 2013;11:184.CrossRefPubMedPubMedCentral
57.
Bottomley A, Efficace F, Thomas R, Vanvoorden V, Ahmedzai SH. Health-related quality of life in non–small-cell lung Cancer: methodologic issues in randomized controlled trials. J Clin Oncol. 2003;21:2982–92.CrossRefPubMed
58.
Efficace F, Kemmler G, Vignetti M, Mandelli F, Molica S, Holzner B. Health-related quality of life assessment and reported outcomes in leukaemia randomised controlled trials – a systematic review to evaluate the added value in supporting clinical decision making. Eur J Cancer. 2008;44:1497–506.CrossRefPubMed
59.
Ganz PA. Assessing the quality and value of quality-of-life measurement in breast Cancer clinical trials. J Natl Cancer Inst. 2011;103:196–9.CrossRefPubMed
60.
Goodwin PJ, Black JT, Bordeleau LJ, Ganz PA. Health-related quality-of-life measurement in randomized clinical trials in breast Cancer—taking stock. J Natl Cancer Inst. 2003;95:263–81.CrossRefPubMed
61.
Hao Y, Krohe M, Yaworsky A, Shields AL, Mazar I, Foley C, Globe D. Clinical trial Patient-reported outcomes data: going beyond the label in oncology. Clin Ther. 2016;38:811–20.CrossRefPubMed
62.
Jacobs M, Macefield RC, Blazeby JM, Korfage IJ, van Berge Henegouwen MI, de Haes HCJM, Smets EM, Sprangers MAG. Systematic review reveals limitations of studies evaluating health-related quality of life after potentially curative treatment for esophageal cancer. Qual Life Res. 2013;22:1787–803.CrossRefPubMed
63.
McNair AG, Blazeby JM. Health-related quality-of-life assessment in GI cancer randomized trials: improving the impact on clinical practice. Expert Rev Pharmacoecon Outcomes Res. 2009;9:559–67.CrossRefPubMed
64.
Mercieca-Bebber RL, Perreca A, King M, Macann A, Whale K, Soldati S, Jacobs M, Efficace F. Patient-reported outcomes in head and neck and thyroid cancer randomised controlled trials: a systematic review of completeness of reporting and impact on interpretation. Eur J Cancer. 2016;56:144–61.CrossRefPubMed
65.
Mott FE. Patient Reported outcomes (PROs) as part of value-based care can shape therapy guidelines: impact on emerging targeted agents and immunotherapy protocols in resource-limited regions. Oncol Ther. 2017;5:69–74.CrossRefPubMed
66.
Basch E, Geoghegan C, Coons S, et al. Patient-reported outcomes in cancer drug development and us regulatory review: perspectives from industry, the food and drug administration, and the patient. JAMA Oncol. 2015;1:375–9.CrossRefPubMed
67.
Au H-J, Ringash J, Brundage M, Palmer M, Richardson H, Meyer RM. Added value of health-related quality of life measurement in cancer clinical trials: the experience of the NCIC CTG. Expert Rev Pharmacoecon Outcomes Res. 2010;10:119–28.CrossRefPubMed
68.
Glasziou PP, Mar CD, Salisbury J. Evidence-Based Practice Workbook: Wiley; 2009.
69.
Kristensen N, Nymann C, Konradsen H. Implementing research results in clinical practice- the experiences of healthcare professionals. BMC Health Serv Res. 2016;16:1–10.
70.
Revicki DA, Osoba D, Fairclough D, Barofsky I, Berzon R, Leidy NK, Rothman M. Recommendations on health-related quality of life research to support labeling and promotional claims in the United States. Qual Life Res. 2000;9:887–900.CrossRefPubMed
71.
Patrick DL, Burke LB, Powers JH, Scott JA, Rock EP, Dawisha S, O'Neill R, Kennedy DL. Patient-reported outcomes to support medical product labeling claims: FDA perspective. Value Health. 2007;10(Suppl 2):S125–37.CrossRefPubMed
72.
73.
DeMuro C, Clark M, Mordin M, Fehnel S, Copley-Merriman C, Gnanasakthy A. Reasons for rejection of Patient-Reported outcome label claims: a compilation based on a review of Patient-Reported outcome Use among new molecular entities and biologic license applications, 2006–2010. Value Health. 2012;15:443–8.CrossRefPubMed
74.
Coon CD. The Use of Patient-reported outcomes in demonstrating safety and efficacy in oncology. Clin Ther. 2016;38:756–8.CrossRefPubMed
75.
Anker SD, Agewall S, Borggrefe M, Calvert M, Jaime Caro J, Cowie MR, Ford I, Paty JA, Riley JP, Swedberg K, et al. The importance of patient-reported outcomes: a call for their comprehensive integration in cardiovascular clinical trials. Eur Heart J. 2014;35:2001–9.CrossRefPubMed
76.
Gnanasakthy A, DeMuro C, Clark M, Haydysch E, Ma E, Bonthapally V. Patient-Reported outcomes labeling for products approved by the Office of Hematology and Oncology Products of the US Food and Drug Administration (2010-2014). J Clin Oncol. 2016;34:1928–34.CrossRefPubMed
77.
78.
Zagadailov E, Fine M, Shields A. Patient-Reported outcomes are changing the landscape in oncology care: challenges and opportunities for payers. Am Health Drug Benefits. 2013;6:264–74.PubMedPubMedCentral
79.
Sztankay M, Giesinger JM, Zabernigg A, Krempler E, Pall G, Hilbe W, Burghuber O, Hochmair M, Rumpold G, Doering S, Holzner B. Clinical decision-making and health-related quality of life during first-line and maintenance therapy in patients with advanced non-small cell lung cancer (NSCLC): findings from a real-world setting. BMC Cancer. 2017;17:565.CrossRefPubMedPubMedCentral
80.
Nixon AWD, Muehlhausen W. Patient reported outcomes to supoort drug development decision making. Farmeconomia Health Econ Ther Pathways. 2015;16:35–7.CrossRef
81.
Rees J, Hurt CN, Gollins S, Mukherjee S, Maughan T, Falk SJ, Staffurth J, Ray R, Bashir N, Geh JI, et al. Patient-reported outcomes during and after definitive chemoradiotherapy for oesophageal cancer. Br J Cancer. 2015;113:603–10.CrossRefPubMedPubMedCentral
82.
Brundage M, Bass B, Davidson J, Queenan J, Bezjak A, Ringash J, Wilkinson A, Feldman-Stewart D. Patterns of reporting health-related quality of life outcomes in randomized clinical trials: implications for clinicians and quality of life researchers. Qual Life Res. 2011;20:653–64.CrossRefPubMed
83.
Dirven L, Taphoorn MJB, Reijneveld JC, Blazeby J, Jacobs M, Pusic A, La Sala E, Stupp R, Fayers P, Efficace F. The level of patient-reported outcome reporting in randomised controlled trials of brain tumour patients: a systematic review. Eur J Cancer. 2014;50:2432–48.CrossRefPubMed
84.
Greenhalgh J, Long AF, Flynn R. The use of patient reported outcome measure in routine clinical practice: lack of impact or lack of theory? Soc Sci Med. 2005;60:833–43.CrossRefPubMed
85.
Valderas JM, Kotzeva A, Espallargues M, Guyatt G, Ferrans CE, Halyard MY, Revicki DA, Symonds T, Parada A, Alonso J. The impact of measuring patient-reported outcomes in clinical practice: a systematic review of the literature. Qual Life Res. 2008;17:179–93.CrossRefPubMed
86.
Williamson PR, Altman DG, Bagley H, Barnes KL, Blazeby JM, Brookes ST, Clarke M, Gargon E, Gorst S, Harman N, et al. The COMET handbook: version 1.0. Trials. 2017;18:280.CrossRefPubMedPubMedCentral
87.
Calvert M, Kyte D, Price G, Valderas JM, Hjollund NH. Maximising the impact of patient reported outcome assessment for patients and society. BMJ. 2019;364:k5267.CrossRefPubMed
88.
Bottomley A, Pe M, Sloan J, Basch E, Bonnetain F, Calvert M, Campbell A, Cleeland C, Cocks K, Collette L, et al. Analysing data from patient-reported outcome and quality of life endpoints for cancer clinical trials: a start in setting international standards. Lancet Oncol. 2016;17:e510–4.CrossRefPubMed
89.
Snyder CF, Smith KC, Bantug ET, Tolbert EE, Blackford AL, Brundage MD, the PRODPSAB. What do these scores mean? Presenting patient-reported outcomes data to patients and clinicians to improve interpretability. Cancer. 2017;123:1848–59.CrossRefPubMed
90.
91.
Use CfMPfH. Reflection paper on the regulatory guidance for the use of health-related quality of life (HRQL) measures in the evaluation of medicinal products. London: European Medicines Agency; 2005.
92.
93.
94.
Metadata
Title
The impact of patient-reported outcome (PRO) data from clinical trials: a systematic review and critical analysis
Authors
Samantha Cruz Rivera
Derek G. Kyte
Olalekan Lee Aiyegbusi
Anita L. Slade
Christel McMullan
Melanie J. Calvert
Publication date
01-12-2019
Publisher
BioMed Central
Published in
Health and Quality of Life Outcomes / Issue 1/2019
Electronic ISSN: 1477-7525
DOI
https://doi.org/10.1186/s12955-019-1220-z

Other articles of this Issue 1/2019

Health and Quality of Life Outcomes 1/2019 Go to the issue

Research

Quality of life reported by survivors after hospitalization for Middle East respiratory syndrome (MERS)

Research

Cross-cultural adaptation and measurement properties of the Nepali version of the DASH (disability of arm, shoulder and hand) in patients with shoulder pain

Research

Do socio-demographic characteristics and/or health status explain the magnitude of differences between patient and general public utility values? A chronic low back pain patients case study

Research

Clostridium difficile Infection–Daily Symptoms (CDI-DaySyms™) questionnaire: psychometric characteristics and responder thresholds

Research

The effects of the combined exercise intervention based on internet and social media software (CEIBISMS) on quality of life, muscle strength and cardiorespiratory capacity in Chinese postoperative breast cancer patients:a randomized controlled trial

Short report

The Dyspnea-ALS-Scale (DALS-15) optimizes individual treatment in patients with amyotrophic lateral sclerosis (ALS) suffering from dyspnea