Real world evidence (RWE) – a disruptive innovation or the quiet evolution of medical evidence generation?

Sajan Khosla; Robert White; Jesús Medina; Mario Ouwens; Cathy Emmas; Tim Koder; Gary Male; Sandra Leonard

doi:10.12688/f1000research.13585.2

Home Browse Real world evidence (RWE) – a disruptive innovation or the quiet evolution...

ALL Metrics

-

Views

-

Downloads

Get PDF

Get XML

Export

▬

✚

Opinion Article

Revised

Real world evidence (RWE) – a disruptive innovation or the quiet evolution of medical evidence generation?

[version 2; peer review: 2 approved]

Sajan Khosla¹, Robert White², Jesús Medina³, [...] Mario Ouwens⁴, Cathy Emmas⁵, Tim Koder⁶, Gary Male⁶, Sandra Leonard²

Sajan Khosla¹, Robert White², [...] Jesús Medina³, Mario Ouwens⁴, Cathy Emmas⁵, Tim Koder⁶, Gary Male⁶, Sandra Leonard²

PUBLISHED 29 Aug 2018

Author details Author details

¹ AstraZeneca Academy House, Cambridge, CB2 1PG, UK
² AstraZeneca, Gaithersburg, MD, 20878, USA
³ AstraZeneca, Madrid, 28033, Spain
⁴ AstraZeneca AB R&D, Mölndal, 431 50, Sweden
⁵ AstraZeneca, Luton, LU1 3LU, UK
⁶ Oxford PharmaGenesis, Tubney Warren Barn, Tubney, Abingdon, OX13 5QJ, UK

Sajan Khosla
Roles: Conceptualization, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing

Robert White
Roles: Writing – Review & Editing

Jesús Medina
Roles: Writing – Original Draft Preparation, Writing – Review & Editing

Mario Ouwens
Roles: Writing – Review & Editing

Cathy Emmas
Roles: Writing – Original Draft Preparation, Writing – Review & Editing

Tim Koder
Roles: Writing – Original Draft Preparation, Writing – Review & Editing

Gary Male
Roles: Writing – Original Draft Preparation, Writing – Review & Editing

Sandra Leonard
Roles: Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

Abstract

Stakeholders in healthcare are increasingly turning to real world evidence (RWE) to inform their decisions, alongside evidence from randomized controlled trials. RWE is generated by analysing data gathered from routine clinical practice, and can be used across the product lifecycle, providing insights into areas including disease epidemiology, treatment effectiveness and safety, and health economic value and impact. Recently, the US Food and Drug Administration and the European Medicines Agency have stated their ambition for greater use of RWE to support applications for new indications, and are now consulting with their stakeholders to formalize standards and expected methods for generating RWE.
Pharmaceutical companies are responding to the increasing demands for RWE by developing standards and processes for each stage of the evidence generation pathway. Some conventions are already in place for assuring quality, whereas other processes are specific to the research question and data sources available. As evidence generation increasingly becomes a core role of medical affairs divisions in large pharmaceutical companies, standards of rigour will continue to evolve and improve. Senior pharmaceutical leaders can drive this change by making RWE a core element of their corporate strategy, providing top-level direction on how their respective companies should approach RWE for maximum quality.
Here, we describe the current and future areas of RWE application within the pharmaceutical industry, necessary access to data to generate RWE, and the challenges in communicating RWE. Supporting and building on viewpoints from industry and publicly funded research, our perspective is that at each stage of RWE generation, quality will be critical to the impact that RWE has on healthcare decision-makers; not only where RWE is an established and evolving tool, but also in new areas that have the potential to disrupt and to improve drug development pathways.

Keywords

Real world evidence, Drug Discovery methods, Drug Industry methods

Corresponding author: Sajan Khosla

Competing interests: Sajan Khosla, Robert White, Jesús Medina and Sandra Leonard are employees of AstraZeneca, in the Medical Evidence and Observational Research team. Mario Ouwens is an employee of AstraZeneca in the Advanced Analytics team. Cathy Emmas is an employee of AstraZeneca in the Patient Centricity team. Tim Koder and Gary Male are employees of Oxford PharmaGenesis, whose work on this manuscript has been funded by AstraZeneca.

Grant information: Development of this manuscript was funded by AstraZeneca; writing was undertaken as part of an internal project by employees of AstraZeneca and Oxford PharmaGenesis.

Copyright: © 2018 Khosla S et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

How to cite: Khosla S, White R, Medina J et al. Real world evidence (RWE) – a disruptive innovation or the quiet evolution of medical evidence generation? [version 2; peer review: 2 approved]. F1000Research 2018, 7:111 (https://doi.org/10.12688/f1000research.13585.2) First published: 25 Jan 2018, 7:111 (https://doi.org/10.12688/f1000research.13585.1) Latest published: 29 Aug 2018, 7:111 (https://doi.org/10.12688/f1000research.13585.2)

Revised Amendments from Version 1

Based on peer review comments, we have included additional references to our manuscript and made two minor alterations to the main text – none of which alters the opinions we are sharing in our article. We have included an additional industry perspective to Table 1, and included minor additions to Tables 3 and 4.

To read any peer review reports and author responses for this article, follow the "read" links in the Open Peer Review table.

Introduction

In March 2016, the US Food and Drug Administration (FDA) released a statement outlining the goals and procedures for the Prescription Drug User Fee Act (PDUFA) VI for 2018–2022, with notice that this would include the use of real world evidence (RWE) in regulatory decision-making. In December 2016, the 21st Century Cures Act became law in the USA, aiming to expedite approval for new medicines. Towards that aim, it included provision for RWE to be used in place of evidence from randomized controlled trials (RCTs), if judged appropriate by the FDA.

RWE is derived from the analysis of data collected from a healthcare setting, outside the context of prescriptive RCTs¹. One of the key objectives of RWE is to understand observations and events in patients in routine clinical practice. RWE complements RCTs, which are carefully controlled experiments to test specific hypotheses on the efficacy and safety of new drugs, and which by design do not reflect current clinical practice. Owing to the mechanism of data collection and experimental design, RWE studies generally may not yield definitive causal inference because of the many confounders of variability^2,3.

The FDA aims to publish draft guidance for the use of RWE by October 2021, and consultation is already underway with healthcare sector stakeholders including the pharmaceutical industry. Both the FDA and the European Medicines Agency (EMA) have stated their wish to see increased use of RWE in supporting indications. In Asia, the growing maturity of real world data sources has led to the recent use of RWE in regulatory discussions, for example, in the decision in Japan on the use of raloxifene for the treatment of osteoperosis⁴. Indian regulatory authorities are also looking to embed routinely collected electronic health records into their decision-making process⁵.

To date, use of RWE by the pharmaceutical industry has primarily focused on the peri-launch period just before, and immediately after, marketing approval of a drug, to describe patient populations, to contribute towards knowledge of patient safety and to make judgements on comparative effectiveness between drugs. RWE is also regularly used in economic modelling and when establishing appropriate pricing for new therapeutic interventions. While details of the methods used vary between agencies, RWE is central to the healthcare technology assessments (HTA) by which payers judge if a new drug is cost-effective in their healthcare system⁶.

Earlier stages of the clinical drug development pipeline are now starting to use RWE to support critical decisions. Experts in RWE from several large pharmaceutical companies have previously published their views on this topic, as have companies that specialize in generating RWE for the pharmaceutical industry, and an overview of results from an unstructured literature search is provided in Table 1^7–13. Pharmaceutical companies have recognized the demand for RWE from national regulators and other healthcare decision-makers; however, few to date have gained committed support at board level for RWE generation being a critical part of the business. With the focus on specialty care, progress in technology and increasing availability of real world data, the time is right to provide this support to ensure that patients can access the medicines they need.

Table 1. Selected industry perspectives on RWE.

Reference	Year	Company	Summary
*Epstein et al.⁷*	2012	Sanofi	A cross-functional approach to evidence generation in the pharmaceutical industry, with a real world perspective, will streamline regulatory approval
Hughes and Kessler⁸	2014	IMS Health	1. RWE capabilities should converge in a platform 2. Narrow focus on specific therapeutic areas and markets leads to differential value from RWE 3. The commercial organization must champion efforts to broaden RWE’s application and value 4. Speed is a goal
Berger et al.⁹	2015	Pfizer	The potential for success of big data to improve healthcare depends on the development of suitable policies, especially regarding openness and quality
Ronicke, Ruhl and Solbach¹⁰	2015	Strategy&, a division of PwC	Companies need to develop specific strategies for RWE, either driven by senior leaders top-down or encouraged to grow in local companies bottom-up
GSK U.S. Public Policy¹¹	2015	GSK	Proactive communication of RWE will improve healthcare but regulatory guidance is required to define robust research standards
Galson and Simon¹²	2016	Amgen and Group Health Research Institute	RWE generation must be driven by the needs of healthcare decision-makers. Integration of clinical research with clinical practice must be facilitated to improve healthcare
QuintilesIMS¹³	2017	QuintilesIMS	Big data, powerful analytics and a strategic approach to collaborations will generate the best insights with greatest transparency
Andersson and Kyhlstedt¹⁵	2017	Synergus	A structured search to identify the relevant RWD sources is an essential step for successful execution of RWE studies

GSK, GlaxoSmithKline; PwC, PricewaterhouseCoopers; RWD, real world data; RWE, real world evidence.

This perspectives paper is aimed primarily at an industry audience with an interest in forming or expanding an RWE function, and looks to describe the planning, generation and communication of RWE (Figure 1), specifically:

• the current and future areas of RWE application in the pharmaceutical industry
• the source, quality of and access to data that are necessary to generate RWE
• challenges in communicating RWE.

Figure 1. The RWE journey – from concept to use.

An outline of the main steps and some key questions in the planning, generation and communication of RWE. RWE, real world evidence.

Potential for RWE in drug lifecycles

The pharmaceutical industry faces fresh challenges in finding ways to make its innovative medicines available to patients. The interests of healthcare system payers and regulators, and the need to measure disease burden, create a complex environment for quantifying clinical value.

Continual observation of disease epidemiology, treatment patterns and outcomes in the real world can help to prioritize and to streamline medicine development, with the potential for accelerating evidence generation to support label expansion for specific products. All phases of medicine development can benefit from increased observation of the real world (Figure 2 and Table 2).

Figure 2. RWE is used throughout the product lifecycle.

The questions that can be addressed by RWE and the functions involved in the generation or use of RWE at each stage of product development. RWE, real world evidence.

Table 2. Evidence needs across the product lifecycle.

Purpose	Evidence needed
Shape target product profile	• Understanding unmet needs and outcomes that matter to patients • Describing the potential market, which with pricing decisions leads to forecasts of future earnings and information that affects development investment
Shape design of phase 3 studies	• RWE on the patient population, unmet need and standard of care • Evidence on pharmacology, toxicology and efficacy
Achieve registration/approval	• Preclinical, toxicology and pharmaceutical chemistry data • Robust efficacy and safety data • Early clinical studies
Achieve access and reimbursement	• RWE on the patient population, unmet need and burden of disease • RWE on comparative effectiveness and costs of comparator or standard of care • Cost-effectiveness and budget impact models
Maintain access and demonstrate continued value	• RWE on comparative clinical effectiveness and costs • Long-term outcomes data • Patient safety data • Patient adherence and utilization data

RWE, real world evidence.

Disease strategy and early discovery disease segmentation

RWE has the potential to be used early in drug discovery and development programmes, facilitating product development by identifying diseases or indications that represent a significant burden in populations. Electronic health records to support differentiation of patients’ needs have been used within the National Institutes of Health (NIH), and the ability to characterize patient populations before conducting a trial has enabled the NIH to design trials that accelerate innovative interventions to testing phase in patient subgroups of particular need¹⁴.

Phase 1–3 clinical study design

To ensure a clinical trial protocol has internal validity, trial design teams will often use a set of restrictive eligibility criteria that may remove from the trial large segments of a population with the disease of interest. The impact of these eligibility criteria is often not understood or in most cases is not tested until the question of generalizability is raised at the stage of regulatory or reimbursement submission¹⁶. This has been recognized as a limitation of RCTs by many regulators, including the FDA, in response to many approved medicines being withdrawn owing to safety problems being identified once a therapy has been exposed to a broad patient population¹⁷. RWE can aid clinical study design; for example, in assessing the population size of patients with different sets of inclusion or exclusion criteria.

Indication-seeking and label-expanding studies

In order to license a therapy in a new indication or to expand the label into a new population, it is mandatory to establish evidence to support the efficacy claim. Traditionally, explanatory trials determine whether the intervention produces the expected result under controlled circumstances, generated through careful design of RCTs. As the need for larger RCTs increases, owing to low-rate event endpoints, potentially differential efficacy throughout subpopulations of patients and the need to observe larger populations for rare adverse events after intervention, the cost of running the trials increases. The time to run these trials also impacts on the potential profitability of indication expansion. Therefore, new thinking is required on how and if explanatory trials can leverage some of the features of real world trials to deliver accelerated efficacy studies.

Pragmatic clinical trials – a hybrid approach

The main features of an RCT are the randomization of patients, enrolment into a controlled trial setting and follow-up specified in a study protocol. Applying this concept while also using real world data may provide a hybrid approach to running pragmatic clinical trials. The levels of pragmatism can be understood within the context of the PRagmatic Explanatory Continuum Indicator Summary (PRECIS)-2 framework¹⁸. In the regulatory context, a balanced approach of using real world data to execute large-cohort phase 3 trials may generate enough of a reward to risk taking the step towards an innovative execution model. This hybrid approach to running studies has been taken in examples such as the Salford Lung Study (see Box 1)^19,20.

Box 1. Case study: The Salford Lung Study

The Salford Lung Study assessed the effectiveness and safety of fluticasone furoate in patients with chronic obstructive pulmonary disease (COPD). In this 12 month, open-label, phase 3, multicentre study, 2799 patients with COPD were randomized 1:1 to a once-daily inhaled combination of fluticasone furoate 100 μg and vilanterol 25 μg, or to continuation of their existing therapy. This collaborative study collected data using electronic health records of consenting patients across all of their interactions with general practitioners (GPs), pharmacists and hospitals. In total, 75 GP practices, 128 community pharmacies in Salford and South Manchester, and two hospitals participated in the study^19,20. The primary objective of the Salford Lung Study pragmatic approach was to assess the effectiveness of the treatment, and pragmatic features were not primarily used to decrease costs or to increase speed of delivery. Although the cost of the Salford Lung Study has not been published, the expenditure incurred by the training of healthcare professionals and the development of a bespoke data collection system is likely to be high. The cost of such an approach should therefore be carefully evaluated before it is used to implement such a study.

Post-authorization studies

Regulators including the FDA, EMA and China Food and Drug Administration increasingly ask pharmaceutical companies to implement ‘post-marketing commitment’ studies as a condition of approval. In some cases, these commitments are requested after a product launch, for example, in light of new safety concerns. The studies may cover safety, efficacy, effectiveness or optimal use. One specific type of study, a post-authorization safety study, is usual for product authorizations: a large group of patients receiving the new medicine is tracked, often for a longer time period than covered by the registrational trial. Pharmaceutical companies are also obliged to enforce systems for spontaneous safety reporting, capturing and assessing adverse event data received from prescribing physicians. These data are consolidated into reports for regulators, and are typically used for pharmacovigilance rather than for public reimbursement by each country’s national and local bodies, based on its effectiveness and safety, value for money and affordability. These are the key questions covered by health technology assessments, answered by health economic models that use data from RCTs and RWE studies, plus financial estimates and calculations⁶.

Physicians also need to know how best to use new treatments in the broad patient population, not just in the restricted clinical trial sample. To give prescribers, guideline committees and formularies confidence to offer the medicines to patients, companies and independent investigators run retrospective and prospective RWE studies, showing outcomes from treatments in their region²¹.

Real world data

Data collected in a routine healthcare setting must be stringently curated, validated and standardized to enable the generation of robust RWE¹. Primary real world data are generated specifically for the purposes of the research, through prospective collection from diagnostic or monitoring procedures. Secondary real world studies use data that were routinely collected for medical or administrative purposes – such as electronic health (or medical) records and administrative claims databases – for the generation of RWE. More recently, a complementary source of real world data, generated directly by patients, has emerged from the growth of health-focused online communities and research networks. Sources such as the PatientsLikeMe research network, in which patients are encouraged to share health data in a structured, standardized format, give more scope for formal research. While structured patient-generated data sources such as PatientsLikeMe can lead to useful evidence (see Box 2), researchers should be aware of the limitations of data generated outside the healthcare environment, such as the challenge of how to validate the data. Examples of available real world data sources are provided in Table 3, and a detailed overview of their benefits and limitations is provided in the GetReal RWE Navigator.

Box 2. Case study: PatientsLikeMe study in amyotrophic lateral sclerosis22

PatientsLikeMe has an established and engaged community of patients with amyotrophic lateral sclerosis (ALS), a rapidly progressive and fatal neurodegenerative condition with no effective treatments. Approximately 9% (348) of patients with ALS in the PatientsLikeMe community reported using lithium carbonate, a drug which had shown promise in a small study (16 treated patients, 28 controls)²³, but which did not have regulatory approval. This offered the opportunity to conduct an observational study of drug usage and disease progression from quantitative data recorded by members of the PatientsLikeMe community. The 149 patients who fulfilled inclusion criteria for the study were matched with multiple controls (447 patients in total) based on their prior disease progression. Disease progression was measured using the Revised ALS Functional Rating Scale, which measures patient-reported functional impairment in domains such as speech, swallowing, walking, arm function and respiratory function. No difference in disease progression was observed after 12 months between the overall study group and those patients in the lithium carbonate treatment group (78 patients). Subsequent randomized studies reached the same conclusion that there was no clinical effect in the overall population, although genotype subgroups were associated with variations in response to treatment²⁴. The approach described in this case study has many limitations and cannot be considered a substitute for double-blind RCTs. However, it does suggest that data reported by patients in online health communities may be useful for accelerating clinical discovery and evaluating the effectiveness of drugs already in use.

Table 3. Example sources of real world data^a.

Data source	Data owners/curators	Typical coverage (patient records)	Typical time to data access
Administrative claims databases	HealthCore, Japanese Medical Claims Database, NHS, Optum, Truven,	> 10 million	Immediate
Electronic health/medical records	CPRD, Evidera, Flatiron Health, NorthWest eHealth, Optum, Parexel, PCORnet, QuintilesIMS	2–10 million	Immediate
Clinical registries	American College of Cardiology, SwedeHeart, CALIBER, CancerLinQ, Health Data Insight, Severe Asthma Registry	< 2 million	Within 1 year
Prospective studies and hybrid approaches	CROs/AROs, academic partnerships	> 1000	Over 1 year
Patient-generated data (e.g. social media or patient-powered research networks)	PatientsLikeMe, Carenity, PCORnet	> 100 000	Immediate^b

^aFurther information on data sources available at: https:///rwe-navigator.eu.

^bTaking account of privacy and unstructured data considerations.

ARO, academic research organization; CPRD, Clinical Practice Research Datalink; CRO, contract research organization; NHS, National Health Service; PCORnet, National Patient-Centered Clinical Research Network.

A scan for data availability and curation for research projects is a necessary step to ensure that the correct choices are made before designing study concepts. This review step is a well-defined process providing knowledge of vendors or research organizations that are able to provide access to data for research purposes. These data can be procured by the pharmaceutical industry and managed and governed within the industry. Data are gathered into an organization by specific RWE functions, which have been formalized by drawing relevant knowledge, processes and people from more established functions such as epidemiology, health economics and observational research, market access/payer divisions, medical affairs, patient safety and health informatics.

Data access

Access to real world data can be categorized into three forms: commercial, research collaborations and developmental collaborations. Each form of data access has implications for budget, time and the research objective. In commercial data access, a data asset is already available and may be able to address a research question; a vendor will therefore allow access to the data asset through commercial contracts. If an established commercial process is not defined, owing to local regulations regarding the commercialization of either a data asset or an academic affiliation, research collaborations can facilitate access to the data. This might be the case for access to clinical registry data and to general data in Europe and Asia. Finally, in a developmental collaboration there is a focus on working with a group to develop a data asset that can meet the needs of a research project or the design of a prospective study that will enable the curation of specific data elements from patients in a real world setting.

Data quality

Real world data must be robust and of high quality to generate valuable evidence that meets the need of healthcare decision-makers. Several guidelines have been developed in recent years to aid investigators in the design and execution of real world studies. Currently, there is still no widely accepted consensus as to which one should be used²⁵. From the perspective of industry sponsors of studies that generate real world evidence, there are many considerations that go beyond scientific, medical or methodological quality in the planning, generation and communication of RWE (Table 4)^26–29. Ultimately, the pharmaceutical industry must conduct research that provides the data or evidence that is required, that is acceptable to healthcare decision-makers and that leads to optimal health outcomes for patients while avoiding the misuse of resources.

Table 4. Considerations in the generation of high-quality real world data.

Stage of the RWE journey	Considerations
Planning	• Understand the needs of local healthcare decision-makers • Collaborate with external experts for advice on study designs and access to real world data • Generate a comprehensive RWE study (and publications) plan aligned to company strategy and local evidence needs • Be aware of local limitations: constraints in budget, time, drug exposure; logistics and study delivery; availability/willingness of patients and investigators to participate in the study • Identify a priori the potential sources of bias or confounding factors and identify measures to minimize them • Determine if data are required from a single country or multiple countries • Define research question a priori following FINER and PICO criteria³³
Generation	• Select the most appropriate study design and data source to address the research question, considering the strengths and limitations of each: – primary versus secondary data collection – prospective, retrospective or hybrid approach – randomization – descriptive versus analytic – cohort, case-control or cross-sectional study • Evaluate the benefits, risks and consequences to healthcare decision-makers of the selected study design and data source
	• Clearly define the primary and secondary objectives or endpoints • Evaluate the potential for missing data
	• Train internal and external study teams and investigators • Assess availability of suitable data extraction, management and analytics resources • Follow the FAIR data principles when appropriate (e.g. when machines are used to find and use reusable data)²⁶ • Evaluate site monitoring, source data verification and quality aspects in primary data collection studies
Communication	• Report methodology and data sources • Ensure transparency in publications strategy • Commit to publication, regardless of the results • Follow best practice guidelines and recommendations (e.g. STROBE, MOOSE, RECORD)^{27–29,34,35}

FAIR, Findability, Accessibility, Interoperability and Reusability; FINER, Feasible to answer, Interesting, Novel, Ethical, Relevant; MOOSE, Meta-analysis Of Observational Studies in Epidemiology; PICO, Patients, Intervention, Comparators, Outcomes; RECORD, REporting of studies Conducted using Observational Routinely-collected health Data; RWE, real world evidence; STROBE, STrengthening the Reporting of OBservational studies in Epidemiology.

Communication of RWE

Generating robust, high-quality RWE is not sufficient on its own; the pharmaceutical industry must also use this evidence effectively and within frameworks defined by regulators at a country or regional level. In 2017, the FDA released draft guidance on proactive communication of healthcare economic information from the pharmaceutical industry to payers; this was a response to the revision of Section 114 of the FDA Modernization Act through the 21st Century Cures Act^30,31. Section 114 was written to enable the pharmaceutical industry to communicate healthcare economic information more readily to payers and formulary decision-makers, lowering the threshold required for proactive communication from ‘substantial evidence’ to ‘competent and reliable scientific evidence’. The scope of the legislation does not extend to the proactive communication of clinical comparisons; here, the ‘substantial evidence’ threshold still applies, requiring evidence from RCTs. The proactive use of healthcare economic information permitted through Section 114 does not extend to communication with healthcare professionals or patients. In addition to regulatory limitations, and in contrast to the evidence generated by RCTs, healthcare decision-makers may not be aware of what RWE is or how to interpret it. The pharmaceutical industry may be challenged on the robustness of their RWE, perhaps owing to concerns over a lack of randomization in the study of interest or a perception that bias cannot be addressed in real world studies³². Variations in terminology and a lack of transparency in reporting real world studies add to the challenges in communicating the value of RWE to healthcare decision-makers. Several organizations have established initiatives with the objective of raising awareness of RWE and providing training. The GetReal project, established in Europe by the Innovative Medicines Initiative, brought together representatives from the pharmaceutical industry and other healthcare stakeholders to develop resources and training that provide guidance in the planning, generation and communication of RWE. A recent editorial has highlighted how certain challenges in communicating RWE can be overcome³⁶. There is a need for greater transparency in reporting how evidence from a real world study is generated, such as explaining the choice of data source or methodology applied. These efforts will help to ensure that healthcare decision-makers can make informed decisions when assessing RWE alongside evidence from RCTs.

Conclusions

RWE complements the evidence generated by RCTs and provides healthcare decision-makers with the confidence to choose the right treatment options for patients. Established types of RWE, such as post-marketing safety surveillance, will continue to evolve, adding value to the evidence base for marketed products, and RWE is now embedded and evolving in the reimbursement and regulatory spaces. Beyond this, however, there is an opportunity for positive disruption in pharmaceutical organizations, where decisions and the execution of clinical development, pipeline prioritization and early development can be driven by RWE. This disruption may reduce barriers for drug development, pushing the pharmaceutical industry to become more agile and innovative as it targets increasingly specific patient populations at an unprecedented pace. While companies recognize the need for RWE, greater strategic direction is needed to maximize its impact on health outcomes and commercial success. The challenge for industry is to adapt in order to utilize the full range of RWE, appropriately, in an environment of changing technology and regulations. Industry also has a responsibility, together with academic support, to make use of its knowledge in ambitions to drive the evolution of medicine development and to disrupt the way evidence is generated. Strategic coordination among local markets, global organizations and external collaborators will raise data quality standards and build international confidence in the planning, generation and communication of RWE.

Grant information

Development of this manuscript was funded by AstraZeneca; writing was undertaken as part of an internal project by employees of AstraZeneca and Oxford PharmaGenesis.

Acknowledgements

We are grateful for the review contributions of Howard G Hutchinson, James L. Gaskill and Frangiscos Sifakis, who are employees of AstraZeneca. We would also like to thank Claire Stoker and Colin Glen, who are employees of Oxford PharmaGenesis, for their review contributions. Scientific editorial support was provided to the authors by editors at Oxford PharmaGenesis, Oxford, UK, funded by AstraZeneca.

Faculty Opinions recommended

References

1. Berger M, Daniel G, Frank K, et al.: A framework for regulatory use of real-world evidence. Duke Margolis Center for Health Policy White Paper. 2017; (Accessed 26 October 2017). Reference Source
2. Eichler HG, Abadie E, Breckenridge A, et al.: Bridging the efficacy-effectiveness gap: a regulator's perspective on addressing variability of drug response. Nat Rev Drug Discov. 2011; 10(7): 495–506. PubMed Abstract | Publisher Full Text
3. Berger ML, Dreyer N, Anderson F, et al.: Prospective observational studies to assess comparative effectiveness: the ISPOR good research practices task force report. Value Health. 2012; 15(2): 217–30. PubMed Abstract | Publisher Full Text
4. Tanaka S, Yamamoto T, Oda E, et al.: Real-world evidence of raloxifene versus alendronate in preventing non-vertebral fractures in Japanese women with osteoporosis: retrospective analysis of a hospital claims database. J Bone Miner Metab. 2018; 36(1): 87–94. PubMed Abstract | Publisher Full Text
5. Dang A, Vallish BN: Real world evidence: An Indian perspective. Perspect Clin Res. 2016; 7(4): 156–60. PubMed Abstract | Publisher Full Text | Free Full Text
6. Makady A, Ham RT, de Boer A, et al.: Policies for use of real-world data in health technology assessment (HTA): a comparative study of six HTA agencies. Value Health. 2017; 20(4): 520–32. PubMed Abstract | Publisher Full Text
7. Epstein RS, Sidorov J, Lehner JP, et al.: Integrating scientific and real-world evidence within and beyond the drug development process. J Comp Eff Res. 2012; 1(1 Suppl): 9–13. PubMed Abstract | Publisher Full Text
8. Hughes B, Kessler M: Breaking new ground with RWE: how some pharmacos are poised to realize a $1 billion opportunity. IMS Health White Paper. 2014; (Accessed 9 November 2017). Reference Source
9. Berger ML, Lipset C, Gutteridge A, et al.: Optimizing the leveraging of real-world data to improve the development and use of medicines. Value Health. 2015; 18(1): 127–30. PubMed Abstract | Publisher Full Text
10. Ronicke V, Ruhl M, Solbach T: Revitalizing pharmaceutical R&D. The value of real word evidence. Strategy&. 2015; (Accessed 9 November 2017). Reference Source
11. Communicating comparative effectiveness research and real world evidence with population health decision makers. GSK U.S. Public Policy Position Paper. (Accessed 9 November 2017). Reference Source
12. Galson S, Simon G: Real-world evidence to guide approval and use of new treatments. National Academy of Medicine, Washington, DC. 2015; (Accessed 9 November 2017). Reference Source
13. News, views and insights from leading international RWE experts. Quintiles IMS. AccessPoint. (Accessed 9 November 2017). Reference Source
14. Johnson KE, Tachibana C, Coronado GD, et al.: A guide to research partnerships for pragmatic clinical trials. BMJ. 2014; 349: g6826. PubMed Abstract | Publisher Full Text | Free Full Text
15. Andersson SW, Kyhlstedt M: Deriving more value from RWE to ensure timely access of medicines by patients. J Comp Eff Res. 2017; 6(5): 391–395. PubMed Abstract | Publisher Full Text
16. He Z, Chandar P, Ryan P, et al.: Simulation-based evaluation of the generalizability index for study traits. AMIA Annu Symp Proc. 2015; 2015: 594–603. PubMed Abstract | Free Full Text
17. Schmidt AF, Groenwold RH, van Delden JJ, et al.: Justification of exclusion criteria was underreported in a review of cardiovascular trials. J Clin Epidemiol. 2014; 67(6): 635–44. PubMed Abstract | Publisher Full Text
18. Loudon K, Treweek S, Sullivan F, et al.: The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015; 350: h2147. PubMed Abstract | Publisher Full Text
19. Bakerly ND, Woodcock A, New JP, et al.: The Salford Lung Study protocol: a pragmatic, randomised phase III real-world effectiveness trial in chronic obstructive pulmonary disease. Respir Res. 2015; 16(1): 101. PubMed Abstract | Publisher Full Text | Free Full Text
20. Vestbo J, Leather D, Diar Bakerly N, et al.: Effectiveness of fluticasone furoate-vilanterol for COPD in clinical practice. N Engl J Med. 2016; 375(13): 1253–1260. PubMed Abstract | Publisher Full Text
21. Janson C, Larsson K, Lisspers KH, et al.: Pneumonia and pneumonia related mortality in patients with COPD treated with fixed combinations of inhaled corticosteroid and long acting β₂ agonist: observational matched cohort study (PATHOS). BMJ. 2013; 346: f3306. PubMed Abstract | Publisher Full Text | Free Full Text
22. Wicks P, Vaughan TE, Massagli MP, et al.: Accelerated clinical discovery using self-reported patient data collected online and a patient-matching algorithm. Nat Biotechnol. 2011; 29(5): 411–14. PubMed Abstract | Publisher Full Text
23. Fornai F, Longone P, Cafaro L, et al.: Lithium delays progression of amyotrophic lateral sclerosis. Proc Natl Acad Sci U S A. 2008; 105(6): 2052–57. PubMed Abstract | Publisher Full Text | Free Full Text
24. van Eijk RPA, Jones AR, Sproviero W, et al.: Meta-analysis of pharmacogenetic interactions in amyotrophic lateral sclerosis clinical trials. Neurology. 2017; 89(18): 1915–1922. PubMed Abstract | Publisher Full Text | Free Full Text
25. Morton SC, Costlow MR, Graff JS, et al.: Standards and guidelines for observational studies: quality is in the eye of the beholder. J Clin Epidemiol. 2016; 71: 3–10. PubMed Abstract | Publisher Full Text
26. Wilkinson MD, Dumontier M, Aalbersberg IJ, et al.: The FAIR Guiding Principles for scientific data management and stewardship. Sci Data. 2016; 3: 160018. PubMed Abstract | Publisher Full Text | Free Full Text
27. von Elm E, Altman DG, Egger M, et al.: The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol. 2008; 61(4): 344–9. PubMed Abstract | Publisher Full Text
28. Stroup DF, Berlin JA, Morton SC, et al.: Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA. 2000; 283(15): 2008–12. PubMed Abstract | Publisher Full Text
29. Benchimol EI, Smeeth L, Guttmann A, et al.: The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement. PLoS Med. 2015; 12(10): e1001885. PubMed Abstract | Publisher Full Text | Free Full Text
30. Perfetto EM, Burke L, Oehrlein EM, et al.: FDAMA Section 114: Why the Renewed Interest? J Manag Care Spec Pharm. 2015; 21(5): 368–74. PubMed Abstract | Publisher Full Text
31. Drug and device manufacturer communications with payors, formulary committees, and similar entities – questions and answers guidance for industry and review staff draft guidance. U.S. Department of Health and Human Services Food and Drug Administration. (Accessed 9 November 2017). Reference Source
32. White R, Carter G, Willet J: RWE: A brave new world for the medical publications professional. The map newsletter (International Society for Medical Publication Professionals (ISMPP). 2017; (Accessed 9 November 2017). Reference Source
33. Farrugia P, Petrisor BA, Farrokhyar F, et al.: Practical tips for surgical research: research questions, hypotheses and objectives. Can J Surg. 2010; 53(4): 278–81. PubMed Abstract | Free Full Text
34. Berger ML, Sox H, Willke RJ, et al.: Good practices for real-world data studies of treatment and/or comparative effectiveness: Recommendations from the joint ISPOR-ISPE special task force on real-world evidence in health care decision making. Value Health. 2017; 20(8): 1003–1008. PubMed Abstract | Publisher Full Text
35. Wang SV, Schneeweiss S, Berger ML, et al.: Reporting to improve reproducibility and facilitate validity assessment for healthcare database studies V1.0. Value Health. 2017; 20(8): 1009–1022. PubMed Abstract | Publisher Full Text
36. White R: Building trust in real-world evidence and comparative effectiveness research: the need for transparency. J Comp Eff Res. 2017; 6(1): 5–7. PubMed Abstract | Publisher Full Text

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 25 Jan 2018

Author details Author details

¹ AstraZeneca Academy House, Cambridge, CB2 1PG, UK
² AstraZeneca, Gaithersburg, MD, 20878, USA
³ AstraZeneca, Madrid, 28033, Spain
⁴ AstraZeneca AB R&D, Mölndal, 431 50, Sweden
⁵ AstraZeneca, Luton, LU1 3LU, UK
⁶ Oxford PharmaGenesis, Tubney Warren Barn, Tubney, Abingdon, OX13 5QJ, UK

Sajan Khosla
Roles: Conceptualization, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing

Robert White
Roles: Writing – Review & Editing

Jesús Medina
Roles: Writing – Original Draft Preparation, Writing – Review & Editing

Mario Ouwens
Roles: Writing – Review & Editing

Cathy Emmas
Roles: Writing – Original Draft Preparation, Writing – Review & Editing

Tim Koder
Roles: Writing – Original Draft Preparation, Writing – Review & Editing

Gary Male
Roles: Writing – Original Draft Preparation, Writing – Review & Editing

Sandra Leonard
Roles: Writing – Review & Editing

Competing interests

Sajan Khosla, Robert White, Jesús Medina and Sandra Leonard are employees of AstraZeneca, in the Medical Evidence and Observational Research team. Mario Ouwens is an employee of AstraZeneca in the Advanced Analytics team. Cathy Emmas is an employee of AstraZeneca in the Patient Centricity team. Tim Koder and Gary Male are employees of Oxford PharmaGenesis, whose work on this manuscript has been funded by AstraZeneca.

Grant information

Development of this manuscript was funded by AstraZeneca; writing was undertaken as part of an internal project by employees of AstraZeneca and Oxford PharmaGenesis.

Article Versions (2)

version 2

Revised

Published: 29 Aug 2018, 7:111

https://doi.org/10.12688/f1000research.13585.2

version 1

Published: 25 Jan 2018, 7:111

https://doi.org/10.12688/f1000research.13585.1

Copyright

© 2018 Khosla S et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

0

SEE MORE DETAILS

CITE

how to cite this article

Khosla S, White R, Medina J et al. Real world evidence (RWE) – a disruptive innovation or the quiet evolution of medical evidence generation? [version 2; peer review: 2 approved] F1000Research 2018, 7:111 (https://doi.org/10.12688/f1000research.13585.2)

NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status: ?

Key to Reviewer Statuses VIEW HIDE

ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions

Version 1

VERSION 1

PUBLISHED 25 Jan 2018

Views

36

Reviewer Report 02 Jul 2018

Mattias Kyhlstedt, Synergus AB, Stockholm, Sweden

Approved

https://doi.org/10.5256/f1000research.14756.r35542

The article provides a good overview of the topic.
I would suggest the following adjustments / comments:

Following this statement “Owing to the mechanism of data collection and experimental design, RWE studies generally cannot yield

The article provides a good overview of the topic.
I would suggest the following adjustments / comments:

Following this statement “Owing to the mechanism of data collection and experimental design, RWE studies generally cannot yield definitive causal inference because of the many confounders of variability.”
- I think it would be good to reference Eichler et al., “Bridging the Efficacy–Effectiveness Gap.”¹. About the implications of moving from traditional RCT to RWE and the implications that comes with this. i.e one should actively consider the confounders, not only identify that is a potential problem.

Table 1:
- Being the co-author of the article : “Deriving more value from RWE to ensure timely access of medicines by patients”². I think this would be an appropriate reference. I would suggest that this would add the following perspective:
  - The structured search to identify the relevant real world data sources as an essential step for successful RWE execution.
Table 4:
- In the planning phase it is essential to understand the confounding factors that may impact the outcome question of interest in order to take this in to account in the study design / evaluation of the RWE study.

Is the topic of the opinion article discussed accurately in the context of the current literature?

Yes
Are all factual statements correct and adequately supported by citations?

Yes
Are arguments sufficiently supported by evidence from the published literature?

Yes
Are the conclusions drawn balanced and justified on the basis of the presented arguments?

Yes

References

1. Eichler H, Abadie E, Breckenridge A, Flamion B, et al.: Bridging the efficacy–effectiveness gap: a regulator's perspective on addressing variability of drug response. Nature Reviews Drug Discovery. 2011; 10 (7): 495-506 Publisher Full Text
2. Andersson SW, Kyhlstedt M: Deriving more value from RWE to ensure timely access of medicines by patients.J Comp Eff Res. 2017; 6 (5): 391-395 PubMed Abstract | Publisher Full Text

Competing Interests: No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Respond or Comment

Views

53

Reviewer Report 29 Jan 2018

Marc L. Berger, SHYFT Analytics, Waltham, MA, USA

Approved

https://doi.org/10.5256/f1000research.14756.r30250

This is a nice summary of the state of play with respect to real world evidence.

Some minor changes and additions would be worthwhile:

Page 3: 2nd paragraph, last sentence: Owing to the mechanism

This is a nice summary of the state of play with respect to real world evidence.

Some minor changes and additions would be worthwhile:

Page 3: 2nd paragraph, last sentence: Owing to the mechanism of data collection and experimental design, RWE studies generally cannot yield definitive causal inference...." The word cannot should be changed to may not. It is a matter of discussion whether one can impute causality vs causation from observational studies (see Berger et al.¹)
Page 5: Discussion of Phase 1-3 Clinical Study Design and Table 2: RWE can inform clinical study design in other ways including assessing size of recruitment pool of patients with different sets of inclusion or exclusion criteria; estimating treatment effect size for sample size estimation; simulating trials in advance; and recruitment of sites.
Page 7: Table 3: Next to Electronic health/medical records, Flatiron Health should be added to the list data owner/curators. They are currently working with the FDA on data quality issues.
Page 8, Table 4: Next to communication, the authors should cite the various ISPOR publications on the subject Including:
- Caro et al.²;
- Recommendations from the joint ISPOR‐ISPE Special Task Force on real‐world evidence in health care decision making, co-published in 2 papers in ViH³ and PEpi and Drug Safety⁴;
- Wang et al.⁵^,⁶.

Is the topic of the opinion article discussed accurately in the context of the current literature?

Yes
Are all factual statements correct and adequately supported by citations?

Yes
Are arguments sufficiently supported by evidence from the published literature?

Yes
Are the conclusions drawn balanced and justified on the basis of the presented arguments?

Yes

References

1. Berger ML, Dreyer N, Anderson F, Towse A, et al.: Prospective observational studies to assess comparative effectiveness: the ISPOR good research practices task force report.Value Health. 15 (2): 217-30 PubMed Abstract | Publisher Full Text
2. Jaime Caro J, Eddy DM, Kan H, Kaltz C, et al.: Questionnaire to assess relevance and credibility of modeling studies for informing health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report.Value Health. 2014; 17 (2): 174-82 PubMed Abstract | Publisher Full Text
3. Berger ML, Sox H, Willke RJ, Brixner DL, et al.: Good Practices for Real-World Data Studies of Treatment and/or Comparative Effectiveness: Recommendations from the Joint ISPOR-ISPE Special Task Force on Real-World Evidence in Health Care Decision Making.Value Health. 2017; 20 (8): 1003-1008 PubMed Abstract | Publisher Full Text
4. Berger ML, Sox H, Willke RJ, Brixner DL, et al.: Good practices for real-world data studies of treatment and/or comparative effectiveness: Recommendations from the joint ISPOR-ISPE Special Task Force on real-world evidence in health care decision making.Pharmacoepidemiol Drug Saf. 2017; 26 (9): 1033-1039 PubMed Abstract | Publisher Full Text
5. Wang SV, Schneeweiss S, Berger ML, Brown J, et al.: Reporting to Improve Reproducibility and Facilitate Validity Assessment for Healthcare Database Studies V1.0.Pharmacoepidemiol Drug Saf. 26 (9): 1018-1032 PubMed Abstract | Publisher Full Text
6. Wang SV, Schneeweiss S, Berger ML, Brown J, et al.: Reporting to Improve Reproducibility and Facilitate Validity Assessment for Healthcare Database Studies V1.0.Value Health. 2017; 20 (8): 1009-1022 PubMed Abstract | Publisher Full Text

Competing Interests: No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Respond or Comment

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 25 Jan 2018

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 2 (revision) 29 Aug 18
Version 1 25 Jan 18	read	read

Marc L. Berger, SHYFT Analytics, Waltham, USA
Mattias Kyhlstedt, Synergus AB, Stockholm, Sweden

Comments on this article

All Comments(0)

Add a comment

Sign up for content alerts

Browse by related subjects

Back to all reports

Reviewer Report

36 Views

02 Jul 2018 | for Version 1

Mattias Kyhlstedt, Synergus AB, Stockholm, Sweden

36 Views Cite this report Responses(0)

Approved

The article provides a good overview of the topic.
I would suggest the following adjustments / comments:

Following this statement “Owing to the mechanism of data collection and experimental design, RWE studies generally cannot yield definitive causal inference because of the many confounders of variability.”
- I think it would be good to reference Eichler et al., “Bridging the Efficacy–Effectiveness Gap.”¹. About the implications of moving from traditional RCT to RWE and the implications that comes with this. i.e one should actively consider the confounders, not only identify that is a potential problem.

Table 1:
- Being the co-author of the article : “Deriving more value from RWE to ensure timely access of medicines by patients”². I think this would be an appropriate reference. I would suggest that this would add the following perspective:
  - The structured search to identify the relevant real world data sources as an essential step for successful RWE execution.
Table 4:
- In the planning phase it is essential to understand the confounding factors that may impact the outcome question of interest in order to take this in to account in the study design / evaluation of the RWE study.

Is the topic of the opinion article discussed accurately in the context of the current literature?

Yes
Are all factual statements correct and adequately supported by citations?

Yes
Are arguments sufficiently supported by evidence from the published literature?

Yes
Are the conclusions drawn balanced and justified on the basis of the presented arguments?

Yes

References

1. Eichler H, Abadie E, Breckenridge A, Flamion B, et al.: Bridging the efficacy–effectiveness gap: a regulator's perspective on addressing variability of drug response. Nature Reviews Drug Discovery. 2011; 10 (7): 495-506 Publisher Full Text
2. Andersson SW, Kyhlstedt M: Deriving more value from RWE to ensure timely access of medicines by patients.J Comp Eff Res. 2017; 6 (5): 391-395 PubMed Abstract | Publisher Full Text

Competing Interests

No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

53 Views

29 Jan 2018 | for Version 1

Marc L. Berger, SHYFT Analytics, Waltham, MA, USA

53 Views Cite this report Responses(0)

Approved

This is a nice summary of the state of play with respect to real world evidence.

Some minor changes and additions would be worthwhile:

Page 3: 2nd paragraph, last sentence: Owing to the mechanism of data collection and experimental design, RWE studies generally cannot yield definitive causal inference...." The word cannot should be changed to may not. It is a matter of discussion whether one can impute causality vs causation from observational studies (see Berger et al.¹)
Page 5: Discussion of Phase 1-3 Clinical Study Design and Table 2: RWE can inform clinical study design in other ways including assessing size of recruitment pool of patients with different sets of inclusion or exclusion criteria; estimating treatment effect size for sample size estimation; simulating trials in advance; and recruitment of sites.
Page 7: Table 3: Next to Electronic health/medical records, Flatiron Health should be added to the list data owner/curators. They are currently working with the FDA on data quality issues.
Page 8, Table 4: Next to communication, the authors should cite the various ISPOR publications on the subject Including:
- Caro et al.²;
- Recommendations from the joint ISPOR‐ISPE Special Task Force on real‐world evidence in health care decision making, co-published in 2 papers in ViH³ and PEpi and Drug Safety⁴;
- Wang et al.⁵^,⁶.

Is the topic of the opinion article discussed accurately in the context of the current literature?

Yes
Are all factual statements correct and adequately supported by citations?

Yes
Are arguments sufficiently supported by evidence from the published literature?

Yes
Are the conclusions drawn balanced and justified on the basis of the presented arguments?

Yes

References

1. Berger ML, Dreyer N, Anderson F, Towse A, et al.: Prospective observational studies to assess comparative effectiveness: the ISPOR good research practices task force report.Value Health. 15 (2): 217-30 PubMed Abstract | Publisher Full Text
2. Jaime Caro J, Eddy DM, Kan H, Kaltz C, et al.: Questionnaire to assess relevance and credibility of modeling studies for informing health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report.Value Health. 2014; 17 (2): 174-82 PubMed Abstract | Publisher Full Text
3. Berger ML, Sox H, Willke RJ, Brixner DL, et al.: Good Practices for Real-World Data Studies of Treatment and/or Comparative Effectiveness: Recommendations from the Joint ISPOR-ISPE Special Task Force on Real-World Evidence in Health Care Decision Making.Value Health. 2017; 20 (8): 1003-1008 PubMed Abstract | Publisher Full Text
4. Berger ML, Sox H, Willke RJ, Brixner DL, et al.: Good practices for real-world data studies of treatment and/or comparative effectiveness: Recommendations from the joint ISPOR-ISPE Special Task Force on real-world evidence in health care decision making.Pharmacoepidemiol Drug Saf. 2017; 26 (9): 1033-1039 PubMed Abstract | Publisher Full Text
5. Wang SV, Schneeweiss S, Berger ML, Brown J, et al.: Reporting to Improve Reproducibility and Facilitate Validity Assessment for Healthcare Database Studies V1.0.Pharmacoepidemiol Drug Saf. 26 (9): 1018-1032 PubMed Abstract | Publisher Full Text
6. Wang SV, Schneeweiss S, Berger ML, Brown J, et al.: Reporting to Improve Reproducibility and Facilitate Validity Assessment for Healthcare Database Studies V1.0.Value Health. 2017; 20 (8): 1009-1022 PubMed Abstract | Publisher Full Text

Competing Interests

No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

[1] 1. Berger M, Daniel G, Frank K, et al.: A framework for regulatory use of real-world evidence. Duke Margolis Center for Health Policy White Paper. 2017; (Accessed 26 October 2017). Reference Source

[2] 2. Eichler HG, Abadie E, Breckenridge A, et al.: Bridging the efficacy-effectiveness gap: a regulator's perspective on addressing variability of drug response. Nat Rev Drug Discov. 2011; 10(7): 495–506. PubMed Abstract | Publisher Full Text

[3] 3. Berger ML, Dreyer N, Anderson F, et al.: Prospective observational studies to assess comparative effectiveness: the ISPOR good research practices task force report. Value Health. 2012; 15(2): 217–30. PubMed Abstract | Publisher Full Text

[4] 4. Tanaka S, Yamamoto T, Oda E, et al.: Real-world evidence of raloxifene versus alendronate in preventing non-vertebral fractures in Japanese women with osteoporosis: retrospective analysis of a hospital claims database. J Bone Miner Metab. 2018; 36(1): 87–94. PubMed Abstract | Publisher Full Text

[5] 5. Dang A, Vallish BN: Real world evidence: An Indian perspective. Perspect Clin Res. 2016; 7(4): 156–60. PubMed Abstract | Publisher Full Text | Free Full Text

[6] 6. Makady A, Ham RT, de Boer A, et al.: Policies for use of real-world data in health technology assessment (HTA): a comparative study of six HTA agencies. Value Health. 2017; 20(4): 520–32. PubMed Abstract | Publisher Full Text

[7] 7. Epstein RS, Sidorov J, Lehner JP, et al.: Integrating scientific and real-world evidence within and beyond the drug development process. J Comp Eff Res. 2012; 1(1 Suppl): 9–13. PubMed Abstract | Publisher Full Text

[8] 8. Hughes B, Kessler M: Breaking new ground with RWE: how some pharmacos are poised to realize a $1 billion opportunity. IMS Health White Paper. 2014; (Accessed 9 November 2017). Reference Source

[9] 9. Berger ML, Lipset C, Gutteridge A, et al.: Optimizing the leveraging of real-world data to improve the development and use of medicines. Value Health. 2015; 18(1): 127–30. PubMed Abstract | Publisher Full Text

[10] 10. Ronicke V, Ruhl M, Solbach T: Revitalizing pharmaceutical R&D. The value of real word evidence. Strategy&. 2015; (Accessed 9 November 2017). Reference Source

[11] 11. Communicating comparative effectiveness research and real world evidence with population health decision makers. GSK U.S. Public Policy Position Paper. (Accessed 9 November 2017). Reference Source

[12] 12. Galson S, Simon G: Real-world evidence to guide approval and use of new treatments. National Academy of Medicine, Washington, DC. 2015; (Accessed 9 November 2017). Reference Source

[13] 13. News, views and insights from leading international RWE experts. Quintiles IMS. AccessPoint. (Accessed 9 November 2017). Reference Source

[14] 14. Johnson KE, Tachibana C, Coronado GD, et al.: A guide to research partnerships for pragmatic clinical trials. BMJ. 2014; 349: g6826. PubMed Abstract | Publisher Full Text | Free Full Text

[15] 15. Andersson SW, Kyhlstedt M: Deriving more value from RWE to ensure timely access of medicines by patients. J Comp Eff Res. 2017; 6(5): 391–395. PubMed Abstract | Publisher Full Text

[16] 16. He Z, Chandar P, Ryan P, et al.: Simulation-based evaluation of the generalizability index for study traits. AMIA Annu Symp Proc. 2015; 2015: 594–603. PubMed Abstract | Free Full Text

[17] 17. Schmidt AF, Groenwold RH, van Delden JJ, et al.: Justification of exclusion criteria was underreported in a review of cardiovascular trials. J Clin Epidemiol. 2014; 67(6): 635–44. PubMed Abstract | Publisher Full Text

[18] 18. Loudon K, Treweek S, Sullivan F, et al.: The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015; 350: h2147. PubMed Abstract | Publisher Full Text

[19] 19. Bakerly ND, Woodcock A, New JP, et al.: The Salford Lung Study protocol: a pragmatic, randomised phase III real-world effectiveness trial in chronic obstructive pulmonary disease. Respir Res. 2015; 16(1): 101. PubMed Abstract | Publisher Full Text | Free Full Text

[20] 20. Vestbo J, Leather D, Diar Bakerly N, et al.: Effectiveness of fluticasone furoate-vilanterol for COPD in clinical practice. N Engl J Med. 2016; 375(13): 1253–1260. PubMed Abstract | Publisher Full Text

[21] 21. Janson C, Larsson K, Lisspers KH, et al.: Pneumonia and pneumonia related mortality in patients with COPD treated with fixed combinations of inhaled corticosteroid and long acting β₂ agonist: observational matched cohort study (PATHOS). BMJ. 2013; 346: f3306. PubMed Abstract | Publisher Full Text | Free Full Text

[22] 22. Wicks P, Vaughan TE, Massagli MP, et al.: Accelerated clinical discovery using self-reported patient data collected online and a patient-matching algorithm. Nat Biotechnol. 2011; 29(5): 411–14. PubMed Abstract | Publisher Full Text

[23] 23. Fornai F, Longone P, Cafaro L, et al.: Lithium delays progression of amyotrophic lateral sclerosis. Proc Natl Acad Sci U S A. 2008; 105(6): 2052–57. PubMed Abstract | Publisher Full Text | Free Full Text

[24] 24. van Eijk RPA, Jones AR, Sproviero W, et al.: Meta-analysis of pharmacogenetic interactions in amyotrophic lateral sclerosis clinical trials. Neurology. 2017; 89(18): 1915–1922. PubMed Abstract | Publisher Full Text | Free Full Text

[25] 25. Morton SC, Costlow MR, Graff JS, et al.: Standards and guidelines for observational studies: quality is in the eye of the beholder. J Clin Epidemiol. 2016; 71: 3–10. PubMed Abstract | Publisher Full Text

[26] 26. Wilkinson MD, Dumontier M, Aalbersberg IJ, et al.: The FAIR Guiding Principles for scientific data management and stewardship. Sci Data. 2016; 3: 160018. PubMed Abstract | Publisher Full Text | Free Full Text

[27] 27. von Elm E, Altman DG, Egger M, et al.: The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol. 2008; 61(4): 344–9. PubMed Abstract | Publisher Full Text

[28] 28. Stroup DF, Berlin JA, Morton SC, et al.: Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA. 2000; 283(15): 2008–12. PubMed Abstract | Publisher Full Text

[29] 29. Benchimol EI, Smeeth L, Guttmann A, et al.: The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement. PLoS Med. 2015; 12(10): e1001885. PubMed Abstract | Publisher Full Text | Free Full Text

[30] 30. Perfetto EM, Burke L, Oehrlein EM, et al.: FDAMA Section 114: Why the Renewed Interest? J Manag Care Spec Pharm. 2015; 21(5): 368–74. PubMed Abstract | Publisher Full Text

[31] 31. Drug and device manufacturer communications with payors, formulary committees, and similar entities – questions and answers guidance for industry and review staff draft guidance. U.S. Department of Health and Human Services Food and Drug Administration. (Accessed 9 November 2017). Reference Source

[32] 32. White R, Carter G, Willet J: RWE: A brave new world for the medical publications professional. The map newsletter (International Society for Medical Publication Professionals (ISMPP). 2017; (Accessed 9 November 2017). Reference Source

[33] 33. Farrugia P, Petrisor BA, Farrokhyar F, et al.: Practical tips for surgical research: research questions, hypotheses and objectives. Can J Surg. 2010; 53(4): 278–81. PubMed Abstract | Free Full Text

[34] 34. Berger ML, Sox H, Willke RJ, et al.: Good practices for real-world data studies of treatment and/or comparative effectiveness: Recommendations from the joint ISPOR-ISPE special task force on real-world evidence in health care decision making. Value Health. 2017; 20(8): 1003–1008. PubMed Abstract | Publisher Full Text

[35] 35. Wang SV, Schneeweiss S, Berger ML, et al.: Reporting to improve reproducibility and facilitate validity assessment for healthcare database studies V1.0. Value Health. 2017; 20(8): 1009–1022. PubMed Abstract | Publisher Full Text

[36] 36. White R: Building trust in real-world evidence and comparative effectiveness research: the need for transparency. J Comp Eff Res. 2017; 6(1): 5–7. PubMed Abstract | Publisher Full Text

Real world evidence (RWE) – a disruptive innovation or the quiet evolution of medical evidence generation?

Abstract

Keywords

Revised Amendments from Version 1

Introduction

Table 1. Selected industry perspectives on RWE.

Figure 1. The RWE journey – from concept to use.

Potential for RWE in drug lifecycles

Figure 2. RWE is used throughout the product lifecycle.

Table 2. Evidence needs across the product lifecycle.

Disease strategy and early discovery disease segmentation

Phase 1–3 clinical study design

Indication-seeking and label-expanding studies

Pragmatic clinical trials – a hybrid approach

Box 1. Case study: The Salford Lung Study

Post-authorization studies

Real world data

Box 2. Case study: PatientsLikeMe study in amyotrophic lateral sclerosis22

Table 3. Example sources of real world dataa.

Data access

Data quality

Table 4. Considerations in the generation of high-quality real world data.

Communication of RWE

Conclusions

Grant information

Acknowledgements

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

Competing Interests Policy

Stay Updated

Table 3. Example sources of real world data^a.