Routine provision of information on patient‐reported outcome measures to healthcare providers and patients in clinical practice
Abstract
This is a protocol for a Cochrane Review (Intervention). The objectives are as follows:
To assess the impact of the routine use of patient‐reported outcomes (PROs) in clinical practice on the process of care (including patient‐physician communication, professionals’ awareness of patients' quality of life, diagnosis and recognition rates, treatment rates, health services and resource use, as well as patient behaviour); patients' and professionals' experiences of care; and health outcomes (both generic and disease‐specific, using both routinely‐used clinical measures and PROs).
Background
Description of the condition
Definition of patient‐reported outcome measures
Patient‐reported outcomes (PROs) measure the patient’s subjective appraisal of outcomes from their own perspective (Valderas 2008b). PRO measures offer complementary information to the objective measurements usually collected in clinical practice to support decision making (e.g. blood tests, imaging, functional tests). PROs allow for systematised collection of subjective information on health status and offer an opportunity for improving processes and outcomes of health care. The potential benefits of their use in clinical practice range from screening, diagnosing, and monitoring to promoting patient‐centred care (Greenhalgh 2009).
Historically, the use of PRO measures has been far less common in clinical practice than in research, where PROs are often selected as outcome measures in clinical trials (Fitzpatrick 1998; FDA 2009). At an individual level and within the clinician‐patient interface, PRO measures have been used for screening and monitoring a condition, such as depression symptoms; for monitoring the progress of the patient during the course of treatment or throughout time; and for promoting patient‐centred care, by explicitly assessing the patient’s perspective (Greenhalgh 2009).
Description of the intervention
PROs have been defined as assessments of any aspect of a patient’s health status which are provided directly by the patient (Valderas 2008b; FDA 2009 ), usually through a questionnaire scale. PRO is an umbrella term: it can be applied to an array of different outcomes, including symptoms, functioning, perceived health status and health‐related quality of life (McKenna 2011).
PROs can measure generic aspects of health. One such example is the Short Form 36, which assesses physical functioning and psychological wellbeing, as well as evaluating overall health (Garratt 1993). In theory, such generic measures can be used within and between populations, regardless of their age, gender, and disease or condition. Greater effort, however, has been devoted to the development of PRO measures for specific diseases or conditions (Garratt 2002), from common conditions such as diabetes (Bradley 1999), to less frequent ones, including amyotrophic lateral sclerosis (Gibbons 2011), and haemophilia (Arranz 2004).
The use of PRO measures in clinical practice at a patient level can be defined as a complex intervention, including different components (Craig 2008). In its most basic form, in a typical PRO intervention the patient is given one or more questionnaires to complete, usually about their health status or health‐related quality of life, and results are then fed back to the healthcare professional. The International Society for Quality of Life Research recently operationalised all the aspects that should be taken into account when trying to implement PROs in clinical practice, suggesting a set of eight methodological steps to be followed (Snyder 2012). These steps are establishing goals; identifying patients and settings; selecting questionnaires; defining the administration and scoring procedures; reporting results; facilitating score interpretation; establishing protocols to address issues raised by the questionnaires; and assessing the eventual impact of the questionnaire in clinical practice.
While these standardised steps can be found in almost all interventions using PRO measures, considerable variation exists between trials. For instance, instruments can be self‐completed (Rand 1988) or interviewer‐administered (German 1987); completed in the clinical setting (Christensen 2005) or posted to the patient’s home (Lewis 1996); and supported by an electronic format such as online or tablet administration (Velikova 2004) or rely on pencil and paper (Trowbridge 1997). As for the feedback, discrepancies might exist between trials as to when the information is given to healthcare professionals, e.g. immediately before the visit (Berry 2011); how it is given, e.g. printed form (Saitz 2003); and by whom, e.g. available in the notes (Linn 1980). More importantly, considerable differences occur regarding the amount of feedback provided. For example, in some studies the only information fed back to healthcare professionals were the scores each patient obtained in the PRO measure (Bergus 2005), whereas in other studies professionals were given information on how to apply interpretation guidelines for the scores (Rosenbloom 2007), or treatment guidelines for the conditions detected by the PRO measure (Saitz 2003). The number of times the patient completes the measure and the information is then fed back to the professional can also vary considerably, from single responses (Hoeper 1984) to multiple feedbacks (Klinkhammer‐Schalke 2012). Reflecting this, there is also variation in whether the clinician receives the scores at a single point in time or the patient scores over a period of time. Finally, the endpoints used to assess the impact of PROs in clinical practice have also been a source of considerable discrepancy, with trials inconsistently reporting on processes of healthcare (e.g. patient‐clinician communication), outcomes of healthcare (e.g. changes in the number or rate of symptoms or complaints), and patient experience (e.g. overall satisfaction with care).
How the intervention might work
The Feedback Intervention Theory (FIT) relies on the assumption that behaviour is regulated through comparison with standards or goals, and that feedback can draw attention to existing gaps (Kluger 1996). If a patient scores above the established cut‐off point in a depression screening scale, then the healthcare professional will be made aware of this discrepancy between the desired state of psychological wellbeing and the current distress experienced by the patient. FIT further postulates that once the gap has been identified, different methods can be followed in order to decrease it and attain the standard, including increasing the effort currently done (Kluger 1996). This could be substantiated by the professional using several strategies, including providing advice, referring to other services, or altering the medication plan. All of these are proximal outcomes that would, potentially, trigger more distal outcomes, such as improved functioning and increased health‐related quality of life. However, whether these outcomes do materialise depends on a range of other contextual factors such as the patient’s acceptance of, and adherence to, any treatment changes and the effectiveness of that treatment.
Why it is important to do this review
In the UK, PROs are one of the cornerstones of the current reform of the National Health Service for the transition towards an outcomes‐oriented performance model. In the US, initiatives such as the Patient Reported Outcomes Measurement Information System (PROMIS 2007), funded by the National Institutes of Health, or the inclusion of PROs in electronic health record software, such as EpicCare (EpicCare 2015) held by Group Health Cooperative, highlight the progressive relevance these outcome measures play in healthcare contexts. The US Department of Health and Human Services also plans to incorporate PRO into meaningful use standards, which is likely to prompt more widespread use (Hostetter 2011).
The level of evidence for the impact of using PROs in clinical practice has been mixed (Espallargues 2000; Gilbody 2001; Greenhalgh 1999; Marshall 2006; Valderas 2008a). Valderas 2008a found that there was more evidence for impact upon the processes rather than the outcomes of care). Specifically, there was an increase for the rate of diagnoses and chart notations for the conditions targeted by the interventions (e.g. diagnosis of depression in primary care). Similarly, there was also a positive effect on the advice and education provided by the healthcare professionals. Furthermore, Valderas 2008a identified a total of 36 endpoints for the 28 randomised controlled trials (RCTs) included in their systematic review, which seems to reiterate the lack of consensus amongst researchers of how the intervention should work and thus what constitutes a relevant indicator when using PROs in clinical practice.
Notwithstanding the potential benefits for clinical practice, several objections have been raised in relation to their routine use. Healthcare professionals have expressed doubts about the clinical utility of PRO measures, as they consider that little value is added to their clinical judgement (Leydon 2011; Taylor 1996). Healthcare professionals have also described how burdensome the use of PROs can be, as it requires time to administer the measures and time to learn how to analyse and interpret the results (Brown 2006), and also to integrate them into clinical practice in an efficient and non‐disruptive manner (Nelson 1990). Clinicians have voiced concerns that the PRO measures might represent a threat to the holistic nature of the patient‐doctor relationship (Leydon 2011). It has also been suggested that PRO measures increase the healthcare professional’s responsibility and burden of care, as they might detect problems that could otherwise go unnoticed (Tavabie 2009).
Taking both the potential benefits and risks and the current health policy initiatives into account, it becomes essential to ascertain to what extent the use of PRO measures in clinical practice does actually improve processes and outcomes of care. Previous reviews have provided mixed evidence and a number of relevant studies have been subsequently published (Valderas 2010).
Objectives
To assess the impact of the routine use of patient‐reported outcomes (PROs) in clinical practice on the process of care (including patient‐physician communication, professionals’ awareness of patients' quality of life, diagnosis and recognition rates, treatment rates, health services and resource use, as well as patient behaviour); patients' and professionals' experiences of care; and health outcomes (both generic and disease‐specific, using both routinely‐used clinical measures and PROs).
Methods
Criteria for considering studies for this review
Types of studies
Randomised controlled trials (RCTs) and cluster RCTs, where individuals (healthcare professionals or patients) or groups of individuals (including whole hospitals or practices) were randomly allocated to either a control or an intervention group. We will not include studies that follow a non‐randomised controlled design, such as interrupted before‐and‐after studies and interrupted time series.
Types of participants
We will only include studies where participants have been recruited in primary (e.g. health practitioner’s office) or secondary/tertiary (e.g. hospital) care settings in order to ensure that interventions are delivered as part of clinical care. We will exclude studies conducted outside primary and secondary/tertiary healthcare settings (e.g. assisted living facilities) in order to ensure that PRO information is used for clinical purposes only. There will be no age restriction or gender restriction, nor restrictions based on the presence or absence of any specific disease.
Types of interventions
We will include studies if they report a replicable intervention, where standardised or individualised PRO measures are administered to patients and the resulting information on each individual patient is subsequently fed back to healthcare providers or patients, or both. Patient‐reported outcome (PRO) measures will be defined as the assessment of any aspect of a patient’s health status which is provided directly by the patient (FDA 2009), usually through a questionnaire or scale. PROs may be used for a number of different outcomes, including measurements of health status, quality of life, symptoms and functioning (McKenna 2011). We will include studies regardless of whether information was provided to patients only or to healthcare providers only or to both. Studies will be included irrespective of whether the results were fed back along with guidelines regarding their optimal use, or other educational strategies. Studies will be included if they have been conducted either during a specific procedure, for instance a surgical procedure; or during routine care, for example a primary‐care appointment. The comparison (control) condition will consist of routine clinical practice without the feedback of any information to the healthcare professionals.
Types of outcome measures
Primary outcomes
Our primary outcomes will include generic or disease‐specific patient‐reported outcomes such as health‐related quality of life and functioning.
Secondary outcome measures will be considered for the process of care.
For the processes of health care, the following endpoints will be considered:
-
Patient‐physician communication (e.g. patients' ratings of the quality of the communication);
-
Diagnosis and recognition (e.g. number of target diagnoses made);
-
Treatment (e.g. changes to treatment);
-
Health services and resource use (e.g. referral to specialist or social care);
-
Patient behaviour (e.g. compliance with treatment);
-
Patient empowerment (e.g. measured using available self‐reported instruments); and
-
Healthcare professionals’ awareness of patients' quality of life.
Other outcomes: patients' experiences (e.g. overall satisfaction with care) and healthcare professionals' perceptions (e.g. attitude and overall satisfaction with intervention); consultation length; healthcare costs.
Adverse effects: distress following or related to PRO completion.
Search methods for identification of studies
Electronic searches
We will search the following databases: MEDLINE (In‐Process & Other Non‐Indexed Citations and Ovid MEDLINE(R), 1948 to Present, accessed through OvidSP); EMBASE (1974 to present, accessed through OvidSP); PsycINFO (1967 to present, accessed through OvidSP); and CINAHL (from 1981 to present, accessed through EBSCO). The search strategy will be adapted to the specific requirements of each database, namely through the use of different thesaurus terms where applicable and truncation and wildcard characters. Appendix 1 displays the search strategy for MEDLINE. We will also search the Cochrane Effective Practice and Organisation of Care Group specialised register, and the Cochrane Central Register of Controlled Trials (CENTRAL), the Cochrane Database of Systematic Reviews (CDSR), and the Database of Abstracts of Reviews of Effects (DARE). We will not apply language restrictions, but searches will always be conducted in English. Studies in languages other than English will be included.
Searching other resources
Additionally, we will identify on‐going trials using the online trials registry of the National Institutes of Health (US) (interventional studies with interventions: "patient reported outcomes" OR "patient reported outcome" OR "quality of life" OR "functional status", and all default settings); and the World Health Organization International Clinical Trials Registry Platform, using the same search criteria. All the documents deemed as relevant, i.e. those that are chosen to be included in the review after full text evaluation, will be subjected to a forward citation search using Web of Science. Previously published reviews will also be screened for potentially‐relevant references. We will contact authors of the included studies to request information about on‐going studies.
Data collection and analysis
Selection of studies
Two reviewers will independently assess each reference in title and abstract form to ascertain whether they meet the eligibility criteria. We will pilot the eligibility criteria against a random sample of approximately 1% of all the documents received, after which two reviewers will independently screen all of the references. Because we will be aiming for maximum sensitivity at this stage, we will include all references assessed as relevant by at least one team member, and will only exclude references unanimously assessed as irrelevant.
We will follow the same strategy for the full text documents selected for inclusion in the review. Again, we will conduct a sensitivity strategy with a random sample of approximately 1% of the records. As at this stage it is desirable for maximum specificity to be achieved, we will discuss disagreements between the team members until consensus is reached, and we will only include references rated as relevant by all the reviewers. We will contact a third reviewer if consensus is not achievable. Whenever pertinent and possible, we will contact authors for the documents that received a discrepant rating, in order to clarify any queries. We will document the selection process through a PRISMA flow diagram. We will describe all the studies that fulfil the inclusion criteria in the 'Characteristics of included studies' table.
Data extraction and management
We will independently save all the retrieved results to a bibliographic database using reference management software (Reuters 2011). Will will save all the results and remove any duplicates. Two reviewers will independently extract data from the studies assessed as relevant during the stage of study, and we will resolve any disagreements through discussion. We will design the data extraction form according to aspects considered to be relevant for the present systematic review, including those suggested by the Cochrane Effective Practice and Organisation of Care Group (EPOC 2014), and will cover the following domains:
a) Study features: clinical setting (type of setting, academic status, and country); method of randomisation (including allocation concealment and blinding); unit of randomisation and analysis (patient/healthcare professional or practice/hospital); number of arms;
b) Participants' features: inclusion and exclusion criteria; patients' characteristics (sociodemographic information using the PROGRESS framework; health condition; and whether new or known to the healthcare professional); healthcare professionals' characteristics (profession; level of training; and previous experiences with PRO measures); number of participants;
c) Intervention features: design, which may be: Single simple feedback (one PRO at a single time); Multiple simple feedback (one PRO at multiple times); Single complex feedback (multiple PROs at a single time); Multiple complex feedback (multiple PROs at multiple times); and how PROs were used (which may be for the intervention or for assessing outcomes, or both);
d) Administration features: method for data collection (self‐reported; interviewer; other); support used (pencil and paper; computer‐assisted; other); setting of data collection (home; clinical; other); facilitator (no facilitator; clinical facilitator; research facilitator; other); other relevant administration‐related characteristics;
e) Feedback: timing (associated with visits or not; scores given before appointments, during or other); amount of information provided (last score; previous scores; application of interpretation guidelines; application of treatment guidelines; other); support used (printed form; computer‐assisted; other); method for feeding back the information (handed by patients; handed by research staff; available in notes; other);
f) Description of the intervention: narrative description as provided by authors;
g) Results: results as provided by authors, both for processes and outcomes of care;
h) Other features: study identifier; source of funding; ethical approval; sample size calculation; prospectively‐identified barriers to change; methodological quality.
Complex health interventions might pose specific challenges to assessment (Craig 2008); and data synthesis (Shepperd 2009). Specific recommendations on how to overcome these limitations have now been suggested, including identifying key components of the interventions and categorising them according to those components (Shepperd 2009). When extracting data, we will thus also categorise the identified interventions according to their main components.
Given the likely heterogeneity of outcomes in this review we propose to handle the outcome results in a two‐stage approach. In the first stage, we will:
1) Collate data according to the headings outlined in the 'Types of outcome measures' section;
2) Extract the appropriate data for each arm according to the principle of intention to treat (i.e. according to the original random allocation). For dichotomous data: number of patients experiencing outcome/total patient number. For continuous data: total patient number, outcome mean and standard deviation (SD). We will seek continuous data reported as mean and SD for change in outcome from baseline (adjusted for baseline score); and, where not available, mean absolute outcome and SD at follow up will be recorded. For other outcome types (e.g. event rate, time to event) we will extract data appropriately;
3) Extract outcome data for all follow‐up points;
4) Extract outcome data by subgroups according to the characteristics of the intervention (straight feedback of the results to the healthcare professional; or feedback along with guidelines regarding how to interpret results or other educational strategies); and patient characteristics (educational level). When required and feasible, data will be transformed in order to standardise outcomes, for instance for differences in the direction of the scales.
We will pilot the data extraction form with a small sample of articles finally selected. The sample will be purposively selected to ensure heterogeneity in terms of type of studies and interventions. All researchers will participate in this pilot. It is anticipated that information obtained at the pilot level might inform potential changes to the data extraction form. Extracted data will be stored in an electronic database, which will be created using RevMan 5 (RevMan 2012).
Assessment of risk of bias in included studies
We will assess risk of bias based on the parameters suggested by the Cochrane Collaboration (Higgins 2011), which comprises six domains: sequence generation; allocation concealment; participants' blinding (either patients or healthcare providers); incomplete outcome data; selective reporting; and other sources of bias, including whether the used PROs have been previously validated for the specific setting and population. Furthermore, we will take into account the complementary risk of bias parameters for RCTs proposed by the Cochrane Effective Practice and Organisation of Care Group, namely the similarity of baseline measurement, both for outcome measures and participants’ characteristics, and the protection against contamination (EPOC 2014a). We will classify each parameter as high risk of bias, low risk of bias, or unclear, and obtained information will be summarised in tabulated form, using RevMan 5. We will express level of confidence in the evidence for each outcome using the GRADE criteria, by assessing the type of evidence, limitations in study design, indirectness of evidence, unexplained heterogeneity of findings, imprecision of results, and probability of publication bias in accordance with the guidance of Higgins 2011. We will use the EPOC Worksheets for preparing summary of findings using GRADE (EPOC Worksheets 2013). As a guide, we will judge a study as at high risk of bias if more than three of the nine individual items are considered to be high risk.
Measures of treatment effect
We will calculate risk ratios with 95% confidence intervals (CIs) for dichotomous data. Where studies use continuous scales of measurement to assess the effects of the intervention, mean differences (MD) with 95% CIs will be used; or, when studies use different scales or measurements, we will use the the standardised mean difference (SMD). Where studies use other outcome metrics, e.g. rates of events or time to event, we will seek the appropriate overall measure of effect, e.g. rate risk ratio, hazard ratio.
Unit of analysis issues
If any cluster randomised clinical trials are included, we will contact the trial authors to obtain an estimate of the intra‐cluster correlation (ICC) where appropriate adjustments for the correlation between participants within clusters have not been made, or impute it using estimates from the other included trials, or from similar external trials. We will inflate the trial standard errors. We will try to either reduce the size of trials to its ‘effective sample size’ or recalculate the effects using an approximately correct analysis and using design effect calculated from the ICC (Higgins 2011). Whenever studies include more than one intervention arm, we will seek to combine arms to create a single pair‐wise comparison or we will conduct pair‐wise comparisons by comparing each intervention arm to the control arm (splitting the control arm sample size).
Dealing with missing data
We will attempt to obtain missing data by contacting the authors of the trials. If data remain unavailable (allowing for a maximum waiting period of one month for a reply), the impact of the missing data will be discussed (see 'Sensitivity analysis' below).
For dichotomous outcomes, we will make analyses according to the intention‐to‐treat method (Higgins 2011), which includes all participants irrespective of compliance or follow‐up. For the primary analyses, we will assume that participants lost to follow up are alive, and have no serious adverse events. For continuous outcomes we will perform available patient analysis and include data only on those for whom results are known (Higgins 2011). If it is not possible to obtain SDs either from authors or by calculation, the missing data will be imputed by using SDs from other included trials, specifically trials with a low risk of bias (Furukawa 2006).
Assessment of heterogeneity
First, we will explore clinical heterogeneity across studies by comparing the population, intervention and control arms. We will then explore statistical heterogeneity observed in the trials both by visual inspection of a forest plot, and by using a standard Chi² value with a significance level of P = 0.10. We will assess heterogeneity using the I² statistic. An I² estimate greater than or equal to 50% with a statistically significant value for Chi², will be interpreted as evidence of a substantial problem with heterogeneity (Higgins 2011). If this is the case, we will explore reasons for heterogeneity. If there is high inconsistency, and clear reasons for this are found, we will present data separately.
Data synthesis
We will perform data synthesis according to recommendations in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011), using RevMan 5 (RevMan 2012) and STATA v13 (StataCorp 2013). Given the likely heterogeneity of data in this review we propose to handle the outcome results in a two‐stage approach. In the first stage, we will: (1) collate data according to the headings outlined in the 'Types of outcome measures' section; (2) according to outcome, extract the appropriate data for each arm according to the principle of intention to treat (i.e. according to the original random allocation). For dichotomous data: number of patients experiencing outcome/total patient number. For continuous data: total patient number, outcome mean and standard deviation (SD). We will seek continuous data reported as mean and SD for change in outcome from baseline (adjusted for baseline score) and where not available, mean absolute outcome and SD at follow up will be recorded. For other outcome types (e.g. event rate, time to event) we will extract data appropriately; (3) we will extract outcome data at all follow‐up points; (4) where reported, we will also extract this outcome data by subgroups according to the characteristics of the intervention (straight feedback of the results to the healthcare professional; feedback along with guidelines regarding how to interpret results or other educational strategies) and patient characteristics (educational level).
In the second stage, based on the quality and consistency of outcome reporting, we will decide to synthesise results across studies using either a formal quantitative meta‐analytic approach or a more descriptive approach that focuses on summarising the size and direction of treatment effect separately for each individual study. If enough information is provided by the studies included in the review, the potential impact of moderator variables will be considered through meta‐regression analysis. When required and feasible, data will be transformed in order to homogenise outcomes, for instance for differences in the direction of the scales. We will employ the I² statistic to assess heterogeneity (Higgins 2003). Due to the expected heterogeneity of the data, we will employ random‐effects methods (Deeks 2008). We will also perform the meta‐analysis using a fixed‐effect model and if there are discrepancies between results from the two models, both sets of results will be presented, otherwise we will report the results from the random‐effects model only. Further specification of the methods for analysis (e.g. MD versus WMD) will be tailored to the type of outcome data. If the heterogeneity of studies is found to be substantial, i.e. I² above 50%, we will not perform a meta‐analysis, although we will still quantify the results by calculating effect sizes and will apply a structured synthesis approach (EPOC 2014b).
Subgroup analysis and investigation of heterogeneity
No interactions or effect modifiers are being hypothesised in this review, and therefore we do not pre‐specify stratified meta‐analysis or meta‐regression analyses (with exception of risk of bias – see 'Sensitivity analysis' below). However, where conducted, we will seek to data extract and report trial level subgroup analyses to inform hypothetical models of subgroup analysis for future meta‐analyses.
Sensitivity analysis
We will conduct a sensitivity analysis by verifying the impact that the exclusion of certain studies (e.g. those with high overall risk of bias (see definition above), and those with large samples) has on the overall results. Whenever relevant and possible we will contact study authors in order to obtain missing information, allowing for a maximum waiting period of one month for a reply. Where authors fail to provide missing information, existing data will be analysed and the hypothetical impact of the missing data examined as sensitivity analysis. Finally a sensitivity analysis will be undertaken to examine the impact varying the ICC for reanalysis of cluster randomised trials.
