Scolaris Content Display Scolaris Content Display

Cochrane Database of Systematic Reviews Protocol - Intervention

Interventions for promoting consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material

Authors' declarations of interest

Version published: 26 January 2004 Version history

https://doi.org/10.1002/14651858.CD004563

This is not the most recent version

view the current version 19 July 2006
Collapse all Expand all

Abstract

This is a protocol for a Cochrane Review (Intervention). The objectives are as follows:

To determine the effects of different ways of promoting consumer involvement in collective decisions about healthcare policy, clinical practice guidelines, patient information material and healthcare research.

Background

The importance of consumer involvement at all levels of the health services is widely recognized. Consumers may offer different and complementary perspectives to those of professionals. They also do not have a conflict of interest and divided loyalties that often shape professional perspectives.

In collective decisions about health care, consumer participation can be viewed as a goal in itself ‐ encouraging participative democracy, public accountability and transparency. For example, the World Health Organization'sDeclaration of Alma Ata states that "The people have the right and duty to participate individually and collectively in the planning and implementation of their health care" (WHO 1978).

At the same time, consumer participation in collective decisions about health care is assumed to have desirable effects. For example, it is believed that input from consumers in planning health care can lead to more accessible and acceptable health services, and improve health and quality of life (Crawford 2002). Consumer involvement is also thought to lead to health research of greater quality and clinical relevance (Boote 2002). These beliefs are widely accepted, despite the lack of research that systematically evaluates the effects of involving consumers. "Few studies have explored the effects of involving patients. Involving patients has contributed to changes in service provision, but the effects of these on quality of care have not been reported" (Crawford 2002).

Consumer involvement in health care is also an idea that faces considerable resistance. Although most professionals are dedicated, they face many other challenging demands and are hierarchically socialized and organized to view themselves as the real authorities. It is claimed, for instance, that consumer involvement can make research projects costlier and longer than some researchers and grant‐awarding bodies would expect, and that consumers may have biased views on certain health issues, which may threaten the traditional academic impartiality of knowledge development (Boote 2002).

Our review will focus on the effects of different ways of promoting consumer involvement in collective decisions about health care, including decisions about healthcare policies and planning (eg. health technology assessment, pay‐back systems for consumers, inequalities in health care); clinical policies (clinical practice guidelines); patient information materials (that aim to inform patients about personal healthcare decisions); and healthcare research (eg. design of clinical or epidemiological studies, priority setting, etc.). These types of decisions are collective decisions that affect a population of people. Consumer involvement is intended to reflect the interests of the population rather than the interests of the individuals who participate.

Consumer involvement varies according to its purpose, the consumers involved, the degree of involvement, the methods employed to support this involvement, and the context. An early attempt to characterise consumer involvement, initiated largely by metropolitan institutions in high income countries, proposed a 'ladder of participation'. Rungs on the ladder represented increasing degrees of participation, from: non‐participation or manipulation and therapy; through the tokenism of informing, consulting and placating; to citizen power through partnership, delegated power and citizen control (Arnstein 1969). A similar scale has been described, drawing on participatory processes for research in low and middle income countries (Cornwall 1996). This acknowledges the tokenism that occurs at one end of the scale and the co‐learning or independent collective action at the other.

As well as distinguishing the degree of involvement, methods vary largely as to whether they involve individuals or groups of consumers. A two‐dimensional representation of models of involvement combining degrees of involvement (from information to control), and distinguishing individual and group involvement, has recently been employed to describe consumer involvement in mental health services in England (Glasby 2003).

How consumer involvement develops and how it is viewed by health professionals and by different sectors of society is also influenced by who initiated specific encounters: the consumers or the professional services (Mullen 1984).

We have chosen a framework for describing consumer involvement constructed for a systematic review of consumer involvement in setting research agendas (Oliver in press). When employed in that review, the framework accommodated the diverse methods spanning all these dimensions (degree of involvement, individuals or groups of consumers, and initiated by consumers or professionals) across low, middle and high income countries. For this framework, Arnstein's ladder of participation was simplified to the three steps employed by Consumers in NHS Research: consultation, collaboration and consumer control (Hanley 2000). Consultation was defined as asking consumers for their views and using these views to inform decision‐making. For example, funders of research have held one‐off meetings with consumers to ask them about their priorities for research, or written to consumers in accessible terms to invite their views. Consumers' views were not necessarily adopted, although they may inform decisions. Collaboration was described as active, on‐going partnership with consumers. For example, consumers have been committee members or collaborated less formally to complete a task. Again, there is no guarantee that consumers' views will influence decisions, but there is more opportunity for them to be heard than in consultations. Consumer‐controlled research was described as consumers designing, undertaking and disseminating the results of a research project. 'Professionals' were only involved at the invitation of the consumers.

Within this framework, methods are further distinguished by descriptions of the forum for participation (such as one‐to‐one interviews, focus groups, citizens' juries, town meetings, committee meetings, working groups) and methods for decision making (such as informal committee consensus, voting, ranking, scoring, visual scales and Delphi surveys). The presence or absence of transparent descriptions of methods for decision‐making can distinguish implied involvement in decisions (such as in committee meetings), some examples of which are misleading, and explicit involvement in decisions. This can inform interpretations of tokenism that is so widespread in consumer involvement.

Given such diversity, interventions in this field can be conceived broadly such as collaboration or consultation; or more narrowly such as fora for discussions; or even more specifically, such as methods for recruitment, training for professionals and/or consumers, or payment policies. All of these interventions are worthy of review.

Based on previous reviews (Boote 2002; Crawford 2002; Oliver in press) we anticipate finding few, if any evaluations. The scope of this review will enable us to search broadly for evaluations. Although our framework for consumer involvement will help us characterise the interventions, care needs to be taken with interpretation, considering the diverse purposes of consumer involvement and the different cultural and political contexts.

Comparative evaluations are necessary to reliably estimate the effects of interventions to promote consumer involvement. While there are many descriptive studies of interventions to promote consumer involvement, there is a paucity of comparative evaluations. Descriptive studies can provide valuable information about how or why an intervention might have both intended and unintended effects, but their study design is inappropriate for answering the question addressed in this review: What are the effects of interventions to promote consumer involvement in collective decisions about health care?

Objectives

To determine the effects of different ways of promoting consumer involvement in collective decisions about healthcare policy, clinical practice guidelines, patient information material and healthcare research.

Methods

Criteria for considering studies for this review

Types of studies

Randomised trials, non‐randomised trials, interrupted time series analyses, and controlled before‐after studies.

Types of participants

Healthcare consumers and others involved in collective decisions about health care. Healthcare consumers include patients and potential patients, long‐term users of services, carers and parents, organisations that represent consumers' interests, members of the public who are the targets of health promotion programmes, and groups asking for research because they believe they have been exposed to potentially harmful circumstances, products or services (Hanley 2000). Depending on the context, they may be described with any of the following terms: 'lay', 'non‐expert', 'service user', 'survivor' or 'member of the general public'.

Types of interventions

Methods of involving consumers in collective decisions about healthcare policy, clinical practice guidelines, patient information material, or healthcare research. A framework for describing consumer involvement distinguishes methods in terms of the degree of consumer involvement, the forum for communication, how consumers are involved in actual decision‐making, and how involvement is implemented (Oliver in press). Methods of implementing involvement include how consumers are recruited, trained and supported.

Healthcare policy is defined as laws, rules, financial and administrative orders made by governments, non government organisations or private organisations, that are intended to directly affect the provision and use of health services. Clinical practice guidelines are defined as "systematically developed statements to assist both practitioner and patient decisions in specific circumstances" (Institute 1992). Patient information material includes printed, audio‐visual and electronic information that is intended to help patients to make informed decisions about health care. Healthcare research includes clinical research, epidemiological research and health services research (need, demand, supply, use, and outcome of health services).

Types of outcome measures

To be included a study must include an objective measure of at least one of the following outcomes: participation or response rates; effects of consumer involvement on decisions, healthcare outcomes or resource utilization; consumers' or professionals' satisfaction; impact on the participating consumers; costs.

Search methods for identification of studies

We will search the Cochrane Consumers and Communication Review Group's specialised register, the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, EMBASE, Sociological Abstracts, SIGLE (System for Information on Grey Literature in Europe), NTIS (the USA government's National Technical Information Service), PAIS (Public Affairs Information Service) International and CSA Worldwide Political Science Abstracts. The strategy for MEDLINE is given below, and we will adjust it for searching other databases.

The search strategy includes medical subject headings (MeSH) and text words.

  • Consumers (lines 1 ‐ 7)

  • Areas of consumer involvement: healthcare policy, clinical practice guidelines, patient information material, healthcare research (lines 8 ‐ 26)

  • Modified version of the EPOC search strategy to limit the search to trials, time series analysis and controlled before‐after studies (lines 27 ‐ 38).

Search strategy MEDLINE (1966 to May 2003) Ovid

1.Consumer Participation/
2.Patient Participation/
3.Patient Advocacy/
4.Consumer Advocacy/
5.Consumer Organizations/
6.((consumer? or patient?) adj2 organi#ation?).tw.
7.((consumer? stakeholder? or patient? or user? or lay or client? or disab$ or citizen? or communit$ or public or advoca$ or carer? or caregiver? or parent? or relative?) adj2 (particip$ or involv$ or represent$ or collaborat$ or consult$ or contribut$)).tw.
8.Health Policy/
9.Health Planning/
10.Health Care Rationing/
11.Health Care Reform/
12.Health Priorities/
13.Community Health Planning/
14.State Health Plans/
15.(health$ adj3 (policy or policies or plan$ or reform$ or priorit$)).tw.
16.Health Services Research/
17.Peer Review, Research/
18.Research/
19.(health$ adj3 research).tw.
20.(research adj3 (agenda? or priorit$ or program?)).tw.
21.Guidelines/
22.Practice Guidelines/
23.guideline?.tw.
24.Information Services/
25.health information.tw.
26.((patient? or consumer?) adj1 information).tw.
27.randomized controlled trial.pt.
28.controlled clinical trial.pt.
29.intervention studies/
30.experiment$.tw.
31.(time adj series).tw.
32.(pre test or pretest or (posttest or post test)).tw.
33.random allocation/
34.evaluation studies/
35.evaluat$.tw.
36.comparative studies/
37.(randomized or randomised).tw.
38.(random$ adj1 (allocat$ or assign$)).tw.
39.or/1‐7
40.or/8‐26
41.or/27‐38
42.39 and 40 and 41

We shall screen the reference lists of all of the relevant reports that we retrieve. We shall also search Science Citation Index for articles citing key references that are identified using the above search strategies. Authors of relevant papers, relevant organizations, and discussion lists will be contacted to identify additional studies, including unpublished and ongoing studies. Translation will be sought for studies identified in languages other than English. Finally, the original database search strategies will be revised based on the yield of the above searches, and updated.

Data collection and analysis

Two reviewers will review all of the search results and reference lists of relevant reports (two of ESN, HTM and MJ). The full text of potentially relevant reports will be retrieved and two (of the same) reviewers will assess the relevance of those studies, assess the quality of included studies and extract data from included studies independently. Disagreements will be resolved by discussion, when necessary including another reviewer.

We will use standard criteria to assess the methodological quality of studies (protection against bias) (EPOC 2003: Table 1). Overall quality (risk of bias) for each main outcome within each study will be assessed by each of the data extractors using the following guidelines:

Open in table viewer
Table 1. Cochrane EPOC Group: Quality assessment

Cochrane Effective Practice and Organisation of Care Group

ASSESSMENT OF METHODOLOGICAL QUALITY

Standard criteria are used to assess the methodological quality of studies included in EPOC reviews (protection against bias). Each criterion is scored as DONE, NOT CLEAR, or NOT DONE. Details regarding the application of these criteria are available from the editorial base.
Seven standard criteria are used to assess the methodological quality of RCTs and CCTs:

1. Concealment of allocation (protection against selection bias). This is scored as DONE if the unit of allocation was by institution, team or professional and any random process was described explicitly; or if the unit of allocation was by patient or episode of care and there was some form of centralised randomisation scheme, an on‐site computer system or sealed opaque envelopes were used.

2. Follow‐up of professionals (protection against exclusion bias). This is scored as DONE if outcome measures were obtained for 80‐100% of subjects randomised.

3. Follow‐up of patients. This is scored as DONE if outcome measures were obtained for 80‐100% of patients randomised, or for patients who entered the trial.

4. Blinded assessment of primary outcome(s) (protection against detection bias). This is scored as DONE if the authors state explicitly that the primary outcome variables were assessed blindly, or the outcome variables are objective, e.g. length of hospital stay, drug levels as assessed by a standardised test. Primary outcome(s) are those variables that correspond to the primary hypothesis or question as defined by the authors. In the event that some of the primary outcome variables were assessed in a blind fashion and others were not, each is scored separately.

5. Baseline measurement. This is scored as DONE if performance or patient outcomes were measured prior to the intervention, and no substantial differences were present across study groups.

6. Reliable primary outcome measure(s). This is scored as DONE if there were two or more raters with at least 90% agreement or kappa greater than or equal to 0.8 OR the outcome data were obtained from some automated system, e.g. length of hospital stay, drug levels as assessed by a standardised test.

7. Protection against contamination. This is scored as DONE if allocation was by community, institution or practice and it is unlikely that the control group received the intervention.

Seven standard criteria are used to assess the methodological quality of CBA studies:

1. Baseline measurement. This is scored as DONE if performance or patient outcomes were measured prior to the intervention and no substantial differences were present across study groups.

2. Baseline characteristics for studies using second site as control. This is scored as DONE if the baseline characteristics of the study and control providers are reported and similar.

3. Blinded assessment of primary outcome(s) (protection against detection bias). This is scored as DONE if the authors state explicitly that the primary outcome variables were assessed blindly OR the outcome variables are objective, e.g. length of hospital stay, drug levels as assessed by a standardised test. Primary outcome(s) are those variables that correspond to the primary hypothesis or question as defined by the authors. In the event that some of the primary outcome variables were assessed in a blind fashion and others were not, each is scored separately.

4. Protection against contamination. For studies using second site as control, this is scored as DONE if allocation was by community, institution or practice and it is unlikely that the control group received the intervention.

5. Reliable primary outcome measure(s). This is scored as DONE if there were two or more raters with at least 90% agreement or kappa greater than or equal to 0.8 OR the outcome data were obtained from some automated system, e.g. length of hospital stay, drug levels as assessed by a standardised test.

6. Follow‐up of professionals (protection against exclusion bias). This is scored as DONE if outcome measures were obtained for 80‐100% of subjects allocated to groups.

7. Follow‐up of patients. This is scored as DONE if outcome measures were obtained for 80‐100% of patients allocated to groups or for patients who entered the study.

Seven standard criteria are used to assess the methodological quality of ITS studies:

1. The intervention is independent of other changes. This is scored as DONE if the intervention occurred independently of other changes.

2. There are sufficient data points to enable reliable statistical inference. This is scored as DONE if at least 20 data points are recorded before the intervention AND the authors have done a traditional time series analysis (ARIMA model), OR if at least 3 data points are recorded pre and post intervention AND the authors have done a repeated measures analysis, OR if at least 3 data points are recorded pre and post intervention AND the authors have used ANOVA or multiple t‐tests AND there are at least 30 observations per data point.

3. Formal test for trend. This is scored as DONE if a formal test for trend is reported.

4. Intervention unlikely to affect data collection. This is scored as DONE if it is reported that the intervention was unlikely to affect data collection, e.g. sources and methods of data collection were the same before and after the intervention.

5. Blinded assessment of primary outcome(s). This is scored as DONE if the authors state explicitly that the primary outcome variables were assessed blindly or the outcome variables are objective, e.g. length of hospital stay, drug levels as assessed by a standardised test. Primary outcome(s) are those variables that correspond to the primary hypothesis or question as defined by the authors. In the event that some of the primary outcome variables were assessed in a blind fashion and others were not, each is scored separately.

6. Completeness of data set. This is scored as DONE if data set covers 80‐100% of total number of participants or episodes of care in the study.

7. Reliable primary outcome measure(s). This is scored as DONE if there were two or more raters with at least 90% agreement or kappa greater than or equal to 0.8 OR the outcome data were obtained from some automated system, e.g. length of hospital stay, drug levels as assessed by a standardised test.

  • Low risk of bias: all seven criteria scored as 'done'

  • Moderate risk of bias: one or two criteria scored as 'not clear' or 'not done'

  • High risk of bias: more than two criteria scored as 'not clear' or 'not done'

The following additional information will be extracted from included studies using a standardised data extraction form:

  • Type of process (development of healthcare policy, clinical practice guidelines, patient information material, or design/priority setting of healthcare research)

  • Purpose and scope for the activity in which consumers are involved

  • Study setting (country, key features of the system in which the process is undertaken)

  • Characteristics of the participants (consumers and professionals)

  • Characteristics of the interventions that are compared

  • Type of study (randomized trial, non‐randomized trial, interrupted time series, controlled before‐after)

  • Main outcome measures and study duration

  • The results for the main outcome measure

Comparisons will be grouped together as outlined in Table 2.

Open in table viewer
Table 2. Methods of the Review: Comparisons

Intervention

Comparison

DEGREE OF CONSUMER INVOLVEMENT
(1) Consultation
Consultation as a method of involving consumers is asking consumers for their views and using these views to inform decision‐making (Oliver 2003).
(2) Cooperation
Participate in an activity (such as speaker at a conference, or resource person in a workshop) where consumers are not involved in the decision‐making.
(3) Collaboration
Active, on‐going partnership. For example, partnership with consumers has included committee membership or less formal collaboration to complete a task, as in team working (Oliver 2003).
(4) Collaboration

DEGREE OF CONSUMER INVOLVEMENT
(1) No involvement
(2) No involvement
(3) No involvement
(4) Consultation

FORUM FOR COMMUNICATION
(1) Consultative fora, eg. town meeting, written consultation, interviews, focus groups
(2) Collaborative, eg. committee membership, permanent consumer panels
(3) Collaborative forum

FORUM FOR COMMUNICATION
(1) No consumers in consultative forum ‐ different/multiple fora for consultation
(2) No consumers in collaborative forum ‐ different/multiple fora for collaboration
(3) Consultative forum (eg. invitation to a single committee meeting)

INVOLVEMENT IN DECISION MAKING
(1) Involvement in decision‐making implied, eg. committee membership
(2) Involvement in decision‐making explicit, eg. voting, ranking
(3) Explicit involvement

INVOLVEMENT IN DECISION MAKING
(1) No involvement in decisions
(2) No involvement in decisions
(3) Implicit involvement

IMPLEMENTING INVOLVEMENT: RECRUITMENT OF PROFESSIONALS/CONSUMERS
(1) Targeted, personal invitations
(2) Wide advertisement
(a) Mass media
(b) Telephone
(c) Mail
(d) E‐mail

IMPLEMENTING INVOLVEMENT: RECRUITMENT OF PROFESSIONALS/CONSUMERS
(1) Target, personal invitations versus wide advertisement
(2) Different/multiple methods of wide advertisement

IMPLEMENTING INVOLVEMENT: TRAINING AND SUPPORT FOR PROFESSIONALS/CONSUMERS
(1) Education
(2) Counselling
(3) Introduction day

IMPLEMENTING INVOLVEMENT: TRAINING AND SUPPORT FOR PROFESSIONALS/CONSUMERS
(1) Training versus no training
(2) Different/multiple methods of training
(3) Different/multiple trainers
(4) Different timing of training (introductory, on‐going)

IMPLEMENTING INVOLVEMENT: FINANCIAL SUPPORT
(1) Funding/staffing specifically to support consumer involvement enterprises
(2) Reimbursement of consumer expensees
(3) Fee or honoraria
(4) No financial support

IMPLEMENTING INVOLVEMENT: FINANCIAL SUPPORT
(1) Financial support versus no financial support
(2) Different/multiple policies for financial support

IMPLEMENTING INVOLVEMENT: PRACTICAL SUPPORT
(1) For example, administrative support for consumer groups

IMPLEMENTING INVOLVEMENT: PRACTICAL SUPPORT
(1) Practical support versus no practical support
(2) Different/multiple types of practical support

We will not undertake a meta‐analysis because of the expected low number of included studies and the heterogeneity in study design, interventions and outcome measures. We will summarise the results of the included studies qualitatively. A results table will be prepared categorizing any comparisons that are found as indicated above, including the study ID, characteristics of the interventions and the results of the evaluations. We will report the mechanisms through which the interventions might affect consumer involvement and other outcomes, both intended and unintended. We shall summarise what is known about the effects of alternative ways of involving consumers, including important options for which no evaluations are found. Our confidence in any estimates of effects that are found will be graded using the approach recommended by the GRADE Working Group (GRADE 2003). The following potential explanatory factors will be considered in exploring potential variation in effects: differences in the characteristics of the interventions, settings, types of process, study quality.

We shall identify important factors that should be taken into consideration by anyone contemplating using any of the identified means of promoting consumer involvement, including: possible trade‐offs (of the expected benefits versus harms, if any, and costs), the quality of the available evidence, possible differences in baseline levels of consumer involvement, and other important factors that might affect the translation of the available evidence into practice in specific settings. Because we do not expect to find a great deal of evidence, we shall also aim to provide a framework for future evaluations and guidance for evaluating interventions to involve consumers in collective decisions about health care.

Consumer participation

We shall establish a consumer panel (e‐mail discussion list) consisting of consumers from The Cochrane Collaboration and relevant consumer organisations. The consumer panel will be asked to undertake different tasks, such as identifying unpublished studies that could be considered for inclusion, and commenting on drafts.

Table 1. Cochrane EPOC Group: Quality assessment

Cochrane Effective Practice and Organisation of Care Group

ASSESSMENT OF METHODOLOGICAL QUALITY

Standard criteria are used to assess the methodological quality of studies included in EPOC reviews (protection against bias). Each criterion is scored as DONE, NOT CLEAR, or NOT DONE. Details regarding the application of these criteria are available from the editorial base.
Seven standard criteria are used to assess the methodological quality of RCTs and CCTs:

1. Concealment of allocation (protection against selection bias). This is scored as DONE if the unit of allocation was by institution, team or professional and any random process was described explicitly; or if the unit of allocation was by patient or episode of care and there was some form of centralised randomisation scheme, an on‐site computer system or sealed opaque envelopes were used.

2. Follow‐up of professionals (protection against exclusion bias). This is scored as DONE if outcome measures were obtained for 80‐100% of subjects randomised.

3. Follow‐up of patients. This is scored as DONE if outcome measures were obtained for 80‐100% of patients randomised, or for patients who entered the trial.

4. Blinded assessment of primary outcome(s) (protection against detection bias). This is scored as DONE if the authors state explicitly that the primary outcome variables were assessed blindly, or the outcome variables are objective, e.g. length of hospital stay, drug levels as assessed by a standardised test. Primary outcome(s) are those variables that correspond to the primary hypothesis or question as defined by the authors. In the event that some of the primary outcome variables were assessed in a blind fashion and others were not, each is scored separately.

5. Baseline measurement. This is scored as DONE if performance or patient outcomes were measured prior to the intervention, and no substantial differences were present across study groups.

6. Reliable primary outcome measure(s). This is scored as DONE if there were two or more raters with at least 90% agreement or kappa greater than or equal to 0.8 OR the outcome data were obtained from some automated system, e.g. length of hospital stay, drug levels as assessed by a standardised test.

7. Protection against contamination. This is scored as DONE if allocation was by community, institution or practice and it is unlikely that the control group received the intervention.

Seven standard criteria are used to assess the methodological quality of CBA studies:

1. Baseline measurement. This is scored as DONE if performance or patient outcomes were measured prior to the intervention and no substantial differences were present across study groups.

2. Baseline characteristics for studies using second site as control. This is scored as DONE if the baseline characteristics of the study and control providers are reported and similar.

3. Blinded assessment of primary outcome(s) (protection against detection bias). This is scored as DONE if the authors state explicitly that the primary outcome variables were assessed blindly OR the outcome variables are objective, e.g. length of hospital stay, drug levels as assessed by a standardised test. Primary outcome(s) are those variables that correspond to the primary hypothesis or question as defined by the authors. In the event that some of the primary outcome variables were assessed in a blind fashion and others were not, each is scored separately.

4. Protection against contamination. For studies using second site as control, this is scored as DONE if allocation was by community, institution or practice and it is unlikely that the control group received the intervention.

5. Reliable primary outcome measure(s). This is scored as DONE if there were two or more raters with at least 90% agreement or kappa greater than or equal to 0.8 OR the outcome data were obtained from some automated system, e.g. length of hospital stay, drug levels as assessed by a standardised test.

6. Follow‐up of professionals (protection against exclusion bias). This is scored as DONE if outcome measures were obtained for 80‐100% of subjects allocated to groups.

7. Follow‐up of patients. This is scored as DONE if outcome measures were obtained for 80‐100% of patients allocated to groups or for patients who entered the study.

Seven standard criteria are used to assess the methodological quality of ITS studies:

1. The intervention is independent of other changes. This is scored as DONE if the intervention occurred independently of other changes.

2. There are sufficient data points to enable reliable statistical inference. This is scored as DONE if at least 20 data points are recorded before the intervention AND the authors have done a traditional time series analysis (ARIMA model), OR if at least 3 data points are recorded pre and post intervention AND the authors have done a repeated measures analysis, OR if at least 3 data points are recorded pre and post intervention AND the authors have used ANOVA or multiple t‐tests AND there are at least 30 observations per data point.

3. Formal test for trend. This is scored as DONE if a formal test for trend is reported.

4. Intervention unlikely to affect data collection. This is scored as DONE if it is reported that the intervention was unlikely to affect data collection, e.g. sources and methods of data collection were the same before and after the intervention.

5. Blinded assessment of primary outcome(s). This is scored as DONE if the authors state explicitly that the primary outcome variables were assessed blindly or the outcome variables are objective, e.g. length of hospital stay, drug levels as assessed by a standardised test. Primary outcome(s) are those variables that correspond to the primary hypothesis or question as defined by the authors. In the event that some of the primary outcome variables were assessed in a blind fashion and others were not, each is scored separately.

6. Completeness of data set. This is scored as DONE if data set covers 80‐100% of total number of participants or episodes of care in the study.

7. Reliable primary outcome measure(s). This is scored as DONE if there were two or more raters with at least 90% agreement or kappa greater than or equal to 0.8 OR the outcome data were obtained from some automated system, e.g. length of hospital stay, drug levels as assessed by a standardised test.

Figures and Tables -
Table 1. Cochrane EPOC Group: Quality assessment
Navigate to table in Protocol
Table 2. Methods of the Review: Comparisons

Intervention

Comparison

DEGREE OF CONSUMER INVOLVEMENT
(1) Consultation
Consultation as a method of involving consumers is asking consumers for their views and using these views to inform decision‐making (Oliver 2003).
(2) Cooperation
Participate in an activity (such as speaker at a conference, or resource person in a workshop) where consumers are not involved in the decision‐making.
(3) Collaboration
Active, on‐going partnership. For example, partnership with consumers has included committee membership or less formal collaboration to complete a task, as in team working (Oliver 2003).
(4) Collaboration

DEGREE OF CONSUMER INVOLVEMENT
(1) No involvement
(2) No involvement
(3) No involvement
(4) Consultation

FORUM FOR COMMUNICATION
(1) Consultative fora, eg. town meeting, written consultation, interviews, focus groups
(2) Collaborative, eg. committee membership, permanent consumer panels
(3) Collaborative forum

FORUM FOR COMMUNICATION
(1) No consumers in consultative forum ‐ different/multiple fora for consultation
(2) No consumers in collaborative forum ‐ different/multiple fora for collaboration
(3) Consultative forum (eg. invitation to a single committee meeting)

INVOLVEMENT IN DECISION MAKING
(1) Involvement in decision‐making implied, eg. committee membership
(2) Involvement in decision‐making explicit, eg. voting, ranking
(3) Explicit involvement

INVOLVEMENT IN DECISION MAKING
(1) No involvement in decisions
(2) No involvement in decisions
(3) Implicit involvement

IMPLEMENTING INVOLVEMENT: RECRUITMENT OF PROFESSIONALS/CONSUMERS
(1) Targeted, personal invitations
(2) Wide advertisement
(a) Mass media
(b) Telephone
(c) Mail
(d) E‐mail

IMPLEMENTING INVOLVEMENT: RECRUITMENT OF PROFESSIONALS/CONSUMERS
(1) Target, personal invitations versus wide advertisement
(2) Different/multiple methods of wide advertisement

IMPLEMENTING INVOLVEMENT: TRAINING AND SUPPORT FOR PROFESSIONALS/CONSUMERS
(1) Education
(2) Counselling
(3) Introduction day

IMPLEMENTING INVOLVEMENT: TRAINING AND SUPPORT FOR PROFESSIONALS/CONSUMERS
(1) Training versus no training
(2) Different/multiple methods of training
(3) Different/multiple trainers
(4) Different timing of training (introductory, on‐going)

IMPLEMENTING INVOLVEMENT: FINANCIAL SUPPORT
(1) Funding/staffing specifically to support consumer involvement enterprises
(2) Reimbursement of consumer expensees
(3) Fee or honoraria
(4) No financial support

IMPLEMENTING INVOLVEMENT: FINANCIAL SUPPORT
(1) Financial support versus no financial support
(2) Different/multiple policies for financial support

IMPLEMENTING INVOLVEMENT: PRACTICAL SUPPORT
(1) For example, administrative support for consumer groups

IMPLEMENTING INVOLVEMENT: PRACTICAL SUPPORT
(1) Practical support versus no practical support
(2) Different/multiple types of practical support

Figures and Tables -
Table 2. Methods of the Review: Comparisons
Navigate to table in Protocol