Types of studies

We included randomised controlled trials (RCTs), cluster RCTs (cRCTs), controlled before‐after (CBA) studies, and interrupted time‐series (ITS) studies.

We expected that the availability of RCTs for this topic would be limited, as these interventions are very different from clinical interventions. The reporting system for occupational diseases is mostly closely linked to compensation systems, which are often part of social security or liability systems, or both. This makes it very difficult to randomise interventions.

CBA studies are easy to perform and they take into account that the intervention is carried out at group level. Despite the lack of randomisation, CBA studies can still have reasonable validity. We define CBA studies as studies in which measurements of the outcome are available both before and after the implementation of the intervention for both the intervention and control group. We define a control group as a group that is similar to the intervention group but has not undergone the experimental intervention or an alternative intervention. Here, the hypothesis is that after the intervention, the intervention group will have increased reporting more than the control group.

ITS studies are studies with or without a control group in which the outcome has been measured at least three times before the intervention and at least three times after the intervention (according to the criteria of the Cochrane Effective Practice and Organisation of Care Group) (EPOC). The intervention is applied at a specific well‐defined moment in time and is supposed to have either an immediate effect or a long‐term effect. Because the outcome is measured several times before and after the intervention, it is possible to take time trends into account and thus make up for the lack of a control group.

Types of participants

Study participants include physicians (i.e. occupational physicians, general practitioners, other physicians).

Although we expected that interventions would be focused primarily on occupational physicians, because in most countries they are the main reporters of occupational diseases, we considered interventions conducted with other physicians as well. We applied no restrictions on area of specialisation.

Types of interventions

We included any type of intervention aimed at increasing the reporting of occupational diseases by physicians. We compared the actual intervention with an alternative intervention or no intervention.

Some interventions could operate primarily at society level, while others could act at the individual level. We intended to include interventions on procedures or techniques of reporting (for example transfer of reporting from paper and pencil to computer‐based systems) as a separate category of intervention.

An intervention can have several of these features at the same time and could act directly or indirectly to influence the behaviour of physicians. For this reason, we also intended to include interventions focused on workers, considering them to be mediators acting on physicians.

We have provided below a non‐exhaustive list of possible types of intervention.

• Interventions at society level

• • Legislation (through national or state laws, or both)

  • Surveillance systems

  • Communication campaigns

• Interventions at individual level

  • Educational materials

  • Educational meetings

  • Formal continuing medical education

  • Audit and feedbacks

  • Reminders

  • Visits by physician educators

  • Guidelines

  • Economic incentives

  • Quality improvement in reporting

• Interventions on procedures or techniques, or both

  • Transition to web‐based reporting

  • Easy tools adaptation (i.e. simplifying the reporting form, offering online reporting, clarifying the information to be reported)

Types of outcome measures

Electronic searches

We searched the following electronic databases from the first day of entries until January 2015.

• Cochrane Occupational Safety and Health Group Specialised Register

• Cochrane Central Register of Controlled Trials (CENTRAL)

• MEDLINE through PubMed

• EMBASE

• OSH UPDATE

We searched the following websites to identify additional studies.

• Database of Abstracts of Reviews of Effects (DARE) produced by the Centre for Reviews and Dissemination, University of York (www.crd.york.ac.uk)

• OpenSIGLE (System for Information on Grey Literature in Europe), which collects grey literature produced in the European Community (opensigle.inist.fr)

• Health Evidence, which is an online registry of systematic reviews on the effectiveness of public health and health promotion interventions (healthevidence.org, formerly health‐evidence.ca)

Searching other resources

We looked for additional studies by checking the reference lists of relevant articles. We contacted experts in the field to identify additional unpublished materials.

Selection of studies

We divided the identified studies between three pairs of review authors (SM, MV; RS, AD; DS, SR) to examine each reference twice. Each review author independently screened titles and abstracts of the articles the search strategy retrieved to identify the articles for potential inclusion. We discussed disagreements within pairs until we reached consensus. We obtained the full text of all articles potentially qualifying for inclusion, and the same pairs of review authors who screened titles and abstracts assessed whether each full article met the inclusion criteria. We discussed disagreements within pairs until we reached consensus. If disagreement persisted, another review author (SC) made the final decision.

Data extraction and management

Two review authors (RS and SR) independently extracted data based on methods, study participant characteristics, intervention type, outcomes, and main results of the included studies. We resolved disagreements by discussion within pair. If disagreement persisted, a third review author (SC) made the final decision. If information was insufficient for data extraction, we contacted the study authors to request additional information. If this failed, we excluded the study due to insufficient information.

Assessment of risk of bias in included studies

Two review authors (RS and AD) independently assessed the risk of bias of the included studies.

We evaluated the risk of bias of RCTs and CBA studies using the checklist developed by Downs 1998. We used only the items on internal validity of the checklist and not those on reporting quality or external validity. The 13 items of the checklist include the domains of the 'Risk of bias' tool recommended in Chapter 8 of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011a): random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, and selective reporting. We adapted the answers to the questions of the checklist slightly so that they fitted the 'Risk of bias' tool as implemented in RevMan 2014 by using the rating of 'high risk of bias', 'low risk of bias', or 'unclear risk of bias' instead of using scores 0 or 1 as proposed by the checklist authors.

We judged an RCT or a CBA study to have a low overall risk of bias if we rated all of the following seven items as having a low risk of bias: blinding of outcome assessor, follow‐up, outcome measure, selection bias (population), selection bias (time), adjustment for confounding, and incomplete outcome data. We judged all CBA studies to have an unclear risk of bias for the item allocation concealment because this item is not applicable to non‐randomised studies.

For ITS studies, we intended to use the quality criteria developed by the Cochrane Effective Practice and Organisation of Care Group (EPOC). The quality assessment for ITS designs consists of: protection against secular changes (three items), protection against detection bias (two items), completeness of data set (one item), and reliable primary outcome measures (one item). Each item is scored as 'done', 'not clear', or 'not done'.

We resolved disagreements by discussion. If disagreement persisted, a third review author (DS) made the final decision.

Measures of treatment effect

We plotted the results of each RCT and CBA study as point estimates, such as risk ratios for the number of physicians reporting occupational diseases or rate ratios for the rate of reporting occupational diseases. In studies where the number of occupational diseases per physician were reported, we calculated the natural logarithm of the rate ratios and their standard errors in an Excel spreadsheet, as recommended by the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011b). We used the natural logarithm of the rate ratios and their standard errors as input for RevMan (RevMan 2014), where we combined them using the generic inverse variance method.

When the results could not be plotted, we described them in the 'Characteristics of included studies' table.

For ITS studies, we intended to extract data from the original papers and re‐analyse them according to the recommended methods for analysis of ITS designs for inclusion in systematic reviews (Ramsay 2003).

Unit of analysis issues

For studies that employed a cluster‐randomised design and that reported sufficient data to be included in the meta‐analysis but did not make an allowance for the design effect, we intended to calculate the design effect based on a fairly large assumed intracluster correlation of 0.10. We based this assumption of 0.10 being a realistic estimate by analogy on studies about implementation research (Campbell 2001). We intended to follow the methods stated in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011b).

For studies with multiple intervention arms and same control arm, we decided to proceed as follows:

1. When they belonged to the same comparison, we divided the number of events and participants in the control group equally over the study arms to prevent double counting of study participants in the meta‐analysis (Brissette 2006a; Brissette 2006b).

2. When they did not belong to the same comparisons (Lenderink 2010a; Lenderink 2010b; Lenderink 2010d; Lenderink 2010e), we used the control groups as reported by the authors.

Dealing with missing data

We dealt with missing data according to the recommendations in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011b). To obtain data missing from their reports but needed for assessment of eligibility of the studies or needed for meta‐analysis, or both, we contacted the authors of the following studies: Chu 2010, Gelberg 2011, Lenderink 2010c, Orriols 2010, Spreeuwers 2008, Spreeuwers 2012. We obtained additional data from five of them (Chu 2010; Gelberg 2011; Lenderink 2010c; Orriols 2010; Spreeuwers 2012), but we were able to use these data for two studies only (Gelberg 2011; Lenderink 2010c). The additional data provided by the authors of the other three studies did not allow us to classify them as an ITS or CBA study (Chu 2010; Orriols 2010; Spreeuwers 2012). We could not obtain data from the authors of Spreeuwers 2008.

Assessment of heterogeneity

We assessed clinical homogeneity based on similarity of population, intervention, control condition, outcome, and follow‐up. We considered physicians as similar irrespective of area of specialisation. We considered interventions as similar if they fell into one of the predefined categories of interventions (as stated in the 'Types of interventions' section). We took into account the control condition and considered 'no intervention' and 'less intensive intervention' control groups as different. We considered the various under‐reporting outcome measures as different. We regarded follow‐up times of less than three months, from three months to one year, and more than one year as different.

In addition, we tested for statistical heterogeneity by means of the Chi² test as implemented in the forest plot in Review Manager 5.3 software (RevMan 2014). We used a significance level of P less than 0.10 to indicate whether there was a problem with heterogeneity. Moreover, we quantified the degree of heterogeneity using the I² statistic, where an I² value of 25% to 50% indicated a low degree of heterogeneity, 50% to 75% a moderate degree of heterogeneity, and greater than 75% a high degree of heterogeneity (Higgins 2003).

Assessment of reporting biases

We reduced the effect of reporting bias by including studies and not publications, thereby avoiding the introduction of duplicated data (that is two articles could represent duplicate publications of the same study). Following the Cho 2000 statement on redundant publications, we attempted to detect duplicate studies so that if more articles had reported on the same study, we would extract data only once. We prevented location bias by searching across multiple databases. We prevented language bias by not excluding any article based on language. We intended to assess publication bias with a funnel plot in comparisons within which we could include five or more studies.

Data synthesis

We pooled data from studies we judged to be clinically homogeneous using Review Manager 5.3 software (RevMan 2014). If sufficient data were available, we performed meta‐analyses. When studies were statistically heterogeneous, we used a random‐effects model; otherwise, we used a fixed‐effect model. When using the random‐effects model, we conducted a sensitivity check by using the fixed‐effect model to reveal differences in results. We included a 95% confidence interval (CI) for all estimates.

We present results separately for RCTs and CBA studies.

For ITS studies, we intended to use the standardised change in level and change in slope as effect measures. We intended to perform meta‐analyses using the generic inverse variance method. We intended to enter the standardised outcomes into Review Manager 5.3 software as effect sizes, along with their standard errors (RevMan 2014).

We used the GRADE approach as described in the Cochrane Handbook for Systematic Reviews of Interventions to present the evidence quality for each combination of intervention and outcome (Higgins 2011b).

We based the downgrading of the quality of a body of evidence for a specific outcome on five factors.

1. Limitations of study

2. Indirectness of evidence

3. Inconsistency of results

4. Imprecision of results

5. Publication bias

The GRADE approach specifies four levels of quality, that is, high, moderate, low, and very low quality evidence.

For RCTs to yield high quality evidence, they have to be judged to have a low risk of bias, produce consistent, precise and directly applicable results measured with a valid non‐proxy outcome and they should present no evidence of reporting bias. The judgment regarding the quality of evidence is then reduced by a level for each of the factors not met. For non‐randomised studies, the overall quality of evidence is low to begin with but this can be upgraded if the studies have special strengths or downgraded if the studies have important limitations.

The interpretation of the quality of evidence is as follows.

• High quality: It is unlikely that further research will change our confidence in the estimate of effect.

• Moderate quality: Further research is likely to have an impact and may change the estimates.

• Low quality: Further research is very likely to have an important impact on our confidence in the estimate of effect.

• Very low quality: Any estimate of effect is very uncertain.

We intended to use the GRADEpro 3.6 software to generate 'Summary of findings' tables, but this was not possible as it was difficult to combine evidence from RCTs and non‐randomised studies statistically. Hence, we present the evidence quality and our considerations for each combination of intervention and outcome in an additional table.

Subgroup analysis and investigation of heterogeneity

We intended to perform subgroup analyses based on:

1. type of disease;

2. type of targeted physician (occupational physicians, general practitioners, specialists); and

3. occupational sector (e.g. construction, agriculture).

However, we could not include enough studies to conduct any subgroup analyses.

Sensitivity analysis

We intended to conduct a sensitivity analysis to test the robustness of our meta‐analysis results by omitting studies judged to have a high risk of bias. However, we could not include enough studies to conduct any sensitivity analyses.