Types of studies

Since the circumstances for evaluation studies are difficult during epidemics, we anticipated including a broad range of study designs.

We included simulation studies of different full‐body PPE types for donning and doffing procedures that use marker chemicals that make contamination visible. We included any study that compared different types of PPE or different procedures of donning and doffing, or different types of education and training.

We included field studies that compared outcomes between hospitals or treatment centres that used different types of PPE, different procedures, or different types of education and training. These studies were observational and retrospective, and measured infection rates and compliance rates. This included cohort studies, defined as studies that followed HCWs over time and compared the effect of PPE, procedures, or training on infection or compliance rates. This also included case‐control studies that compared PPE, procedures, or training retrospectively between cases that had become infected and comparable controls that did not get infected. We also included randomised controlled trials that compared different types of PPE, procedures, or training.

We intended to also include uncontrolled audit reports or case reports of PPE failure for descriptive purposes, but we did not find any. If we find any such reports in future updates of this review, we will not use them for drawing conclusions, but only to compare with findings produced by the above study types.

Types of participants

For simulation studies, we included participants using PPE designed for EVD or comparable highly infectious diseases with serious consequences.

For field studies, we included studies conducted with HCWs and ancillary staff exposed to body fluids in the form of splashes, droplets or aerosols contaminated with particles of highly infectious diseases that have serious consequences for health such as EVD or SARS. We excluded studies conducted with laboratory staff because there preventive measures would be more detailed and easier to comply with.

Types of interventions

1. We included studies that evaluated the effectiveness of different types of whole‐body protection, comparing different types, compositions, or amounts of the following:

• body protection such as gowns, coveralls or hazmat suits;

• eye and face protection such as glasses, goggles, face shields or visors, or masks or hoods that cover the entire head;

• hand protection: gloves; and

• foot protection: overshoes or boots.

We defined PPE as any of the above equipment designed or intended to protect healthcare staff from contamination with infected patients' body fluids.

We especially sought to include studies that had compared the use of gowns with coveralls, different types of fabrics, such as less breathable fabrics with more breathable fabrics, goggles versus visors, various forms of hoods in combination with goggles, single versus multiple layers of gloves, and taping versus no taping of separate elements of PPE.

2. We included studies that evaluated the effectiveness of different procedures for donning and doffing of the PPE.

We especially sought to include studies that had compared a single person or two person procedure, procedures with and without spraying disinfectants, procedures for changing gloves or PPE after gross contamination or breach of barrier protection versus no change.

3. We included any intervention to increase compliance with guidance for selection and proper use of PPE, including but not limited to:

• education,

• training,

• supervision during donning and doffing,

• information only such as posters, guideline leaflets, etc.,

• audit and feedback, or

• monetary or organisational incentives.

Types of outcome measures

Electronic searches

We conducted a systematic literature search to identify all published and unpublished trials that could be considered eligible for inclusion in this review. We adapted the search strategy we developed for PubMed (see Appendix 1) for use in the other electronic databases. The literature search identified potential studies in all languages. We asked native speakers to assess the papers in Russian (AP) and in Chinese (CCC) for potential inclusion in the review.

We searched the following electronic databases from inception to present for identifying potential studies:

• Cochrane Central Register of Controlled Trials (CENTRAL) (Wiley Online Library) until 20 January 2016;

• MEDLINE (PubMed) (Appendix 1) until 8 January 2016;

• EMBASE (embase.com) to 8 January 2016;

• CINAHL (EBSCOhost) to 20 January 2016;

• NIOSHTIC (OSH‐UPDATE) to 8 January 2016;

• NIOSHTIC‐2 (OSH‐UPDATE) to 8 January 2016;

• HSELINE (OSH‐UPDATE) to 8 January 2016;

• CISDOC (OSH‐UPDATE) to 8 January 2016;

We also conducted a search of ClinicalTrials.gov (www.ClinicalTrials.gov) and the WHO trials portal (www.who.int/ictrp/en/) which includes the Pan African Registry for potential studies on EVD. We searched all databases from their inception to the present and we did not impose a restriction on language of publication.

Searching other resources

We checked reference lists of all primary studies and reviewed articles for additional references. We contacted non‐governmental organisations involved in medical relief operations in the high risk EVD areas to identify additional unpublished materials:

• Médécins Sans Frontières (MSF)

• Save the Children

We also used Twitter to ask for unpublished reports from people in the field. Evidence Aid helped in locating relevant organisations and asking them for unpublished reports.

We further contacted the following manufacturers of PPE to request unpublished studies:

• DuPont

• 3M

• Alpha Pro Tech

Selection of studies

Pairs of two review authors (JV, CM, SI, JR, KN and RS) independently screened titles and abstracts of all the potential studies that we identified with our systematic search, to identify studies for inclusion. The same authors coded them as 'retrieve' (eligible or potentially eligible/unclear) or 'do not retrieve'. We retrieved the full‐text study reports/publication and pairs of two review authors (JV, CM, SI, JR, KN and RS) independently screened the full‐text and identified studies for inclusion, and identified and recorded reasons for exclusion of the ineligible studies. We used the computer programme Covidence for the selection of references and full‐text studies. We resolved any disagreement through discussion so there was no need to consult a third review author. We identified and excluded duplicates and collated multiple reports of the same study so that each study rather than each report is the unit of interest in the review. We recorded the selection process in sufficient detail and completed a PRISMA flow diagram (see Figure 2) and a 'Characteristics of excluded studies' table.

Figure 2

---

---

PRISMA study flow diagram

Data extraction and management

We used a data collection form for study characteristics and outcome data which had been piloted on one included study. Two review authors (JV, CM, JR, SI, ME, KN, RS) independently extracted the following study characteristics from included studies:

1. Methods: study design, total duration of study, study location, study setting, withdrawals, and date of study.

2. Participants: N, mean age or age range, gender, severity of condition, diagnostic criteria if applicable, inclusion criteria, and exclusion criteria.

3. Interventions: description of intervention, comparison, duration, intensity, content of both intervention and control condition, and co‐interventions.

4. Outcomes: description of primary and secondary outcomes specified and collected, and at which time points reported.

5. Notes: funding for trial, and notable conflicts of interest of trial authors, country where trial was conducted.

Pairs of two review authors (JV, CM, SI, JR, KN, ME, RS) independently extracted outcome data from included studies. We noted in the 'Characteristics of included studies' table if outcome data were not reported in a usable way. We resolved disagreements by consensus so there was no need to involve a third review author. One review author (JV) transferred the data into Review Manager (RevMan 2014). We double‐checked that data were entered correctly by comparing the data presented in the systematic review with the study reports. A second review author (CM) spot‐checked study characteristics for accuracy against the trial report.

Assessment of risk of bias in included studies

Pairs of two review authors (JV, CM, SI, JR, KN, ME, RS) independently assessed risk of bias for each randomised study using the criteria outlined in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011). We resolved any disagreements by discussion so there was no need to involve another author. We assessed the risk of bias according to the following domains in all randomised controlled trials.

1. Random sequence generation,

2. Allocation concealment,

3. Blinding of participants and personnel,

4. Blinding of outcome assessment,

5. Incomplete outcome data,

6. Selective outcome reporting, and

7. Other bias

We rated each potential source of bias as high, low or unclear and provided a quote from the study report or author together with a justification for our judgment in the 'Risk of bias' table. We summarised the risk of bias judgements across different studies for each of the domains listed. For compliance, we considered blinding to PPE type significant for the outcome assessor only. Where information on risk of bias relates to unpublished data or correspondence with a trial author, we noted this in the 'Risk of bias' table.

We considered randomised studies to have a low overall risk of bias when we judged random sequence generation and blinded outcome assessment to have a low risk of bias and none of the other domains to have a high risk of bias.

We used domains three to seven listed above for all non‐randomised studies. Instead of the domains one and two ‐ random sequence generation and allocation concealment ‐ we used the following items as suggested in the Cochrane Acrobat tool for the assessment of risk of bias in non‐randomised intervention studies:

1. Bias due to confounding. We made an overall assessment of risk of bias based on the following questions if the signalling question "Is confounding of the effect of intervention unlikely in this study?" was answered with no.

   • Did the authors use an appropriate analysis method that adjusted for all the critically important confounding domains?

   • Were confounding domains that were adjusted for measured validly and reliably by the variables available in this study? For this review question, we considered baseline differences between compared groups in the following factors significant: prior experience with PPE, healthcare qualification or education of HCWs, age and sex, ambient temperatures, and stressful activities.

2. Bias due to selection of participants into the study. We made an overall assessment of this risk of bias based on the following questions if the signalling questions "Was selection into the study unrelated to intervention or unrelated to outcome?" and "Do start of follow‐up and start of intervention coincide for most subjects?" were answered with no.

   • Were adjustment techniques used that are likely to correct for the presence of selection biases?

   • For case‐control studies: Were the controls sampled from the population that gave rise to the cases, or using another method that avoids selection bias?

We considered the domains of confounding and selection of participants to yield high, low or unclear risk of bias. For a non‐randomised study as a whole, we considered the study to have a low risk of bias if all domains received a judgment of low risk of bias comparable to an RCT. This means receiving a low risk of bias judgment on the two domains listed above as well as domains three to seven in the previous section.

When considering treatment effects, we took into account the risk of bias for the studies that contributed to that outcome.

Measures of treatment effect

We entered the outcome data for each study into the data tables in RevMan 2014 to calculate the treatment effects. We used risk ratios (RRs) for dichotomous outcomes, and mean differences (MDs) or standardised mean differences (SMDs) for continuous outcomes. When only effect estimates and their 95% confidence intervals or standard errors were reported in studies, we entered these data into RevMan 2014 using the generic inverse variance method. When authors used multivariate analyses, we used the most adjusted OR (Odds Ratios) or RRs. We ensured that higher scores for continuous outcomes had the same meaning for the particular outcome, explained the direction and reported where the directions were reversed, if this was necessary. If in future updates of this review we come across studies reporting results that we cannot enter in either way, we will describe them in the 'Characteristics of included studies' table, or we will enter the data into Additional tables. For cohort studies that compare an exposed to a non‐exposed population we intended to report both the RR for the intervention versus the control at baseline and at follow‐up for dichotomous outcomes to indicate the change brought about by the intervention but we did not find any such studies.

Unit of analysis issues

If in future updates of this review we come across studies that employ a cluster‐randomised design and that report sufficient data to be included in the meta‐analysis but do not make an allowance for the design effect, we will calculate the design effect based on a fairly large assumed intra‐cluster correlation of 0.10. We based this assumption of 0.10 being a realistic estimate by analogy on studies about implementation research (Campbell 2001). We will follow the methods stated in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011) for the calculations.

We intended to take the paired nature of the cross‐over design in the included studies into account in our data analysis. However, the included studies did not present suffidient data to do so and the results presented here are based on the unpaired test that is implemented in RevMan 2014 which resulted in wider confidence intervals than with the use of a paired t‐test.

Dealing with missing data

We contacted investigators in order to verify key study characteristics and obtain missing numerical outcome data where possible (e.g. when a study was identified as abstract only). If in future updates of this review we come across studies where this is not possible, and the missing data are thought to introduce serious bias, we will explore the impact of including such studies in the overall assessment of results by a sensitivity analysis.

Similarly, If in future updates of this review we come across studies where numerical outcome data are missing, such as SDs or correlation coefficients and they cannot be obtained from the authors, we will calculate them from other available statistics such as P values, according to the methods described in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011).

Assessment of heterogeneity

We assessed the clinical homogeneity of the results of included studies based on similarity of population, intervention, outcome and follow‐up. We considered populations as similar when they were HCWs directly engaged in patient treatment (nurses, doctors, paramedics) versus those who were not involved in patient therapy directly (cleaning and transport staff).

We considered interventions as similar when they fell into one of the intervention categories as stated in Types of interventions.

We considered any assessment of contamination of the skin or mucous membranes as similar enough to combine.

We considered the following follow‐up times as similar: from immediately following a procedure up until the end of the work shift (short term), and any time after the incubation time (long‐term).

We used the I² statistic to measure heterogeneity among the trials in each analysis. Where we identified substantial heterogeneity, we reported it and explored possible causes by prespecified subgroup analysis. We regarded an I² value above 50% as substantial heterogeneity.

We made a distinction between any kind of protection and protection that had been certified based on testing for biological hazards like CEN (European Committee for Standardization) or comparable standards.

Data synthesis

We pooled data from studies we judged to be clinically homogeneous using Review Manager 5.3 software (RevMan 2014). Where more than one study provided usable data in any single comparison, we performed a meta‐analysis. We used a random‐effects model when I² was above 40%; otherwise we used a fixed‐effect model. Where I² was higher than 75% we did not pool results of studies in meta‐analyses. We included a 95% confidence interval (CI) for all estimates.

If in future updates of this review, we come across studies with skewed data reported as medians and interquartile ranges, we will simply describe the data.

If in future updates of this review, we come across studies reporting multiple trial arms in a single trial, we will include only the relevant arms. If we need to combine two comparisons in the same meta‐analysis, we will halve the control group to avoid double‐counting.

Subgroup analysis and investigation of heterogeneity

If future updates of this review find a sufficient number of studies, we will carry out the following subgroup analyses: high income versus low and middle income countries. We will also use our primary outcomes in subgroup analyses, and we will use the X² test, as implemented in RevMan 2014, to test for subgroup interactions. At this time, we did not identify enough studies to allow for such a subgroup analysis.

Sensitivity analysis

If future updates of this review find a sufficient number of studies, we will perform sensitivity analyses defined a priori to assess the robustness of our conclusions. This involves including only studies we judge to have a low risk of bias. At this time we did not identify enough studies to allow such a sensitivity analysis. However, we did check if using a fixed‐effect model instead of a random‐effects model had an influence on our conclusions when heterogeneity was between 10% and 50%.