Types of studies

This review includes reports of empirical studies that investigate samples of primary research studies. To avoid confusion we will use the terms 'studies' for the primary research studies and 'papers' for the reports of empirical studies of primary research studies. We will use the term trials to describe studies of a randomized clinical trial design.

We included papers of cross‐sectional studies, cohort studies, systematic reviews or meta‐analyses that quantitatively compared primary research of human drug or medical device studies sponsored by the pharmaceutical or device industries with studies that had other sources of sponsorship. These papers could report the results of methodological studies or systematic reviews that had a pre‐specified subgroup or sensitivity analysis by sponsorship source. We also included papers investigating sources of heterogeneity (e.g. using meta‐regression) if sponsorship was investigated. Drugs were defined as medications that require approval by a regulatory authority as a prescription drug, recognizing that these approval standards vary worldwide. Devices were defined based on the Food and Drug Administration (FDA) definition as instruments intended for use in the diagnosis, treatment or prevention of disease.

We excluded papers without quantitative data related to our primary or secondary outcomes. We excluded papers of the effects of sponsorship by non‐pharmaceutical or non‐device (e.g. tobacco, food or chemical) industries, and papers that evaluated the effectiveness of herbal supplements or medical procedures. Papers examining mixed interventions (e.g. pharmaceuticals and educational interventions) were included if drug or device data were reported separately or could be obtained from the authors.

We excluded papers that quantitatively compared the association of sponsorship and results of syntheses of research studies (i.e. systematic reviews or meta‐analyses) or pharmacoeconomic studies of drugs or devices. We also excluded analyses of pharmacokinetic studies and studies restricted to non humans (e.g. animal or cell cultures).

Only papers published in full, including structured research letters, were included. We excluded unstructured letters to the editor and conference abstracts. This decision was based on the poor reporting quality of data in letters and conference abstracts encountered in a previous version of our review (Lexchin 2003). A comment to the previous version of our review (Lundh 2012) suggested that the exclusion of conference abstracts and letters could have introduced publication bias. Therefore, we included conference abstracts and all types of letters in a sensitivity analysis (see below). We had no language restrictions.

Types of data

Drug and device papers including human research studies comparing drug to placebo, device to sham, drug to drug, drug to device, device to device, or mixed comparisons where the effectiveness, efficacy or harms of the drug or device were evaluated. A few papers included data from both unpublished and published studies. If data were reported separately for the published studies or were available from the authors we used data from published studies only. The reason for this decision was that published studies represent what is available to users of the medical literature and our focus was on assessing biases in published studies.

Types of methods

We defined sponsorship as funding or provision of free drug or devices. Drug or device studies with pharmaceutical or device industry funding versus those with other or undisclosed funding were included. We extracted the definition of industry funding verbatim from the included papers (see Data extraction and management) and reported this in the 'Characteristics of included studies' table. For analysis, we grouped the definitions into a variety of categories, including 100% pharmaceutical or device company funding, 100% non‐industry funding, mixed funding (e.g. non‐industry and industry collaboration), free provision of drug or device only, and undisclosed funding.

We included papers that compared industry sponsored studies with non‐industry sponsored studies and also papers that compared studies of products by competing manufacturers (i.e. studies sponsored by the manufacturer of the test treatment with studies sponsored by the manufacturer of the control treatment); we analyzed the two types of papers separately.

Types of outcome measures

Electronic searches

In this update, we searched Ovid MEDLINE (R) In‐Process and other non‐indexed citations and Ovid MEDLINE (R) (2010 to February 2015), Ovid Embase (2010 to February 2015) and the Cochrane Methodology Register (2015, Issue 2) (Wiley InterScience Online). We searched the Web of Science (June 2015) for papers that cited any of the papers included in our review.

Searching other resources

Other sources of data included author files, searches of reference lists of included papers and previous systematic reviews.

Selection of studies

Two pairs of assessors  (LB and JS or BM and JL) screened the titles and abstracts, when available, of all retrieved records for obvious exclusions, and assessed the remaining papers based on full text. Any disagreements were resolved by consensus and reasons for exclusions of potentially eligible papers are described in the 'Characteristics of excluded studies' table. There was no need for translation of non‐English papers.

Data extraction and management

Two pairs of assessors (AL and JS or BM and JL) independently extracted data from included papers; differences in data extraction were resolved by consensus. 

We extracted data on the following.

• Year published.

• Country of corresponding author.

• Study objective.

• Study design used in the paper (cohort, cross‐sectional, systematic review or meta‐analysis, other).

• Study domain ‐ descriptive (e.g. oncology drug trials).

• Study domain ‐ category (drug/device class, specific disease, medical specialty/type of diseases, mixed).

• Type of studies (drug, device, drug and device, mixed).

• Type of comparisons (drug versus drug, drug versus placebo, device versus device, device versus sham, device versus drug, mixed, other).

• Sample strategy used to locate research studies (electronic search only, electronic plus other, sampling of journals, sampling by venue (e.g. conference abstracts)).

• Whether there were language restrictions on the search.

• Number of studies included in the sample.

• Time period covered by studies in the paper.

• Sponsorship categories coded in the paper. Categories were:

• • 100% pharmaceutical/device company funded;

  • 100% non‐profit funded;

  • mixed funding ‐ e.g. non‐industry and industry collaboration;

  • provision of drug or device only; and

  • undisclosed funding.

• Sponsorship categories used in analysis in the paper (e.g. 100% industry funded grouped with mixed funding for industry category).

• Description of role of the sponsor (if any). For example, definition of the sponsor’s role in the design, implementation or reporting in the sample of studies.

• Criteria used to assess risk of bias of the studies included in the paper.

• Primary purpose of the study.

• Whether the paper commented on appropriateness of comparators.

• Data on sponsorship and results.

• Data on sponsorship and conclusions.

• Data on sponsorship and effect size.

• Data on sponsorship and risk of bias.

• Data on sponsorship and concordance between study results and conclusions.

• Additional relevant data.

• Funding source of included paper. As this item was not included in the previous version of the review we also extracted data from papers included in the previous version.

Assessment of risk of bias in included studies

Since there are no validated criteria for assessing risk of bias in these types of papers, we developed our own criteria. We reviewed papers for high, low or unclear risk of bias for each of four criteria. If a criterion was met, it was regarded as having low risk of bias, and high risk of bias otherwise. If we could not determine whether a criterion was met, we coded it as unclear. We used the following criteria:

• whether explicit and well‐defined criteria that could be replicated by others were used to select studies for inclusion/exclusion;

• whether there was an adequate study inclusion method, with two or more assessors selecting studies;

• whether the search for studies was comprehensive; and

• whether methodological differences and other characteristics that could introduce bias were controlled for or explored.

Measures of the effect of the methods

We performed a meta‐analysis of the papers that reported the association of sponsorship with favorable study outcomes in cases where a pooled risk ratio (RR) and its 95% confidence interval could be computed.

The definition of a favorable outcome varied among papers. In some papers favorable outcomes were defined as those that were favorable to the sponsor's product and in others favorable to the test treatment. This difference in terminology did not matter when the comparison was between active treatment and placebo, since the sponsor's product was the active treatment and not placebo. For head‐to‐head comparisons, however, the sponsor could be either the manufacturer of the test treatment or the control treatment. In these cases, when data were available, we recoded outcomes as to whether they were favorable to the sponsor's product.

We separately analyzed papers of industry sponsored head‐to‐head studies, comparing studies sponsored by the manufacturer of the test treatment with studies sponsored by the manufacturer of the comparator treatment. This was done by assigning the newest treatment (most recent FDA approval date) as the 'test' treatment and the older treatment as the 'comparator' treatment using similar methods as described by Bero and colleagues (Bero 2007) and comparing the number of studies favorable to the test treatment in the two groups (i.e. sponsor produces test treatment or sponsor produces comparator treatment).

At the time many of the papers were published, the approach was to assess the methodological quality of studies as opposed to an assessment of the risk of bias of studies. We therefore recoded the data on methodological quality into 'Risk of bias' categories. So, for example, a trial with adequate concealment of allocation was coded as low risk of bias and a trial with inadequate concealment of allocation as high risk of bias. Some papers assessed risk of bias by summarizing results for individual domains into an overall methodological quality score (i.e. a scale approach). There are substantial methodological problems related to quality scales (Jüni 1999), and their use is not recommended. We therefore did not combine the results obtained with these scales, but report the results descriptively. The included papers assessed blinding using different approaches. Some papers rated blinding for the study overall, for example whether a study used matching placebo tablets (which could be considered blinded) and some papers assessed who was blinded, similar to the Cochrane 'Risk of bias' tool. The Cochrane 'Risk of bias' tool assesses blinding to protect against performance bias (e.g. clinicians or patients are blinded) and to protect against detection bias (e.g. outcome assessors are blinded). We therefore categorized each 'Risk of bias' assessment into the items blinding‐overall, blinding‐performance bias and blinding‐detection bias.

Dealing with missing data

We contacted authors of the original papers in an attempt to obtain missing data. If papers included studies reporting conflicts of interest, but not the source of funding, we contacted the authors in order to obtain separate data for funding. In total, we contacted authors of eight papers included in this update and received additional data for five of these papers.

Assessment of heterogeneity

We assessed heterogeneity using I². We defined substantial statistical heterogeneity as an I² > 50% (Higgins 2011a).

Data synthesis

We used Review Manager (RevMan 2014) to analyze data. For dichotomous data we used the Mantel‐Haenszel random‐effects model to create a pooled RR. In the previous version of this review (Lundh 2012), we used a fixed‐effect model as default and a random‐effects model when substantial heterogeneity was observed. However, due to the large clinical heterogeneity between papers (e.g. study domains, study designs and definition of outcomes), we decided that a random‐effects model was more appropriate.

Subgroup analysis and investigation of heterogeneity

We considered the following factors as potential explanations for heterogeneity and investigated them in separate subgroup analyses for our primary outcomes.    

• We hypothesized that the association of industry sponsorship and favorable outcomes may be larger in high risk of bias papers. We assessed overall risk of bias of the included papers using the criteria described in 'Assessment of risk of bias in included studies'. We regarded papers with adequate study inclusion, a comprehensive search and controlling for bias as having a low risk of bias; others as having a high risk. We compared low risk of bias papers with high risk of bias papers in a subgroup analysis.

• We compared papers of drug studies with device studies, as the mechanisms of influencing study outcomes may differ between the industries. For example, drug trials are more regulated than device trials, which could have an influence on biases in the design, conduct and reporting of the trials. We compared this in a subgroup analysis.   

• As the study domain might contribute to heterogeneity, we compared papers on specific treatments or diseases with papers of mixed domains in another subgroup analysis.

Sensitivity analysis

We undertook the following sensitivity analyses to test the robustness of our findings for our primary outcomes.

• The primary analyses compared the number of favorable results and conclusions in papers with industry sponsorship to those with other sources of sponsorship; 'industry sponsorship' included 100% pharmaceutical/device company funding, mixed funding and provision of drug or device only. 'Non‐industry sponsorship' included 100% government funding, 100% non‐industry funding and undisclosed funding. In a sensitivity analysis, we excluded those studies with mixed funding sources and those with funding consisting solely of free product from the 'industry sponsorship' category, and excluded studies with undisclosed funding from the category of 'non‐industry sponsorship', to determine if these had an impact on the initial analysis. As noted under 'Data extraction and management', we were reliant on how the studies in our review defined 'funding'.

• A sensitivity analysis restricted to papers that adjusted for confounders (e.g. adjusted for sample size and concealment of allocation using logistic regression) using adjusted estimates. We used the generic inverse variance method to pool adjusted odds ratios in a random‐effects model.

• A sensitivity analysis where all analyses were based on a fixed‐effect model.

• One paper (Finucane 2004), included unpublished abstracts from conference proceedings. One paper (Lynch 2007), included manuscripts submitted to a medical journal of which the majority were never published. However, based on the reported data it was not possible to extract data from the published studies separately. As these two papers included unpublished data and we planned to analyze only published data, we conducted a sensitivity analysis that excluded these two papers.

• Many of the included papers investigated similar domains (e.g. antidepressants or oncology drugs) or included studies from similar journals in overlapping time‐periods. This is likely to lead to double counting if data from the same studies are included more than once and to an overestimate of the precision of effect estimates. Due to the way data were reported in the papers, it was not possible ensure that data was not double counted. Instead we performed a sensitivity analysis restricted to papers on specific treatments or diseases where none of the other included papers were related to the same domain.

• We included letters and conference abstracts reporting quantitative data in a sensitivity analysis.

Quality of Evidence

We created a 'Summary of findings' table using the GRADE approach (Higgins 2011a) and using the MAGICapp software (MAGICapp; Vandvik 2013).