Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on medication adherence

LIVE: Thursday 27th June 2024, 18:00-19:30 (CEST)
Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.
ADVANCED SEARCH
Log in
Top
Trials
Published in: Trials 1/2010

Open Access 01-12-2010 | Research

Applicability and generalisability of the results of systematic reviews to public health practice and policy: a systematic review

Authors: Nizar Ahmad, Isabelle Boutron, Agnès Dechartres, Pierre Durieux, Philippe Ravaud

Published in: Trials | Issue 1/2010

Login to get access

Abstract

Background

The purpose of the study was to evaluate systematic reviews of research into two public health priorities, tobacco consumption and HIV infection, in terms of the reporting of data related to the applicability of trial results (i.e., whether the results of a trial can be reasonably applied or generalized to a definable group of patients in a particular setting in routine practice, also called external validity or generalisability).

Methods

All systematic reviews of interventions aimed at reducing or stopping tobacco use and treating or preventing HIV infection published in the Cochrane database of systematic reviews and in journals indexed in MEDLINE between January 1997 and December 2007 were selected. We used a standardized data abstraction form to extract data related to applicability in terms of the context of the trial, (country, centres, settings), participants (recruitment, inclusion and exclusion criteria, baseline characteristics of participants such as age, sex, ethnicity, coexisting diseases or co-morbidities, and socioeconomic status), treatment (duration, intensity/dose of treatment, timing and delivery format), and the outcomes assessment from selected reviews.

Results

A total of 98 systematic reviews were selected (57 Cochrane reviews and 41 non-Cochrane reviews); 49 evaluated interventions aimed at reducing or stopping tobacco use and 49 treating or preventing HIV infection. The setting of the individual studies was reported in 45 (46%) of the systematic reviews, the number of centres in 21 (21%), and the country where the trial took place in 62 (63%). Inclusion and exclusion criteria of the included studies were reported in 16 (16%) and 13 (13%) of the reviews, respectively. Baseline characteristics of participants in the included studies were described in 59 (60%) of the reviews. These characteristics concerned age in about half of the reviews, sex in 46 (47%), and ethnicity in 9 (9%).
Applicability of results was discussed in 13 (13%) of the systematic reviews. The reporting was better in systematic reviews by the Cochrane Collaboration than by non-Cochrane groups.

Conclusions

Our study highlighted the lack of consideration of applicability of results in systematic reviews of research into 2 public health priorities: tobacco consumption and HIV infection.
Appendix
Available only for authorised users
Literature
1.
Silagy CA, Stead LF, Lancaster T: Use of systematic reviews in clinical practice guidelines: case study of smoking cessation. Bmj. 2001, 323 (7317): 833-6. 10.1136/bmj.323.7317.833.CrossRefPubMedPubMedCentral
2.
3.
Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG: Epidemiology and reporting characteristics of systematic reviews. PLoS Med. 2007, 4 (3): e78-10.1371/journal.pmed.0040078.CrossRefPubMedPubMedCentral
4.
Dekkers OM, von Elm E, Algra A, Romijn JA, Vandenbroucke JP: How to assess the external validity of therapeutic trials: a conceptual approach. Int J Epidemiol. 2009, 39 (1): 89-94. 10.1093/ije/dyp174.CrossRefPubMed
5.
Berlin JA, Rennie D: Measuring the quality of trials: the quality of quality scales. JAMA. 1999, 282 (11): 1083-5. 10.1001/jama.282.11.1083.CrossRefPubMed
6.
Downs SH, Black N: The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health. 1998, 52 (6): 377-84. 10.1136/jech.52.6.377.CrossRefPubMedPubMedCentral
7.
Juni P, Altman DG, Egger M: Systematic reviews in health care: Assessing the quality of controlled clinical trials. Bmj. 2001, 323 (7303): 42-6. 10.1136/bmj.323.7303.42.CrossRefPubMedPubMedCentral
8.
Juni P, Witschi A, Bloch R, Egger M: The hazards of scoring the quality of clinical trials for meta-analysis. JAMA. 1999, 282 (11): 1054-60. 10.1001/jama.282.11.1054.CrossRefPubMed
9.
Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S: Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials. 1995, 16 (1): 62-73. 10.1016/0197-2456(94)00031-W.CrossRefPubMed
10.
Verhagen AP, de Vet HC, de Bie RA, Kessels AG, Boers M, Bouter LM, Knipschild PG: The Delphi list: a criteria list for quality assessment of randomized clinical trials for conducting systematic reviews developed by Delphi consensus. J Clin Epidemiol. 1998, 51 (12): 1235-41. 10.1016/S0895-4356(98)00131-0.CrossRefPubMed
11.
Lundh A, Gotzsche PC: Recommendations by Cochrane Review Groups for assessment of the risk of bias in studies. BMC Med Res Methodol. 2008, 8 (22): 22-10.1186/1471-2288-8-22.CrossRefPubMedPubMedCentral
12.
Hartling L, Ospina M, Liang Y, Dryden DM, Hooton N, Krebs Seida J, Klassen TP: Risk of bias versus quality assessment of randomised controlled trials: cross sectional study. BMJ. 2009, 339 (339): b4012-10.1136/bmj.b4012.CrossRefPubMedPubMedCentral
13.
Straus SE, McAlister F: Applying the results of trials and systematic reviews to our individual patients. Evid Based Ment Health. 2001, 4 (1): 6-7. 10.1136/ebmh.4.1.6.CrossRefPubMed
14.
15.
Naylor CD: Grey zones of clinical practice: some limits to evidence-based medicine. Lancet. 1995, 345 (8953): 840-2. 10.1016/S0140-6736(95)92969-X.CrossRefPubMed
16.
Rothwell PM: External validity of randomised controlled trials: "to whom do the results of this trial apply?". Lancet. 2005, 365 (9453): 82-93. 10.1016/S0140-6736(04)17670-8.CrossRefPubMed
17.
Pibouleau L, Boutron I, Reeves BC, Nizard R, Ravaud P: Applicability and generalisability of published results of randomised controlled trials and non-randomised studies evaluating four orthopaedic procedures: methodological systematic review. BMJ. 2009, 339 (339): b4538-10.1136/bmj.b4538.CrossRefPubMedPubMedCentral
18.
Ahmad N, Boutron I, Moher D, Pitrou I, Roy C, Ravaud P: Neglected external validity in reports of randomized trials: the example of hip and knee osteoarthritis. Arthritis Rheum. 2009, 61 (3): 361-9. 10.1002/art.24279.CrossRefPubMedPubMedCentral
19.
Glasziou P, Meats E, Heneghan C, Shepperd S: What is missing from descriptions of treatment in trials and reviews?. BMJ. 2008, 336 (7659): 1472-4. 10.1136/bmj.39590.732037.47.CrossRefPubMedPubMedCentral
20.
Jadad AR, Cook DJ, Jones A, Klassen TP, Tugwell P, Moher M, Moher D: Methodology and reports of systematic reviews and meta-analyses: a comparison of Cochrane reviews with articles published in paper-based journals. JAMA. 1998, 280 (3): 278-80. 10.1001/jama.280.3.278.CrossRefPubMed
21.
Lopez AD, Mathers CD, Ezzati M, Jamison DT, Murray CJ: Global and regional burden of disease and risk factors, 2001: systematic analysis of population health data. Lancet. 2006, 367 (9524): 1747-57. 10.1016/S0140-6736(06)68770-9.CrossRefPubMed
22.
WHO: WHO Report on the Global Tobacco Epidemic, 2008 - The MPOWER package. 2008, WHO
23.
Murray CJ, Lopez AD: Alternative projections of mortality and disability by cause 1990-2020: Global Burden of Disease Study. Lancet. 1997, 349 (9064): 1498-504. 10.1016/S0140-6736(96)07492-2.CrossRefPubMed
24.
Gilmore A, Pomerleau J, McKee M, Rose R, Haerpfer CW, Rotman D, Tumanov S: Prevalence of smoking in 8 countries of the former Soviet Union: results from the living conditions, lifestyles and health study. Am J Public Health. 2004, 94 (12): 2177-87. 10.2105/AJPH.94.12.2177.CrossRefPubMedPubMedCentral
25.
Wu Z, Sullivan SG, Wang Y, Rotheram-Borus MJ, Detels R: Evolution of China's response to HIV/AIDS. Lancet. 2007, 369 (9562): 679-90. 10.1016/S0140-6736(07)60315-8.CrossRefPubMed
26.
WHO: WHO, World Health Statistics Report, 2008. 2008, WHO
27.
Patrick K, Scutchfield FD, Woolf SH: External validity reporting in prevention research. Am J Prev Med. 2008, 34 (3): 260-2. 10.1016/j.amepre.2007.11.011.CrossRefPubMed
28.
Klesges LM, Dzewaltowski DA, Glasgow RE: Review of external validity reporting in childhood obesity prevention research. Am J Prev Med. 2008, 34 (3): 216-23. 10.1016/j.amepre.2007.11.019.CrossRefPubMed
29.
Bonell C, Oakley A, Hargreaves J, Strange V, Rees R: Assessment of generalisability in trials of health interventions: suggested framework and systematic review. BMJ. 2006, 333 (7563): 346-9. 10.1136/bmj.333.7563.346.CrossRefPubMedPubMedCentral
30.
31.
Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, Gøtzsche PC, Lang T, CONSORT GROUP (Consolidated Standards of Reporting Trials): The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med. 2001, 134 (8): 663-94.CrossRefPubMed
32.
Ethgen M, Boutron L, Steg PG, Roy C, Ravaud P: Quality of reporting internal and external validity data from randomized controlled trials evaluating stents for percutaneous coronary intervention. BMC Med Res Methodol. 2009, 9: 24-10.1186/1471-2288-9-24.CrossRefPubMedPubMedCentral
33.
Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P, CONSORT Group: Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration. Ann Intern Med. 2008, 148 (4): 295-309.CrossRefPubMed
34.
Giraudeau B, Mary JY: Planning a reproducibility study: how many subjects and how many replicates per subject for an expected width of the 95 per cent confidence interval of the intraclass correlation coefficient. Stat Med. 2001, 20 (21): 3205-14. 10.1002/sim.935.CrossRefPubMed
35.
Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D: The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009, 339: b2700-10.1136/bmj.b2700.CrossRefPubMedPubMedCentral
36.
White AR, Rampes H, Campbell JL: Acupuncture and related interventions for smoking cessation. Cochrane Database Syst Rev. 2006, 25 (1): CD000009-
37.
38.
Biau DJ, Porcher R, Boutron I: The account for provider and center effects in multicenter interventional and surgical randomized controlled trials is in need of improvement: a review. J Clin Epidemiol. 2008, 61 (5): 435-9. 10.1016/j.jclinepi.2007.10.018.CrossRefPubMed
39.
Glasgow RE, Bull SS, Gillette C, Klesges LM, Dzewaltowski DA: Behavior change intervention research in healthcare settings: a review of recent reports with emphasis on external validity. Am J Prev Med. 2002, 23 (1): 62-9. 10.1016/S0749-3797(02)00437-3.CrossRefPubMed
40.
Jaeschke R, Guyatt GH, Dellinger P, Schünemann H, Levy MM, Kunz R, Norris S, Bion J, GRADE Working Group: Use of GRADE grid to reach decisions on clinical practice guidelines when consensus is elusive. BMJ. 2008, 337: a744-10.1136/bmj.a744.CrossRefPubMed
41.
Atkins D, Best D, Briss PA, Eccles M, Falck-Ytter Y, Flottorp S, Guyatt GH, Harbour RT, Haugh MC, Henry D, Hill S, Jaeschke R, Leng G, Liberati A, Magrini N, Mason J, Middleton P, Mrukowicz J, O'Connell D, Oxman AD, Phillips B, Schünemann HJ, Edejer TT, Varonen H, Vist GE, Williams JW, Zaza S, GRADE Working Group: Grading quality of evidence and strength of recommendations. BMJ. 2004, 328 (7454): 1490-10.1136/bmj.328.7454.1490.CrossRefPubMed
42.
McAlister FA, van Diepen S, Padwal RS, Johnson JA, Majumdar SR: How evidence-based are the recommendations in evidence-based guidelines?. PLoS Med. 2007, 4 (8): e250-10.1371/journal.pmed.0040250.CrossRefPubMedPubMedCentral
43.
Moja LP, Telaro E, D'Amico R, Moschetti I, Coe L, Liberati A: Assessment of methodological quality of primary studies by systematic reviews: results of the metaquality cross sectional study. BMJ. 2005, 330 (7499): 1053-10.1136/bmj.38414.515938.8F.CrossRefPubMedPubMedCentral
44.
Moher D, Cook DJ, Jadad AR, Tugwell P, Moher M, Jones A, Pham B, Klassen TP: Assessing the quality of reports of randomised trials: implications for the conduct of meta-analyses. Health Technol Assess. 1999, 3 (12): i-iv. 1-98PubMed
45.
Merson MH, O'Malley J, Serwadda D, Apisuk C: The history and challenge of HIV prevention. Lancet. 2008, 372 (9637): 475-88. 10.1016/S0140-6736(08)60884-3.CrossRefPubMed
Metadata
Title
Applicability and generalisability of the results of systematic reviews to public health practice and policy: a systematic review
Authors
Nizar Ahmad
Isabelle Boutron
Agnès Dechartres
Pierre Durieux
Philippe Ravaud
Publication date
01-12-2010
Publisher
BioMed Central
Published in
Trials / Issue 1/2010
Electronic ISSN: 1745-6215
DOI
https://doi.org/10.1186/1745-6215-11-20

Other articles of this Issue 1/2010

Trials 1/2010 Go to the issue

Study protocol

A prospective evaluation of the safety and efficacy of the TAXUS Element paclitaxel-eluting coronary stent system for the treatment of de novo coronary artery lesions: Design and statistical methods of the PERSEUS clinical program

Methodology

Quality of reporting of trial abstracts needs to be improved: using the CONSORT for abstracts to assess the four leading Chinese medical journals of traditional Chinese medicine

Study protocol

Comparison of the collagen haemostat Sangustop® versus a carrier-bound fibrin sealant during liver resection; ESSCALIVER-Study

Study protocol

Losartan therapy in adults with Marfan syndrome: study protocol of the multi-center randomized controlled COMPARE trial

Study protocol

Rationale and design of the randomised clinical trial comparing early medication change (EMC) strategy with treatment as usual (TAU) in patients with Major Depressive Disorder - the EMC trial

Research

Fluvastatin in the first-line therapy of acute coronary syndrome: results of the multicenter, randomized, double-blind, placebo-controlled trial (the FACS-trial)