Top

Published in:

Open Access 01-12-2019 | Research article

Age-treatment subgroup analyses in Cochrane intervention reviews: a meta-epidemiological study

Authors: Patrick Liu, John P. A. Ioannidis, Joseph S. Ross, Sanket S. Dhruva, Anita T. Luxkaranayagam, Vasilis Vasiliou, Joshua D. Wallach

Published in: BMC Medicine | Issue 1/2019

Abstract

Background

There is growing interest in evaluating differences in healthcare interventions across routinely collected demographic characteristics. However, individual subgroup analyses in randomized controlled trials are often not prespecified, adjusted for multiple testing, or conducted using the appropriate statistical test for interaction, and therefore frequently lack credibility. Meta-analyses can be used to examine the validity of potential subgroup differences by collating evidence across trials. Here, we characterize the conduct and clinical translation of age-treatment subgroup analyses in Cochrane reviews.

Methods

For a random sample of 928 Cochrane intervention reviews of randomized trials, we determined how often subgroup analyses of age are reported, how often these analyses have a P < 0.05 from formal interaction testing, how frequently subgroup differences first observed in an individual trial are later corroborated by other trials in the same meta-analysis, and how often statistically significant results are included in commonly used clinical management resources (BMJ Best Practice, UpToDate, Cochrane Clinical Answers, Google Scholar, and Google search).

Results

Among 928 Cochrane intervention reviews, 189 (20.4%) included plans to conduct age-treatment subgroup analyses. The vast majority (162 of 189, 85.7%) of the planned analyses were not conducted, commonly because of insufficient trial data. There were 22 reviews that conducted their planned age-treatment subgroup analyses, and another 3 reviews appeared to perform unplanned age-treatment subgroup analyses. These 25 (25 of 928, 2.7%) reviews conducted a total of 97 age-treatment subgroup analyses, of which 65 analyses (in 20 reviews) had non-overlapping subgroup levels. Among the 65 age-treatment subgroup analyses, 14 (21.5%) did not report any formal interaction testing. Seven (10.8%) reported P < 0.05 from formal age-treatment interaction testing; however, none of these seven analyses were in reviews that discussed the potential biological rationale or clinical significance of the subgroup findings or had results that were included in common clinical practice resources.

Conclusion

Age-treatment subgroup analyses in Cochrane intervention reviews were frequently planned but rarely conducted, and implications of detected interactions were not discussed in the reviews or mentioned in common clinical resources. When subgroup analyses are performed, authors should report the findings, compare the results to previous studies, and outline any potential impact on clinical care.

Available only for authorised users

Kent DM, Rothwell PM, Ioannidis JP, et al. Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal. Trials. 2010;11:85.PubMedPubMedCentralCrossRef

Hamburg MA, Collins FS. The path to personalized medicine. N Engl J Med. 2010;363(4):301–4.PubMedCrossRef

Collins FS, Varmus H. A new initiative on precision medicine. N Engl J Med. 2015;372(9):793–5.PubMedPubMedCentralCrossRef

Assmann SF, Pocock SJ, Enos LE, et al. Subgroup analysis and other (mis)uses of baseline data in clinical trials. Lancet. 2000;355(9209):1064–9.PubMedCrossRef

Pocock SJ, Hughes MD, Lee RJ. Statistical problems in the reporting of clinical trials. A survey of three medical journals. N Engl J Med. 1987;317(7):426–32.PubMedCrossRef

Wallach JD, Sullivan PG, Trepanowski JF, et al. Evaluation of evidence of statistical support and corroboration of subgroup claims in randomized clinical trials. JAMA Intern Med. 2017;177(4):554–60.PubMedPubMedCentralCrossRef

Wang R, Lagakos SW, Ware JH, et al. Statistics in medicine--reporting of subgroup analyses in clinical trials. N Engl J Med. 2007;357(21):2189–94.PubMedCrossRef

Pocock SJ, Assmann SE, Enos LE, et al. Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practice and problems. Stat Med. 2002;21(19):2917–30.PubMedCrossRef

Oxman AD, Guyatt GH. A consumer’s guide to subgroup analyses. Ann Intern Med. 1992;116(1):78–84.PubMedCrossRef

10.

Kent DM, Steyerberg E, van Klaveren D. Personalized evidence based medicine: predictive approaches to heterogeneous treatment effects. BMJ. 2018;363:k4245.PubMedCrossRefPubMedCentral

11.

Centers for Medicare and Medicaid Services (CMS). CMS releases quality data showing racial, ethnic and gender differences in Medicare Advantage health care during National Minority Health Month. Press release. 13 April 2017. https://www.cms.gov/newsroom/press-releases/cms-releases-quality-data-showing-racial-ethnic-and-gender-differences-medicare-advantagehealth-care. Accessed 25 Feb 2019.

12.

U.S. Food and Drug Administration (FDA). Investigational new drug applications and new drug applications. 21 CFR 312, 63 FR 6854. 11 Feb 1998. https://www.govinfo.gov/app/details/FR-1998-02-11/98-3422. Accessed 25 Feb 2019.

13.

U.S. Food and Drug Administration (FDA). FDA action plan to enhance collection and availability of demographic subgroup data. FDA report. Silver Spring, MD: FDA; August 2014. https://www.fda.gov/downloads/RegulatoryInformation/Legislation/FederalFoodDrugandCosmeticActFDCAct/SignificantAmendmentstotheFDCAct/FDASIA/UCM410474.pdf. Accessed 25 Feb 2019.

14.

U.S. Food and Drug Administration (FDA). Evaluating inclusion and exclusion criteria in clinical trials. FDA report. Washington, DC: FDA; 16 April 2018. https://www.fda.gov/downloads/RegulatoryInformation/LawsEnforcedbyFDA/SignificantAmendmentstotheFDCAct/FDARA/UCM613054.pdf. Accessed 25 Feb 2019.

15.

Tugwell P, Petkovic J, Welch V, et al. Setting priorities for knowledge translation of Cochrane reviews for health equity: evidence for equity. Int J Equity Health. 2017;16(1):208.PubMedPubMedCentralCrossRef

16.

Welch VA, Norheim OF, Jull J, et al. CONSORT-Equity 2017 extension and elaboration for better reporting of health equity in randomised trials. BMJ. 2017;359:j5085.PubMedCrossRef

17.

Campbell and Cochrane Equity Methods Group. About us. Ottawa: Cochrane; 2019. https://methods.cochrane.org/equity/about-us. Accessed 25 Feb 2019.

18.

National Institutes of Health (NIH). NIH policy and guidelines on the inclusion of individuals across the lifespan as participants in research involving human subjects. NIH notice. Bethesda, MD: NIH; 19 December 2017. https://grants.nih.gov/grants/guide/notice-files/NOT-OD-18-116.html. Accessed 25 Feb 2019.

19.

U.S. Food and Drug Administration (FDA). Evaluation and reporting of age-, race-, and ethnicity-specific data in medical device clinical studies. Washington, DC: U.S. Department of Health and Human Services, Food and Drug Administration, Center for Devices and Raiological Health (CDRH), Center for Biologics Evaluation Research (CBER), Office of the Commissioner (OC): 12 September 2017. https://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM507278.pdf. Accessed 25 Feb 2019.

20.

Preston RA, Materson BJ, Reda DJ, et al. Age-race subgroup compared with renin profile as predictors of blood pressure response to antihypertensive therapy. Department of Veterans Affairs Cooperative Study Group on antihypertensive agents. JAMA. 1998;280(13):1168–72.PubMedCrossRef

21.

Zhang S, Liang F, Li W, et al. Subgroup analyses in reporting of phase III clinical trials in solid tumors. J Clin Oncol. 2015;33(15):1697–702.PubMedCrossRef

22.

Deeks JJ, Higgins JPT, Altman DG (editors). Chapter 9: Analyzing and undertaking meta-analyses. In: Higgins JPT, Green S (editors). Cochrane handbook for systematic reviews of interventions version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from www.handbook.cochrane.org. Accessed 25 Feb 2019.

23.

U.S. General Accountability Office (GAO). Better oversight needed to help ensure continued progress including women in health research. GAO report. Washington, DC: GAO; 22 October 2015. https://www.gao.gov/assets/680/673276.pdf. Accessed 25 Feb 2019.

24.

Campbell and Cochrane Equity Methods Group. Sex/Gender Cochrane Corner. Ottawa: Cochrane; 2019. https://methods.cochrane.org/equity/igh-cochrane-corner. Accessed 25 Feb 2019.

25.

Wallach JD, Sullivan PG, Trepanowski JF, et al. Sex based subgroup differences in randomized controlled trials: empirical evidence from Cochrane meta-analyses. BMJ. 2016;355:i5826.PubMedPubMedCentralCrossRef

26.

U.S. Food and Drug Administration (FDA). Collection, analysis, and availability of demographic subgroup data for FDA-approved medical products. FDA report. Silver Spring, MD: FDA; August 2013. https://www.fda.gov/downloads/RegulatoryInformation/Legislation/FederalFoodDrugandCosmeticActFDCAct/SignificantAmendmentstotheFDCAct/FDASIA/UCM365544.pdf. Accessed 25 Feb 2019.

27.

Rochon P. Drug prescribing for older adults. Section Editor: Schmader KE, Deputy Editor: Givens, J. UpToDate. Last updated 19 Feb 2019. https://www.uptodate.com/contents/drug-prescribing-for-older-adults. Accessed 25 Feb 2019.

28.

Williams K, Thomson D, Seto I, et al. Standard 6: age groups for pediatric trials. Pediatrics. 2012;129(Suppl 3):S153–60.PubMedCrossRef

29.

Contopoulos-Ioannidis DG, Seto I, Hamm MP, et al. Empirical evaluation of age groups and age-subgroup analyses in pediatric randomized trials and pediatric meta-analyses. Pediatrics. 2012;129(Suppl 3):S161–84.PubMedCrossRef

30.

Vandermeer B, van der Tweel I, Jansen-van der Weide MC, et al. Comparison of nuisance parameters in pediatric versus adult randomized trials: a meta-epidemiologic empirical evaluation. BMC Med Res Methodol. 2018;18(1):7.PubMedPubMedCentralCrossRef

31.

Contopoulos-Ioannidis DG, Baltogianni MS, Ioannidis JP. Comparative effectiveness of medical interventions in adults versus children. J Pediatr. 2010;157(2):322–30 e17.PubMedCrossRef

32.

Raz I, Ceriello A, Wilson PW, et al. Post hoc subgroup analysis of the HEART2D trial demonstrates lower cardiovascular risk in older patients targeting postprandial versus fasting/premeal glycemia. Diabetes Care. 2011;34(7):1511–3.PubMedPubMedCentralCrossRef

33.

Meade TW, Brennan PJ. Determination of who may derive most benefit from aspirin in primary prevention: subgroup results from a randomised controlled trial. BMJ. 2000;321(7252):13–7.PubMedPubMedCentralCrossRef

34.

Kwon Y, Lemieux M, McTavish J, Wathen N. Identifying and removing duplicate records from systematic review searches. J Med Libr Assoc. 2015;103(4):184–8.PubMedCrossRefPubMedCentral

35.

Sun X, Ioannidis JPA, Agoritsas T, Alba AC, Guyatt G. How to use a subgroup analysis: users’ guide to the medical literature. JAMA. 2014;311(4):405–11.PubMedCrossRef

36.

Boreinstein M, Higgins JPT. Meta-analysis and subgroups. Prev Sci. 2013;14:134–43.CrossRef

37.

Sedgwick P. Meta-analyses: heterogeneity and subgroup analysis. BMJ. 2013;346:f4040.CrossRef

38.

Ioannidis JPA. The proposal to lower P value thresholds to .005. JAMA. 2018;319(14):1429–30.PubMedCrossRef

39.

Adams N, Bestall JM, Jones PW. Budesonide at different doses for chronic asthma. Cochrane Database Syst Rev. 2001;4:CD003271.

40.

Adams N, Bestall JM, Jones PW. Fluticasone versus beclomethasone or budesonide for chronic asthma. Cochrane Database Syst Rev. 2007;4:CD002310.

41.

Mbuagbaw L, Morgano GP, Lawson DO, et al. Subgroup analyses are seldom possible and subgroup effects are rare in Cochrane HIV systematic reviews. J Clin Epidemiol. 2018;104:143–4.PubMedCrossRef

42.

Evans AT, Mints G. Evidence-based medicine. UpToDate; 2019.

43.

Haynes RB. What kind of evidence is it that Evidence-Based Medicine advocates want health care providers and consumers to pay attention to? BMC Health Serv Res. 2002;2:3.PubMedPubMedCentralCrossRef

44.

Jadad AR, Haynes RB. The Cochrane Collaboration—advances and challenges in improving evidence-based decision making. Med Decis Mak. 1998;18(1):2–9 discussion 16-8.CrossRef

45.

Tse T, Williams RJ, Zarin DA. Reporting “basic results” in ClinicalTrials.gov. Chest. 2009;136(1):295–303.PubMedPubMedCentralCrossRef

46.

U.S. Food and Drug Administration. Guidance for industry: collection of race and ethnicity data in clinical trials. Washington, DC: U.S. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research (CDER), Center for Biologics Evaluation and Research (CBER), Center for Devices and Radiologic Health (CDRH), Office of the Commissioner (OC); September 2005. http://www.fda.gov/downloads/RegulatoryInformation/Guidances/ucm126396.pdf. Accessed 25 Feb 2019.

47.

da Costa BR, Juni P. Systematic reviews and meta-analyses of randomized trials: principles and pitfalls. Eur Heart J. 2014;35(47):3336–45.PubMedCrossRef

48.

Sun X, Briel M, Busse JW, et al. Credibility of claims of subgroup effects in randomised controlled trials: systematic review. BMJ. 2012;344:e1553.PubMedCrossRef

49.

Sun X, Briel M, Walter SD, et al. Is a subgroup effect believable? Updating criteria to evaluate the credibility of subgroup analyses. BMJ. 2010;340:c117.PubMedCrossRef

50.

Rothwell PM. Treating individuals 2. Subgroup analysis in randomised controlled trials: importance, indications, and interpretation. Lancet. 2005;365(9454):176–86.PubMedCrossRef

51.

Moher D, Schulz KF, Altman DG, et al. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. J Am Podiatr Med Assoc. 2001;91(8):437–42.PubMedCrossRef

52.

Dunne J, Rodriguez WJ, Murphy MD, et al. Extrapolation of adult data and other data in pediatric drug-development programs. Pediatrics. 2011;128(5):e1242–9.PubMedCrossRef

53.

Downing NS, Shah ND, Neiman JH, et al. Participation of the elderly, women, and minorities in pivotal trials supporting 2011–2013 U.S. Food and Drug Administration approvals. Trials. 2016;17:199.PubMedPubMedCentralCrossRef

54.

Chandler J, Lasserson T, Higgins JPT, Tovey D, Churchill R. Standard for the planning, conduct and reporting of updates of Cochrane Intervention Reviews. In: JPT H, Lasserson T, Chandler J, Tovey D, Churchill R, editors. Methodological Expectations of Cochrane Intervention Reviews. London: Cochrane; 2016.

55.

Schuit E, Li AH, Ioannidis JPA. How often can meta-analyses of individual-level data individualize treatment? A meta-epidemiologic study. Int J Epidemiol. 2019;48(2):596–608.CrossRef

56.

Song F, Bachmann MO. Cumulative subgroup analysis to reduce waste in clinical research for individualised medicine. BMC Med. 2016;14(1):197.PubMedPubMedCentralCrossRef

Title: Age-treatment subgroup analyses in Cochrane intervention reviews: a meta-epidemiological study
Authors: Patrick Liu
John P. A. Ioannidis
Joseph S. Ross
Sanket S. Dhruva
Anita T. Luxkaranayagam
Vasilis Vasiliou
Joshua D. Wallach
Publication date: 01-12-2019
Publisher: BioMed Central
Published in: BMC Medicine / Issue 1/2019
Electronic ISSN: 1741-7015
DOI: https://doi.org/10.1186/s12916-019-1420-8

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Age-treatment subgroup analyses in Cochrane intervention reviews: a meta-epidemiological study

Abstract

Background

Methods

Results

Conclusion

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Background

Methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 1/2019

Tobacco smoking and risk of 36 cardiovascular disease subtypes: fatal and non-fatal outcomes in a large prospective Australian study

Translational research on reserve against neurodegenerative disease: consensus report of the International Conference on Cognitive Reserve in the Dementias and the Alzheimer’s Association Reserve, Resilience and Protective Factors Professional Interest Area working groups

From high to low malaria transmission in Zanzibar—challenges and opportunities to achieve elimination

Prospective association between ultra-processed food consumption and incident depressive symptoms in the French NutriNet-Santé cohort

The haematological consequences of Plasmodium vivax malaria after chloroquine treatment with and without primaquine: a WorldWide Antimalarial Resistance Network systematic review and individual patient data meta-analysis

The global impact and cost-effectiveness of a melioidosis vaccine