Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
Trials
Published in: Trials 1/2009

Open Access 01-12-2009 | Research

Dealing with heterogeneity of treatment effects: is the literature up to the challenge?

Authors: Nicole B Gabler, Naihua Duan, Diana Liao, Joann G Elmore, Theodore G Ganiats, Richard L Kravitz

Published in: Trials | Issue 1/2009

Login to get access

Abstract

Background

Some patients will experience more or less benefit from treatment than the averages reported from clinical trials; such variation in therapeutic outcome is termed heterogeneity of treatment effects (HTE). Identifying HTE is necessary to individualize treatment. The degree to which heterogeneity is sought and analyzed correctly in the general medical literature is unknown. We undertook this literature sample to track the use of HTE analyses over time, examine the appropriateness of the statistical methods used, and explore the predictors of such analyses.

Methods

Articles were selected through a probability sample of randomized controlled trials (RCTs) published in Annals of Internal Medicine, BMJ, JAMA, The Lancet, and NEJM during odd numbered months of 1994, 1999, and 2004. RCTs were independently reviewed and coded by two abstractors, with adjudication by a third. Studies were classified as reporting: (1) HTE analysis, utilizing a formal test for heterogeneity or treatment-by-covariate interaction, (2) subgroup analysis only, involving no formal test for heterogeneity or interaction; or (3) neither. Chi-square tests and multiple logistic regression were used to identify variables associated with HTE reporting.

Results

319 studies were included. Ninety-two (29%) reported HTE analysis; another 88 (28%) reported subgroup analysis only, without examining HTE formally. Major covariates examined included individual risk factors associated with prognosis, responsiveness to treatment, or vulnerability to adverse effects of treatment (56%); gender (30%); age (29%); study site or center (29%); and race/ethnicity (7%). Journal of publication and sample size were significant independent predictors of HTE analysis (p < 0.05 and p < 0.001, respectively).

Conclusion

HTE is frequently ignored or incorrectly analyzed. An iterative process of exploratory analysis followed by confirmatory HTE analysis will generate the data needed to facilitate an individualized approach to evidence-based medicine.
Appendix
Available only for authorised users
Literature
1.
Longford NT: Selection bias and treatment heterogeneity in clinical trials. Stat Med. 1999, 18: 1467-1474. 10.1002/(SICI)1097-0258(19990630)18:12<1467::AID-SIM149>3.0.CO;2-H.CrossRefPubMed
2.
Kravitz RL, Duan N, Braslow J: Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages. Milbank Q. 2004, 82: 661-687. 10.1111/j.0887-378X.2004.00327.x.CrossRefPubMedPubMedCentral
3.
Greenfield S, Kravitz R, Duan N, Kaplan SH: Heterogeneity of treatment effects: implications for guidelines, payment, and quality assessment. Am J Med. 2007, 120: S3-9. 10.1016/j.amjmed.2007.02.002.CrossRefPubMed
4.
Spear BB, Heath-Chiozzi M, Huff J: Clinical application of pharmacogenetics. Trends Mol Med. 2001, 7: 201-204. 10.1016/S1471-4914(01)01986-4.CrossRefPubMed
5.
Connor S: Glaxo chief: Our drugs do not work on most patients. The Independent. 2003
6.
Kent DM, Ruthazer R, Selker HP: Are some patients likely to benefit from recombinant tissue-type plasminogen activator for acute ischemic stroke even beyond 3 hours from symptom onset?. Stroke. 2003, 34: 464-467. 10.1161/01.STR.0000051506.43212.8B.CrossRefPubMed
7.
Rothwell PM: Treating individuals 2. Subgroup analysis in randomised controlled trials: importance, indications, and interpretation. Lancet. 2005, 365: 176-186. 10.1016/S0140-6736(05)17709-5.CrossRefPubMed
8.
Rothwell PM, Mehta Z, Howard SC, Gutnikov SA, Warlow CP: Treating individuals 3: from subgroups to individuals: general principles and the example of carotid endarterectomy. Lancet. 2005, 365: 256-265.CrossRefPubMed
9.
Kent DM, Hayward RA: Limitations of applying summary results of clinical trials to individual patients: the need for risk stratification. JAMA. 2007, 298: 1209-1212. 10.1001/jama.298.10.1209.CrossRefPubMed
10.
Guyatt G, Sackett D, Adachi J, Roberts R, Chong J, Rosenbloom D, Keller J: A clinician's guide for conducting randomized trials in individual patients. Cmaj. 1988, 139: 497-503.PubMedPubMedCentral
11.
Guyatt G, Sackett D, Taylor DW, Chong J, Roberts R, Pugsley S: Determining optimal therapy – randomized trials in individual patients. N Engl J Med. 1986, 314: 889-892.CrossRefPubMed
12.
13.
Senn S: Controversies concerning randomization and additivity in clinical trials. Stat Med. 2004, 23: 3729-3753. 10.1002/sim.2074.CrossRefPubMed
14.
Hayward RA, Kent DM, Vijan S, Hofer TP: Reporting clinical trial results to inform providers, payers, and consumers. Health Aff (Millwood). 2005, 24: 1571-1581. 10.1377/hlthaff.24.6.1571.CrossRef
15.
Kent DM, Hayward RA, Griffith JL, Vijan S, Beshansky JR, Califf RM, Selker HP: An independently derived and validated predictive model for selecting patients with myocardial infarction who are likely to benefit from tissue plasminogen activator compared with streptokinase. Am J Med. 2002, 113: 104-111. 10.1016/S0002-9343(02)01160-9.CrossRefPubMed
16.
Cook DI, Gebski VJ, Keech AC: Subgroup analysis in clinical trials. Med J Aust. 2004, 180: 289-291.PubMed
17.
Pocock SJ, Assmann SE, Enos LE, Kasten LE: Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practice and problems. Stat Med. 2002, 21: 2917-2930. 10.1002/sim.1296.CrossRefPubMed
18.
Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, Gotzsche PC, Lang T: The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med. 2001, 134: 663-694.CrossRefPubMed
19.
Stallones RA: The use and abuse of subgroup analysis in epidemiological research. Prev Med. 1987, 16: 183-194. 10.1016/0091-7435(87)90082-X.CrossRefPubMed
20.
Nissen SE, Tuzcu EM, Schoenhagen P, Brown BG, Ganz P, Vogel RA, Crowe T, Howard G, Cooper CJ, Brodie B: Effect of intensive compared with moderate lipid-lowering therapy on progression of coronary atherosclerosis: a randomized controlled trial. JAMA. 2004, 291: 1071-1080. 10.1001/jama.291.9.1071.CrossRefPubMed
21.
Norman PE, Jamrozik K, Lawrence-Brown MM, Le MT, Spencer CA, Tuohy RJ, Parsons RW, Dickinson JA: Population based randomised controlled trial on impact of screening on mortality from abdominal aortic aneurysm. BMJ. 2004, 329: 1259-10.1136/bmj.38272.478438.55.CrossRefPubMedPubMedCentral
22.
Ruggenenti P, Perna A, Gherardi G, Garini G, Zoccali C, Salvadori M, Scolari F, Schena FP, Remuzzi G: Renoprotective properties of ACE-inhibition in non-diabetic nephropathies with non-nephrotic proteinuria. Lancet. 1999, 354: 359-364. 10.1016/S0140-6736(98)10363-X.CrossRefPubMed
23.
Torriani FJ, Rodriguez-Torres M, Rockstroh JK, Lissen E, Gonzalez-Garcia J, Lazzarin A, Carosi G, Sasadeusz J, Katlama C, Montaner J: Peginterferon Alfa-2a plus ribavirin for chronic hepatitis C virus infection in HIV-infected patients. N Engl J Med. 2004, 351: 438-450. 10.1056/NEJMoa040842.CrossRefPubMed
24.
25.
Kraemer HC, Frank E, Kupfer DJ: Moderators of treatment outcomes: clinical, research, and policy importance. JAMA. 2006, 296: 1286-1289. 10.1001/jama.296.10.1286.CrossRefPubMed
26.
Lu M, Lyden PD, Brott TG, Hamilton S, Broderick JP, Grotta JC: Beyond subgroup analysis: improving the clinical interpretation of treatment effects in stroke research. J Neurosci Methods. 2005, 143: 209-216. 10.1016/j.jneumeth.2004.10.002.CrossRefPubMed
27.
Parker AB, Naylor CD: Subgroups, treatment effects, and baseline risks: some lessons from major cardiovascular trials. Am Heart J. 2000, 139: 952-961. 10.1067/mhj.2000.106610.CrossRefPubMed
28.
Horwitz RI, Singer BH, Makuch RW, Viscoli CM: Clinical versus statistical considerations in the design and analysis of clinical research. J Clin Epidemiol. 1998, 51: 305-307. 10.1016/S0895-4356(98)00006-7.CrossRefPubMed
29.
Feinstein AR: The problem of cogent subgroups: a clinicostatistical tragedy. J Clin Epidemiol. 1998, 51: 297-299. 10.1016/S0895-4356(98)00004-3.CrossRefPubMed
30.
Kraemer HC, Stice E, Kazdin A, Offord D, Kupfer D: How do risk factors work together? Mediators, moderators, and independent, overlapping, and proxy risk factors. Am J Psychiatry. 2001, 158: 848-856.CrossRefPubMed
31.
Kraemer HC, Wilson GT, Fairburn CG, Agras WS: Mediators and moderators of treatment effects in randomized clinical trials. Arch Gen Psychiatry. 2002, 59: 877-883. 10.1001/archpsyc.59.10.877.CrossRefPubMed
32.
Hernandez AV, Boersma E, Murray GD, Habbema JD, Steyerberg EW: Subgroup analyses in therapeutic cardiovascular clinical trials: are most of them misleading?. Am Heart J. 2006, 151: 257-264. 10.1016/j.ahj.2005.04.020.CrossRefPubMed
33.
Assmann SF, Pocock SJ, Enos LE, Kasten LE: Subgroup analysis and other (mis)uses of baseline data in clinical trials. Lancet. 2000, 355: 1064-1069. 10.1016/S0140-6736(00)02039-0.CrossRefPubMed
34.
Moreira ED, Stein Z, Susser E: Reporting on methods of subgroup analysis in clinical trials: a survey of four scientific journals. Braz J Med Biol Res. 2001, 34: 1441-1446.PubMed
35.
Wang R, Lagakos SW, Ware JH, Hunter DJ, Drazen JM: Statistics in medicine – reporting of subgroup analyses in clinical trials. N Engl J Med. 2007, 357: 2189-2194. 10.1056/NEJMsr077003.CrossRefPubMed
36.
Chew M, Villanueva EV, Weyden Van Der MB: Life and times of the impact factor: retrospective analysis of trends for seven medical journals (1994–2005) and their Editors' views. J R Soc Med. 2007, 100: 142-150. 10.1258/jrsm.100.3.142.CrossRefPubMedPubMedCentral
37.
38.
39.
SAS: Cary, North Carolina, USA: SAS Institute, 9
40.
Antman EM, Cohen M, Bernink PJ, McCabe CH, Horacek T, Papuchis G, Mautner B, Corbalan R, Radley D, Braunwald E: The TIMI risk score for unstable angina/non-ST elevation MI: A method for prognostication and therapeutic decision making. Jama. 2000, 284: 835-842. 10.1001/jama.284.7.835.CrossRefPubMed
41.
Ioannidis JP, Lau J: Heterogeneity of the baseline risk within patient populations of clinical trials: a proposed evaluation algorithm. Am J Epidemiol. 1998, 148: 1117-1126.CrossRefPubMed
42.
Morrow DA, Antman EM, Snapinn SM, McCabe CH, Theroux P, Braunwald E: An integrated clinical approach to predicting the benefit of tirofiban in non-ST elevation acute coronary syndromes. Application of the TIMI Risk Score for UA/NSTEMI in PRISM-PLUS. Eur Heart J. 2002, 23: 223-229. 10.1053/euhj.2001.2738.CrossRefPubMed
43.
Rothwell PM, Warlow CP: Prediction of benefit from carotid endarterectomy in individual patients: a risk-modelling study. European Carotid Surgery Trialists' Collaborative Group. Lancet. 1999, 353: 2105-2110. 10.1016/S0140-6736(98)11415-0.CrossRefPubMed
44.
The ESPRIM trial: short-term treatment of acute myocardial infarction with molsidomine. European Study of Prevention of Infarct with Molsidomine (ESPRIM) Group. Lancet. 1994, 344: 91-97.
45.
Bjornson CL, Klassen TP, Williamson J, Brant R, Mitton C, Plint A, Bulloch B, Evered L, Johnson DW: A randomized trial of a single dose of oral dexamethasone for mild croup. N Engl J Med. 2004, 351: 1306-1313. 10.1056/NEJMoa033534.CrossRefPubMed
46.
Milpied N, Deconinck E, Gaillard F, Delwail V, Foussard C, Berthou C, Gressin R, Lucas V, Colombat P, Harousseau JL: Initial treatment of aggressive lymphoma with high-dose chemotherapy and autologous stem-cell support. N Engl J Med. 2004, 350: 1287-1295. 10.1056/NEJMoa031770.CrossRefPubMed
47.
Cui L, Hung HM, Wang SJ, Tsong Y: Issues related to subgroup analysis in clinical trials. J Biopharm Stat. 2002, 12: 347-358. 10.1081/BIP-120014565.CrossRefPubMed
48.
Hayward RA, Kent DM, Vijan S, Hofer TP: Multivariable risk prediction can greatly enhance the statistical power of clinical trial subgroup analysis. BMC Med Res Methodol. 2006, 6: 18-10.1186/1471-2288-6-18.CrossRefPubMedPubMedCentral
49.
Haiman CA, Stram DO, Wilkens LR, Pike MC, Kolonel LN, Henderson BE, Le Marchand L: Ethnic and racial differences in the smoking-related risk of lung cancer. N Engl J Med. 2006, 354: 333-342. 10.1056/NEJMoa033250.CrossRefPubMed
50.
Conjeevaram HS, Fried MW, Jeffers LJ, Terrault NA, Wiley-Lucas TE, Afdhal N, Brown RS, Belle SH, Hoofnagle JH, Kleiner DE, Howell CD: Peginterferon and ribavirin treatment in African American and Caucasian American patients with hepatitis C genotype 1. Gastroenterology. 2006, 131: 470-477. 10.1053/j.gastro.2006.06.008.CrossRefPubMed
51.
McDowell SE, Coleman JJ, Ferner RE: Systematic review and meta-analysis of ethnic differences in risks of adverse reactions to drugs used in cardiovascular medicine. BMJ. 2006, 332: 1177-1181. 10.1136/bmj.38803.528113.55.CrossRefPubMedPubMedCentral
52.
Department of Health: A first class service: quality in the new NHS. 1998, London: Department of Health
53.
Department of Health: "Faster Access to Modern Treatment": how NICE appraisal will work. 1999, Leeds: NHS Executive
54.
Rawlins M: In pursuit of quality: the National Institute for Clinical Excellence. Lancet. 1999, 353: 1079-1082. 10.1016/S0140-6736(99)02381-8.CrossRefPubMed
55.
56.
Trusheim MR, Berndt ER, Douglas FL: Stratified medicine: strategic and economic implications of combining drugs and clinical biomarkers. Nat Rev Drug Discov. 2007, 6: 287-293. 10.1038/nrd2251.CrossRefPubMed
Metadata
Title
Dealing with heterogeneity of treatment effects: is the literature up to the challenge?
Authors
Nicole B Gabler
Naihua Duan
Diana Liao
Joann G Elmore
Theodore G Ganiats
Richard L Kravitz
Publication date
01-12-2009
Publisher
BioMed Central
Published in
Trials / Issue 1/2009
Electronic ISSN: 1745-6215
DOI
https://doi.org/10.1186/1745-6215-10-43

Other articles of this Issue 1/2009

Trials 1/2009 Go to the issue

Research

Over 50 years of trial in Acta Psychiatrica Scandinavica: a survey

Study protocol

The Toronto prehospital hypertonic resuscitation-head injury and multi organ dysfunction trial (TOPHR HIT) - Methods and data collection tools

Study protocol

DOMINO-AD protocol: donepezil and memantine in moderate to severe Alzheimer's disease – a multicentre RCT

Study protocol

The Tilburg double blind randomised controlled trial comparing inguinal hernia repair according to Lichtenstein and the transinguinal preperitoneal technique

Study protocol

Clinical e valuation of magnetic resonance imaging in coronary heart disease: The CE-MARC study

Methodology

Improved quality monitoring of multi-center acupuncture clinical trials in China