Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
Trials
Published in: Trials 1/2009

Open Access 01-12-2009 | Review

Making trials matter: pragmatic and explanatory trials and the problem of applicability

Authors: Shaun Treweek, Merrick Zwarenstein

Published in: Trials | Issue 1/2009

Login to get access

Abstract

Randomised controlled trials are the best research design for decisions about the effect of different interventions but randomisation does not, of itself, promote the applicability of a trial's results to situations other than the precise one in which the trial was done. While methodologists and trialists have rightly paid great attention to internal validity, much less has been given to applicability.
This narrative review is aimed at those planning to conduct trials, and those aiming to use the information in them. It is intended to help the former group make their trials more widely useful and to help the latter group make more informed decisions about the wider use of existing trials. We review the differences between the design of most randomised trials (which have an explanatory attitude) and the design of trials more able to inform decision making (which have a pragmatic attitude) and discuss approaches used to assert applicability of trial results.
If we want evidence from trials to be used in clinical practice and policy, trialists should make every effort to make their trial widely applicable, which means that more trials should be pragmatic in attitude.
Appendix
Available only for authorised users
Literature
1.
2.
Guyatt GH, Haynes RB, Jaeschke RZ, Cook DJ, Green L, Naylor CD, Wilson MC, Richardson WS: Users' Guides to the Medical Literature: XXV. Evidence-based medicine: principles for applying the Users' Guides to patient care. Evidence-Based Medicine Working Group. JAMA. 2000, 284: 1290-1296. 10.1001/jama.284.10.1290.CrossRefPubMed
3.
Cochrane AL: Effectiveness and efficiency. Random reflections on health services. 1972, London, UK: Nuffield Provincial Hospitals Trust
4.
Schwartz D, Lellouch J: Explanatory and pragmatic attitudes in therapeutical trials. J Chronic Dis. 1967, 20: 637-648. 10.1016/0021-9681(67)90041-0.CrossRefPubMed
5.
Coca SG, Krumholz HM, Garg AX, Parikh CR: Underrepresentation of renal disease in randomized controlled trials of cardiovascular disease. JAMA. 2006, 296: 1377-1384. 10.1001/jama.296.11.1377.CrossRefPubMed
6.
Lee PY, Alexander KP, Hammill BG, Pasquali SK, Peterson ED: Representation of elderly persons and women in published randomized trials of acute coronary syndromes. JAMA. 2001, 286: 708-713. 10.1001/jama.286.6.708.CrossRefPubMed
7.
Glasgow R, Davidson K, Dobkin P, Ockene J, Spring B: Practical behavioural trials to advance evidence-based behavioural medicine. Ann Behav Med. 2006, 31: 5-13. 10.1207/s15324796abm3101_3.CrossRefPubMed
8.
Rothwell PM: External validity of randomised controlled trials: "To whom do the results of this trial apply?". Lancet. 2005, 365: 82-93. 10.1016/S0140-6736(04)17670-8.CrossRefPubMed
9.
Bosch J, Yusuf S, Pogue J, Sleight P, Lonn E, Rangoonwala B, Davies R, Ostergren J, Probstfield J, HOPE Investigators: Heart outcomes prevention evaluation. Use of ramipril in preventing stroke: double blind randomised trial. BMJ. 2002, 324: 699-10.1136/bmj.324.7339.699.CrossRefPubMedPubMedCentral
10.
Rothwell PM: Factors that can affect the external validity of randomised controlled trials. PLoS Clinical Trials. 2006, 1: e9-10.1371/journal.pctr.0010009.CrossRefPubMedPubMedCentral
11.
Hotopf M, Churchill R, Glyn L: Pragmatic randomised controlled trials in psychiatry. Br J Psychiatry. 1999, 175: 217-223. 10.1192/bjp.175.3.217.CrossRefPubMed
12.
13.
Lavis JN, Posada FB, Haines A, Osei E: Use of research to inform public policymaking. Lancet. 2004, 364: 1615-1621. 10.1016/S0140-6736(04)17317-0.CrossRefPubMed
14.
Tunis SR, Stryer DB, Clancy CM: Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA. 2003, 290: 1624-1632. 10.1001/jama.290.12.1624.CrossRefPubMed
15.
Godwin M, Ruhland L, Casson I, MacDonald S, Delva D, Birtwhistle R, Lam M, Seguin R: Pragmatic controlled clinical trials in primary care: the struggle between external and internal validity. BMC Med Res Methodol. 2003, 3: 28-10.1186/1471-2288-3-28.CrossRefPubMedPubMedCentral
16.
Zwarenstein M, Treweek S: What kind of randomized trials do we need?. J Clin Epi. 2009, 62 (2): 461-463. 10.1016/j.jclinepi.2009.01.011.CrossRef
17.
18.
Zwarenstein M, Treweek S: What kind of randomized trials do patients and clinicians need?. Ann Intern Med. 2009, 150: JC5-2, JC5-3.
19.
Vallvé C: A critical review of the pragmatic clinical trial [in Spanish]. Med Clin (Barc). 2003, 27: 384-388. 10.1157/13052554.CrossRef
20.
21.
Bombardier C, Laine L, Reicin A, Shapiro D, Burgos-Vargas R, Davis B, Day R, Ferraz MB, Hawkey CJ, Hochberg MC, Kvien TK, Schnitzer TJ, VIGOR Study Group: Comparison of upper gastrointestinal toxicity of rofecoxib and naproxen in patients with rheumatoid arthritis. N Engl J Med. 2000, 343: 1520-1528. 10.1056/NEJM200011233432103.CrossRefPubMed
22.
Bresalier RS, Sandler RS, Quan H, Bolognese JA, Oxenius B, Horgan K, Lines C, Riddell R, Morton D, Lanas A, Konstam MA, Baron JA, Adenomatous Polyp Prevention on Vioxx (APPROVe) Trial Investigators: Cardiovascular events associated with rofecoxib in a colorectal adenoma chemoprevention trial. N Engl J Med. 2005, 352: 1092-1102. 10.1056/NEJMoa050493.CrossRefPubMed
23.
The National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group: Tissue plasminogen activator for acute ischemic stroke. New Engl J Med. 1995, 333: 1581-1587. 10.1056/NEJM199512143332401.CrossRef
24.
Hoffman JR: Tissue plasminogen activator for acute ischemic stroke: is the CAEP Position Statement too negative?. CJEM. 2001, 3 (3): 183-185.PubMed
25.
Sandercock P, Lindley R, Wardlaw J, Dennis M, Lewis S, Venables G, Kobayashi A, Czlonkowska A, Berge E, Slot KB, Murray V, Peeters A, Hankey G, Matz K, Brainin M, Ricci S, Celani MG, Righetti E, Cantisani T, Gubitz G, Phillips S, Arauz A, Prasad K, Correia M, Lyrer P, the IST-3 collaborative group: The third international stroke trial (IST-3) of thrombolysis for acute ischaemic stroke. Trials. 2008, 9: 37-CrossRefPubMedPubMedCentral
26.
Travers J, Marsh S, Williams M, Weatherall M, Caldwell B, Shirtcliffe P, Aldington S, Beasley R: External validity of randomised controlled trials in asthma: to whom do the results of the trials apply?. Thorax. 2007, 62: 219-223. 10.1136/thx.2006.066837.CrossRefPubMed
27.
Travers J, Marsh S, Caldwell B, Williams M, Aldington S, Weatherall M, Shirtcliffe P, Beasley R: External validity of randomized controlled trials in COPD. Respir Med. 2007, 101: 1313-1320. 10.1016/j.rmed.2006.10.011.CrossRefPubMed
28.
Sackett DL: Explanatory vs. management trials. Clinical Epidemiology: How to Do Clinical Practice Research. Edited by: Haynes RB, Sackett DL, Guyatt GH, Tugwell P. 2005, Philadelphia, PA: Lippincott, Williams and Wilkins
29.
Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, Tunis S, Bergel E, Harvey I, Magid DJ, Chalkidou K: A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. CMAJ. 2009, 180: E47-57.CrossRefPubMedPubMedCentral
30.
Gartlehner G, Hansen RA, Nissman D, Lohr KN, Carey TS: A simple and valid tool distinguished efficacy from effectiveness studies. J Clin Epidemiol. 2006, 59: 1040-1048. 10.1016/j.jclinepi.2006.01.011.CrossRefPubMed
31.
Alford L: On differences between explanatory and pragmatic clinical trials. NZ J Physiother. 2007, 35: 12-16.
32.
Hotopf M: The pragmatic randomised controlled trial. Advan Psychiatr Treat. 2002, 8 (5): 326-333. 10.1192/apt.8.5.326.CrossRef
33.
Cuthbertson BH, Rattray J, Johnston M, Wildsmith A, Wilson E, Hernendez R, Ramsey C, Hull AM, Norrie J, Campbell M: A pragmatic randomised, controlled trial of intensive care follow up programmes in improving longer-term outcomes from critical illness. The PRACTICAL study (study protocol). BMC Health Serv Res. 2007, 7: 116-10.1186/1472-6963-7-116.CrossRefPubMedPubMedCentral
34.
Hutchings J, Bywater T, Daley D, Gardner F, Whitaker C, Jones K, Eames C, Edwards RT: Parenting intervention in Sure Start services for children at risk of developing conduct disorder: pragmatic randomised controlled trial. BMJ. 2007, 334: 678-10.1136/bmj.39126.620799.55.CrossRefPubMedPubMedCentral
35.
Mutrie N, Campbell AM, Whyte F, McConnachie A, Emslie C, Lee L, Kearney N, Walker A, Ritchie D: Benefits of supervised group exercise programme for women being treated for early stage breast cancer: pragmatic randomised controlled trial. BMJ. 2007, 334: 517-10.1136/bmj.39094.648553.AE.CrossRefPubMedPubMedCentral
36.
Hay EM, Paterson SM, Lewis M, Hosie G, Croft P: Pragmatic randomised controlled trial of local corticosteroid injection and naproxen for treatment of lateral epicondylitis of elbow in primary care. BMJ. 1999, 319: 964-968.CrossRefPubMedPubMedCentral
37.
Steinsbekk A, Fonnebo V, Lewith G, Bentzen N: Homeopathic care for the prevention of upper respiratory tract infections in children: a pragmatic, randomised, controlled trial comparing individualised homeopathic care and waiting-list controls. Complement Ther Med. 2005, 13: 231-238. 10.1016/j.ctim.2005.06.007.CrossRefPubMed
38.
Zwarenstein M, Schoeman JH, Vundule C, Lombard CJ, Tatley M: Randomised controlled trial of self-supervised and directly observed treatment of tuberculosis. Lancet. 1998, 352: 1340-1343. 10.1016/S0140-6736(98)04022-7.CrossRefPubMed
39.
North American Symptomatic Carotid Endarterectomy Trial Collaborators: Beneficial effect of carotid endarterectomy in symptomatic patients with high-grade carotid stenosis. N Engl J Med. 1991, 325: 445-453.CrossRef
40.
Karanicolas PJ, Montori VM, Devereaux PJ, Schünermann H, Guyatt GH: A new "mechanistic-practical" framework for designing and interpreting randomized trials. J Clin Epidemiol. 2009, 62: 479-484. 10.1016/j.jclinepi.2008.02.009.CrossRefPubMed
41.
Oxman AD, Lombard C, Treweek S, Gagnier JJ, Maclure M, Zwarenstein M: Why we will remain pragmatists: four problems with the impractical mechanistic framework and a better solution. J Clin Epidemiol. 2009, 62: 485-488. 10.1016/j.jclinepi.2008.08.015.CrossRefPubMed
42.
Karanicolas PJ, Montori VM, Devereaux PJ, Schünermann H, Guyatt GH: The practicalists' response. J Clin Epidemiol. 2009, 62: 489-494. 10.1016/j.jclinepi.2008.08.013.CrossRefPubMed
43.
Oxman AD, Lombard C, Treweek S, Gagnier JJ, Maclure M, Zwarenstein M: A pragmatic resolution. J Clin Epidemiol. 2009, 62: 495-498. 10.1016/j.jclinepi.2008.08.014.CrossRefPubMed
44.
Riper H, Kramer J, Smit F, Conijn B, Schippers G, Cuijpers P: Web-based self-help for problem drinkers: a pragmatic randomized trial. Addiction. 2007, 103 (2): 218-227. 10.1111/j.1360-0443.2007.02063.x.CrossRef
45.
46.
47.
Lalonde L, Martineau J, Blais N, Montigny M, Ginsberg J, Fournier M, Berbiche D, Vanier MC, Blais L, Perreault S, Rodrigues I: Is long-term pharmacist-managed anticoagulation service efficient? A pragmatic randomized controlled trial. Am Heart J. 2008, 156: 148-154. 10.1016/j.ahj.2008.02.009.CrossRefPubMed
48.
Zwarenstein M, Treweek S, Gagnier J, Altman DG, Maclure M, Tunis S, Haynes B, Oxman AD, Moher D: Improving the reporting of pragmatic trials: an extension of the CONSORT Statement. BMJ. 2008, 337: a2390-10.1136/bmj.a2390.CrossRefPubMedPubMedCentral
49.
Glasziou P, Meats E, Heneghan C, Shepperd S: What is missing from descriptions of treatment in trials and reviews?. BMJ. 2008, 336: 1472-1474. 10.1136/bmj.39590.732037.47.CrossRefPubMedPubMedCentral
50.
51.
Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P: Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration. Ann Intern Med. 2008, 148: W60-66.PubMed
52.
Lindsay B: Randomized controlled trials of socially complex nursing interventions: creating bias and unreliability?. J Adv Nurs. 2004, 45: 84-94. 10.1046/j.1365-2648.2003.02864.x.CrossRefPubMed
53.
54.
Ogrinc G, Mooney SE, Estrada C, Foster T, Goldmann D, Hall LW, Huizinga MM, Liu SK, Mills P, Neily J, Nelson W, Pronovost PJ, Provost L, Rubenstein LV, Speroff T, Splaine T, Thomson R, Tomolo AM, Watts B: The SQUIRE (Standards for QUality Improvement Reporting Excellence) guidelines for quality improvement reporting: explanation and elaboration. Qual Saf Health Care. 2008, 17: i13-32. 10.1136/qshc.2008.029058.CrossRefPubMedPubMedCentral
55.
May C: A rational model for assessing and evaluating complex interventions in health care. BMC Health Serv Res. 2006, 6: 1-11. 10.1186/1472-6963-6-86.CrossRef
56.
May C, Finch T, Mair F, Ballini L, Dowrick C, Eccles M, Gask L, MacFarlane A, Murray E, Rapley T, Rogers A, Treweek S, Wallace P, Anderson G, Burns J, Heaven B: Understanding the implementation of complex interventions in health care: the normalisation process model. BMC Health Serv Res. 2007, 7: 148-10.1186/1472-6963-7-148.CrossRefPubMedPubMedCentral
57.
May C, Mair FS, Dowrick CF, Finch TL: Process evaluation of complex interventions in primary care: understanding trials using the normalization process model. BMC Fam Pract. 2007, 8: 42-10.1186/1471-2296-8-42.CrossRefPubMedPubMedCentral
58.
May C, Finch T: Implementation, embedding, and integration: an outline of normalization process theory. Sociology. 2009,
59.
Greenhouse JB, Kaizar EE, Kelleher K, Seltman H, Gardner W: Generalizing from clinical trial data: a case study. The risk of suicidality among pediatric antidepressant users. Stat Med. 2008, 27: 1801-1813. 10.1002/sim.3218.CrossRefPubMedPubMedCentral
60.
Yamaguchi T, Ohashi Y: Investigating centre effects in a multi-centre clinical trial of superficial bladder cancer. Stat Med. 1999, 18: 1961-1971. 10.1002/(SICI)1097-0258(19990815)18:15<1961::AID-SIM170>3.0.CO;2-3.CrossRefPubMed
61.
Wells EM: Behind the scenes of randomised trials of complex interventions – insiders reveal the importance of context. PhD Thesis. 2007, University of Dundee; School of Nursing and Midwifery
Metadata
Title
Making trials matter: pragmatic and explanatory trials and the problem of applicability
Authors
Shaun Treweek
Merrick Zwarenstein
Publication date
01-12-2009
Publisher
BioMed Central
Published in
Trials / Issue 1/2009
Electronic ISSN: 1745-6215
DOI
https://doi.org/10.1186/1745-6215-10-37

Other articles of this Issue 1/2009

Trials 1/2009 Go to the issue

Study protocol

Comparison of embedded and added motor imagery training in patients after stroke: study protocol of a randomised controlled pilot trial using a mixed methods approach

Study protocol

Clinical trial to assess the effect of physical exercise on endothelial function and insulin resistance in pregnant women

Research

N-acetylcysteine does not prevent contrast-induced nephropathy after cardiac catheterization in patients with diabetes mellitus and chronic kidney disease: a randomized clinical trial

Study protocol

REFINE (Reducing Falls in In-patient Elderly) - a randomised controlled trial

Study protocol

Periodontal treatment to improve glycaemic control in diabetic patients: study protocol of the randomized, controlled DIAPERIO trial

Research

Over 50 years of trial in Acta Psychiatrica Scandinavica: a survey