Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
Trials
Published in: Trials 1/2015

Open Access 01-12-2015 | Research

The natural history of conducting and reporting clinical trials: interviews with trialists

Authors: Rebecca MD Smyth, Ann Jacoby, Douglas G Altman, Carrol Gamble, Paula R Williamson

Published in: Trials | Issue 1/2015

Login to get access

Abstract

Background

To investigate the nature of the research process as a whole, factors that might influence the way in which research is carried out, and how researchers ultimately report their findings.

Methods

Semi-structured qualitative telephone interviews with authors of trials, identified from two sources: trials published since 2002 included in Cochrane systematic reviews selected for the ORBIT project; and trial reports randomly sampled from 14,758 indexed on PubMed over the 12-month period from August 2007 to July 2008.

Results

A total of 268 trials were identified for inclusion, 183 published since 2002 and included in the Cochrane systematic reviews selected for the ORBIT project and 85 randomly selected published trials indexed on PubMed. The response rate from researchers in the former group was 21% (38/183) and in the latter group was 25% (21/85). Overall, 59 trialists were interviewed from the two different sources. A number of major but related themes emerged regarding the conduct and reporting of trials: establishment of the research question; identification of outcome variables; use of and adherence to the study protocol; conduct of the research; reporting and publishing of findings. Our results reveal that, although a substantial proportion of trialists identify outcome variables based on their clinical experience and knowing experts in the field, there can be insufficient reference to previous research in the planning of a new trial. We have revealed problems with trial recruitment: not reaching the target sample size, over-estimation of recruitment potential and recruiting clinicians not being in equipoise. We found a wide variation in the completeness of protocols, in terms of detailing study rationale, outlining the proposed methods, trial organisation and ethical considerations.

Conclusion

Our results confirm that the conduct and reporting of some trials can be inadequate. Interviews with researchers identified aspects of clinical research that can be especially challenging: establishing appropriate and relevant outcome variables to measure, use of and adherence to the study protocol, recruiting of study participants and reporting and publishing the study findings. Our trialists considered the prestige and impact factors of academic journals to be the most important criteria for selecting those to which they would submit manuscripts.
Literature
1.
2.
Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273:408–12.CrossRefPubMed
3.
Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA. 2010;303:2058–64.CrossRefPubMed
4.
Chan AW, Altman DG. Epidemiology and reporting of randomised trials published in PubMed journals. Lancet. 2005;365:1159–62.CrossRefPubMed
5.
Hopewell S, Dutton S, Yu LM, Chan AW, Altman DG. The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed. BMJ. 2010;340:c723.CrossRefPubMedPubMedCentral
6.
Peron J, Pond GR, Gan HK, Chen EX, Almufti R, Maillet D, et al. Quality of reporting of modern randomized controlled trials in medical oncology: a systematic review. J Natl Cancer Inst. 2012;104:982–9.CrossRefPubMed
7.
8.
Campbell MK, Snowdon C, Francis D, Elbourne D, McDonald AM, Knight R, et al. Recruitment to randomised trials: strategies for trial enrollment and participation study. The STEPS study. Health Technol Assess. 2007;11:iii, ix-105.CrossRefPubMed
9.
Duley L, Antman K, Arena J, Avezum A, Blumenthal M, Bosch J, et al. Specific barriers to the conduct of randomized trials. Clin Trials. 2008;5:40–8.CrossRefPubMed
10.
Smith A, Palmer S, Johnson DW, Navaneethan S, Valentini M, Strippoli GF. How to conduct a randomized trial. Nephrology (Carlton). 2010;15:740–6.CrossRef
11.
12.
Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K. Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev. 2009;1:MR000006.
13.
Dickersin K, Chan S, Chalmers TC, Sacks HS, Smith Jr H. Publication bias and clinical trials. Control Clin Trials. 1987;8:343–53.CrossRefPubMed
14.
Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan AW, Cronin E, et al. Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One. 2008;3:e308.CrossRef
15.
Altman DG, Furberg CD, Grimshaw JM, Rothwell PM. Lead editorial: trials - using the opportunities of electronic publishing to improve the reporting of randomised trials. Trials. 2006;7:6.CrossRefPubMedPubMedCentral
16.
Kirkham JJ, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, et al. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ. 2010;340:c365.CrossRefPubMed
17.
Smyth RM, Kirkham JJ, Jacoby A, Altman DG, Gamble C, Williamson PR. Frequency and reasons for outcome reporting bias in clinical trials: interviews with trialists. BMJ. 2011;342:c7153.CrossRefPubMedPubMedCentral
18.
Richie J, Spencer L. Qualitative data analysis for applied policy research. In: Bryman A, Burgess RG, editors. Analysis qualitative data. London: Routledge; 1994. p. 173–94.
19.
Jones AP, Conroy E, Williamson PR, Clarke M, Gamble C. The use of systematic reviews in the planning, design and conduct of randomised trials: a retrospective cohort of NIHR HTA funded trials. BMC Med Res Methodol. 2013;13:50.CrossRefPubMedPubMedCentral
20.
Clark M. Standardising outcomes for clinical trials and systematic reviews. Trials. 2007;8:39.CrossRef
21.
Cooney RM, Warren BF, Altman DG, Abreu MT, Travis SP. Outcome measurement in clinical trials for Ulcerative Colitis: towards standardisation. Trials. 2007;8:17.CrossRefPubMedPubMedCentral
22.
Tugwell P, Boers M, Brooks P, Simon L, Strand V, Idzerda L. OMERACT: an international initiative to improve outcome measurement in rheumatology. Trials. 2007;8:38.CrossRefPubMedPubMedCentral
23.
Sinha I, Jones L, Smyth RL, Williamson PR. A systematic review of studies that aim to determine which outcomes to measure in clinical trials in children. PLoS Med. 2008;5:0569–77.CrossRef
24.
Sinha IP, Gallagher R, Williamson PR, Smyth RL. Development of a core outcome set for clinical trials in childhood asthma: a survey of clinicians, parents, and young people. Trials. 2012;13:103.CrossRefPubMedPubMedCentral
25.
Williamson PR, Altman DG, Blazeby JM, Clarke M, Devane D, Gargon E, et al. Developing core outcome sets for clinical trials: issues to consider. Trials. 2012;13:132.CrossRefPubMedPubMedCentral
26.
27.
Caldwell PHY, Hamilton S, Tan A, Craig JC. Strategies for increasing recruitment to randomised controlled trias: systematic review. PLoS Med. 2010;7:e1000368.CrossRefPubMedPubMedCentral
28.
Prescott RJ, Counsel CE, Gillepsie WJ, Grant A, Russell IT, Kiauka S, et al. Factors that limit the quality, number and progress of randomised controlled trials. Health Technol Assess. 1999;3:1–143.PubMed
29.
30.
Raftery J, Kerr C, Hawker S, Powell J. Paying clinicians to join clinical trials: a review of guidelines and interview study with trialists. Trials. 2009;10:15.CrossRefPubMedPubMedCentral
31.
Chan A-W, Tetzlaff JM, Gøtzsche PC, Altman DG, Mann H, Berlin J, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013;346:e7576.
32.
33.
Pildal J, Chan A-W, Hróbjartsson A, Forfang E, Altman DG, Gøtzsche PC. Comparison of descriptions of allocation concealment in trial protocols and the published reports: cohort study. BMJ. 2005;330:1049.CrossRefPubMedPubMedCentral
34.
Hróbjartsson A, Pildal J, Chan AW, Haahr MT, Altman DG, Gøtzsche PC. Reporting on blinding in trial protocols and corresponding publications was often inadequate but rarely contradictory. J Clin Epidemiol. 2009;62:967–73.CrossRefPubMed
35.
Al-Marzouki S, Roberts I, Evans S, Marshall T. Selective reporting in clinical trials: analysis of trial protocols accepted by The Lancet. Lancet. 2008;372:201.CrossRefPubMed
36.
Chan A-W, Tetzlaff JM, Altman DG, Laupacis A, Gøtzsche PC, Krleža-Jerić K, et al. SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med. 2013;158:200–7.CrossRefPubMedPubMedCentral
37.
38.
39.
40.
41.
Editorial. Best tests for candidates. Nature. 2008;451:605–6.
42.
Purcell GP, Donovan SL, Davidoff F. Changes to manuscripts during the editorial process: characterizing the evolution of a clinical paper. JAMA. 1998;280:227–8.CrossRefPubMed
43.
Roberts JC, Fletcher RH, Fletcher SW. Effects of peer review and editing on the readability of articles published in Annals of Internal Medicine. JAMA. 1994;272:119–21.CrossRefPubMed
44.
Dickersin K, Ssemanda E, Mansell C, Rennie D. What do the JAMA editors say when they discuss manuscripts that they are considering for publication? Developing a schema for classifying the content of editorial discussion. BMC Med Res Methodol. 2007;7:44.CrossRefPubMedPubMedCentral
45.
Hamm MP, Scott SD, Klassen TP, Moher D, Hartling L. Do health care institutions value research? A mixed methods study of barriers and facilitators to methodological rigor in pediatric randomized trials. BMC Med Res Methodol. 2012;12:158.CrossRefPubMedPubMedCentral
46.
Ziebland S, Featherstone K, Snowdon C, Barker K, Frost H, Fairbank J. Does it matter if clinicians recruiting for a trial don’t understand what the trial is really about? Qualitative study of surgeons’ experiences of participation in a pragmatic multi-centre RCT. Trials. 2007;8:4.CrossRefPubMedPubMedCentral
47.
Steneck NH, Bulger RE. The history, purpose, and future of instruction in the responsible conduct of research. Acad Med. 2007;82:829–34.CrossRefPubMed
48.
49.
50.
51.
NIHR. A guide to efficient trial management. In: Trials Managers’ Network. The Fourth Edition (2014) of the Guide to Efficient Trial Management. University of Southampton Science Park, Southampton, UK: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House; 2014.
52.
53.
54.
Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869.CrossRefPubMedPubMedCentral
55.
56.
Metadata
Title
The natural history of conducting and reporting clinical trials: interviews with trialists
Authors
Rebecca MD Smyth
Ann Jacoby
Douglas G Altman
Carrol Gamble
Paula R Williamson
Publication date
01-12-2015
Publisher
BioMed Central
Published in
Trials / Issue 1/2015
Electronic ISSN: 1745-6215
DOI
https://doi.org/10.1186/s13063-014-0536-6

Other articles of this Issue 1/2015

Trials 1/2015 Go to the issue

Study protocol

Transcutaneous electrical stimulation at auricular acupoints innervated by auricular branch of vagus nerve pairing tone for tinnitus: study protocol for a randomized controlled clinical trial

Research

Reliability of photographic analysis of wound epithelialization assessed in human skin graft donor sites and epidermolysis bullosa wounds

Study protocol

Mindfulness-based exposure and response prevention for obsessive compulsive disorder: study protocol for a pilot randomised controlled trial

Study protocol

Effects of postural specific sensorimotor training in patients with chronic low back pain: study protocol for randomised controlled trial

Study protocol

The NAtional randomised controlled Trial of Tonsillectomy IN Adults (NATTINA): a clinical and cost-effectiveness study: study protocol for a randomised control trial

Study protocol

DRAPA trial – closed-suction drains versus closed gravity drains in pancreatic surgery: study protocol for a randomized controlled trial