Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2017

Open Access 01-12-2017 | Research article

Research waste in diagnostic trials: a methods review evaluating the reporting of test-treatment interventions

Authors: Lavinia Ferrante di Ruffano, Jacqueline Dinnes, Sian Taylor-Phillips, Clare Davenport, Chris Hyde, Jonathan J. Deeks

Published in: BMC Medical Research Methodology | Issue 1/2017

Login to get access

Abstract

Background

The most rigorous method for evaluating the effectiveness of diagnostic tests is through randomised trials that compare test-treatment interventions: complex interventions comprising episodes of testing, decision-making and treatment. The multi-staged nature of these interventions, combined with the need to relay diagnostic decision-making and treatment planning, has led researchers to hypothesise that test-treatment strategies may be very challenging to document. However, no reviews have yet examined the reporting quality of interventions used in test-treatment RCTs. In this study we evaluate the completeness of intervention descriptions in a systematically identified cohort of test-treatment RCTs.

Methods

We ascertained all test-treatment RCTs published 2004–2007, indexed in CENTRAL. Included trials randomized patients to diagnostic tests and measured patient outcomes after treatment. Two raters examined the completeness of test-treatment intervention descriptions in four components: 1) the test, 2) diagnostic decision-making, 3) management decision-making, 4) treatments.

Results

One hundred and three trials compared 105 control with 119 experimental interventions, most commonly in cardiovascular medicine (35, 34%), obstetrics and gynecology (17%), gastroenterology (14%) or orthopedics (10%). A broad range of tests were evaluated, including imaging (50, 42%), biochemical assays (21%) and clinical assessment (12%). Only five (5%) trials detailed all four components of experimental and control interventions, none of which also provided a complete care pathway diagram. Experimental arms were missing descriptions of tests, diagnostic-decision making, management planning and treatments (36%, 51%, 55% and 79% of trials respectively); control arms were missing the same details in 61%, 66%, 67% and 84% of trials.

Conclusion

Reporting of test-treatment interventions is very poor, inadequate for understanding the results of these trials, and for comparing or translating results into clinical practice. Reporting needs to improve, with greater emphasis on describing the decision-making components of care pathways in both pragmatic and explanatory trials.
Please see the companion paper to this article: http://​bmcmedresmethodo​l.​biomedcentral.​com/​articles/​10.​1186/​s12874-016-0287-z.
Literature
1.
Glasziou P, Chalmers I, Altman DG, Bastian H, Boutron I, Brice A, et al. Taking healthcare interventions from trial to practice. BMJ. 2010;341:c3852.CrossRefPubMed
2.
3.
Douet L, Milne R, Anstee S, et al. The completeness of intervention descriptions in published National Institute of Health Research HTA-funded trials: a cross-sectional study. BMJ Open. 2014;4:e003713.CrossRefPubMedPubMedCentral
4.
Hoffmann TC, Erueti C, Glasziou PP. Poor description of non-pharmacological interventions: a remediable barrier to evidence use in practice? BMJ. 2013;347:f3755.CrossRefPubMedPubMedCentral
5.
Jacquier I, Boutron I, Moher D, Roy C, Ravaud P. The reporting of randomizes clinical trials using a surgical intervention is in need of immediate improvement. A systematic review. Ann Surg. 2006;244:677–83.CrossRefPubMedPubMedCentral
6.
Boutron I, Tubach F, Giraudeau B, Ravaud P. Methodological differences in clinical trials evaluating nonpharmacological and pharmacological treatments of hip and knee osteoarthritis. JAMA. 2003;290:1062–70.CrossRefPubMed
7.
Abell B, Glasziou P, Hoffmann T. Reporting and replicating trials of exercise-based cardiac rehabilitation: do we know what the researchers actually did? Circ Cardiovasc Qual Outcomes. 2015;8:187–94.CrossRefPubMed
8.
Moher D, Hopewell S, Schultz, Montori V, Gøtzsche P, Devereaux PJ, et al. CONSORT 2010 Explanation and elaboration: updated guidelines for reporting parallel group randomized trials. BMJ. 2010;340:c869.CrossRefPubMedPubMedCentral
9.
Chan AW, Tetzlaff JM, Gøtzsche PC, Altman DG, Mann H, Berlin JA, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013;346:e7586.CrossRefPubMedPubMedCentral
10.
Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, et al. The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med. 2001;134:663–94.CrossRefPubMed
11.
Hoffmann TC, Glasziou PP, Boutron I, Perera R, Moher D, Altman DG, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014;348:g1687.CrossRefPubMed
12.
Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P, Group CONSORT. Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration. Ann Intern Med. 2008;148:295–309.CrossRefPubMed
13.
14.
Schünemann HJ, Oxman AD, Brozek J, Glasziou P, Jaeschke R, Vist GE, et al. Grading quality of evidence and strength of recommendations for diagnostic tests and strategies. BMJ. 2008;336:1106e10.CrossRef
15.
16.
Bossuyt PM, Lijmer JG, Mol BW. Randomized comparisons of medical tests: sometimes invalid, not always efficient. Lancet. 2000;356:1844–7.CrossRefPubMed
17.
Ferrante di Ruffano L, Dinnes J, Sitch A, Hyde C, Deeks J. Test-treatment RCTs are susceptible to bias: a review of the methodological quality of randomized trials that evaluate diagnostic tests. Submitted companion paper on methodological quality.
18.
Lee CC, Golub R, Singer AJ, Cantu R, Levinson H. Routine versus selective abdominal computed tomography scan in the evaluation of right lower quadrant pain: a randomized controlled trial. Acad Emerg Med. 2007;14:117–22.CrossRefPubMed
19.
Guyatt GH, Tugwell PX, Feeny DH, Haynes RB, Drummond M. A framework for clinical evaluation of diagnostic technologies. Can Med Assoc J. 1986;134:587–94.
20.
Hunink MGM, Krestin GP. Study design for concurrent development, assessment, and implementation of new diagnostic imaging technology. Radiology. 2002;222:604–14.CrossRefPubMed
21.
Ransohoff DF. Challenges and opportunities in evaluating diagnostic tests. J Clin Epidemiol. 2002;55:1178–82.CrossRefPubMed
22.
Bossuyt PMM, McCaffery K. Multiple pathways and additional patient outcomes in evaluations of testing. Med Decis Making. 2009;29:E30–8.CrossRefPubMed
23.
Ferrante di Ruffano L, Hyde CJ, McCaffery KJ, Bossuyt PMM, Deeks JJ. Assessing the value of diagnostic tests-A framework for designing and evaluating trials. BMJ. 2012;344:e686.CrossRefPubMed
24.
Alfirevic Z, Neilson P. Doppler ultrasonography in high-risk pregnancies: systematic review,vith meta-analysis. Am J Obstet GynecoI. 1995;172:1397–87.
25.
Mol BW, Lijmer JG, Evers JL, Bossuyt PM. Characteristics of good diagnostic studies. Semin Reprod Med. 2003;21:17–25.CrossRefPubMed
26.
Vis JY, Wilms FF, Oudijk MA, Bossuyt PMM, van der Post JAM, Grobman WA, et al. Why were the results of randomized trials on the clinical utility of Fetal Fibronectin negative? A systematic review of their study. Am J Perinatol. 2011;28:145–50.CrossRefPubMed
27.
Deeks JJ. Assessing outcomes following tests. In: Price CP, Christenson RH, editors. Evidence-based laboratory medicine: principles, practice and outcomes. 2nd ed. Washington DC: AACC Press; 2007. p. 95–111.
28.
Lijmer JG, Bossuyt PMM. Various randomized designs can be used to evaluate medical tests. J Clin Epidemiol. 2009;62:364–73.CrossRefPubMed
29.
Ferrante di Ruffano L, Davenport C, Eisinga A, Hyde C, Deeks J. A capture-recapture analysis demonstrated that randomized controlled trials evaluating the impact of diagnostic tests on patient outcomes are rare. J Clin Epi. 2012;65:282–7.CrossRef
30.
Bossuyt PMM, Lijmer JG. Traditional health outcomes in the evaluation of diagnostic tests. Acad Radiol. 1999;6 suppl 1:S77–80.CrossRefPubMed
31.
Lord SJ, Irwig L, Bossuyt PMM. Evaluating new tests: when can comparative evidence of test accuracy and other intermediate outcomes be used as an alternative to randomized controlled trials. Med Decis Making. 2009;29:E1–12.CrossRefPubMed
32.
33.
Nikken JJ, Oei EH, Ginai AZ, Krestin GP, Verhaar JA, van Vugt AB, et al. Acute peripheral joint injury: cost and effectiveness of low-field-strength MR imaging-results of randomized controlled trial. Radiology. 2005;236:958–67.CrossRefPubMed
34.
Kearon C, Ginsberg JS, Douketis J, Turpie AG, Bates SM, Lee AY, et al. An evaluation of D-dimer in the diagnosis of pulmonary embolism: a randomized trial. Ann Intern Med. 2006;144:812–21.CrossRefPubMed
35.
Marzio L, Coraggio D, Capodicasa S, Grossi L, Cappello G. Role of the preliminary susceptibility testing for initial and after failed therapy of Helicobacter pylori infection with levofloxacin, amoxicillin, and esomeprazole. Helicobacter. 2006;11:237–42.CrossRefPubMed
36.
Faltin DL, Boulvain M, Floris LA, Irion O. Diagnosis of anal sphincter tears to prevent fecal incontinence: a randomized controlled trial. Obstet Gynecol. 2005;106:6–13.CrossRefPubMed
37.
Glenton C, Underland V, Kho M, Pennick V, Oxman AD. Summaries of findings, descriptions of interventions, and information about adverse effects would make reviews more informative. J Clin Epidemiol. 2006;59:770–8.CrossRefPubMed
38.
Djais N, Kalim H. The role of lumbar spine radiography in the outcomes of patients with simple acute low back pain. APLAR J Rheumatol. 2005;8:45–50.CrossRef
39.
Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383:267–76.CrossRefPubMed
40.
Hofmann T, English T, Glasziou P. Reporting of interventions in randomized trials: an audit of journal Instructions to Authors. Trials. 2014;15:20.CrossRef
41.
Tunis R, Stryer DB, Clancy CM. Practical Clinical Trials: Increasing the Value of Clinical Research for Decision Making in Clinical and Health Policy. JAMA. 2003;290(12):1624–32.CrossRefPubMed
42.
Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015;350:h2147.CrossRefPubMed
43.
44.
Zwarenstein M, Treweek S, Gagnier JJ, Altman DG, Tunis S, Haynes B, et al. Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ. 2008;337:a2390. Roland M, Torgerson DJ. What are pragmatic trials? BMJ 1998;316(7127):285.CrossRefPubMedPubMedCentral
45.
46.
Boutron I, Moher D, Tugwell P, Giraudeau B, Poiraudeau S, Nizard R, et al. A checklist to evaluate a report of a nonpharmacological trial (CLEAR NPT) was developed using consensus. J Clin Epidemiol. 2005;58(12):1233–40.CrossRefPubMed
Metadata
Title
Research waste in diagnostic trials: a methods review evaluating the reporting of test-treatment interventions
Authors
Lavinia Ferrante di Ruffano
Jacqueline Dinnes
Sian Taylor-Phillips
Clare Davenport
Chris Hyde
Jonathan J. Deeks
Publication date
01-12-2017
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2017
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-016-0286-0

Other articles of this Issue 1/2017

BMC Medical Research Methodology 1/2017 Go to the issue

Research Article

Basic parametric analysis for a multi-state model in hospital epidemiology

Research article

Self-controlled designs in pharmacoepidemiology involving electronic healthcare databases: a systematic review

Research article

Estimation of treatment preference effects in clinical trials when some participants are indifferent to treatment choice

Research article

Sample size and power determination when limited preliminary information is available

Research article

A statistical method utilizing information of imported cases to estimate the transmissibility for an influenza pandemic

Research article

Changes in affect after completing a mailed survey about trauma: two pre- and post-test studies in former disability applicants for posttraumatic stress disorder