Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2017

Open Access 01-12-2017 | Technical advance

Sample size re-estimation in paired comparative diagnostic accuracy studies with a binary response

Authors: Gareth P. J. McCray, Andrew C. Titman, Paula Ghaneh, Gillian A. Lancaster

Published in: BMC Medical Research Methodology | Issue 1/2017

Login to get access

Abstract

Background

The sample size required to power a study to a nominal level in a paired comparative diagnostic accuracy study, i.e. studies in which the diagnostic accuracy of two testing procedures is compared relative to a gold standard, depends on the conditional dependence between the two tests - the lower the dependence the greater the sample size required. A priori, we usually do not know the dependence between the two tests and thus cannot determine the exact sample size required. One option is to use the implied sample size for the maximal negative dependence, giving the largest possible sample size. However, this is potentially wasteful of resources and unnecessarily burdensome on study participants as the study is likely to be overpowered. A more accurate estimate of the sample size can be determined at a planned interim analysis point where the sample size is re-estimated.

Methods

This paper discusses a sample size estimation and re-estimation method based on the maximum likelihood estimates, under an implied multinomial model, of the observed values of conditional dependence between the two tests and, if required, prevalence, at a planned interim. The method is illustrated by comparing the accuracy of two procedures for the detection of pancreatic cancer, one procedure using the standard battery of tests, and the other using the standard battery with the addition of a PET/CT scan all relative to the gold standard of a cell biopsy. Simulation of the proposed method illustrates its robustness under various conditions.

Results

The results show that the type I error rate of the overall experiment is stable using our suggested method and that the type II error rate is close to or above nominal. Furthermore, the instances in which the type II error rate is above nominal are in the situations where the lowest sample size is required, meaning a lower impact on the actual number of participants recruited.

Conclusion

We recommend multinomial model maximum likelihood estimation of the conditional dependence between paired diagnostic accuracy tests at an interim to reduce the number of participants required to power the study to at least the nominal level.

Trial registration

ISRCTN ISRCTN73852054. Registered 9th of January 2015. Retrospectively registered.
Appendix
Available only for authorised users
Literature
1.
Knottnerus JA, van Weel C, Muris JWM. Evaluation of diagnostic procedures. Br Med J. 2002;324:477–80.CrossRef
2.
Swets JA, Pickett RM. Evaluation of Diagnostic Systems. New York: Academic Press; 1982.
3.
Zhou XH, Obuchowski NA, DK MC. Statistical Methods in Diagnostic Medicine. New York: Wiley; 2002.CrossRef
4.
Freedman LS. Evaluating and comparing imaging techniques: a review and classification of study designs. Br J Radiol. 1987;60:1071–81.CrossRefPubMed
5.
6.
Gould AL. Sample size re-estimation: recent developments and practical considerations. Stat Med. 2001;20:2625–43.CrossRefPubMed
7.
Wittes J, Brittain E. The role of internal pilot studies in increasing the efficiency of clinical trials. Stat Med. 1990;9:65–72.CrossRefPubMed
8.
Shih WJ. Sample size reestimation in clinical trials. In: Peace K, editor. Biopharm. Seq. Stat. Appl. New York: Marcel Dekker; 1992. p. 285–301.
9.
Shih WJ. Sample size reestimation for triple blind clinical trials. Drug Inf J. 1993;27:761–4.CrossRef
10.
Gould AL, Shih WJ. Sample size re-estimation without unblinding for normally distributed outcomes with unknown variance. Commun Stat Methods. 1992;21:2833–53.CrossRef
11.
Birkett MA, Day SJ. Internal Pilot Studies for Estimating Sample Size. Stat Med. 1994;13:2455–63.CrossRefPubMed
12.
Gould AL. Interim Analyses for Monitoring Clinical Trails that do not Affect Type I Error Rates. Stat Med. 1992;11:55–66.CrossRefPubMed
13.
Herson J, Wittes J. The Use of Interim Analysis for Sample Size Adjustment. Drug Inf J. 1993;27:761–4.CrossRef
14.
Shih WJ, Zhao P. Design for Sample Size Re-estimation with Interim Data for Double-Blind Clinical Trails. Stat Med. 1997;16:1913–23.CrossRefPubMed
15.
Proschan MA. Two-stage sample size re-estimation based on a nuisance parameter: a review. J Biopharm Stat. 2005;15:559–74.CrossRefPubMed
16.
Gerke O, Høilund-carlsen PF, Poulsen MH, Vach W. Interim analyses in diagnostic versus treatment studies : differences and similarities. Am J Nucl Med Mol Imaging. 2012;2:344–52.PubMedPubMedCentral
17.
18.
Lord SJ, Staub LP, Bossuyt PMM, Irwig LM. Target practice : choosing target conditions for test accuracy studies that are relevant to clinical practice. Br Med J. 2011;343:1–5.CrossRef
19.
Newcombe RG. Improved Confidence Intervals for the Difference between Binomial Proportions Based on Paired Data. Stat Med. 1998;17:2635–50.CrossRefPubMed
20.
Tango T. Equivalence Test and Confidence Interval for the Difference in the Proportions Based on Paired Data. Stat Med. 1998;17:891–908.CrossRefPubMed
21.
Alonzo TA, Pepe MS, Moskowitz CS. Sample Size Calculations for Comparative Studies of Medical Tests for Detecting Presence of Disease. Stat Med. 2002;21:835–52.CrossRefPubMed
22.
Lu Y, Jin H, Genant HK. On the Non-Inferiority of a Diagnostic Test Based on Paired Observations. Stat Med. 2006;3:227–79.
23.
Moskowitz CS, Pepe MS. Comparing the Predictive Values of Diagnostic Tests: Sample Size and Analysis for Paired Study Designs. Clin Trials. 2006;3:272–9.CrossRefPubMed
24.
Bonett DG, Price RM. Confidence Intervals for a Ratio of Binomial Proportions Based on Paired Data. Stat Med. 2006;25:3039–47.CrossRefPubMed
25.
Vacek P. The effect of conditional dependence on the evaluation of diagnostic tests. Biometrics. 1985;41:959–68.CrossRefPubMed
26.
van Smeden M, Naaktgeboren CA, Reitsma JB, Moons KGM, de Groot JA. Latent Class Models in Diagnostic Studies When There is No Reference Standard — A Systematic Review. Am J Epidemiol. 2014;179:423–31.CrossRefPubMed
27.
Schiller I, van Smeden M, Hadgu A, Libman M, Reitsma B, Dendukuri N. Bias due to composite reference standards in diagnostic accuracy studies. Stat Med. 2016;35:1454–70.CrossRefPubMed
28.
Royston P. Exact conditional and unconditional sample size for pair-matched studies with binary outcome: a practical guide. Stat Med. 1993;12:699–712.CrossRefPubMed
29.
van Enst WA, Naaktgeboren CA, Ochodo EA, de Groot JA, Leeflang MM, Reitsma JB, et al. Small-study effects and time trends in diagnostic test accuracy meta-analyses : a meta-epidemiological study. Syst Rev. 2015;4:66.CrossRefPubMedPubMedCentral
Metadata
Title
Sample size re-estimation in paired comparative diagnostic accuracy studies with a binary response
Authors
Gareth P. J. McCray
Andrew C. Titman
Paula Ghaneh
Gillian A. Lancaster
Publication date
01-12-2017
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2017
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-017-0386-5

Other articles of this Issue 1/2017

BMC Medical Research Methodology 1/2017 Go to the issue

Research Article

Item response models for the longitudinal analysis of health-related quality of life in cancer clinical trials

Research article

Impact of correlation of predictors on discrimination of risk models in development and external populations

Research Article

Comparison of surveillance-based metrics for the assessment and monitoring of disease detection: simulation study about type 2 diabetes

Research article

Centralising and optimising decentralised stroke care systems: a simulation study on short-term costs and effects

Research article

Performance of the marginal structural cox model for estimating individual and joined effects of treatments given in combination

Research Article

Choice of futility boundaries for group sequential designs with two endpoints