Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2016

Open Access 01-12-2016 | Research article

A systematic approach to designing statistically powerful heteroscedastic 2 × 2 factorial studies while minimizing financial costs

Authors: Show-Li Jan, Gwowen Shieh

Published in: BMC Medical Research Methodology | Issue 1/2016

Login to get access

Abstract

Background

The 2 × 2 factorial design is widely used for assessing the existence of interaction and the extent of generalizability of two factors where each factor had only two levels. Accordingly, research problems associated with the main effects and interaction effects can be analyzed with the selected linear contrasts.

Methods

To correct for the potential heterogeneity of variance structure, the Welch-Satterthwaite test is commonly used as an alternative to the t test for detecting the substantive significance of a linear combination of mean effects. This study concerns the optimal allocation of group sizes for the Welch-Satterthwaite test in order to minimize the total cost while maintaining adequate power. The existing method suggests that the optimal ratio of sample sizes is proportional to the ratio of the population standard deviations divided by the square root of the ratio of the unit sampling costs. Instead, a systematic approach using optimization technique and screening search is presented to find the optimal solution.

Results

Numerical assessments revealed that the current allocation scheme generally does not give the optimal solution. Alternatively, the suggested approaches to power and sample size calculations give accurate and superior results under various treatment and cost configurations.

Conclusions

The proposed approach improves upon the current method in both its methodological soundness and overall performance. Supplementary algorithms are also developed to aid the usefulness and implementation of the recommended technique in planning 2 × 2 factorial designs.
Appendix
Available only for authorised users
Literature
1.
3.
go back to reference Kent DM, Rothwell PM, Ioannidis JP, Altman DG, Hayward RA. Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal. Trials. 2010;11:85.CrossRefPubMedPubMedCentral Kent DM, Rothwell PM, Ioannidis JP, Altman DG, Hayward RA. Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal. Trials. 2010;11:85.CrossRefPubMedPubMedCentral
4.
go back to reference McAlister FA, Straus SE, Sackett DL, Altman DG. Analysis and reporting of factorial trials: A systematic review. J Am Med Assoc. 2003;289:2545–53.CrossRef McAlister FA, Straus SE, Sackett DL, Altman DG. Analysis and reporting of factorial trials: A systematic review. J Am Med Assoc. 2003;289:2545–53.CrossRef
5.
go back to reference Montgomery AA, Astin MP, Peters TJ. Reporting of factorial trials of complex interventions in community settings: A systematic review. Trials. 2011;12:179.CrossRefPubMedPubMedCentral Montgomery AA, Astin MP, Peters TJ. Reporting of factorial trials of complex interventions in community settings: A systematic review. Trials. 2011;12:179.CrossRefPubMedPubMedCentral
6.
go back to reference Kutner MH, Nachtsheim CJ, Neter J, Li W. Applied linear statistical models. 5th ed. New York, NY: McGraw Hill; 2005. Kutner MH, Nachtsheim CJ, Neter J, Li W. Applied linear statistical models. 5th ed. New York, NY: McGraw Hill; 2005.
7.
go back to reference Maxwell SE, Delaney HD. Designing experiments and analyzing data: A model comparison perspective. 2nd ed. Mahwah, NJ: Lawrence Erlbaum Associates; 2004. Maxwell SE, Delaney HD. Designing experiments and analyzing data: A model comparison perspective. 2nd ed. Mahwah, NJ: Lawrence Erlbaum Associates; 2004.
8.
go back to reference Golinski C, Cribbie RA. The expanding role of quantitative methodologists in advancing psychology. Can Psychol. 2009;50:83–90.CrossRef Golinski C, Cribbie RA. The expanding role of quantitative methodologists in advancing psychology. Can Psychol. 2009;50:83–90.CrossRef
9.
10.
go back to reference Keselman HJ, Huberty CJ, Lix LM, Olejnik S, Cribbie R, Donahue B, Kowalchuk RK, Lowman LL, Petoskey MD, Keselman JC, Levin JR. Statistical practices of educational researchers: An analysis of their ANOVA, MANOVA, and ANCOVA analyses. Rev Educ Res. 1998;68:350–86.CrossRef Keselman HJ, Huberty CJ, Lix LM, Olejnik S, Cribbie R, Donahue B, Kowalchuk RK, Lowman LL, Petoskey MD, Keselman JC, Levin JR. Statistical practices of educational researchers: An analysis of their ANOVA, MANOVA, and ANCOVA analyses. Rev Educ Res. 1998;68:350–86.CrossRef
11.
go back to reference Ruscio J, Roche B. Variance heterogeneity in published psychological research: A review and a new index. Methodology. 2012;8:1–11.CrossRef Ruscio J, Roche B. Variance heterogeneity in published psychological research: A review and a new index. Methodology. 2012;8:1–11.CrossRef
12.
go back to reference Satterthwaite FE. An approximate distribution of estimate of variance components. Biom Bull. 1946;2:110–4.CrossRef Satterthwaite FE. An approximate distribution of estimate of variance components. Biom Bull. 1946;2:110–4.CrossRef
13.
go back to reference Welch BL. The generalization of Students’ problem when several different population variances are involved. Biometrika. 1947;34:28–35.PubMed Welch BL. The generalization of Students’ problem when several different population variances are involved. Biometrika. 1947;34:28–35.PubMed
14.
go back to reference Kirk RE. Experimental Design: Procedures for the behavioral sciences (4th ed.). CA: Sage.: Thousand Oaks; 2013.CrossRef Kirk RE. Experimental Design: Procedures for the behavioral sciences (4th ed.). CA: Sage.: Thousand Oaks; 2013.CrossRef
15.
go back to reference Beck GJ, Berg RL, Coggins CH, Gassman JJ, Hunsicker LG, Schluchter MD, Williams GW. Design and statistical issues of the Modification of Diet in Renal Disease Trial. The Modification of Diet in Renal Disease Study Group. Control Clin Trials. 1991;12:566–86.CrossRefPubMed Beck GJ, Berg RL, Coggins CH, Gassman JJ, Hunsicker LG, Schluchter MD, Williams GW. Design and statistical issues of the Modification of Diet in Renal Disease Trial. The Modification of Diet in Renal Disease Study Group. Control Clin Trials. 1991;12:566–86.CrossRefPubMed
16.
go back to reference Brookes ST, Whitely E, Egger M, Smith GD, Mulheran PA, Peters TJ. Subgroup analyses in randomized trials: Risks of subgroup-specific analyses; power and sample size for the interaction test. J Clin Epidemiol. 2004;57:229–36.CrossRefPubMed Brookes ST, Whitely E, Egger M, Smith GD, Mulheran PA, Peters TJ. Subgroup analyses in randomized trials: Risks of subgroup-specific analyses; power and sample size for the interaction test. J Clin Epidemiol. 2004;57:229–36.CrossRefPubMed
17.
go back to reference Gonen M. Planning for subgroup analysis: a case study of treatment-marker interaction in metastatic colorectal cancer. Control Clin Trials. 2003;24:355–63.CrossRefPubMed Gonen M. Planning for subgroup analysis: a case study of treatment-marker interaction in metastatic colorectal cancer. Control Clin Trials. 2003;24:355–63.CrossRefPubMed
18.
go back to reference Natarajan R, Turnbull BW, Slate EH, Clark LC. A computer program for sample size and power calculations in the design of multi-arm and factorial clinical trials with survival time endpoints. Comput Methods Prog Biomed. 1996;49:137–47.CrossRef Natarajan R, Turnbull BW, Slate EH, Clark LC. A computer program for sample size and power calculations in the design of multi-arm and factorial clinical trials with survival time endpoints. Comput Methods Prog Biomed. 1996;49:137–47.CrossRef
19.
go back to reference Wolbers M, Heemskerk D, Chau TT, Yen NT, Caws M, Farrar J, Day J. Sample size requirements for separating out the effects of combination treatments: Randomised controlled trials of combination therapy vs. standard treatment compared to factorial designs for patients with tuberculous meningitis. Trials. 2011;12:26.CrossRefPubMedPubMedCentral Wolbers M, Heemskerk D, Chau TT, Yen NT, Caws M, Farrar J, Day J. Sample size requirements for separating out the effects of combination treatments: Randomised controlled trials of combination therapy vs. standard treatment compared to factorial designs for patients with tuberculous meningitis. Trials. 2011;12:26.CrossRefPubMedPubMedCentral
20.
go back to reference Shieh G, Jan SL. Power and sample size calculations for testing linear combinations of group means under variance heterogeneity with applications to meta and moderation analyses. Psicológica. 2015;36:367–90. Shieh G, Jan SL. Power and sample size calculations for testing linear combinations of group means under variance heterogeneity with applications to meta and moderation analyses. Psicológica. 2015;36:367–90.
23.
go back to reference Allison DB, Allison RL, Faith MS, Paultre F, Pi-Sunyer X. Power and money: Designing statistically powerful studies while minimizing financial costs. Psychol Methods. 1997;2:20–33.CrossRef Allison DB, Allison RL, Faith MS, Paultre F, Pi-Sunyer X. Power and money: Designing statistically powerful studies while minimizing financial costs. Psychol Methods. 1997;2:20–33.CrossRef
24.
go back to reference Luh WM, Guo JH. Allocating sample sizes to reduce budget for fixed-effect 2 × 2 heterogeneous analysis of variance. J Exp Educ. 2016;84:197–211.CrossRef Luh WM, Guo JH. Allocating sample sizes to reduce budget for fixed-effect 2 × 2 heterogeneous analysis of variance. J Exp Educ. 2016;84:197–211.CrossRef
25.
go back to reference Guo JH, Luh WM. Optimum sample size allocation to minimize cost or maximize power for the two-sample trimmed mean test. Br J Math Stat Psychol. 2009;62:283–98.CrossRefPubMed Guo JH, Luh WM. Optimum sample size allocation to minimize cost or maximize power for the two-sample trimmed mean test. Br J Math Stat Psychol. 2009;62:283–98.CrossRefPubMed
26.
go back to reference Jan SL, Shieh G. Optimal sample sizes for Welch’s test under various allocation and cost considerations. Behav Res Methods. 2011;43:1014–22.CrossRefPubMed Jan SL, Shieh G. Optimal sample sizes for Welch’s test under various allocation and cost considerations. Behav Res Methods. 2011;43:1014–22.CrossRefPubMed
27.
go back to reference Institute SAS. SAS/IML User’s Guide, Version 9.3. Cary, NC: SAS Institute Inc.; 2013. Institute SAS. SAS/IML User’s Guide, Version 9.3. Cary, NC: SAS Institute Inc.; 2013.
28.
go back to reference Barnett SBL, Nurmagambetov TA. Costs of asthma in the United States: 2002–2007. J Allergy Clin Immunol. 2010;127:145–52.CrossRef Barnett SBL, Nurmagambetov TA. Costs of asthma in the United States: 2002–2007. J Allergy Clin Immunol. 2010;127:145–52.CrossRef
29.
go back to reference Greaves CJ, Eiser C, Seamark D, Halpin DMG. Attack context: An important mediator of the relationship between psychological status and asthma outcomes. Thorax. 2002;57:217–21.CrossRefPubMedPubMedCentral Greaves CJ, Eiser C, Seamark D, Halpin DMG. Attack context: An important mediator of the relationship between psychological status and asthma outcomes. Thorax. 2002;57:217–21.CrossRefPubMedPubMedCentral
Metadata
Title
A systematic approach to designing statistically powerful heteroscedastic 2 × 2 factorial studies while minimizing financial costs
Authors
Show-Li Jan
Gwowen Shieh
Publication date
01-12-2016
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2016
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-016-0214-3

Other articles of this Issue 1/2016

BMC Medical Research Methodology 1/2016 Go to the issue