Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2020

Open Access 01-12-2020 | Research article

Modeling observations with a detection limit using a truncated normal distribution with censoring

Authors: Justin R. Williams, Hyung-Woo Kim, Catherine M. Crespi

Published in: BMC Medical Research Methodology | Issue 1/2020

Login to get access

Abstract

Background

When data are collected subject to a detection limit, observations below the detection limit may be considered censored. In addition, the domain of such observations may be restricted; for example, values may be required to be non-negative.

Methods

We propose a method for estimating population mean and variance from censored observations that accounts for known domain restriction. The method finds maximum likelihood estimates assuming an underlying truncated normal distribution.

Results

We show that our method, tcensReg, has lower bias, Type I error rates, and mean squared error than other methods commonly used for data with detection limits such as Tobit regression and single imputation under a range of simulation settings from mild to heavy censoring and truncation. We further demonstrate the consistency of the maximum likelihood estimators. We apply our method to analyze vision quality data collected from ophthalmology clinical trials comparing different types of intraocular lenses implanted during cataract surgery. All of the methods yield similar conclusions regarding non-inferiority, but estimates from the tcensReg method suggest that there may be greater mean differences and overall variability.

Conclusions

In the presence of detection limits, our new method tcensReg provides a way to incorporate known domain restrictions in dependent variables that substantially improves inferences.
Literature
1.
go back to reference Hornung RW, Reed LD. Estimation of average concentration in the presence of nondetectable values. Appl Occup Environ Hyg. 1990; 5(1):46–51. Hornung RW, Reed LD. Estimation of average concentration in the presence of nondetectable values. Appl Occup Environ Hyg. 1990; 5(1):46–51.
2.
go back to reference Lubin JH, Colt JS, Camann D, Davis S, Cerhan JR, Severson RK, Bernstein L, Hartge P. Epidemiologic evaluation of measurement data in the presence of detection limits. Environ Health Perspect. 2004; 112(17):1691–6.PubMedPubMedCentral Lubin JH, Colt JS, Camann D, Davis S, Cerhan JR, Severson RK, Bernstein L, Hartge P. Epidemiologic evaluation of measurement data in the presence of detection limits. Environ Health Perspect. 2004; 112(17):1691–6.PubMedPubMedCentral
3.
go back to reference Schisterman EF, Vexler A, Whitcomb BW, Liu A. The limitations due to exposure detection limits for regression models. Am J Epidemiol. 2006; 163(4):374–83.PubMedPubMedCentral Schisterman EF, Vexler A, Whitcomb BW, Liu A. The limitations due to exposure detection limits for regression models. Am J Epidemiol. 2006; 163(4):374–83.PubMedPubMedCentral
4.
go back to reference Helsel DR. Less than obvious-statistical treatment of data below the detection limit. Environ Sci Technol. 1990; 24(12):1766–74. Helsel DR. Less than obvious-statistical treatment of data below the detection limit. Environ Sci Technol. 1990; 24(12):1766–74.
5.
go back to reference Analytical Methods Committee. Recommendations for the definition, estimation and use of the detection limit. Analyst. 1987; 112(2):199–204. Analytical Methods Committee. Recommendations for the definition, estimation and use of the detection limit. Analyst. 1987; 112(2):199–204.
6.
go back to reference Zaugg SD, Sandstrom MW, Smith SG, Fehlberg KM. Methods of Analysis by the U.S. Geological Survey National Water Quality Laboratory-Determination of Pesticides in Water by C-18 Solid Phase Extraction and Capillary-Column Gas Chromatography/Mass Spectrometry with Selected-Ion Monitoring. Open-File Report 95-181, U.S. Geological Survey, Denver, Colorado. 1995. http://pubs.er.usgs.gov/publication/ofr95181. Zaugg SD, Sandstrom MW, Smith SG, Fehlberg KM. Methods of Analysis by the U.S. Geological Survey National Water Quality Laboratory-Determination of Pesticides in Water by C-18 Solid Phase Extraction and Capillary-Column Gas Chromatography/Mass Spectrometry with Selected-Ion Monitoring. Open-File Report 95-181, U.S. Geological Survey, Denver, Colorado. 1995. http://​pubs.​er.​usgs.​gov/​publication/​ofr95181.
7.
go back to reference McDonald JF, Moffitt RA. The uses of tobit analysis. Rev Econ Stat. 1980; 62(2):318–21. McDonald JF, Moffitt RA. The uses of tobit analysis. Rev Econ Stat. 1980; 62(2):318–21.
8.
go back to reference Greene WH. Econometric Analysis. In: Chap. 19. "Limited Dependent Variables, Truncation, Censoring and Sample Selection". 8th edn. United Kingdom: Pearson: 2018. Greene WH. Econometric Analysis. In: Chap. 19. "Limited Dependent Variables, Truncation, Censoring and Sample Selection". 8th edn. United Kingdom: Pearson: 2018.
9.
go back to reference Hald A. Maximum likelihood estimation of the parameters of a normal distribution which is truncated at a known point. Skandinavisk Aktuarietidskrift. 1949; 32:119–34. Hald A. Maximum likelihood estimation of the parameters of a normal distribution which is truncated at a known point. Skandinavisk Aktuarietidskrift. 1949; 32:119–34.
10.
go back to reference Gupta AK. Estimation of the mean and standard deviation of a normal population from a censored sample. Biometrika. 1952; 39(3):260–73. Gupta AK. Estimation of the mean and standard deviation of a normal population from a censored sample. Biometrika. 1952; 39(3):260–73.
11.
go back to reference Harter HL, Moore AH. Iterative maximum-likelihood estimation of the parameters of normal population from singly and doubly censored samples. Biometrika. 1966; 53(1-2):205–13.PubMed Harter HL, Moore AH. Iterative maximum-likelihood estimation of the parameters of normal population from singly and doubly censored samples. Biometrika. 1966; 53(1-2):205–13.PubMed
12.
go back to reference Tiku M. Estimating the mean and standard deviation from a censored normal sample. Biometrika. 1967; 54(1):155–65.PubMed Tiku M. Estimating the mean and standard deviation from a censored normal sample. Biometrika. 1967; 54(1):155–65.PubMed
13.
go back to reference Sarhan AE, Greenberg BG. Estimation of location and scale parameters by order statistics from singly and doubly censored samples. Ann Math Stat. 1956; 27(2):427–51. Sarhan AE, Greenberg BG. Estimation of location and scale parameters by order statistics from singly and doubly censored samples. Ann Math Stat. 1956; 27(2):427–51.
14.
go back to reference Dixon WJ. Simplified estimation from censored normal samples. Ann Math Stat. 1960; 31(2):385–91. Dixon WJ. Simplified estimation from censored normal samples. Ann Math Stat. 1960; 31(2):385–91.
15.
go back to reference Tobin J. Estimation of relationships for limited dependent variables. Econometrica. 1958; 26(1):24–36. Tobin J. Estimation of relationships for limited dependent variables. Econometrica. 1958; 26(1):24–36.
16.
go back to reference Buckley J, James I. Linear regression with censored data. Biometrika. 1979; 66(3):429–36. Buckley J, James I. Linear regression with censored data. Biometrika. 1979; 66(3):429–36.
17.
go back to reference Cohen Jr AC. Estimating the mean and variance of normal populations from singly truncated and doubly truncated samples. Ann Math Stat. 1950; 21(4):557–69. Cohen Jr AC. Estimating the mean and variance of normal populations from singly truncated and doubly truncated samples. Ann Math Stat. 1950; 21(4):557–69.
18.
go back to reference Halperin M. Maximum likelihood estimation in truncated samples. Ann Math Stat. 1952; 23(2):226–38. Halperin M. Maximum likelihood estimation in truncated samples. Ann Math Stat. 1952; 23(2):226–38.
19.
go back to reference Amemiya T. Regression analysis when the dependent variable is truncated normal. Econometrica. 1973; 41(6):997–1016. Amemiya T. Regression analysis when the dependent variable is truncated normal. Econometrica. 1973; 41(6):997–1016.
20.
go back to reference Owsley C, Sloane ME. Contrast sensitivity, acuity, and the perception of ‘real-world’ targets. Br J Ophthalmol. 1987; 71(10):791–6.PubMedPubMedCentral Owsley C, Sloane ME. Contrast sensitivity, acuity, and the perception of ‘real-world’ targets. Br J Ophthalmol. 1987; 71(10):791–6.PubMedPubMedCentral
21.
go back to reference Pelli DG, Bex P. Measuring contrast sensitivity. Vis Res. 2013; 90:10–14.PubMed Pelli DG, Bex P. Measuring contrast sensitivity. Vis Res. 2013; 90:10–14.PubMed
22.
go back to reference ISO 11979-7. Ophthalmic implants – Intraocular lenses – Part 7: Clinical investigations of intraocular lenses for the correction of aphakia. Vernier, Geneva, CH: Standard, International Organization for Standardization; 2018. ISO 11979-7. Ophthalmic implants – Intraocular lenses – Part 7: Clinical investigations of intraocular lenses for the correction of aphakia. Vernier, Geneva, CH: Standard, International Organization for Standardization; 2018.
23.
go back to reference Lange K. Numerical Analysis for Statisticians. 2nd Edn.New York: Springer; 2010. Lange K. Numerical Analysis for Statisticians. 2nd Edn.New York: Springer; 2010.
24.
go back to reference Ypma TJ. Historical development of the newton-raphson method. Soc Ind Appl Math. 1995; 37(4):531–51. Ypma TJ. Historical development of the newton-raphson method. Soc Ind Appl Math. 1995; 37(4):531–51.
25.
go back to reference Broyden CG. The Convergence of a Class of Double-rank Minimization Algorithms 1. General Considerations. IMA J Appl Math. 1970; 6(1):76–90. Broyden CG. The Convergence of a Class of Double-rank Minimization Algorithms 1. General Considerations. IMA J Appl Math. 1970; 6(1):76–90.
26.
go back to reference Fletcher R. A new approach to variable metric algorithms. Comput J. 1970; 13(3):317–22. Fletcher R. A new approach to variable metric algorithms. Comput J. 1970; 13(3):317–22.
27.
go back to reference Goldfarb D. A family of variable-metric methods derived by variational means. Math Comput. 1970; 24(109):23–26. Goldfarb D. A family of variable-metric methods derived by variational means. Math Comput. 1970; 24(109):23–26.
28.
go back to reference Shanno DF. Conditioning of quasi-Newton methods for function minimization. Math Comput. 1970; 24(111):647–56. Shanno DF. Conditioning of quasi-Newton methods for function minimization. Math Comput. 1970; 24(111):647–56.
29.
go back to reference Fletcher R, Reeves CM. Function minimization by conjugate gradients. Comput J. 1964; 7(2):149–54. Fletcher R, Reeves CM. Function minimization by conjugate gradients. Comput J. 1964; 7(2):149–54.
30.
go back to reference Henningsen A, Toomet O. maxLik: A package for maximum likelihood estimation in R. Comput Stat. 2011; 26(3):443–58. Henningsen A, Toomet O. maxLik: A package for maximum likelihood estimation in R. Comput Stat. 2011; 26(3):443–58.
31.
go back to reference Henningsen A. Estimating Censored Regression Models in R using the censReg Package. R Package vignettes. 2010; 5(2):12. Henningsen A. Estimating Censored Regression Models in R using the censReg Package. R Package vignettes. 2010; 5(2):12.
33.
go back to reference R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2019. https://www.R-project.org/, R Foundation for Statistical Computing. R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2019. https://​www.​R-project.​org/​, R Foundation for Statistical Computing.
35.
go back to reference Veall MR, Zimmermann KF. Pseudo-r2 measures for some common limited dependent variable models. J Econ Surv. 1996; 10(3):241–59. Veall MR, Zimmermann KF. Pseudo-r2 measures for some common limited dependent variable models. J Econ Surv. 1996; 10(3):241–59.
36.
go back to reference McKelvey RD, Zavoina W. A statistical model for the analysis of ordinal level dependent variables. J Math Sociol. 1975; 4(1):103–20. McKelvey RD, Zavoina W. A statistical model for the analysis of ordinal level dependent variables. J Math Sociol. 1975; 4(1):103–20.
37.
go back to reference Lindstrom MJ, Bates DM. Newton-raphson and em algorithms for linear mixed-effects models for repeated-measures data. J Am Stat Assoc. 1988; 83(404):1014–22. Lindstrom MJ, Bates DM. Newton-raphson and em algorithms for linear mixed-effects models for repeated-measures data. J Am Stat Assoc. 1988; 83(404):1014–22.
38.
go back to reference Zhang X, Wan AT, Zhou SZ. Focused information criteria, model selection, and model averaging in a Tobit model with a nonzero threshold. J Bus Econ Stat. 2012; 30(1):132–42. Zhang X, Wan AT, Zhou SZ. Focused information criteria, model selection, and model averaging in a Tobit model with a nonzero threshold. J Bus Econ Stat. 2012; 30(1):132–42.
Metadata
Title
Modeling observations with a detection limit using a truncated normal distribution with censoring
Authors
Justin R. Williams
Hyung-Woo Kim
Catherine M. Crespi
Publication date
01-12-2020
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2020
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-020-01032-9

Other articles of this Issue 1/2020

BMC Medical Research Methodology 1/2020 Go to the issue