Skip to main content
Top
Published in: Journal of Gambling Studies 3/2014

01-09-2014 | Original Paper

Setting Criterion Thresholds for Estimating Prevalence: What is Being Validated?

Author: Blase Gambino

Published in: Journal of Gambling Studies | Issue 3/2014

Login to get access

Abstract

Much of the debate over how best to estimate the prevalence of problem gambling in the general population is driven by a number of misconceptions, misinterpretations, and questionable, sometimes erroneous assumptions. Among the latter is the failure to understand that what is being validated is not the test but the interpretation of test scores for a specific purpose. In addition there has been a lack of attention to defining the clinical and/or epidemiologic relevance of case definitions in terms of severity and other clinical attributes, a misunderstanding of how test values are interpreted when criterion thresholds or cut-off points are selected, and a failure to replicate the validation of criterion thresholds for defining cases of problem gambling. It is argued further that the distinction between dichotomy and continuum is a false choice, and any emphasis on overestimation is misdirected. Alternative methods for evaluating tests and estimating prevalence are described and a pragmatic empirical approach to the interpretation of prevalence estimates is recommended.
Footnotes
1
The occurrence of errors in gold standards may be due to a number of sources, for example, an alternative condition mimics the biomarker changes of the condition of interest, a laboratory test may be mislabeled, a tumor is missed by MRI because it is too small, an image may be misinterpreted, or a biopsy occurs outside of the affected tumor (Reitsma et al. 2009).
 
2
Since it takes an interval of time to survey all respondents, point prevalence estimates are as a practical and technical matter also period prevalence rates. As a rule of thumb, one-month rates are considered point prevalence estimates.
 
3
The use of the term normal is specific to gambling and does not rule out the presence of other conditions.
 
4
The term reference or criterion standard rather than gold standard is now preferred since gold standard connotes an errorless process whereas in practice this is rarely if ever true. Further, for most disorders there is no gold standard and alternative standards that are error-prone must be utilized. The study reference standard is an operational definition of diagnostic truth for evaluating the new test. Contrary to popular belief the application of a gold standard does not imply the standard is error free; in medicine, for example, some gold standards have error rates as high as 5 % or greater (Valenstein 1990).
 
5
Bias in prevalence estimation refers to whether standard estimates for P given by AP tend, on average, to overestimate (positive bias) or underestimate (negative bias) the true population prevalence (Gambino 1997). If the test is imperfect then AP is a biased estimator for P; that is, in repeated studies the test will tend to overestimate (underestimate) the true P more often than AP will equal or underestimate (overestimate) P. Even biased tests will sometimes provide unbiased results, Table 2, panel B.
 
6
It may be noted that whereas in Table 1 test diagnoses are represented as rows and the disorder or condition is represented as columns, some experts on test evaluation argue that columns also represent diagnoses as made by the reference standard (Kraemer 1992). Other experts disagree with this view (Pepe 2003) and argue that this may confuse readers. In the present paper, rows are identified as test diagnoses and columns are identified with the presence or absence of the target disorder.
 
7
Canadian Problem Gambling Index, Ferris and Wynne (2001).
 
8
Since any column (row) in a 2 × 2 table must sum to 100 %, it follows directly that knowledge of TPR (sensitivity) defines its complement FNR (false negative rate) = 100 % − TPR, and knowledge of FPR defines its complement TNR (true negative rate or specificity) = 100 % − FPR. It is important to recognize the value of these two relations, if TPR and FPR are known or estimable then the test evaluator has all the information about test properties based solely on positive test results.
 
9
Those readers unfamiliar with ROC analysis are referred to the excellent introductions by Obuchowski (2003, 2005), Zou et al. (2007) and Lasko et al. (2005) which is somewhat more technical but well worth the read. I highly recommend the reader unfamiliar with ROC check out “The Magnificent ROC” at www.​anaesthetist.​com/​mnm/​stats/​roc/​ where the author takes the reader on an interactive tour de force of the ROC.
 
10
The counts of normal and problem gamblers were reconstructed from Boudreau and Poulin (2006, Table 1, p. 303) re-validation of SOGS-RA (SOGS Revised for Adolescents, Winters et al. 1993)), and are presented for illustrative purposes only. It is assumed in the illustration that the reference standard, self perceived problem gambler is errorless.
 
11
Mathematically TPR is not considered a rate and statisticians prefer the more precise term fraction or proportion instead; the reader may consider TPR = TPF and FPR = FPF equivalent in meaning and measurement. The reader will find the terms, rate or fraction, will both appear in the literature on ROC.
 
12
There is some debate over whether face validity is equivalent to content validity with the two terms often used as interchangeable. Anastasi (1988) has framed the discussion as: “Content validity should not be confused with face validity. The latter is not validity in the technical sense; it refers, not to what the test actually measures, but to what it appears superficially to measure” (p. 144). Content validity is relatively easy to measure when the construct is ability or skills that are specific to task elements but is more difficult to establish for less specific constructs such as problem gambling.
 
13
In practice any theoretical position that continues to be supported over time will become accepted as established with the often unstated caveat that if new evidence cannot be explained then the theory will be revised, discarded, or replaced. Good theories survive most attacks on the validity of the theory; poor theories are unlikely to survive. Although no doubt there are instances of the latter.
 
14
South Oaks Gambling Screen (Lesieur & Blume 1987).
 
15
Victorian Gambling Screen, Ben Tovin et al. (2001).
 
16
National Opinion [Research Center] Diagnostic Screen (Gerstein et al. 1999).
 
17
PPV is obtained by applying Bayes Theorem as pre-test odds (P/(1 − P)) × test odds (LR+) = post-test odds and converting post-test odds to PPV by applying odds/(1 + odds).
 
Literature
go back to reference Abbott, M., & Volberg, R. (1996). The New Zealand national survey of problem and pathological gambling. Journal of Gambling Studies, 12, 143–160.PubMedCrossRef Abbott, M., & Volberg, R. (1996). The New Zealand national survey of problem and pathological gambling. Journal of Gambling Studies, 12, 143–160.PubMedCrossRef
go back to reference Ahlbom, A., & Norell, S. (1990). Introduction to modern epidemiology. Chestnut Hill, MA: Epidemiology Resources Inc. Ahlbom, A., & Norell, S. (1990). Introduction to modern epidemiology. Chestnut Hill, MA: Epidemiology Resources Inc.
go back to reference Anastasi, A. (1988). Psychological testing. New York, NY: Macmillan. Anastasi, A. (1988). Psychological testing. New York, NY: Macmillan.
go back to reference Bachman, L. M., Juni, P., Reichenbach, S., et al. (2005). Consequences of different diagnostic’gold standards’ in test accuracy research: Carpal tunnel syndrome as an example. International Journal of Epidemiology, 34, 953–955.CrossRef Bachman, L. M., Juni, P., Reichenbach, S., et al. (2005). Consequences of different diagnostic’gold standards’ in test accuracy research: Carpal tunnel syndrome as an example. International Journal of Epidemiology, 34, 953–955.CrossRef
go back to reference Banzato, C. E. M. (2004). Classification in psychiatry: The move towards IICD-11 and DSM-V. Current Opinion in Psychiatry, 17, 497–501.CrossRef Banzato, C. E. M. (2004). Classification in psychiatry: The move towards IICD-11 and DSM-V. Current Opinion in Psychiatry, 17, 497–501.CrossRef
go back to reference Battersby, M. W., Thomas, L. J., Tolchard, B., & Esterman, A. (2002). The South Oaks gambling screen: A review with reference to Australian use. Journal of Gambling Studies, 18, 257–271.PubMedCrossRef Battersby, M. W., Thomas, L. J., Tolchard, B., & Esterman, A. (2002). The South Oaks gambling screen: A review with reference to Australian use. Journal of Gambling Studies, 18, 257–271.PubMedCrossRef
go back to reference Bellringer, M., Abbott, M. W., Volberg, R. A., Garrett, N., & Coombes, R. (2007). Problem gambling assessment and screening instruments: Phase one report. Auckland: Gambling Research Centre, Auckland University of Technology. Bellringer, M., Abbott, M. W., Volberg, R. A., Garrett, N., & Coombes, R. (2007). Problem gambling assessment and screening instruments: Phase one report. Auckland: Gambling Research Centre, Auckland University of Technology.
go back to reference Ben Tovin, D., Esterman, A., & Tolchard, M. (2001). The Victorian gambling screen. Melbourne: Gambling Research Panel. Ben Tovin, D., Esterman, A., & Tolchard, M. (2001). The Victorian gambling screen. Melbourne: Gambling Research Panel.
go back to reference Blaszczynski, A., Ladouceur, R., & Moodie, C. (2008). The Sydney Laval universities gambling screen: Preliminary data. Addiction Research and Theory, 16, 401–411.CrossRef Blaszczynski, A., Ladouceur, R., & Moodie, C. (2008). The Sydney Laval universities gambling screen: Preliminary data. Addiction Research and Theory, 16, 401–411.CrossRef
go back to reference Boudreau, B., & Poulin, C. (2006). The South Oaks gambling screen-revised adolescent (SOGS-RA) revisited: A cut-point analysis. Journal of Gambling Studies, 23, 299–308.CrossRef Boudreau, B., & Poulin, C. (2006). The South Oaks gambling screen-revised adolescent (SOGS-RA) revisited: A cut-point analysis. Journal of Gambling Studies, 23, 299–308.CrossRef
go back to reference Carragher, N., & McWilliams, L. A. (2011). A latent class analysis of DSM-IV criteria for pathological gambling: Results from the national epidemiologic survey on alcohol and related conditions. Psychiatry Research, 187, 185–192.PubMedCrossRef Carragher, N., & McWilliams, L. A. (2011). A latent class analysis of DSM-IV criteria for pathological gambling: Results from the national epidemiologic survey on alcohol and related conditions. Psychiatry Research, 187, 185–192.PubMedCrossRef
go back to reference Choi, B. C. K. (1998). Slopes of receiver operating characteristic curve and likelihood ratios for a diagnostic test. American Journal of Epidemiology, 148, 1127–1132.PubMedCrossRef Choi, B. C. K. (1998). Slopes of receiver operating characteristic curve and likelihood ratios for a diagnostic test. American Journal of Epidemiology, 148, 1127–1132.PubMedCrossRef
go back to reference Cox, B. J., Yu, N., Affifi, T. O., & Ladouceur, R. (2005). A national survey of gambling problems in Canada. Canadian Journal of Psychiatry, 50, 213–217. Cox, B. J., Yu, N., Affifi, T. O., & Ladouceur, R. (2005). A national survey of gambling problems in Canada. Canadian Journal of Psychiatry, 50, 213–217.
go back to reference Culleton, R. P. (1989). The prevalence rates of pathological gambling: A look at methods. Journal of Gambling Behaviors, 5, 22–41.CrossRef Culleton, R. P. (1989). The prevalence rates of pathological gambling: A look at methods. Journal of Gambling Behaviors, 5, 22–41.CrossRef
go back to reference Currie, S. R., Casey, D. M., & Hodgins, D. C. (2010). Improving the psychometric properties of the problem gambling severity index. Submitted to the Canadian Consortium for Gambling Research. Currie, S. R., Casey, D. M., & Hodgins, D. C. (2010). Improving the psychometric properties of the problem gambling severity index. Submitted to the Canadian Consortium for Gambling Research.
go back to reference Department of Justice (2011, June). The Victorian Gambling Study: A longitudinal study of gambling and public health-wave two findings. State of Victoria: Department of Justice. Department of Justice (2011, June). The Victorian Gambling Study: A longitudinal study of gambling and public health-wave two findings. State of Victoria: Department of Justice.
go back to reference Dickerson, M. (1993). A preliminary exploration of a two-stage methodology in the assessment of the extent and degree of gambling related problems in the Australian population. In W. R. Eadington, J. Cornelius, & J. I. Tabor (Eds.), Gambling behavior and problem gambling (pp. 347–363). Reno, Nevada: Institute for the Study of Gambling and Commercial Gaming, University of Nevada. Dickerson, M. (1993). A preliminary exploration of a two-stage methodology in the assessment of the extent and degree of gambling related problems in the Australian population. In W. R. Eadington, J. Cornelius, & J. I. Tabor (Eds.), Gambling behavior and problem gambling (pp. 347–363). Reno, Nevada: Institute for the Study of Gambling and Commercial Gaming, University of Nevada.
go back to reference Dickerson, M. G., Baron, E., Hong, S. M., & Cottrell, D. (1996). Estimating the extent and degree of gambling related problems in the Australian population: A national survey. Journal of Gambling Studies, 12, 161–178.PubMedCrossRef Dickerson, M. G., Baron, E., Hong, S. M., & Cottrell, D. (1996). Estimating the extent and degree of gambling related problems in the Australian population: A national survey. Journal of Gambling Studies, 12, 161–178.PubMedCrossRef
go back to reference Dickerson, M., McMillen, J., Hallebone, E., Volberg, R., & Woolley, R. (1997). Definition and incidence of problem gambling including the socio-economic distribution of gamblers. Melbourne: Victorian Casino and Gaming Authority. Dickerson, M., McMillen, J., Hallebone, E., Volberg, R., & Woolley, R. (1997). Definition and incidence of problem gambling including the socio-economic distribution of gamblers. Melbourne: Victorian Casino and Gaming Authority.
go back to reference Feinstein, A. R. (1985). Clinical epidemiology: The architecture of clinical research. Philadelphia: Saunders Press. Feinstein, A. R. (1985). Clinical epidemiology: The architecture of clinical research. Philadelphia: Saunders Press.
go back to reference Ferris, J., & Wynne, H. (2001). The Canadian problem gambling index: Final report. Ottawa: Canadian Centre on Substance Abuse. Ferris, J., & Wynne, H. (2001). The Canadian problem gambling index: Final report. Ottawa: Canadian Centre on Substance Abuse.
go back to reference Ferris, J., Wynne, H., & Single, E. (1999). Measuring problem gambling in Canada: Final report—Phase I. Ottawa: Canadian Centre on Substance Abuse. Ferris, J., Wynne, H., & Single, E. (1999). Measuring problem gambling in Canada: Final report—Phase I. Ottawa: Canadian Centre on Substance Abuse.
go back to reference Fleiss, J. L. (1981). Statistical methods for rates and proportions. New York: Wiley. Fleiss, J. L. (1981). Statistical methods for rates and proportions. New York: Wiley.
go back to reference Gambino, B. (1997). The correction for bias in prevalence estimation with screening tests. Journal of Gambling Studies, 13, 343–351.PubMedCrossRef Gambino, B. (1997). The correction for bias in prevalence estimation with screening tests. Journal of Gambling Studies, 13, 343–351.PubMedCrossRef
go back to reference Gambino, B. (1999). An epidemiologic note on verification bias: Implications for estimation of rates. Journal of Gambling Studies, 15, 223–232.PubMedCrossRef Gambino, B. (1999). An epidemiologic note on verification bias: Implications for estimation of rates. Journal of Gambling Studies, 15, 223–232.PubMedCrossRef
go back to reference Gambino, B. (2006b). A comment on the utility of prevalence estimates of pathological gambling. Journal of Gambling Studies, 22, 321–328.PubMedCrossRef Gambino, B. (2006b). A comment on the utility of prevalence estimates of pathological gambling. Journal of Gambling Studies, 22, 321–328.PubMedCrossRef
go back to reference Gambino, B. (2012). The validation of screening tests: Meet the new screen same as the old screen? Journal of Gambling Studies, 28, 573–605.PubMedCrossRef Gambino, B. (2012). The validation of screening tests: Meet the new screen same as the old screen? Journal of Gambling Studies, 28, 573–605.PubMedCrossRef
go back to reference Gebauer, L., LaBrie, R., & Shaffer, H. J. (2010). Optimizing DSM-IV-TR classification accuracy: A brief biosocial screen for detecting current gambling disorders among gamblers in the general household population. Canadian Journal of Psychiatry, 55, 82–90. Gebauer, L., LaBrie, R., & Shaffer, H. J. (2010). Optimizing DSM-IV-TR classification accuracy: A brief biosocial screen for detecting current gambling disorders among gamblers in the general household population. Canadian Journal of Psychiatry, 55, 82–90.
go back to reference Gerstein, D., Hoffman, J., Larison, C., Murphy, S., Palmer, A., Chuchro, L., et al. (1999). Gambling impact and behavior study. Report to the National Gambling Impact Study Commission. Chicago: NORC. Gerstein, D., Hoffman, J., Larison, C., Murphy, S., Palmer, A., Chuchro, L., et al. (1999). Gambling impact and behavior study. Report to the National Gambling Impact Study Commission. Chicago: NORC.
go back to reference Greenes, R. A., & Begg, C. B. (1985). Assessment of diagnostic technologies: Methodology for unbiased estimation from samples of selectively verified patients. Investigative Radiology, 20, 751–756.PubMedCrossRef Greenes, R. A., & Begg, C. B. (1985). Assessment of diagnostic technologies: Methodology for unbiased estimation from samples of selectively verified patients. Investigative Radiology, 20, 751–756.PubMedCrossRef
go back to reference Greiner, M., Pfeiffer, D., & Smith, R. D. (2000). Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests. Preventive Veterinary Medicine, 45, 23–41.PubMedCrossRef Greiner, M., Pfeiffer, D., & Smith, R. D. (2000). Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests. Preventive Veterinary Medicine, 45, 23–41.PubMedCrossRef
go back to reference Habbema, J. D. F., Eijkemans, R., Krijnen, P., & Knottnerus, J. A. (2002). Analysis of data on the accuracy of diagnostic tests. In J. A. Knottnerus (Ed.), The evidence base of clinical diagnosis (pp. 117–144). London: BMJ Publishing Group. Habbema, J. D. F., Eijkemans, R., Krijnen, P., & Knottnerus, J. A. (2002). Analysis of data on the accuracy of diagnostic tests. In J. A. Knottnerus (Ed.), The evidence base of clinical diagnosis (pp. 117–144). London: BMJ Publishing Group.
go back to reference Hodgins, D. C., Stea, J. N., & Grant, J. E. (2011). Gambling disorders. Lancet, online first pp. 1–11. Hodgins, D. C., Stea, J. N., & Grant, J. E. (2011). Gambling disorders. Lancet, online first pp. 1–11.
go back to reference Hong, S. I., Sacco, P., & Cunningham-Williams, R. M. (2009). An empirical typology of lifetime and current gambling behaviors: Association with health status of older adults. Aging Mental Health, 13, 265–273.PubMedCrossRef Hong, S. I., Sacco, P., & Cunningham-Williams, R. M. (2009). An empirical typology of lifetime and current gambling behaviors: Association with health status of older adults. Aging Mental Health, 13, 265–273.PubMedCrossRef
go back to reference Ihorst, G., Forster, J., Peterson, G., Werchau, H., Rohwedder, A., & Schumacher, M. (2007). The use of imperfect diagnostic tests had an impact on prevalence estimation. Journal of Clinical Epidemiology, 60, 902–910.PubMedCrossRef Ihorst, G., Forster, J., Peterson, G., Werchau, H., Rohwedder, A., & Schumacher, M. (2007). The use of imperfect diagnostic tests had an impact on prevalence estimation. Journal of Clinical Epidemiology, 60, 902–910.PubMedCrossRef
go back to reference Irwig, L., Bossuyt, P., Glasziou, P., Gatsonis, C., & Lijmer, J. (2002). Designing studies to ensure that estimates of test accuracy are transferable. British Medical Journal, 324, 669–671.PubMedCentralPubMedCrossRef Irwig, L., Bossuyt, P., Glasziou, P., Gatsonis, C., & Lijmer, J. (2002). Designing studies to ensure that estimates of test accuracy are transferable. British Medical Journal, 324, 669–671.PubMedCentralPubMedCrossRef
go back to reference Irwin, R. J., & Irwin, T. C. (2011). A principled approach to setting optimal diagnostic thresholds: Where ROC and indifference curves meet. European Journal of Internal Medicine, 22, 230–234.PubMedCrossRef Irwin, R. J., & Irwin, T. C. (2011). A principled approach to setting optimal diagnostic thresholds: Where ROC and indifference curves meet. European Journal of Internal Medicine, 22, 230–234.PubMedCrossRef
go back to reference Johansson, A., Grant, J. E., Kim, S. W., Odlaug, B. L., & Gotestam, K. G. (2009). Risk factors for problematic gambling: A critical literature review. Journal of Gambling Studies, 25, 67–92.PubMedCrossRef Johansson, A., Grant, J. E., Kim, S. W., Odlaug, B. L., & Gotestam, K. G. (2009). Risk factors for problematic gambling: A critical literature review. Journal of Gambling Studies, 25, 67–92.PubMedCrossRef
go back to reference Kane, M. T. (2006). Validation. In R. L. Brennen (Ed.), Educational measurement (4th ed., pp. 17–64). Westport, CT: American Council on Eduction/Praeger. Kane, M. T. (2006). Validation. In R. L. Brennen (Ed.), Educational measurement (4th ed., pp. 17–64). Westport, CT: American Council on Eduction/Praeger.
go back to reference Kline, P. (2000). The new psychometrics. London: Rutledge. Kline, P. (2000). The new psychometrics. London: Rutledge.
go back to reference Kraemer, H. C. (1992). Evaluating medical tests: objective and quantitative guidelines. Kraemer, H. C. (1992). Evaluating medical tests: objective and quantitative guidelines.
go back to reference Kraemer, H. C. (2007). DSM categories and dimensions in clinical and research contexts. International Journal of Methods in Psychiatric Research, 16, S8–S15.PubMedCrossRef Kraemer, H. C. (2007). DSM categories and dimensions in clinical and research contexts. International Journal of Methods in Psychiatric Research, 16, S8–S15.PubMedCrossRef
go back to reference Kraemer, H. C., Noda, A., & O’Hara, R. (2004). Categorical versus dimensional approaches to diagnosis: methodological challenges. Journal of Psychiatric Research, 38, 17–25.PubMedCrossRef Kraemer, H. C., Noda, A., & O’Hara, R. (2004). Categorical versus dimensional approaches to diagnosis: methodological challenges. Journal of Psychiatric Research, 38, 17–25.PubMedCrossRef
go back to reference LaPlante, D. A., Nelson, S. E., LaBrie, R. A., & Shaffer, H. J. (2008). Stability and progression of disordered gambling: Lessons from longitudinal studies. Canadian Journal of Psychiatry, 53, 52–60. LaPlante, D. A., Nelson, S. E., LaBrie, R. A., & Shaffer, H. J. (2008). Stability and progression of disordered gambling: Lessons from longitudinal studies. Canadian Journal of Psychiatry, 53, 52–60.
go back to reference Lasko, T. A., Bhagwat, J. G., Zou, K. H., & Ohno-Machado, L. (2005). The use of receiver operating characteristic curves in biomedical informatics. Journal of Biomedical Informatics, 38, 404–415.PubMedCrossRef Lasko, T. A., Bhagwat, J. G., Zou, K. H., & Ohno-Machado, L. (2005). The use of receiver operating characteristic curves in biomedical informatics. Journal of Biomedical Informatics, 38, 404–415.PubMedCrossRef
go back to reference Last, J. M. (1988). A dictionary of epidemiology (2nd ed.). New York: Oxford University Press. Last, J. M. (1988). A dictionary of epidemiology (2nd ed.). New York: Oxford University Press.
go back to reference Lesieur, H. R., & Blume, S. B. (1987). South Oaks Gambling Screen (SOGS): A new instrument for the identification of pathological gamblers. American Journal of Psychiatry, 144, 1184–1188.PubMed Lesieur, H. R., & Blume, S. B. (1987). South Oaks Gambling Screen (SOGS): A new instrument for the identification of pathological gamblers. American Journal of Psychiatry, 144, 1184–1188.PubMed
go back to reference Martins, S. S., Ghandour, L. A., & Storr, C. L. (2011). Gambling behavior subtypes among respondents with gambling-related problems in a population-based sample. Journal of Addictive Diseases, 30, 169–180.PubMedCrossRef Martins, S. S., Ghandour, L. A., & Storr, C. L. (2011). Gambling behavior subtypes among respondents with gambling-related problems in a population-based sample. Journal of Addictive Diseases, 30, 169–180.PubMedCrossRef
go back to reference Mayes, R., & Horwitz, A. V. (2005). DSM-III and the revolution in the classification of mental illness. Journal of the History of the Behavioral Sciences, 41, 249–267.PubMedCrossRef Mayes, R., & Horwitz, A. V. (2005). DSM-III and the revolution in the classification of mental illness. Journal of the History of the Behavioral Sciences, 41, 249–267.PubMedCrossRef
go back to reference McBride, O., Adamson, G., & Shevlin, M. (2010). A latent class analysis of DSM-IV pathological gambling criteria in a nationally representative British sample. Psychiatry Research, 178, 401–407.PubMedCrossRef McBride, O., Adamson, G., & Shevlin, M. (2010). A latent class analysis of DSM-IV pathological gambling criteria in a nationally representative British sample. Psychiatry Research, 178, 401–407.PubMedCrossRef
go back to reference McMillen, J., & Wenzel, M. (2006). Measuring problem gambling: Assessment of three prevalence screens. International Gambling Studies, 6, 147–174.CrossRef McMillen, J., & Wenzel, M. (2006). Measuring problem gambling: Assessment of three prevalence screens. International Gambling Studies, 6, 147–174.CrossRef
go back to reference McNamee, R. (2002). Optimal designs of two-stage studies for estimation of sensitivity, specificity and positive predictive value. Statistics in Medicine, 21, 3609–3625.PubMedCrossRef McNamee, R. (2002). Optimal designs of two-stage studies for estimation of sensitivity, specificity and positive predictive value. Statistics in Medicine, 21, 3609–3625.PubMedCrossRef
go back to reference McNamee, R. (2003). Efficiency of two-phase designs for prevalence estimation. International Journal of Epidemiology, 32, 1072–1078.PubMedCrossRef McNamee, R. (2003). Efficiency of two-phase designs for prevalence estimation. International Journal of Epidemiology, 32, 1072–1078.PubMedCrossRef
go back to reference Mechanic, D. (2003). Is the prevalence of mental disorders a good measure of the need for services? Health Affairs, 22, 8–20.PubMedCrossRef Mechanic, D. (2003). Is the prevalence of mental disorders a good measure of the need for services? Health Affairs, 22, 8–20.PubMedCrossRef
go back to reference Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). Washington, DC: American Council on Education and National Council on Measurement in Education. Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). Washington, DC: American Council on Education and National Council on Measurement in Education.
go back to reference Miettinen, O. (1985). Theoretical epidemiology. Boston: Wiley. Miettinen, O. (1985). Theoretical epidemiology. Boston: Wiley.
go back to reference Neal, P., Delfabbro, P., & O’Neil, M. (2005). Problem gambling and harm: Towards a national definition. Adelaide: South Australian Centre for Economic Studies. Neal, P., Delfabbro, P., & O’Neil, M. (2005). Problem gambling and harm: Towards a national definition. Adelaide: South Australian Centre for Economic Studies.
go back to reference Obuchowski, N. A. (2003). Receiver operating characteristic curves and their use in radiology. Radiology, 229, 3–8.PubMedCrossRef Obuchowski, N. A. (2003). Receiver operating characteristic curves and their use in radiology. Radiology, 229, 3–8.PubMedCrossRef
go back to reference Obuchowski, N. A., & Zhou, X. (2002). Prospective studies of diagnostic test accuracy when disease prevalence is low. Biostatistics, 3, 477–492.PubMedCrossRef Obuchowski, N. A., & Zhou, X. (2002). Prospective studies of diagnostic test accuracy when disease prevalence is low. Biostatistics, 3, 477–492.PubMedCrossRef
go back to reference Pepe, M. S. (2003). The statistical evaluation of medical tests for classification and prediction. Oxford UK: Oxford University Press. Pepe, M. S. (2003). The statistical evaluation of medical tests for classification and prediction. Oxford UK: Oxford University Press.
go back to reference Perkins, N. J., & Schisterman, E. F. (2006). The inconsistency of ‘‘optimal’’ cut points obtained using two criteria based on the receiver operating characteristic curve. American Journal of Epidemiology, 163, 670–675.PubMedCentralPubMedCrossRef Perkins, N. J., & Schisterman, E. F. (2006). The inconsistency of ‘‘optimal’’ cut points obtained using two criteria based on the receiver operating characteristic curve. American Journal of Epidemiology, 163, 670–675.PubMedCentralPubMedCrossRef
go back to reference Productivity Commission (2010, February). Gambling: Inquiry report, Volume 1. Productivity Commission, Australia. Productivity Commission (2010, February). Gambling: Inquiry report, Volume 1. Productivity Commission, Australia.
go back to reference Reilly, M. (1996). Optimal sampling strategies for two-stage studies. American Journal of Epidemiology, 143, 92–100.PubMedCrossRef Reilly, M. (1996). Optimal sampling strategies for two-stage studies. American Journal of Epidemiology, 143, 92–100.PubMedCrossRef
go back to reference Reitsma, J. B., Rutjes, A. W. S., Khan, K. S., Coomarasamy, A., & Bossuyt, P. M. (2009). A review of solutions for diagnostic accuracy studies with an imperfect or missing reference standard. Journal of Clinical Epidemiology, 62, 797–806.PubMedCrossRef Reitsma, J. B., Rutjes, A. W. S., Khan, K. S., Coomarasamy, A., & Bossuyt, P. M. (2009). A review of solutions for diagnostic accuracy studies with an imperfect or missing reference standard. Journal of Clinical Epidemiology, 62, 797–806.PubMedCrossRef
go back to reference Robins, L. N. (1985). Epidemiology: Reflections on testing the validity of psychiatric interviews. Archives of General Psychiatry, 42, 918–924.PubMedCrossRef Robins, L. N. (1985). Epidemiology: Reflections on testing the validity of psychiatric interviews. Archives of General Psychiatry, 42, 918–924.PubMedCrossRef
go back to reference Rosenthal, R. J. (2003, Commentaries). Distribution of the DSM-IV criteria for pathological gambling, Addiction, 98:1674–1675. Rosenthal, R. J. (2003, Commentaries). Distribution of the DSM-IV criteria for pathological gambling, Addiction, 98:1674–1675.
go back to reference Sackett, D. L., Haynes, R. B., & Tugwell, P. (1985). Clinical epidemiology: A basic science for clinical medicine. Boston: Little Brown. Sackett, D. L., Haynes, R. B., & Tugwell, P. (1985). Clinical epidemiology: A basic science for clinical medicine. Boston: Little Brown.
go back to reference Salim, A., & Welsh, A. H. (2009). Designing 2-phase prevalence studies in the absence of a “Gold Standard” test. American Journal of Epidemiology, 170, 369–378.PubMedCrossRef Salim, A., & Welsh, A. H. (2009). Designing 2-phase prevalence studies in the absence of a “Gold Standard” test. American Journal of Epidemiology, 170, 369–378.PubMedCrossRef
go back to reference Sassen, M., Kraus, L., & Buhringer, G. (2011). Differences in pathological gambling prevalence estimates: Facts or artefacts? International Journal of Methods in Psychiatric Research, 20, e83–e99.PubMedCrossRef Sassen, M., Kraus, L., & Buhringer, G. (2011). Differences in pathological gambling prevalence estimates: Facts or artefacts? International Journal of Methods in Psychiatric Research, 20, e83–e99.PubMedCrossRef
go back to reference Saunders, J. B., & Cottler, L. B. (2007). The development of the diagnostic and statistical manual of mental disorders version V substance abuse disorders section: Establishing the research framework. Current Opinion Psychiatry, 20, 208–212. Saunders, J. B., & Cottler, L. B. (2007). The development of the diagnostic and statistical manual of mental disorders version V substance abuse disorders section: Establishing the research framework. Current Opinion Psychiatry, 20, 208–212.
go back to reference Shaffer, H. J., & Hall, M. N. (1996). Estimating the prevalence of adolescent gambling disorders: A quantitative synthesis and guide toward standard gambling nomenclature. Journal of Gambling Studies, 12, 193–214.PubMedCrossRef Shaffer, H. J., & Hall, M. N. (1996). Estimating the prevalence of adolescent gambling disorders: A quantitative synthesis and guide toward standard gambling nomenclature. Journal of Gambling Studies, 12, 193–214.PubMedCrossRef
go back to reference Shaffer, H. J., & Hall, M. N. (2001). Updating and refining prevalence estimates of disordered gambling behaviour in the United States and Canada. Canadian Journal of Public Health, 92, 168–172.PubMed Shaffer, H. J., & Hall, M. N. (2001). Updating and refining prevalence estimates of disordered gambling behaviour in the United States and Canada. Canadian Journal of Public Health, 92, 168–172.PubMed
go back to reference Shaffer, H. J., Hall, M. N., & Bilt Vander, J. (1999). Estimating the prevalence of disordered gambling behavior in the United States and Canada: A research synthesis. American Journal of Public Health, 89(9), 1369–1376.PubMedCentralPubMedCrossRef Shaffer, H. J., Hall, M. N., & Bilt Vander, J. (1999). Estimating the prevalence of disordered gambling behavior in the United States and Canada: A research synthesis. American Journal of Public Health, 89(9), 1369–1376.PubMedCentralPubMedCrossRef
go back to reference Shaffer, H. J., Hall, M. N., & Vander Bilt, J. (1997). A meta-analysis estimating the prevalence of disordered gambling behavior in the United States and Canada. Boston, MA: Harvard Medical School Division on Addictions. Shaffer, H. J., Hall, M. N., & Vander Bilt, J. (1997). A meta-analysis estimating the prevalence of disordered gambling behavior in the United States and Canada. Boston, MA: Harvard Medical School Division on Addictions.
go back to reference Shaffer, H. J., & Korn, D. A. (2002). Gambling and related mental disorders: A public health analysis. Annual Review of Public Health, 23, 171–212.PubMedCrossRef Shaffer, H. J., & Korn, D. A. (2002). Gambling and related mental disorders: A public health analysis. Annual Review of Public Health, 23, 171–212.PubMedCrossRef
go back to reference Shaffer, H. J., LaBrie, R. A., LaPlante, D. A., Nelson, S. E., & Stanton, M. V. (2004). The road less traveled: Moving from distribution to determinants in the study of gambling epidemiology. Canadian Journal of Psychiatry, 49, 504–516. Shaffer, H. J., LaBrie, R. A., LaPlante, D. A., Nelson, S. E., & Stanton, M. V. (2004). The road less traveled: Moving from distribution to determinants in the study of gambling epidemiology. Canadian Journal of Psychiatry, 49, 504–516.
go back to reference Slutske, W. S., Piasecki, T. M., Blaszczynski, A., & Martin, N. G. (2010). Pathological gambling recovery in the absence of abstinence. Addiction, 105, 2169–2175.PubMedCentralPubMedCrossRef Slutske, W. S., Piasecki, T. M., Blaszczynski, A., & Martin, N. G. (2010). Pathological gambling recovery in the absence of abstinence. Addiction, 105, 2169–2175.PubMedCentralPubMedCrossRef
go back to reference Smits, N., Smit, F., Cuijpers, P., & de Graaf, R. (2007). Using decision theory to derive optimal cut-off scores of screening instruments: An illustration explicating costs and benefits of mental health screening. International Journal of Methods in Psychiatric Research, 16, 219–229.PubMedCrossRef Smits, N., Smit, F., Cuijpers, P., & de Graaf, R. (2007). Using decision theory to derive optimal cut-off scores of screening instruments: An illustration explicating costs and benefits of mental health screening. International Journal of Methods in Psychiatric Research, 16, 219–229.PubMedCrossRef
go back to reference Sobell, M. B., & Sobell, L. C. (1995). Controlled drinking after 25 years: How important was the great debate? Addiction, 90, 1149–1153.PubMedCrossRef Sobell, M. B., & Sobell, L. C. (1995). Controlled drinking after 25 years: How important was the great debate? Addiction, 90, 1149–1153.PubMedCrossRef
go back to reference Stinchfield, R. (2002). Reliability, validity, classification accuracy of the South Oaks Gambling Screen (SOGS). Addictive Behaviors, 27, 1–19.PubMedCrossRef Stinchfield, R. (2002). Reliability, validity, classification accuracy of the South Oaks Gambling Screen (SOGS). Addictive Behaviors, 27, 1–19.PubMedCrossRef
go back to reference Stinchfield, R. (2003). Reliability, validity, and classification accuracy of a measure of DSM-IV diagnostic criteria for pathological gambling. American Journal of Psychiatry, 160, 180–182.PubMedCrossRef Stinchfield, R. (2003). Reliability, validity, and classification accuracy of a measure of DSM-IV diagnostic criteria for pathological gambling. American Journal of Psychiatry, 160, 180–182.PubMedCrossRef
go back to reference Stinchfield, R., Govoni, R., & Frisch, G. R. (2007). A review of screening and assessment instruments for problem and pathological gambling. In G. Smith, D. Hodgins, & R. J. Williams (Eds.), Research and measurement issues in gambling studies (pp. 179–213). New York: Academic Press. Stinchfield, R., Govoni, R., & Frisch, G. R. (2007). A review of screening and assessment instruments for problem and pathological gambling. In G. Smith, D. Hodgins, & R. J. Williams (Eds.), Research and measurement issues in gambling studies (pp. 179–213). New York: Academic Press.
go back to reference Strong, D. R., & Kahler, C. W. (2007). Evaluation of the continuum of gambling problems. Addiction, 102, 713–721.PubMedCrossRef Strong, D. R., & Kahler, C. W. (2007). Evaluation of the continuum of gambling problems. Addiction, 102, 713–721.PubMedCrossRef
go back to reference Strong, D. R., Lesieur, H. R., Breen, H. B., Stinchfield, R., & Lejuez, C. W. (2004). Using the Rasch model to examine the utility of the South Oaks Gambling Screen across clinical and community samples. Addictive Behaviors, 29, 465–481.PubMedCrossRef Strong, D. R., Lesieur, H. R., Breen, H. B., Stinchfield, R., & Lejuez, C. W. (2004). Using the Rasch model to examine the utility of the South Oaks Gambling Screen across clinical and community samples. Addictive Behaviors, 29, 465–481.PubMedCrossRef
go back to reference Svetieva, E., & Walker, M. (2008). Inconsistency between concept and measurement: The Canadian Problem Gambling Index (CPGI). Journal of Gambling Issues, 22, 157–173.CrossRef Svetieva, E., & Walker, M. (2008). Inconsistency between concept and measurement: The Canadian Problem Gambling Index (CPGI). Journal of Gambling Issues, 22, 157–173.CrossRef
go back to reference Tang, C. S., Wu, A. M. S., Tang, J. Y. C., & Yan, E. C. W. (2010). Reliability, validity, and cut scores of the South Oaks Gambling Screen (SOGS) for Chinese. Journal of Gambling Studies, 26, 145–158.PubMedCentralPubMedCrossRef Tang, C. S., Wu, A. M. S., Tang, J. Y. C., & Yan, E. C. W. (2010). Reliability, validity, and cut scores of the South Oaks Gambling Screen (SOGS) for Chinese. Journal of Gambling Studies, 26, 145–158.PubMedCentralPubMedCrossRef
go back to reference Thomas, S., Jackson, A., & Blaszczynski, A. (2003). Measuring problem gambling: Evaluation of the Victorian gambling screen. Melbourne: Gambling Research Panel. Thomas, S., Jackson, A., & Blaszczynski, A. (2003). Measuring problem gambling: Evaluation of the Victorian gambling screen. Melbourne: Gambling Research Panel.
go back to reference Toce-Gerstein, M., Gerstein, D. R., & Volberg, R. A. (2009). The NODS-CLIP: A rapid for adult pathological and problem gambling. Journal of Gambling Studies, 25, 541–555.PubMedCentralPubMedCrossRef Toce-Gerstein, M., Gerstein, D. R., & Volberg, R. A. (2009). The NODS-CLIP: A rapid for adult pathological and problem gambling. Journal of Gambling Studies, 25, 541–555.PubMedCentralPubMedCrossRef
go back to reference Tolchard, B., & Battersby, M. W. (2010). The Victorian gambling screen: Reliability and validation in a clinical population. Journal of Gambling Studies, 26, 623–638.PubMedCrossRef Tolchard, B., & Battersby, M. W. (2010). The Victorian gambling screen: Reliability and validation in a clinical population. Journal of Gambling Studies, 26, 623–638.PubMedCrossRef
go back to reference Valenstein, P. N. (1990). Evaluating diagnostic tests with imperfect standards. American Journal of Clinical Pathology, 93, 252–258. Valenstein, P. N. (1990). Evaluating diagnostic tests with imperfect standards. American Journal of Clinical Pathology, 93, 252–258.
go back to reference Van Holst, R. J., van den Brink, W., & Veltman, D. J. (2010). Brain imaging studies in pathological gambling. Current Psychiatry Reports, 12, 418–425.PubMedCentralPubMedCrossRef Van Holst, R. J., van den Brink, W., & Veltman, D. J. (2010). Brain imaging studies in pathological gambling. Current Psychiatry Reports, 12, 418–425.PubMedCentralPubMedCrossRef
go back to reference Volberg, R. A. (2002, March). Gambling and problem gambling in Nevada. Report to the Nevada Department of Human Resources. Volberg, R. A. (2002, March). Gambling and problem gambling in Nevada. Report to the Nevada Department of Human Resources.
go back to reference Volberg, R. A. (2003, June). Gambling and Problem Gambling in Arizona. Report to the Arizona Lottery. Volberg, R. A. (2003, June). Gambling and Problem Gambling in Arizona. Report to the Arizona Lottery.
go back to reference Volberg, R. A., & Banks, S. M. (1990). A review of two measures of pathological gambling in the United States. Journal of Gambling Behavior, 6, 153–163. Volberg, R. A., & Banks, S. M. (1990). A review of two measures of pathological gambling in the United States. Journal of Gambling Behavior, 6, 153–163.
go back to reference Volberg, R. A. & Bernhard, B. (2006, June). The 2006 study of gambling and problem gambling in New Mexico. Report to the Responsible Gaming Association of New Mexico. Volberg, R. A. & Bernhard, B. (2006, June). The 2006 study of gambling and problem gambling in New Mexico. Report to the Responsible Gaming Association of New Mexico.
go back to reference Volberg, R. A. & Young, M. M. (2008, October). Using SOGS vs. CPGI in problem gambling screening and assessment. Commissioned by the Ontario Problem Gambling Research Centre. Volberg, R. A. & Young, M. M. (2008, October). Using SOGS vs. CPGI in problem gambling screening and assessment. Commissioned by the Ontario Problem Gambling Research Centre.
go back to reference Wainer, H., & Braun, H. I. (1988). Test validity. Hillsdale, NJ: Lawrence Erlbaum Associates. Wainer, H., & Braun, H. I. (1988). Test validity. Hillsdale, NJ: Lawrence Erlbaum Associates.
go back to reference Walker, M. & Blaszczynski, A. (2011, May). Clinical assessment of problem gamblers identified using the Canadian Problem Gambling Index. Report prepared by the University of Sydney Gambling Treatment Clinic and Gambling Research Unit for the South Australian Independent Gambling Authority. Walker, M. & Blaszczynski, A. (2011, May). Clinical assessment of problem gamblers identified using the Canadian Problem Gambling Index. Report prepared by the University of Sydney Gambling Treatment Clinic and Gambling Research Unit for the South Australian Independent Gambling Authority.
go back to reference Wenzel, M., McMillen, J., Marshall, D., & Ahmed, E. (2004). Validation of the Victorian gambling screen. Melbourne: Australian National University. Wenzel, M., McMillen, J., Marshall, D., & Ahmed, E. (2004). Validation of the Victorian gambling screen. Melbourne: Australian National University.
go back to reference Wiebe, J., Maitland, S. B., Hodgins, D., Davey, A., & Gottlieb, B. (2008). Transition and stability of problem gambling behaviours. Final report to the addiction foundation of Manitoba. Wiebe, J., Maitland, S. B., Hodgins, D., Davey, A., & Gottlieb, B. (2008). Transition and stability of problem gambling behaviours. Final report to the addiction foundation of Manitoba.
go back to reference Wiebe, J., Maitland, S. B., Hodgins, D., Davey, A., & Gottlieb, B. (2009). Transition and stability of problem gambling behaviours. Final report to the addiction foundation of Manitoba. Wiebe, J., Maitland, S. B., Hodgins, D., Davey, A., & Gottlieb, B. (2009). Transition and stability of problem gambling behaviours. Final report to the addiction foundation of Manitoba.
go back to reference Wiebe, J., & Volberg, R. A. (2007, December). Problem gambling prevalence research: A critical overview. A report to the Canadian Gaming Association. Wiebe, J., & Volberg, R. A. (2007, December). Problem gambling prevalence research: A critical overview. A report to the Canadian Gaming Association.
go back to reference Winters, K. C., Stinchfield, R. D., Botzet, A., & Anderson, N. (2002). A prospective study of youth gambling behavior. Psychology of Addictive Behaviors, 16, 3–9.PubMedCrossRef Winters, K. C., Stinchfield, R. D., Botzet, A., & Anderson, N. (2002). A prospective study of youth gambling behavior. Psychology of Addictive Behaviors, 16, 3–9.PubMedCrossRef
go back to reference Winters, K., Stinchfield, R., & Fulkerson, J. (1993). Toward the development of an adolescent gambling problem severity scale. Journal of Gambling Studies, 9, 63–84.CrossRef Winters, K., Stinchfield, R., & Fulkerson, J. (1993). Toward the development of an adolescent gambling problem severity scale. Journal of Gambling Studies, 9, 63–84.CrossRef
go back to reference Yerushalmy, J. (1947). Statistical problems in assessing methods of medical diagnosis, with special reference to X-ray techniques. Public Health Reports, 62, 1432–1449.PubMed Yerushalmy, J. (1947). Statistical problems in assessing methods of medical diagnosis, with special reference to X-ray techniques. Public Health Reports, 62, 1432–1449.PubMed
go back to reference Zhou, X., Obuchowski, N. A., & McClish, D. K. (2002). Statistical methods in diagnostic medicine. New York: Wiley and Sons.CrossRef Zhou, X., Obuchowski, N. A., & McClish, D. K. (2002). Statistical methods in diagnostic medicine. New York: Wiley and Sons.CrossRef
go back to reference Zou, K. H., O’Malley, A. J., & Mauri, L. (2007). Receiver-operating characteristic analysis for evaluating diagnostic tests and predictive models. Circulation, 115, 654–657.PubMedCrossRef Zou, K. H., O’Malley, A. J., & Mauri, L. (2007). Receiver-operating characteristic analysis for evaluating diagnostic tests and predictive models. Circulation, 115, 654–657.PubMedCrossRef
Metadata
Title
Setting Criterion Thresholds for Estimating Prevalence: What is Being Validated?
Author
Blase Gambino
Publication date
01-09-2014
Publisher
Springer US
Published in
Journal of Gambling Studies / Issue 3/2014
Electronic ISSN: 1573-3602
DOI
https://doi.org/10.1007/s10899-013-9380-y

Other articles of this Issue 3/2014

Journal of Gambling Studies 3/2014 Go to the issue