Top

Published in:

01-12-2018 | Original Paper

Test Performance Variation Between Settings and Populations

Author: Blase Gambino

Published in: Journal of Gambling Studies | Issue 4/2018

Abstract

The present paper is the third in a series on the evaluation of new tests designed to detect the disordered gambler. The present paper has two objectives. First, the observed variation in test performance between settings and populations is described in general terms and an explanation of the observed variation is presented in terms of what is referred to as spectrum effects. Second the expected variation in test performance between settings and populations is illustrated with several examples and the implications emphasized for the purpose of test evaluation.

Arguments that some tests do not include certain attributes or behaviors must keep in mind that tests are not designed to capture all those with a specific condition since spreading too wide a loop will catch many non-cases who share similar attributes. Such arguments must therefore rest on making a strong case for the relative clinical importance of the missing attribute or behavior.

It is difficult to characterize the relative severity of college students, should these students be considered along the age continuum, i.e., an extension of adolescent gamblers, or as a special population, e.g., seniors?

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: American Psychiatric Association.

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: American Psychiatric Association.CrossRef

Back, K., Williams, R. J., & Lee, C. (2015). Reliability and validity of three instruments (DSM-IV, CPGI, and PPGM) in the assessment of problem gambling in South Korea. Journal of Gambling Studies, 31, 775–786.CrossRef

Battersby, M. W., Thomas, L. J., Tolchard, B., & Esterman, A. (2002). The South Oaks gambling screen: A review with reference to Australian use. Journal of Gambling Studies, 18, 257–271.CrossRef

Bellringer, M., Abbott, M. W., Volberg, R. A., Garrett, N., & Coombes, R. (2007). Problem gambling assessment and screening instruments: Phase One report. Auckland: Gambling Research Centre, Auckland University of Technology.

Blinn-Pike, L., Worthy, S. L., & Jonkman, J. N. (2007). Disordered gambling among college students: A meta-analytic synthesis. Journal of Gambling Studies, 23, 175–183.CrossRef

Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., et al. (2003). Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD initiative. BMJ, 326, 41–44.CrossRef

Boudreau, B., & Poulin, C. (2006). The South Oaks Gambling Screen-Revised Adolescent (SOGS-RA) revisited: A cut-point analysis. Journal of Gambling Studies, 23, 299–308.CrossRef

Braun, B., Ludwig, M., Sleczka, P., Buhringer, G., & Kraus, L. (2014). Gamblers seeking treatment: Who does and who doesn’t? Journal of Behavioral Addictions, 3, 189–198.CrossRef

Brenner, H., & Gefeller, O. (1997). Variation of sensitivity, specificity, likelihood ratios and predictive values with disease prevalence. Statistics in Medicine, 16, 981–991.CrossRef

Calado, F., & Griffiths, M. D. (2016). Problem gambling worldwide: An update and systematic review of empirical research (2000–2015). Journal of Behavioral Addictions, 5, 592–613.CrossRef

Challet-Bouju, G., Perrot, B., Romo, L., et al. (2016). Harmonizing Screening for gambling problems in epidemiological surveys—Development of the rapid screener for problem gambling (RSPG). Journal of Behavioral Addictions, 5, 239–250.CrossRef

Cowlishaw, S., Merkouris, S., Chapman, A., & Radermacher, H. (2014). Pathological and problem gambling in substance abuse treatment. A systematic review and meta-analysis. Journal of Substance Abuse Treatment, 46, 98–105.CrossRef

Culleton, R. P. (1989). The prevalence rates of pathological gambling: A look at methods. Journal of Gambling Behaviors, 5, 22–41.CrossRef

Currie, S. K., Hodgins, D. C., Wang, J., el-Guebaly, N., Wynne, H., & Chen, S. (2006). Risk of harm among gamblers in the general population as a function of level of participation in gambling activities. Addiction, 101, 570–580.CrossRef

Derevensky, J. L., Gupta, R., & Winters, K. (2003). Prevalence rates of youth gambling problems: Are the current rates inflated? Journal of Gambling Studies, 19, 405–425.CrossRef

Di Ruffano, L. F., Hyde, C. J., McCaffery, K. J., Bossuyt, P. M. M., & Deeks, J. J. (2012). Assessing the value of diagnostic tests: A framework for designing and evaluating trials. BMJ, 344, 1–9.

Dickerson, M. G., Baron, E., Hong, S. M., & Cottrell, D. (1996). Estimating the extent and degree of gambling related problems in the Australian population: A national survey. Journal of Gambling Studies, 12, 161–178.CrossRef

Edgren, R., Castrén, S., Mäkelä, M., et al. (2016). Reliability of instruments measuring at-risk and problem gambling among young individuals: A systematic review covering years 2009–2015. Journal of Adolescent Health, 58, 600–615.CrossRef

Ferris, J., & Wynne, H. (2001). The Canadian Problem Gambling Index: Final report. Ottawa: Canadian Centre on Substance Abuse.

Gambino, B. (1997). The correction for bias in prevalence estimation with screening tests. Journal of Gambling Studies, 13, 343–351.CrossRef

Gambino, B. (2006). Reflections on accuracy. Journal of Gambling Studies, 22, 393–404.CrossRef

Gambino, B. (2007). A test of the false positive bias hypothesis. Journal of Gambling Studies, 23, 55–62.CrossRef

Gambino, B. (2012). The validation of screening tests: Meet the new screen same as the old screen? Journal of Gambling Studies, 28, 573–605.CrossRef

Gambino, B. (2014). Setting criterion thresholds for estimating prevalence: What is being validated? Journal of Gambling Studies, 30, 577–607.CrossRef

Gambino, B., & Lesieur, H. (2006). The South Oaks Gambling Screen (SOGS): A rebuttal to critics. Journal of Gambling Issues, 17. http://www.camh.net/egambling/issue17/gambino.html.

Gebauer, L., LaBrie, R., & Shaffer, H. J. (2010). Optimizing DSM-IV-TR classification accuracy: A brief biosocial screen for detecting current gambling disorders among gamblers in the general household population. Canadian Journal of Psychiatry, 55, 82–90.CrossRef

Gerstein, D., Hoffman, J., Larison, C., Murphy, S., Palmer, A., Chuchro, L. et al. (1999). Gambling impact and behavior study. Report to the National Gambling Impact Study Commission. Chicago: NORC.

Gotestam, K. G., Johansson, A., Wenzel, H. E., & Simensen, I. (2004). Validation of the Lie/Bet screen for pathological gambling on two normal population data sets. Psychological Reports, 95, 1009–1013.CrossRef

Grall-Brennec, M., Wainstein, L., Feuivillet, F., et al. (2012). Clinical profiles as a function of level and type of impulsivity in a sample group of at-risk and pathological gamblers. Journal of Gambling Behavior, 28, 239–252.CrossRef

Himelhoch, S. S., Miles-McLean, H., Medoff, D. R., et al. (2015). Evaluation of brief screens for gambling disorder in the substance use treatment setting. American Journal on Addictions, 24, 460–466.CrossRef

Johansson, A., Grant, J. E., Kim, S. W., Odlang, B. L., & Gotestam, G. (2009). Risk factors for problematic gambling: A critical literature review. Journal of Gambling Studies, 25, 67–92.CrossRef

Johnson, E. E., Hamer, R., Nora, R., Tan, B., Eisenstein, N., & Engelhart, C. (1997). The Lie/Bet questionnaire for screening pathological gamblers. Psychological Reports, 80, 83–88.CrossRef

Knottnerus, J. A., & van Weel, C. (2002). General introduction: Evaluation of diagnostic procedures. In J. A. Knottnerus (Ed.), The evidence base of clinical diagnosis (pp. 1–17). London: BMJ Publishing Group.

Ladouceur, R., Bouchard, C., Rheaume, N., et al. (2000). Is the SOGS an accurate measure of pathological gambling among children, adolescents and adults? Journal of Gambling Studies, 16, 1–24.CrossRef

Lasko, T. A., Bhagwat, J. G., Zou, K. H., & Ohno-Machado, L. (2005). The use of receiver operating characteristic curves in biomedical informatics. Journal of Biomedical Informatics, 38, 404–415.CrossRef

Lesieur, H. R., & Blume, S. B. (1987). South Oaks Gambling Screen (SOGS): A new instrument for the identification of pathological gamblers. American Journal of Psychiatry, 144, 1184–1188.CrossRef

Lijmer, J. G., Leeflang, M., & Bossuyt, P. M. M. (2009). Proposals for a phased evaluation of medical tests. Medical Decision Making, 29, E13–E21.CrossRef

Linnet, K., Bossuyt, P., Moons, K., & Reitsma, J. (2012). Quantifying the accuracy of a diagnostic test or marker. Clinical Chemistry, 58, 1292–1301.CrossRef

Milosevic, A., & Ledgerwood, D. M. (2010). The subtyping of pathological gambling: A comprehensive review. Clinical Psychology Review, 30, 988–998.CrossRef

Mulherin, S. A., & Miller, W. C. (2002). Spectrum bias or spectrum effect? Subgroup variation in diagnostic test evaluation. Annals of Internal Medicine, 137, 598–602.CrossRef

Nower, L., & Blaszczynski, A. (2017). Development and validation of the Gambling Pathways Questionnaire (GPQ). Psychology of Addictive Behaviors, 31, 95–109.CrossRef

Obuchowski, N. A. (2003). Receiver operating characteristic curves and their use in radiology. Radiology, 229, 3–8.CrossRef

Obuchowski, N. A. (2005). ROC analysis. American Journal of Roentgenology, 184, 364–372.CrossRef

Pepe, M. S. (2003). The statistical evaluation of medical tests for classification and prediction. Oxford: Oxford University Press.

Problem Gambling Research and Treatment Centre, PGRTC. (2011). Guidelines for screening, assessment, and treatment in problem gambling. Clayton: Monash University.

Ransohoff, D. F., & Feinstein, A. R. (1978). Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. New England Journal of Medicine, 299, 926–930.CrossRef

Reid, M. C., Lachs, M. S., & Feinstein, A. R. (1995). Use of methodological standards in diagnostic test research: Getting better but still not good. Journal of American Medical Association, 274, 645–651.CrossRef

Rennert, L., Denis, C., Peer, K., et al. (2014). DSM-5 gambling disorder: Prevalence and characteristics in a substance use disorder sample. Experimental and Clinical Psychopharmacology, 22, 50–56.CrossRef

Rodger, M., Ramsay, T., & Fergusson, D. (2012). Diagnostic randomized controlled trials: The final frontier. Trials, 13, 1–7.CrossRef

Sackett, D. L., & Haynes, R. B. (2002). Evidence base of clinical diagnosis: The architecture of diagnostic research. BMJ, 324, 539–541.CrossRef

Samet, S., Waxman, R., Hatzenbuehler, M., & Hasin, D. S. (2007). Assessing addiction: Concepts and instruments. Addiction Science & Clinical Practice, 4, 19–31.CrossRef

Saunders, J. B., & Cottler, L. B. (2007). The development of the Diagnostic and Statistical Manual of Mental Disorders Version V substance abuse disorders section: Establishing the research framework. Current Opinion Psychiatry, 20, 208–212.CrossRef

Smith, D., Harvey, P., Humeniuk, R., Battersby, M., & Pols, R. (2014). Effects of affective and anxiety disorders on outcome in problem gamblers attending routine cognitive behavioural treatment in South Australia. Journal of Gambling Studies, 31, 1069–1085.CrossRef

Spitalnic, S. (2001). Spectrum bias and clinical decision making. Hospital Physician, 37, 22–24.

Stinchfield, R. (2002). Reliability, validity, classification accuracy of the South Oaks Gambling Screen (SOGS). Addictive Behaviors, 27, 1–19.CrossRef

Stinchfield, R. (2010). A critical review of adolescent problem gambling assessment instruments. International Journal of Adolescent Medicine and Health, 22, 77–93.PubMed

Stinchfield, R. (2014). A review of problem gambling assessment instruments and brief screens. In D. C. S. Richard, A. Blaszczynski, & L. Nower (Eds.), Handbook of disordered gambling (pp. 165–203). New York: Wiley-Blackwell.

Stinchfield, R., Govoni, R. J., & Frisch, G. R. (2001). An evaluation of diagnostic criteria for pathological gambling. Windsor, Canada: Problem Gambling Research Group, University of Windsor.

Stinchfield, R., McCready, J., & Govoni, R. (2012a). Cross-validation of the Windsor Ontario Gambling Problem Severity Item pool. Toronto: Healthy Horizons Consulting.

Stinchfield, R., McCready, J., & Turner, N. (2012b). A comprehensive review of problem gambling screens and scales for online self-assessment. Toronto: Healthy Horizons Consulting.

Tang, C. S., Wu, A. M. S., Tang, J. Y. C., & Yan, E. C. W. (2010). Reliability, validity, and cut scores of the South Oaks Gambling Screen (SOGS) for Chinese. Journal of Gambling Studies, 26, 145–158.CrossRef

Temcheff, C. E., Paskus, T. S., Potenza, M. N., & Derevensky, J. L. (2016). Which diagnostic criteria are most useful in discriminating between social gamblers and individuals with gambling problems? An examination of DSM-IV and DSM-V criteria. Journal of Gambling Studies, 32, 957–968.CrossRef

Thomas, S., Jackson, A., & Blaszczynski, A. (2003). Measuring problem gambling: Evaluation of the Victorian gambling screen. Melbourne: Gambling Research Panel.

Toce-Gerstein, M., Gerstein, D. R., & Volberg, R. A. (2009). The NODS-CLIP: A rapid screen for adult pathological and problem gambling. Journal of Gambling Studies, 25, 541–555.CrossRef

Tremayne, K., Masterman-Smith, H., & McMillen, J. (2001). Survey of the nature and extent of gambling and problem gambling in the ACT. Sydney: Australian Institute for Gambling Research, University of Western Sydney.

Usher-Smith, J. A., Sharp, S. J., & Griffin, S. J. (2016). The spectrum effect in tests for risk prediction, screening, and diagnosis. BMJ, 353, 1–5.

Van den Bruel, A., Aertgeerts, B., & Buntix, F. (2006). Results of diagnostic accuracy studies are not always validated. Journal of Clinical Epidemiology, 59, 559–566.PubMed

Volberg, R. A., Munck, I. M., & Petry, N. M. (2011). A quick and simple screening method for pathological and problem gamblers in addiction programs and practices. The American Journal on Addictions, 20, 220–227.CrossRef

Whiting, P., Rutjes, A. W., Reitsma, J. B., Bossuyt, P. M., & Kleijnen, J. (2003). The development of QUADAS: A tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Medical Research Methodology, 3, 25.CrossRef

Williams, R. J., & Volberg, R. A. (2010). Best practices in the population assessment of problem gambling. Report prepared for the Ontario Problem Gambling Research Centre. Guelph, Ontario, Canada. March 31, 2010.

Williams, R. J., Volberg, R. A., & Stevens, R. M. G. (2012). The population prevalence of problem gambling: Methodological influences, standardized rates, jurisdictional differences, and worldwide trends. Report prepared for the Ontario Problem Gambling Research Centre and the Ontario Ministry of Health and Long Term Care. May 8, 2012.

Young, M. M., & Volberg, R. A. (2009). Using the NODS-CLiP to screen for gambling problems in Ontario primary health care settings. Report prepared for the Ontario Problem Gambling Research Centre. November.

Zhou, X., Obuchowski, N. A., & McClish, D. K. (2002). Statistical methods in diagnostic medicine. New York: Wiley.CrossRef

Zou, K. H., O’Malley, A. J., & Mauri, L. (2007). Receiver-operating characteristic analysis for evaluating diagnostic tests and predictive models. Circulation, 115, 654–657.CrossRef

Title: Test Performance Variation Between Settings and Populations
Author: Blase Gambino
Publication date: 01-12-2018
Publisher: Springer US
Published in: Journal of Gambling Studies / Issue 4/2018
Electronic ISSN: 1573-3602
DOI: https://doi.org/10.1007/s10899-017-9728-9

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Test Performance Variation Between Settings and Populations

Abstract

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 4/2018

Narcissistic Implications in Gambling Disorder: The Mediating Role of Emotion Dysregulation

Impulsivity and Gambling Type Among Treatment-Seeking Disordered Gamblers: An Explorative Study

Public Stigma and the Label of Gambling Disorder: Does it Make a Difference?

Self-Directed Gambling Changes: Trajectory of Problem Gambling Severity in Absence of Treatment

Overfeed the Bold, Starve the Cowardly: A Legend or Reality?

Comparative Test Evaluation: Methods and Challenges