Skip to main content
Top
Published in: Journal of Pediatric Neuropsychology 1/2022

01-03-2022 | Review

Overview of Modern Measurement Theory and Examples of Its Use to Measure Execution Function in Children

Authors: Carrie R. Houts, Andrea Savord, R. J. Wirth

Published in: Journal of Pediatric Neuropsychology | Issue 1/2022

Login to get access

Abstract

Researchers and clinicians in neuropsychology are interested in measuring constructs that cannot be directly observed, such as cognitive ability, executive function, or depression. Modern measurement theory (MMT) provides a framework for understanding and measuring these constructs (also called latent variables) through the use of test batteries and other assessments that provide observable responses that are theoretically related to the underlying construct of interest. MMT subsumes a number of statistical methods, including factor analysis and item response theory (IRT), that are focused on using observable responses to model these unobservable constructs. These methods are able to provide information on how test batteries or individual items relate to each other and the latent variable across varying levels of the construct of interest (e.g., mild ADHD and severe ADHD). Finally, we review several applications of IRT to the measurement of executive function in children to highlight the unique information IRT provides when evaluating the relationship of observed responses to unobservable constructs. In summary, this paper seeks to provide an accessible overview of latent variable models within MMT (with a focus on IRT) that could improve measurement in neuropsychology, if more widely used, and offer readers the foundations of a knowledge base to become more critical consumers of measurement-related research in pediatric neuropsychology.
Literature
go back to reference Byrne, B. M. (2005). Factor analytic models: Viewing the structure of an assessment instrument from three perspectives. Journal of personality assessment, 85(1), 17–32.CrossRefPubMed Byrne, B. M. (2005). Factor analytic models: Viewing the structure of an assessment instrument from three perspectives. Journal of personality assessment, 85(1), 17–32.CrossRefPubMed
go back to reference Cai, L., & Houts, C. R. (2021). Longitudinal analysis of patient reported outcomes in clinical trials: Applications of multilevel and multidimensional item response theory. Psychometrika, 86(3), 754–777.CrossRef Cai, L., & Houts, C. R. (2021). Longitudinal analysis of patient reported outcomes in clinical trials: Applications of multilevel and multidimensional item response theory. Psychometrika, 86(3), 754–777.CrossRef
go back to reference Cronbach, L. J. (1947). Test “reliability”: Its meaning and determination. Psychometrika, 12(1), 1–16.CrossRefPubMed Cronbach, L. J. (1947). Test “reliability”: Its meaning and determination. Psychometrika, 12(1), 1–16.CrossRefPubMed
go back to reference De Champlain, A. F. (2010). A primer on classical test theory and item response theory for assessments in medical education. Medical Education, 44, 109–117.CrossRef De Champlain, A. F. (2010). A primer on classical test theory and item response theory for assessments in medical education. Medical Education, 44, 109–117.CrossRef
go back to reference Edwards, M. C. (2009). An introduction to item response theory using the need for cognition scale. Social and Personality Psychology Compass, 3(4), 507–529.CrossRef Edwards, M. C. (2009). An introduction to item response theory using the need for cognition scale. Social and Personality Psychology Compass, 3(4), 507–529.CrossRef
go back to reference Edwards, M. C., & Houts, C. R. (2017). Conceptual overview of computerized adaptive testing. Value and Outcomes Spotlight, 3, 11–12. Edwards, M. C., & Houts, C. R. (2017). Conceptual overview of computerized adaptive testing. Value and Outcomes Spotlight, 3, 11–12.
go back to reference Embretson, S. E. (1996). The new rules of measurement. Psychological Assessment, 8(4), 341–349.CrossRef Embretson, S. E. (1996). The new rules of measurement. Psychological Assessment, 8(4), 341–349.CrossRef
go back to reference Gibbons, R. D., & Hedeker, D. (1992). Full-information item bifactor analysis. Psychometrika, 57, 423–436.CrossRef Gibbons, R. D., & Hedeker, D. (1992). Full-information item bifactor analysis. Psychometrika, 57, 423–436.CrossRef
go back to reference Howard, M. C. (2016). A review of exploratory factor analysis decision and overview of current practices: What are we doing and how can we improve? International Journal of Human-Computer Interaction, 32(1), 51–62.CrossRef Howard, M. C. (2016). A review of exploratory factor analysis decision and overview of current practices: What are we doing and how can we improve? International Journal of Human-Computer Interaction, 32(1), 51–62.CrossRef
go back to reference Karr, J. E., Areshenkoff, C. N., Rast, P., Hofer, S. M., Iverson, G. L., & Garcia-Barrera, M. A. (2018). The unity and diversity of executive functions: A systematic review and re-analysis of latent variable studies. Psychological bulletin, 144(11), 1147. Karr, J. E., Areshenkoff, C. N., Rast, P., Hofer, S. M., Iverson, G. L., & Garcia-Barrera, M. A. (2018). The unity and diversity of executive functions: A systematic review and re-analysis of latent variable studies. Psychological bulletin, 144(11), 1147.
go back to reference Langer, M. M., Hill, C. D., Thissen, D., Burwinkle, T. M., Varni, J. W., & DeWalt, D. A. (2008). Item response theory detected differential functioning between healthy and ill children in quality of life metrics. Journal of Clinical Epidemiology, 61(3), 268–276.CrossRef Langer, M. M., Hill, C. D., Thissen, D., Burwinkle, T. M., Varni, J. W., & DeWalt, D. A. (2008). Item response theory detected differential functioning between healthy and ill children in quality of life metrics. Journal of Clinical Epidemiology, 61(3), 268–276.CrossRef
go back to reference Lecavalier, L., Gadow, K. D., Devincent, C. J., Houts, C. R., & Edwards, M. C. (2011). Validity of DSM-IV syndromes in preschoolers with autism spectrum disorders. Autism, 15(5), 527–543.CrossRef Lecavalier, L., Gadow, K. D., Devincent, C. J., Houts, C. R., & Edwards, M. C. (2011). Validity of DSM-IV syndromes in preschoolers with autism spectrum disorders. Autism, 15(5), 527–543.CrossRef
go back to reference MacCallum, R. C. (2007). Factor analysis. In R. E. Millsap & A. Maydeu-Olivares (Eds.), The Sage Handbook of Quantitative Methods in Psychology (pp. 123–147). Sage. MacCallum, R. C. (2007). Factor analysis. In R. E. Millsap & A. Maydeu-Olivares (Eds.), The Sage Handbook of Quantitative Methods in Psychology (pp. 123–147). Sage.
go back to reference Nunnally, J. C. (1978). Psychometric Theory (2nd ed.). McGraw-Hill. Nunnally, J. C. (1978). Psychometric Theory (2nd ed.). McGraw-Hill.
go back to reference Ryan, R. M., Fauth, R. C., & Brooks-Gunn, J. (2006). Childhood Poverty: Implications for School Readiness and Early Childhood Education. In B. Spodek & O. N. Saracho (Eds.), Handbook of research on the education of young children (pp. 323–346). Lawrence Erlbaum Associates Publishers. Ryan, R. M., Fauth, R. C., & Brooks-Gunn, J. (2006). Childhood Poverty: Implications for School Readiness and Early Childhood Education. In B. Spodek & O. N. Saracho (Eds.), Handbook of research on the education of young children (pp. 323–346). Lawrence Erlbaum Associates Publishers.
go back to reference Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometric Monograph Supplement, 34(4, Pt. 2), 100. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometric Monograph Supplement, 34(4, Pt. 2), 100.
go back to reference Teresi, J. A., Ocepek-Welikson, K., Kleinman, M., Ramirez, M., & Kim, G. (2016). Psychometric properties and performance of the patient reported outcomes measurement information system ® (PROMIS®) depression short forms in ethnically diverse groups. Psychological Test and Assessment Modeling, 58(1), 141–181.PubMedPubMedCentral Teresi, J. A., Ocepek-Welikson, K., Kleinman, M., Ramirez, M., & Kim, G. (2016). Psychometric properties and performance of the patient reported outcomes measurement information system ® (PROMIS®) depression short forms in ethnically diverse groups. Psychological Test and Assessment Modeling, 58(1), 141–181.PubMedPubMedCentral
go back to reference Thissen, D. & Orlando, M., (2001). Item response theory for items scores in two categories. In D. Thissen & HJ. Wainer (Eds). Test Scoring (pp. 73–140). Mahwah, NJ: Lawrence Erlbaum Associates Thissen, D. & Orlando, M., (2001). Item response theory for items scores in two categories. In D. Thissen & HJ. Wainer (Eds). Test Scoring (pp. 73–140). Mahwah, NJ: Lawrence Erlbaum Associates
go back to reference Thissen, D., Steinberg, L., & Wainer, H. (1993). Detection of differential item functioning using the parameters of item response models. In P. W. Holland & H. Wainer (Eds.), Differential Item Functioning (pp. 67–113). Lawrence Erlbaum Associates, Inc. Thissen, D., Steinberg, L., & Wainer, H. (1993). Detection of differential item functioning using the parameters of item response models. In P. W. Holland & H. Wainer (Eds.), Differential Item Functioning (pp. 67–113). Lawrence Erlbaum Associates, Inc.
go back to reference Thurstone, L. L. (1938). Primary mental abilities. Psychometric Monographs, 1, ix + 121. Thurstone, L. L. (1938). Primary mental abilities. Psychometric Monographs, 1, ix + 121.
go back to reference Wainer, H., Dorans, N. J., Green, B. F., Steinberg, L., Flaugher, R., Mislevy, R. J., & Thissen, D. (2000). Computerized adaptive testing: A primer (2nd edition). Lawrence Erlbaum Associates, Inc Wainer, H., Dorans, N. J., Green, B. F., Steinberg, L., Flaugher, R., Mislevy, R. J., & Thissen, D. (2000). Computerized adaptive testing: A primer (2nd edition). Lawrence Erlbaum Associates, Inc
go back to reference Welsh, J. A., Nix, R. L., Blair, C., Bierman, K. L., & Nelson, K. E. (2010). The development of cognitive skills and gains in academic school readiness for children from low-income families. Journal of Educational Psychology, 102(1), 43–53.CrossRef Welsh, J. A., Nix, R. L., Blair, C., Bierman, K. L., & Nelson, K. E. (2010). The development of cognitive skills and gains in academic school readiness for children from low-income families. Journal of Educational Psychology, 102(1), 43–53.CrossRef
go back to reference Widaman, K. F. (2012). Exploratory factor analysis and confirmatory factor analysis. In H. Cooper, P. M. Camic, D. L. Long, A. T. Panter, D. Rindskopf, & K. J. Sher (Eds.), APA handbook of research methods in psychology, Vol. 3. Data analysis and research publication (pp. 361–389). American Psychological Association. https://doi.org/10.1037/13621-018 Widaman, K. F. (2012). Exploratory factor analysis and confirmatory factor analysis. In H. Cooper, P. M. Camic, D. L. Long, A. T. Panter, D. Rindskopf, & K. J. Sher (Eds.), APA handbook of research methods in psychology, Vol. 3. Data analysis and research publication (pp. 361–389). American Psychological Association. https://​doi.​org/​10.​1037/​13621-018
go back to reference Willoughby, M. T., Blair, C. B., Wirth, R. J., Greenberg, M., & The Family Life Project Investigators. (2010). The measurement of executive function at age 3 years: Psychometric properties and criterion validity of a new battery of tasks. Psychological Assessment, 22(2), 306–317.CrossRef Willoughby, M. T., Blair, C. B., Wirth, R. J., Greenberg, M., & The Family Life Project Investigators. (2010). The measurement of executive function at age 3 years: Psychometric properties and criterion validity of a new battery of tasks. Psychological Assessment, 22(2), 306–317.CrossRef
go back to reference Willoughby, M. T., Blair, C. B., Wirth, R. J., Greenberg, M., & The Family Life Project Investigators. (2012). The measurement of executive function at age 5: Psychometric properties and relationship to academic achievement. Psychological Assessment, 24(1), 226–239.CrossRef Willoughby, M. T., Blair, C. B., Wirth, R. J., Greenberg, M., & The Family Life Project Investigators. (2012). The measurement of executive function at age 5: Psychometric properties and relationship to academic achievement. Psychological Assessment, 24(1), 226–239.CrossRef
go back to reference Willoughby, M. T., Wirth, R. J., & Blair, C. B. (2011). Contributions of modern measurement theory to measuring executive function in early childhood: An empirical demonstration. Journal of Experimental Child Psychology, 108, 414–435.CrossRef Willoughby, M. T., Wirth, R. J., & Blair, C. B. (2011). Contributions of modern measurement theory to measuring executive function in early childhood: An empirical demonstration. Journal of Experimental Child Psychology, 108, 414–435.CrossRef
go back to reference Wirth, R. J., & Edwards, M. C. (2007). Item factor analysis: Current approaches and future directions. Psychological Measurement, 67, 73–87. Wirth, R. J., & Edwards, M. C. (2007). Item factor analysis: Current approaches and future directions. Psychological Measurement, 67, 73–87.
Metadata
Title
Overview of Modern Measurement Theory and Examples of Its Use to Measure Execution Function in Children
Authors
Carrie R. Houts
Andrea Savord
R. J. Wirth
Publication date
01-03-2022
Publisher
Springer International Publishing
Published in
Journal of Pediatric Neuropsychology / Issue 1/2022
Print ISSN: 2199-2681
Electronic ISSN: 2199-2673
DOI
https://doi.org/10.1007/s40817-021-00117-7

Other articles of this Issue 1/2022

Journal of Pediatric Neuropsychology 1/2022 Go to the issue