Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2011

Open Access 01-12-2011 | Research article

An accurate and efficient identification of children with psychosocial problems by means of computerized adaptive testing

Authors: Antonius GC Vogels, Gert W Jacobusse, Symen A Reijneveld

Published in: BMC Medical Research Methodology | Issue 1/2011

Login to get access

Abstract

Background

Questionnaires used by health services to identify children with psychosocial problems are often rather short. The psychometric properties of such short questionnaires are mostly less than needed for an accurate distinction between children with and without problems. We aimed to assess whether a short Computerized Adaptive Test (CAT) can overcome the weaknesses of short written questionnaires when identifying children with psychosocial problems.

Method

We used a Dutch national data set obtained from parents of children invited for a routine health examination by Preventive Child Healthcare with 205 items on behavioral and emotional problems (n = 2,041, response 84%). In a random subsample we determined which items met the requirements of an Item Response Theory (IRT) model to a sufficient degree. Using those items, item parameters necessary for a CAT were calculated and a cut-off point was defined. In the remaining subsample we determined the validity and efficiency of a Computerized Adaptive Test using simulation techniques, with current treatment status and a clinical score on the Total Problem Scale (TPS) of the Child Behavior Checklist as criteria.

Results

Out of 205 items available 190 sufficiently met the criteria of the underlying IRT model. For 90% of the children a score above or below cut-off point could be determined with 95% accuracy. The mean number of items needed to achieve this was 12. Sensitivity and specificity with the TPS as a criterion were 0.89 and 0.91, respectively.

Conclusion

An IRT-based CAT is a very promising option for the identification of psychosocial problems in children, as it can lead to an efficient, yet high-quality identification. The results of our simulation study need to be replicated in a real-life administration of this CAT.
Appendix
Available only for authorised users
Literature
1.
Brugman E, Reijneveld SA, Verhulst FC, Verloove-Vanhorick SP: Identification and management of psychosocial problems by preventive child health care. Arch Pediatr Adolesc Med. 2001, 155: 462-469.CrossRefPubMed
2.
Kelleher KJ, McInerny TK, Gardner WP, Childs GE, Wasserman RC: Increasing identification of psychosocial problems: 1979-1996. Pediatrics. 2000, 105: 1313-1321. 10.1542/peds.105.6.1313.CrossRefPubMed
3.
Meltzer H, Gatward R, Goodman R, Ford T: Mental health of children and adolescents in Great Britain. Int Rev Psychiatry. 2003, 15: 185-187. 10.1080/0954026021000046155.CrossRefPubMed
4.
Moffitt TE, Caspi A: Childhood predictors differentiate life-course persistent and adolescence-limited antisocial pathways among males and females. Development and Psychopathology. 2001, 355-375.
5.
Verhulst FC, Van der Ende J, Rietbergen A: Ten-year time trends of psychopathology in Dutch children and adolescents: no evidence for strong trends. Acta Psychiatr Scand. 1997, 96: 7-13.CrossRefPubMed
6.
Durlak JA, Wells AM: Primary prevention mental health programs for children and adolescents: a meta-analytic review. Am J Community Psychol. 1997, 25: 115-152. 10.1023/A:1024654026646.CrossRefPubMed
7.
Costello EJ, Janizewski S: Who gets treated? Factors associated with referral in children with psychiatric disorders. Acta Psychiatr Scand. 1990, 81: 523-529. 10.1111/j.1600-0447.1990.tb05492.x.CrossRefPubMed
8.
Glascoe FP: Using parents' concerns to detect and address developmental and behavioral problems. J Soc Pediatr Nurs. 1999, 4: 24-35. 10.1111/j.1744-6155.1999.tb00077.x.CrossRefPubMed
9.
Glascoe FP: Screening for developmental and behavioral problems. Ment Retard Dev Disabil Res Rev. 2005, 11: 173-179. 10.1002/mrdd.20068.CrossRefPubMed
10.
Lavigne JV, Binns HJ, Christoffel KK, Rosenbaum D, Arend R, Smith K, Hayford JR, McGuire PA: Behavioral and emotional problems among preschool children in pediatric primary care: prevalence and pediatricians' recognition. Pediatric Practice Research Group. Pediatrics. 1993, 91: 649-655.PubMed
11.
Reijneveld SA, Brugman E, Verhulst FC, Verloove-Vanhorick SP: Identification and management of psychosocial problems among toddlers in Dutch preventive Child Health Care. Arch Pediatr Adolesc Med. 2004, 158: 811-817. 10.1001/archpedi.158.8.811.CrossRefPubMed
12.
Wiefferink CH, Reijneveld SA, de Wijs J, Swagerman M, Campman D, Paulussen TG: Screening for psychosocial problems in 5-6-year olds: a randomised controlled trial of routine health assessments. Patient Educ Couns. 2006, 60: 57-65. 10.1016/j.pec.2004.11.013.CrossRefPubMed
13.
Reijneveld SA, Vogels AG, Brugman E, Ede van J, Verhulst FC, Verloove-Vanhorick SP: Early detection of psychosocial problems in adolescents: how useful is the Dutch short indicative questionnaire (KIVPA)?. Eur J Public Health. 2003, 13: 152-159. 10.1093/eurpub/13.2.152.CrossRefPubMed
14.
Vogels T, Reijneveld SA, Brugman E, Hollander-Gijsman M, Verhulst FC, Verloove-Vanhorick SP: Detecting psychosocial problems among 5-6-year-old children in preventive Child Health Care: the validity of a short questionnaire used in an assessment procedure for detecting psychosocial problems among children. Eur J Public Health. 2003, 13: 353-360. 10.1093/eurpub/13.4.353.CrossRefPubMed
15.
Weitzman CC, Leventhal JM: Screening for behavioral health problems in primary care. Curr Opin Pediatr. 2006, 18: 641-648. 10.1097/MOP.0b013e3280108292.CrossRefPubMed
16.
Embretson SE, Reise SP: Item Response Theory for psychologists. 2006, London: Lawrence Elbaum Associate Publishers
17.
Wright D, Masters G: Rating Scale Analysis. 1982, Institute for Objective Measurement
18.
Revicki DA, Cella DF: Health status assessment for the twenty-first century: item response theory, item banking and computer adaptive testing. Qual Life Res. 1997, 6: 595-600. 10.1023/A:1018420418455.CrossRefPubMed
19.
Fliege H, Becker J, Walter OB, Bjorner JB, Klapp BF, Rose M: Development of a computer-adaptive test for depression (D-CAT). Qual Life Res. 2005, 14: 2277-2291. 10.1007/s11136-005-6651-9.CrossRefPubMed
20.
Gardner W, Kelleher KJ, Pajer KA: Multidimensional adaptive testing for mental health problems in primary care. Med Care. 2002, 40: 812-823. 10.1097/00005650-200209000-00010.CrossRefPubMed
21.
Gardner W, Shear K, Kelleher KJ, Pajer KA, Mammen O, Buysse D, Frank E: Computerized adaptive measurement of depression: a simulation study. BMC Psychiatry. 2004, 4: 13-10.1186/1471-244X-4-13.CrossRefPubMedPubMedCentral
22.
Hill CD, Edwards MC, Thissen D, Langer MM, Wirth RJ, Burwinkle TM, Varni JW: Practical issues in the application of item response theory: a demonstration using items from the pediatric quality of life inventory (PedsQL) 4.0 generic core scales. Med Care. 2007, 45: S39-S47. 10.1097/01.mlr.0000259879.05499.eb.CrossRefPubMed
23.
Wainer H: Computerized Adaptive Testing. A Primer. 1990, Hillsdale: Lawrence Erlbaum Associates, Publishers
24.
Vogels AG, Crone MR, Hoekstra F, Reijneveld SA: Comparing three short questionnaires to detect psychosocial dysfunction among primary school children: a randomized method. BMC Public Health. 2009, 9: 489-10.1186/1471-2458-9-489.CrossRefPubMedPubMedCentral
25.
Jellinek M, Little M, Murphy JM, Pagano M: The Pediatric Symptom Checklist. Support for a role in a managed care environment. Arch Pediatr Adolesc Med. 1995, 149: 740-746.CrossRefPubMed
26.
Jellinek M, Murphy JM, Robinson J, Feins A, Lamb S, Fenton T: Pediatric Symptom Checklist: screening schoolage children for psychosocial dysfunction. J Pediatr. 1988, 112: 201-209. 10.1016/S0022-3476(88)80056-8.CrossRefPubMed
27.
Jellinek MS, Murphy JM: Use of the Pediatric Symptom Checklist in outpatient practice. Curr Probl Pediatr. 1990, 602-609.
28.
Bourdon KH, Goodman R, Rae DS, Simpson G, Koretz DS: The Strengths and Difficulties Questionnaire: U.S. normative data and psychometric properties. J Am Acad Child Adolesc Psychiatry. 2005, 44: 557-564. 10.1097/01.chi.0000159157.57075.c8.CrossRefPubMed
29.
Goodman R, Scott S: Comparing the Strengths and Difficulties Questionnaire and the Child Behavior Checklist: is small beautiful?. J Abnorm Child Psychol. 1999, 27: 17-24. 10.1023/A:1022658222914.CrossRefPubMed
30.
Goodman R, Ford T, Simmons H, Gatward R, Meltzer H: Using the Strengths and Difficulties Questionnaire (SDQ) to screen for child psychiatric disorders in a community sample. Br J Psychiatry. 2000, 177: 534-539. 10.1192/bjp.177.6.534.CrossRefPubMed
31.
Goodman R, Ford T, Richards H, Gatward R, Meltzer H: The Development and Well-Being Assessment: description and initial validation of an integrated assessment of child and adolescent psychopathology. J Child Psychol Psychiatry. 2000, 41: 645-655. 10.1111/j.1469-7610.2000.tb02345.x.CrossRefPubMed
32.
Van Ede J, De Rover C, Verhappen MJ, De Vries-Lequin I: PSYBOBA. een pilotstudy naar de mogelijkheden van gebruik van een signaleringsinstrument voor Psychosociale Problematiek in de Bovenbouw van het Basisonderwijs. 2003, Utrecht: GGD Nederland
33.
Crone MR, Vogels AG, Hoekstra F, Treffers PD, Reijneveld SA: A comparison of four scoring methods based on the parent-rated Strengths and Difficulties Questionnaire as used in the Dutch preventive child health care system. BMC Public Health. 2008, 8: 106-10.1186/1471-2458-8-106.CrossRefPubMedPubMedCentral
34.
Van Widenfelt BM, Goedhart AW, Treffers PD, Goodman R: Dutch version of the Strengths and Difficulties Questionnaire (SDQ). Eur Child Adolesc Psychiatry. 2003, 12: 281-289. 10.1007/s00787-003-0341-3.CrossRefPubMed
35.
Guillemin F, Bombardier C, Beaton D: Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993, 46: 1417-1432. 10.1016/0895-4356(93)90142-N.CrossRefPubMed
36.
Reijneveld SA, Vogels AG, Hoekstra F, Crone MR: Use of the Pediatric Symptom Checklist for the detection of psychosocial problems in preventive child healthcare. BMC Public Health. 2006, 6: 197-10.1186/1471-2458-6-197.CrossRefPubMedPubMedCentral
37.
Verhulst FC, Koot HM: Manual for the CBCL/4-18 [in Dutch]. 1996, Rotterdam: EUR/Sophia Kinderziekenhuis
38.
Andrach D: Rasch models for measurement. 1988, Newbury Park: Sage PublicationsCrossRef
39.
Andric D, Luo G: Conditional pairwise estimation in the Rasch model for ordered response categories using principal components. J Appl Meas. 2003, 4 (3): 205-221.
40.
Jodoin MG, Gierl MJ: Evaluating Type I error and power rates using an effect size measure with the logistic regression procedure for DIF detection. Applied Measurement in Education. 2001, 14: 329-349. 10.1207/S15324818AME1404_2.CrossRef
41.
Koot HM, Verhulst FC: Prediction of children's referral to mental health and special education services from earlier adjustment. J Child Psychol Psychiatry. 1992, 33: 717-729. 10.1111/j.1469-7610.1992.tb00907.x.CrossRefPubMed
42.
Verhulst FC, Dekker MC, Van der Ende J: Parent, teacher and self-reports as predictors of signs of disturbance in adolescents: whose information carries the most weight?. Acta Psychiatr Scand. 1997, 96: 75-81. 10.1111/j.1600-0447.1997.tb09909.x.CrossRefPubMed
43.
Verhulst FC, Van der Ende J: Six-year stability of parent-reported problem behavior in an epidemiological sample. J Abnorm Child Psychol. 1992, 20: 595-610. 10.1007/BF00911243.CrossRefPubMed
44.
Verhulst FC, Koot HM, Van der Ende J: Differential predictive value of parents' and teachers' reports of children's problem behaviors: a longitudinal study. J Abnorm Child Psychol. 1994, 22: 531-546. 10.1007/BF02168936.CrossRefPubMed
45.
Vestering N, Buuren Sv: Two New Bayesian item selection methods and their application in Computerized Classification Testing. 2004, Leiden: TNO Quality of Life
46.
Waller NG, Reise SP: Computerized adaptive personality assessment: an illustration with the Absorption scale. J Pers Soc Psychol. 2006, 57: 1051-1058.CrossRef
47.
Ware JE, Bjorner JB, Kosinski M: Practical implications of item response theory and computerized adaptive testing: a brief summary of ongoing studies of widely used headache impact scales. Med Care. 2000, 38: II73-II82.CrossRefPubMed
48.
Hacker KA, Myagmarjav E, Harris V, Suglia SF, Weidner D, Link D: Mental health screening in pediatric practice: factors related to positive screens and the contribution of parental/personal concern. Pediatrics. 2006, 118: 1896-1906. 10.1542/peds.2006-0026.CrossRefPubMed
Metadata
Title
An accurate and efficient identification of children with psychosocial problems by means of computerized adaptive testing
Authors
Antonius GC Vogels
Gert W Jacobusse
Symen A Reijneveld
Publication date
01-12-2011
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2011
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/1471-2288-11-111

Other articles of this Issue 1/2011

BMC Medical Research Methodology 1/2011 Go to the issue

Research article

Bikeability and methodological issues using the active commuting route environment scale (ACRES) in a metropolitan setting

Research article

How the psychosocial context of clinical trials differs from usual care: A qualitative study of acupuncture patients

Research article

Self-perceived quality of life predicts mortality risk better than a multi-biomarker panel, but the combination of both does best

Research article

Methodology in conducting a systematic review of systematic reviews of healthcare interventions

Research article

Parametric methods outperformed non-parametric methods in comparisons of discrete numerical variables

Research article

Pharmacy sales data versus ward stock accounting for the surveillance of broad-spectrum antibiotic use in hospitals