Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on medication adherence

LIVE: Thursday 27th June 2024, 18:00-19:30 (CEST)
Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2016

Open Access 01-12-2016 | Research article

An evaluation of computerized adaptive testing for general psychological distress: combining GHQ-12 and Affectometer-2 in an item bank for public mental health research

Authors: Jan Stochl, Jan R. Böhnke, Kate E. Pickett, Tim J. Croudace

Published in: BMC Medical Research Methodology | Issue 1/2016

Login to get access

Abstract

Background

Recent developments in psychometric modeling and technology allow pooling well-validated items from existing instruments into larger item banks and their deployment through methods of computerized adaptive testing (CAT). Use of item response theory-based bifactor methods and integrative data analysis overcomes barriers in cross-instrument comparison. This paper presents the joint calibration of an item bank for researchers keen to investigate population variations in general psychological distress (GPD).

Methods

Multidimensional item response theory was used on existing health survey data from the Scottish Health Education Population Survey (n = 766) to calibrate an item bank consisting of pooled items from the short common mental disorder screen (GHQ-12) and the Affectometer-2 (a measure of “general happiness”). Computer simulation was used to evaluate usefulness and efficacy of its adaptive administration.

Results

A bifactor model capturing variation across a continuum of population distress (while controlling for artefacts due to item wording) was supported. The numbers of items for different required reliabilities in adaptive administration demonstrated promising efficacy of the proposed item bank.

Conclusions

Psychometric modeling of the common dimension captured by more than one instrument offers the potential of adaptive testing for GPD using individually sequenced combinations of existing survey items. The potential for linking other item sets with alternative candidate measures of positive mental health is discussed since an optimal item bank may require even more items than these.
Appendix
Available only for authorised users
Footnotes
1
The selection of modelling a specific factor for negatively or positively worded items is arbitrary and depends on the selection of “reference wording”. We selected the negative wording as our reference type of wording.
 
2
Subject to considerations regarding copyright permissions.
 
Literature
1.
Goldberg DP, Williams P. A user's guide to the General Health Questionnaire. Windsor UK: NFER-Nelson; 1988.
2.
McDowell I. Measuring health: A guide to rating scales and questionnaires. New York: Oxford University Press; 2006.
3.
Stewart-Brown S. Defining and measuring mental health and wellbeing. In: Knifton L, Quinn N, editors. Public mental health: global perspectives. edn. New York: McGraw Hill Open University Press; 2013. p. 33–42.
4.
Lindert J, Bain PA, Kubzansky LD, Stein C. Well-being measurement and the WHO health policy Health 2010: systematic review of measurement scales. Eur J Public Health. 2015;25(4):731–40.CrossRefPubMed
5.
Wahl I, Löwe B, Bjorner JB, Fischer F, Langs G, Voderholzer U, Aita SA, Bergemann N, Brähler E, Rose M. Standardization of depression measurement: a common metric was developed for 11 self-report depression measures. J Clin Epidemiol. 2014;67(1):73–86
6.
Weich S, Brugha T, King M, McManus S, Bebbington P, Jenkins R, Cooper C, McBride O, Stewart-Brown S. Mental well-being and mental illness: findings from the Adult Psychiatric Morbidity Survey for England 2007. Br J Psychiatry. 2011;199(1):23–8.CrossRefPubMed
7.
Gibbons RD, Perraillon MC, Kim JB. Item response theory approaches to harmonization and research synthesis. Health Serv Outcomes Res Methodol. 2014;14(4):213–31.CrossRefPubMedPubMedCentral
8.
Böhnke JR, Croudace TJ. Calibrating well-being, quality of life and common mental disorder items: psychometric epidemiology in public mental health research. Br J Psychiatry. 2015. doi:10.​1192/​bjp.​bp.​115.​165530.
9.
10.
Bauer DJ, Hussong AM. Psychometric approaches for developing commensurate measures across independent studies: traditional and new models. Psychol Methods. 2009;14(2):101–25.CrossRefPubMedPubMedCentral
11.
Wainer H, Dorans NJ, Flaugher R, Green BF, Mislevy RJ. Computerized adaptive testing: A primer. Hillsdale, NJ: Lawrence Erlbaum; 2000.
12.
Böhnke JR, Lutz W. Using item and test information to optimize targeted assessments of psychological distress. Assessment. 2014;21(6):679–93.CrossRefPubMed
13.
Hankins M. The factor structure of the twelve item General Health Questionnaire (GHQ-12): The result of negative phrasing? Clin Pract Epidemiol Ment Health. 2008;4(1):10.CrossRefPubMedPubMedCentral
14.
Egberink IJL, Meijer RR. An item response theory analysis of Harter’s Self-Perception Profile for children or why strong clinical scales should be distrusted. Assessment. 2011;18(2):201–12.CrossRefPubMed
15.
Goldberg DP. The detection of psychiatric illness by questionnaire. London: Oxford University Press; 1972.
16.
Kammann R, Flett R. Affectometer 2: A scale to measure current level of general happiness. Aust J Psychol. 1983;35(2):259–65.CrossRef
17.
Tennant R, Joseph S, Stewart-Brown S. The Affectometer 2: a measure of positive mental health in UK populations. Qual Life Res. 2007;16(4):687–95.CrossRefPubMed
18.
Reise SP. The rediscovery of bifactor measurement models. Multivar Behav Res. 2012;47(5):667–96.CrossRef
19.
Gibbons RD, Bock RD, Hedeker D, Weiss DJ, Segawa E, Bhaumik DK, Kupfer DJ, Frank E, Grochocinski VJ, Stover A. Full-Information item bifactor analysis of graded response data. Appl Psych Meas. 2007;31(1):4–19.CrossRef
20.
Gibbons R, Hedeker D. Full-information item bi-factor analysis. Psychometrika. 1992;57(3):423–36.CrossRef
21.
Romppel M, Braehler E, Roth M, Glaesmer H. What is the General Health Questionnaire-12 assessing?: Dimensionality and psychometric properties of the General Health Questionnaire-12 in a large scale German population sample. Compr Psychiatry. 2013;54(4):406–13.CrossRefPubMed
22.
Ye S. Factor structure of the General Health Questionnaire (GHQ-12): The role of wording effects. Pers Indiv Differ. 2009;46(2):197–201.CrossRef
23.
Wang W-C, Chen H-F, Jin K-Y. Item response theory models for wording effects in mixed-format scales. Educ Psychol Meas. 2014;75(1):157-78.
24.
Pohl S, Steyer R. Modeling common traits and method effects in multitrait-multimethod analysis. Multivar Behav Res. 2010;45(1):45–72.CrossRef
25.
Geiser C, Lockhart G. A comparison of four approaches to account for method effects in latent state–trait analyses. Psychol Methods. 2012;17(2):255–83.CrossRefPubMedPubMedCentral
26.
Scotland NH. Health Education Population Survey. Colchester, Essex: UK Data Archive; 2006.
27.
Tennant R, Hiller L, Fishwick R, Platt S, Joseph S, Weich S, Parkinson J, Secker J, Stewart-Brown S. The Warwick-Edinburgh Mental Well-being Scale (WEMWBS): development and UK validation. Health Qual Life Outcomes. 2007;5:63.CrossRefPubMedPubMedCentral
28.
Satorra A, Bentler PM. Corrections to test statistics and standard errors in covariance structure analysis. In: von Eye A, Clogg CC, editors. Latent variables analysis: Applications for developmental research. edn. Thousand Oaks: Sage; 1994. p. 399–419.
29.
30.
Tucker LR, Lewis C. A reliability coeffficient for maximum likelihood factor analysis. Psychometrika. 1973;38:1–10.CrossRef
31.
Steiger JH, Lind J. Statistically-based tests for the number of common factors. Paper presented at the annual Spring Meeting of the Psychometric Society in Iowa City. May 30, 1980.
32.
Satorra A. Scaled and adjusted restricted tests in multi-sample analysis of moment structures. In: Heijmans RDH, Pollock DSG, Satorra A, editors. Innovations in multivariate statistical analysis A Festschrift for Heinz Neudecker. edn. London: Kluwer Academic Publishers; 2000. p. 233–47.CrossRef
33.
Muthén L, Muthén B. Mplus: Statistical analysis with latent variables. Version 7.3. Los Angeles, CA: Muthén & Muthén; 1998-2016.
34.
Samejima F. Estimation of latent ability using a response pattern of graded scores, Psychometric Monograph no 17. 1969.
35.
Takane Y, Leeuw J. On the relationships between item response theory and factor analysis of discretized variables. Psychometrika. 1987;52(3):393–408.CrossRef
36.
McDonald RP. Test theory: A unified treatment. Mahwah: Lawrence Erlbaum Associates, Inc.; 1999.
37.
Baker FB, Kim SH. Item response theory: Parameter estimation techniques. New York: Marcell Dekker; 2004.
38.
Veerkamp WJ, Berger MP. Some new item selection criteria for adaptive testing. J Educ Behav Stat. 1997;22(2):203–26.CrossRef
39.
van der Linden W. Bayesian item selection criteria for adaptive testing. Psychometrika. 1998;63(2):201–16.CrossRef
40.
Chang H-H, Ying Z. A global information approach to computerized adaptive testing. Appl Psych Meas. 1996;20(3):213–29.CrossRef
41.
42.
Fliege H, Becker J, Walter OB, Bjorner JB, Klapp BF, Rose M. Development of a computer-adaptive test for depression (D-CAT). Qual Life Res. 2005;14(10):2277–91.CrossRefPubMed
43.
Zinbarg R, Revelle W, Yovel I, Li W. Cronbach’s α, Revelle’s β, and Mcdonald’s ωH: their relations with each other and two alternative conceptualizations of reliability. Psychometrika. 2005;70(1):123–33.CrossRef
44.
Weiss DJ, Gibbons RD. Computerized adaptive testing with the bifactor model. In: Proceedings of the 2007 GMAC Conference on Computerized Adaptive Testing: 2007. 2007.
45.
Dimitrov DM. Marginal true-score measures and reliability for binary items as a function of their IRT parameters. Appl Psych Meas. 2003;27(6):440–58.CrossRef
46.
Green BF, Bock RD, Humphreys LG, Linn RL, Reckase MD. Technical guidelines for assessing computerized adaptive tests. J Educ Meas. 1984;21(4):347–60.CrossRef
47.
Seligman ME, Steen TA, Park N, Peterson C. Positive psychology progress: empirical validation of interventions. Am Psychol. 2005;60(5):410–21.CrossRefPubMed
48.
Ryff CD. Happiness is everything, or is it? Explorations on the meaning of psychological well-being. J Pers Soc Psychol. 1989;57(6):1069.CrossRef
49.
Wood AM, Taylor PJ, Joseph S. Does the CES-D measure a continuum from depression to happiness? Comparing substantive and artifactual models. Psychiatry Res. 2010;177(1):120–3.CrossRefPubMed
50.
Joseph S, Lewis CA. The Depression–Happiness Scale: Reliability and validity of a bipolar self‐report scale. J Clin Psychol. 1998;54(4):537–44.CrossRefPubMed
51.
Kammann R, Farry M, Herbison P. The analysis and measurement of happiness as a sense of well-being. Soc Indic Res. 1984;15(2):91–115.CrossRef
52.
Shevlin M, Adamson G. Alternative factor models and factorial invariance of the GHQ-12: a large sample analysis using confirmatory factor analysis. Psychol Assess. 2005;17(2):231–6.CrossRefPubMed
53.
Werneke U, Goldberg DP, Yalcin I, Ustun BT. The stability of the factor structure of the General Health Questionnaire. Psychol Med. 2000;30(4):823–9.CrossRefPubMed
54.
Hu Y, Stewart-Brown S, Twigg L, Weich S. Can the 12-item General Health Questionnaire be used to measure positive mental health? Psychol Med. 2007;37(7):1005–13.CrossRefPubMed
55.
Molina JG, Rodrigo MF, Losilla JM, Vives J. Wording effects and the factor structure of the 12-item General Health Questionnaire (GHQ-12). Psychol Assess. 2014;26(3):1031–7.CrossRefPubMed
56.
Crawford JR, Henry JD. The positive and negative affect schedule (PANAS): construct validity, measurement properties and normative data in a large non-clinical sample. Br J Clin Psychol. 2004;43(Pt 3):245–65.CrossRefPubMed
57.
Simms LJ, Gros DF, Watson D, O’Hara MW. Parsing the general and specific components of depression and anxiety with bifactor modeling. Depress Anxiety. 2008;25(7):E34–46.CrossRefPubMed
58.
Emons WHM, Sijtsma K, Meijer RR. On the consistency of individual classification using short scales. Psychol Methods. 2007;12(1):105–20.CrossRefPubMed
59.
van der Linden WJ, Glas CAW, editors. Elements of adaptive testing. New York: Springer; 2010.
60.
Urban R, Kun B, Farkas J, Paksi B, Kokonyei G, Unoka Z, Felvinczi K, Olah A, Demetrovics Z. Bifactor structural model of symptom checklists: SCL-90-R and Brief Symptom Inventory (BSI) in a non-clinical community sample. Psychiatry Res. 2014;216(1):146–54.CrossRefPubMed
61.
Glaesmer H, Braehler E, Grande G, Hinz A, Petermann F, Romppel M. The German version of the Hopkins Symptoms Checklist-25 (HSCL-25): Factorial structure, psychometric properties, and population-based norms. Compr Psychiatry. 2014;55(2):396–403.CrossRefPubMed
62.
Stochl J, Khandaker GM, Lewis G, Perez J, Goodyer IM, Zammit S, Sullivan S, Croudace TJ, Jones PB. Mood, anxiety and psychotic phenomena measure a common psychopathological factor. Psychol Med. 2015;45(07):1483–93.CrossRefPubMed
63.
Jovanović V. Structural validity of the Mental Health Continuum-Short Form: The bifactor model of emotional, social and psychological well-being. Pers Indiv Differ. 2015;75:154–9.CrossRef
64.
Camfield L, Skevington SM. On subjective well-being and quality of life. J Health Psychol. 2008;13(6):764–75.CrossRefPubMed
65.
Wood AM, Tarrier N. Positive Clinical Psychology: a new vision and strategy for integrated research and practice. Clin Psychol Rev. 2010;30(7):819–29.CrossRefPubMed
66.
Gibbons RD, Weiss DJ, Pilkonis PA, Frank E, Moore T, Kim JB, Kupfer DJ. Development of a computerized adaptive test for depression. Arch Gen Psychiatry. 2012;69(11):1104–12.CrossRefPubMedPubMedCentral
67.
Gibbons RD, Weiss DJ, Kupfer DJ, Frank E, Fagiolini A, Grochocinski VJ, Bhaumik DK, Stover A, Bock RD, Immekus JC. Using computerized adaptive testing to reduce the burden of mental health assessment. Psych Serv. 2008;59(4):361–8.CrossRef
68.
Gibbons RD, Weiss DJ, Pilkonis PA, Frank E, Moore T, Kim JB, Kupfer DJ. Development of the CAT-ANX: a computerized adaptive test for anxiety. Am J Psychiatry. 2014;171(2):187–94.CrossRefPubMedPubMedCentral
Metadata
Title
An evaluation of computerized adaptive testing for general psychological distress: combining GHQ-12 and Affectometer-2 in an item bank for public mental health research
Authors
Jan Stochl
Jan R. Böhnke
Kate E. Pickett
Tim J. Croudace
Publication date
01-12-2016
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2016
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-016-0158-7

Other articles of this Issue 1/2016

BMC Medical Research Methodology 1/2016 Go to the issue

Research article

Predictors of in-hospital mortality following major lower extremity amputations in type 2 diabetic patients using artificial neural networks

Research article

Least absolute shrinkage and selection operator type methods for the identification of serum biomarkers of overweight and obesity: simulation and application

Research article

ImpRess: an Implementation Readiness checklist developed using a systematic review of randomised controlled trials assessing cognitive stimulation for dementia

Research article

A comparison of methods to adjust for continuous covariates in the analysis of randomised trials

Research article

Mixed-strain housing for female C57BL/6, DBA/2, and BALB/c mice: validating a split-plot design that promotes refinement and reduction

Research Article

Measurement and control of bias in patient reported outcomes using multidimensional item response theory