Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on medication adherence

LIVE: Thursday 27th June 2024, 18:00-19:30 (CEST)
Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2012

Open Access 01-12-2012 | Research article

A proof of principle for using adaptive testing in routine Outcome Monitoring: the efficiency of the Mood and Anxiety Symptoms Questionnaire -Anhedonic Depression CAT

Authors: Niels Smits, Frans G Zitman, Pim Cuijpers, Margien E den Hollander-Gijsman, Ingrid VE Carlier

Published in: BMC Medical Research Methodology | Issue 1/2012

Login to get access

Abstract

Background

In Routine Outcome Monitoring (ROM) there is a high demand for short assessments. Computerized Adaptive Testing (CAT) is a promising method for efficient assessment. In this article, the efficiency of a CAT version of the Mood and Anxiety Symptom Questionnaire, - Anhedonic Depression scale (MASQ-AD) for use in ROM was scrutinized in a simulation study.

Methods

The responses of a large sample of patients (N = 3,597) obtained through ROM were used. The psychometric evaluation showed that the items met the requirements for CAT. In the simulations, CATs with several measurement precision requirements were run on the item responses as if they had been collected adaptively.

Results

CATs employing only a small number of items gave results which, both in terms of depression measurement and criterion validity, were only marginally different from the results of a full MASQ-AD assessment.

Conclusions

It was concluded that CAT improved the efficiency of the MASQ-AD questionnaire very much. The strengths and limitations of the application of CAT in ROM are discussed.
Appendix
Available only for authorised users
Literature
1.
Gardner W, Shear K, Kelleher KJ, Pajer KA, Mammen O, Buysse D, Frank E: Computerized adaptive measurement of depression: A simulation study. BMC Psychiatry. 2004, 4: Article 13-[DOI: 10.1186/1471-244X-4-13]CrossRef
2.
Cella D, Yount S, Rothrock N, Gershon R, Cook K, Reeve B, Ader D, Fries J, Bruce B, Rose M: The Patient-Reported Outcomes Measurement Information System (PROMIS): Progress of an NIH Roadmap Cooperative Group During its First Two Years. Medical Care. 2007, 45: S3-11.CrossRefPubMedPubMedCentral
3.
Smits N, Cuijpers P, Beekman ATF, Smit JH: Reducing the length of mental health instruments through structurally incomplete designs. International Journal of Methods in Psychiatric Research. 2007, 16: 150-160. 10.1002/mpr.223.CrossRefPubMed
4.
De Beurs E, den Hollander-Gijsman M, Van Rood Y, van Der Wee N, Giltay E, Van Noorden M, Van der Lem R, Van Fenema E, Zitman F: Routine outcome monitoring in the Netherlands: practical experiences with a web-based strategy for the assessment of treatment outcome in clinical practice. Clinical Psychology & Psychotherapy. 2011, 18: 1-12. 10.1002/cpp.696.CrossRef
5.
Carlier IVE, Meuldijk D, Van Vliet I, Van Fenema E, Van der Wee N, Zitman F: Routine outcome monitoring and feedback on physical or mental health status: evidence and theory. Journal of Evaluation in Clinical Practice. 2010, [DOI: 10.1111/j.1365-2753.2010.01543.x]
6.
Fliege H, Becker J, Walter OB, Bjorner JB, Klapp BF, Rose M: Development of a computer-adaptive test for depression (D-CAT). Quality of Life Research. 2005, 14: 2277-2291. 10.1007/s11136-005-6651-9.CrossRefPubMed
7.
Forkmann T, Boecker M, Norra C, Eberle N, Kircher T, Schauerte P, Mischke K, Westhofen M, Gauggel S, Wirtz M: Development of an Item Bank for the Assessment of Depression in Persons With Mental Illnesses and Physical Diseases Using Rasch Analysis. Rehabilitation Psychology. 2009, 54: 186-197.CrossRefPubMed
8.
Walter OB, Becker J, Bjorner JB, Fliege H, Klapp BF, Rose M: Development and evaluation of a computer adaptive test for 'Anxiety' (Anxiety-CAT). Quality of Life Research. 2007, 16: 143-155.CrossRefPubMed
9.
Watson D, Clark LA: The Mood and Anxiety Symptoms Questionnaire. 1991, Iowa City: University of Iowa: Department of Psychology
10.
Smits N, Cuijpers P, van Straten A: Applying Computerized Adaptive Testing to the CES-D scale: A Simulation Study. Psychiatry Research. 2011, 188: 147-155. 10.1016/j.psychres.2010.12.001.CrossRefPubMed
11.
Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, Hergueta T, Baker R, Dunbar GC: The Mini-International Neuropsychiatric Interview (MINI): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. Journal of Clinical Psychiatry. 1998, 59 (suppl 20): 22-57.PubMed
12.
Clark LA, Watson D: A tripartite model of anxiety and depression: Psychometric evidence and taxonomic implications. Journal of Abnormal Psychology. 1991, 100: 316-336.CrossRefPubMed
13.
De Beurs E, den Hollander-Gijsman M, Helmich S, Zitman F: The tripartite model for assessing symptoms of anxiety and depression: Psychometrics of the Dutch version of the mood and anxiety symptoms questionnaire. Behaviour Research and Therapy. 2007, 45 (7): 1609-1617. 10.1016/j.brat.2006.07.004.CrossRefPubMed
14.
Weiss DJ, Kingsbury GG: Application of Computerized Adaptive Testing to Educational Problems. Journal of Educational Measurement. 1984, 21: 361-375. 10.1111/j.1745-3984.1984.tb01040.x.CrossRef
15.
Wainer H: Computerized Adaptive Testing: A Primer. 2000, Mahwah NJ: Lawrence Erlbaum Associates, 2
16.
Samejima F: Estimation of latent ability using a pattern of graded resonses. Psychometrika Monograph Supplement No. 17. 1969
17.
Mellenbergh GJ: Conceptual notes on models for discrete polytomous item responses. Applied Psychological Measurement. 1995, 19: 91-100. 10.1177/014662169501900110.CrossRef
18.
Reeve BB, Hays RD, Bjorner JB, Cook KF, Crane PK, Teresi JA, Thissen D, Revicki DA, Weiss DJ, Hambleton RK, Liu H, Gershon R, Reise SP, Lai J, Cella D: Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Medical Care. 2007, 45: S22-31. 10.1097/01.mlr.0000250483.85507.04.CrossRefPubMed
19.
Embretson S, Reise SP: Item Response Theory for Psychologists. 2000, Mahwah, NJ: Lawrence Erlbaum
20.
Hastie T, Tibshirani R, Friedman JH: The Elements of Statistical Learning: Data mining, inference and prediction. 2001, New York: SpringerCrossRef
21.
Stone M: Cross-validation choice and assessment of statistical predictions. Journal of the Royal Statistical Society B. 1974, 36: 111-147.
22.
McDonald RP: Test Theory: A Unified Treatment. 1999, Mahwah, NJ: Lawrence Erlbaum
23.
Thissen D: MULTILOG User's Guide. 1991, Mooresville, IN: Scientific Software
24.
R Development Core Team: R: A language and environment for statistical computing. 2010, R Foundation for Statistical Computing, Vienna, Austria, [ISBN 3-900051-07-0], [http://​www.​R-project.​org]
25.
Kraemer H, Morgan G, Leech N, Gliner J, Vaske J, Harmon R: Measures of clinical significance. Journal of the American Academy of Child and Adolescent Psychiatry. 2003, 42 (12): 1524-10.1097/00004583-200312000-00022.CrossRefPubMed
26.
Zweig MH, Campbell G: Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clinical Chemistry. 1993, 39: 561-577.PubMed
27.
Jöreskog KG, Sörbom D: LISREL 8.8 for Windows. 2006, Lincolnwood, IL: Scientific Software
28.
Rizopoulos D: ltm: An R Package for Latent Variable Modeling and Item Response Theory Analyses. Journal of Statistical Software. 2006, 17 (5): 1-25.CrossRef
29.
Rizopoulos D: The ltm Package: Latent trait models under IRT. 2007, [Library of the R package]
30.
Fox J: The polycor Package: Polychoric and Polyserial Correlations. 2007, [Library of the R package]
31.
32.
Van der Ark LA: Package 'mokken'. 2009, [Library of the R package]
33.
Choi SW, Gibbons LE, Crane PK: lordif: An R Package for Detecting Differential Item Functioning Using Iterative Hybrid Ordinal Logistic Regression/Item Response Theory and Monte Carlo Simulations. Journal of Statistical Software. 2011, 39 (8): 1-30.CrossRefPubMedPubMedCentral
34.
Choi SW: Package 'lordif'. 2011, [Library of the R package]
35.
Green BF, Bock RD, Humphreys LG, Linn RL, Reckase MD: Technical Guidelines for Assessing Computerized Adaptive Tests. Journal of Educational Measurement. 1984, 21: 347-360. 10.1111/j.1745-3984.1984.tb01039.x.CrossRef
36.
Kraemer HC, Kupfer DJ: Size of treatment effects and their importance to clinical research and practice. Biological Psychiatry. 2006, 59: 990-996. 10.1016/j.biopsych.2005.09.014.CrossRefPubMed
37.
Kocalevent RD, Rose M, Becker J, Walter OB, Fliege H, Bjorner JB, Kleiber D, Klapp BF: An evaluation of patient-reported outcomes found computerized adaptive testing was efficient in assessing stress perception. Journal of Clinical Epidemiology. 2009, 62: 278-287. 10.1016/j.jclinepi.2008.03.003.CrossRefPubMed
38.
Finkelman M, He Y, Kim W, Lai A: Stochastic curtailment of health questionnaires: A method to reduce respondent burden. Statistics in Medicine. 2011, 30: 1989-2004. 10.1002/sim.4231.CrossRefPubMed
39.
Waller NG, Reise SP: Computerized adaptive personality assessment: An illustration with the absorption scale. Journal of Personality and Social Psychology. 1989, 57: 1051-1058.CrossRefPubMed
40.
Thompson N: Item selection in computerized classification testing. Educational and Psychological Measurement. 2009, 69 (5): 778-10.1177/0013164408324460.CrossRef
41.
Finkelman M, Weiss D, Kim-Kang G: Item Selection and Hypothesis Testing for the Adaptive Measurement of Change. Applied Psychological Measurement. 2010, 34 (4): 238-254. 10.1177/0146621609344844.CrossRef
42.
McArdle JJ, Woodcock RW: Expanding test-retest designs to include developmental time-lag components. Psychological Methods. 1997, 2: 403-435.CrossRef
43.
Smits N, Vorst HCM: Reducing test length through structurally incomplete designs: An Illustration. Learning and Individual Differences. 2007, 17: 25-34. 10.1016/j.lindif.2006.12.005.CrossRef
44.
Dillman DA, Sinclair MD, Clark JR: Effects of questionnaire length, respondent-friendly design, and a difficult question on response rates for occupant-addressed census mail surveys. Public Opinion Quarterly. 1993, 57: 289-304. 10.1086/269376.CrossRef
45.
Wacholder S, Carroll RJ, Pee D, Gail MH: The partial questionnaire design for case-control studies. Statistics in Medicine. 1994, 13: 623-634. 10.1002/sim.4780130523.CrossRefPubMed
46.
Bollen K: Structural Equations with Latent Variables. 1989, New York: John WileyCrossRef
47.
Reckase MD: Unifactor Latent Trait Models Applied to Multifactor Tests: Results and Implications. Journal of Educational Statistics. 1979, 4: 207-230.CrossRef
48.
Hambleton RK: Principles and selected applications of Item Response Theory. 1988, New York: American Council on Education, 147-200. 3
49.
Yen WM: Scaling performance assessments: Strategies for managing local item dependence. Journal of Educational Measurement. 1993, 30: 187-213. 10.1111/j.1745-3984.1993.tb00423.x.CrossRef
50.
Cohen J: Statistical power analysis for the behavioral sciences. 1988, New Jersey: Lawrence Erlbaum Associates, 2
51.
Mokken RJ: A theory and procedure of scale analysis. 1971, The Hague: MoutonCrossRef
52.
McKinley RL, Mills CN: A comparison of several goodness-of-fit statistics. Applied Psychological Measurement. 1985, 9: 49-57. 10.1177/014662168500900105.CrossRef
53.
Holland PW, Wainer H: Differential Item Functioning. 1993, Hillsdale, NJ: Lawrence Erlbaum Associates, Inc
54.
Crane PK, Gibbons LE, Jolley L, van Belle G: Differential Item Functioning Analysis With Ordinal Logistic Regression Techniques. Medical Care. 2006, 44: S115-23. 10.1097/01.mlr.0000245183.28384.ed.CrossRefPubMed
Metadata
Title
A proof of principle for using adaptive testing in routine Outcome Monitoring: the efficiency of the Mood and Anxiety Symptoms Questionnaire -Anhedonic Depression CAT
Authors
Niels Smits
Frans G Zitman
Pim Cuijpers
Margien E den Hollander-Gijsman
Ingrid VE Carlier
Publication date
01-12-2012
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2012
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/1471-2288-12-4

Other articles of this Issue 1/2012

BMC Medical Research Methodology 1/2012 Go to the issue

Research article

Systematic reviews need to consider applicability to disadvantaged populations: inter-rater agreement for a health equity plausibility algorithm

Software

Artificial neural networks versus proportional hazards Cox models to predict 45-year all-cause mortality in the Italian Rural Areas of the Seven Countries Study

Research article

Bias-corrected estimator for intraclass correlation coefficient in the balanced one-way random effects model

Debate

'You give us rangoli, we give you talk': using an art-based activity to elicit data from a seldom heard group

Research article

Including mixed methods research in systematic reviews: Examples from qualitative syntheses in TB and malaria control

Research article

Inclusion of mobile phone numbers into an ongoing population health survey in New South Wales, Australia: design, methods, call outcomes, costs and sample representativeness