Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on sleep in brain health

LIVE: Thursday 2nd May 2024, 18:00-19:30 (CEST)

Quality sleep is essential for health. But what happens to our brains when sleep patterns are disturbed? Join our experts to explore the interplay between sleep disruption and neurological diseases, and the questions that you need to be asking your patients to help you prevent the harmful effects of sleep deprivation.
ADVANCED SEARCH
Log in
Top
Quality of Life Research
Published in: Quality of Life Research 6/2017

Open Access 01-06-2017

Longitudinal measurement invariance of the Dutch Outcome Questionnaire-45 in a clinical sample

Authors: Ruslan Jabrayilov, Wilco H. M. Emons, Kim de Jong, Klaas Sijtsma

Published in: Quality of Life Research | Issue 6/2017

Login to get access

Abstract

Purpose

In the absence of measurement invariance across measurement occasions, change scores based on pretest–posttest measurements may be inaccurate representations of real change on the latent variable. In this study, we examined whether measurement invariance held in the Dutch version of Outcome Questionnaire-45 (OQ-45).

Method

Using secondary data analysis of a sample of N = 540 Dutch outpatients, we tested the stability of the factorial structure (gamma change) and the metric and scalar invariance (beta change) across pretest and posttest measurements using a combination of factor analysis and item response theory methodology.

Results

Results revealed a stable factorial structure from pretest to posttest and minor violations of metric invariance for two items in the Dutch OQ-45.

Conclusion

Even though for two items the assumption of invariance was violated, results suggest that the effects of these violations on practical change assessment using the OQ-45 were negligible.
Literature
1.
Millsap, R. (2010). Testing measurement invariance using item response theory in longitudinal data: An introduction. Child Development Perspectives, 4, 5–9.CrossRef
2.
Schmitt, N. (1982). The use of analysis of covariance structures to assess beta and gamma change. Multivariate Behavioral Research, 17, 343–358.CrossRefPubMed
3.
Alvares, K. M., & Hulin, C. L. (1972). Two explanations of temporal changes in ability-skill relationships: A literature review and theoretical analysis. Human Factors, 14, 295–308.CrossRef
4.
Henry, R. A., & Hulin, C. L. (1987). Stability of skilled performance across time: Some generalizations and limitations on utilities. Journal of Applied Psychology, 72, 457–462.CrossRef
5.
Golembiewski, R. T., Billingsley, K., & Yeager, S. (1976). Measuring change and persistence in human affairs: Types of change generated by OD designs. Journal of Applied Behavioral Science, 12, 133–157.CrossRef
6.
Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, recommendations for organizational research. Organizational Research Methods, 3, 4–70.CrossRef
7.
Howard, G. S., Ralph, K. M., Gulanick, N. A., Maxwell, S. E., Nance, S. W., & Gerber, S. K. (1979). Internal invalidity in pre-test-posttest self-report evaluations and a re-evaluation of retrospective pre-tests. Applied Psychological Measurement, 3, 1–23.CrossRef
8.
Schwartz, C., Bode, R., Repucci, N., Becker, J., Sprangers, M., & Fayers, P. (2006). The clinical significance of adaptation to changing health: A meta-analysis of response shift. Quality of Life Research, 15, 1533–1550.CrossRefPubMed
9.
Nieuwkerk, P. T., Tollenaar, M. S., Oort, F. J., & Sprangers, M. A. G. (2007). Are retrospective measures of change in quality of life more valid than prospective measures? Medical Care, 45, 199–205.CrossRefPubMed
10.
Lambert, M. J., Hansen, N. B., Umphress, V., Lunnen, K., Okiishi, J., Burlingame, G., & Reisinger, C. W. (1996). Administration and scoring manual for the Outcome Questionnaire (OQ45.2). Wilmington, DE: American Professional Credentialing Services.
11.
De Jong, K., Nugter, M. A., Polak, M. G., Wagenborg, J. E. A., Spinhoven, P., & Heiser, W. J. (2007). The Outcome Questionnaire (OQ-45) in a Dutch populations: A cross-cultural validation. Clinical Psychology and Psychotherapy, 14, 288–301.CrossRef
12.
Hatfield, D. R., & Ogles, B. M. (2004). The use of outcome measures by psychologists in clinical practice. Professional Psychology—Research and Practice, 35, 485–491.CrossRef
13.
De Jong, K., Van Sluis, P., Nugter, M. A., Heiser, W. J., & Spinhoven, P. (2012). Understanding the differential impact on outcome monitoring: Therapist variables that moderate feedback effects in a randomized clinical trial. Psychotherapy Research, 22, 464–474.CrossRefPubMed
14.
Beretvas, S. N., & Kearney, L. K. (2003). A shortened form of the Outcome Questionnaire: A validation of scores across groups (A research report of the Research Consortium of Counseling and Psychological Services for Higher Education). Austin, TX: University of Texas at Austin, Counseling and Mental Health Center.
15.
Chapman, J. E. (2003). Reliability and validity of the progress questionnaire: An adaptation of the Outcome Questionnaire. Philadelphia, PA: Drexel University.
16.
Kim, S.-H., Beretvas, N., & Sherry, A. R. (2010). A validation of the factor structure of OQ-45 scores using factor mixture modeling. Measurement and Evaluation in Counseling and Development, 42, 275–295.CrossRef
17.
Mueller, R. M., Lambert, M. J., & Burlingame, G. M. (1998). Construct validity of the outcome questionnaire: A confirmatory factor analysis. Journal of Personality Assessment, 70, 248–262.CrossRef
18.
Conijn, J. M., Emons, W. H. M., De Jong, K., & Sijtsma, K. (2015). Detecting and explaining aberrant responding to the Outcome Questionnaire–45. Assessment, 22, 513–524.CrossRefPubMed
19.
Meade, A. W., Lautenschlager, G. J., & Hecht, J. E. (2005). Establishing measurement equivalence and invariance in longitudinal data with item response theory. International Journal of Testing, 5, 279–300.CrossRef
20.
Oort, F. J. (2005). Using structural equation modeling to detect response shifts and true change. Quality of Life Research, 14, 587–598.CrossRefPubMed
21.
Schaubroeck, J., & Green, S. G. (1989). Confirmatory factor analytic procedures for assessing change during organizational entry. Journal of Applied Psychology, 74, 892–900.CrossRef
22.
Widaman, K. F., Ferrer, E., & Conger, R. D. (2010). Factorial invariance within longitudinal structural equation models: Measuring the same construct across time. Child Development Perspectives, 4, 10–18.CrossRefPubMedPubMedCentral
23.
Muthén, L. K., & Muthén, B. O. (1998–2011). Mplus User’s Guide. Sixth Edition. Los Angeles, CA: Muthén & Muthén.
24.
Gorsuch, R. L. (1997). Exploratory factor analysis: Its role in item analysis. Journal of Personality Assessment, 68, 532–560.CrossRefPubMed
25.
Van der Linden, W. J. (2017). Handbook of modern item response theory (vol. 1). Boca Raton, FL: Chapman and Hall/CRC.
26.
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph, 17.
27.
Morizot, J., Ainsworth, A. T., & Reise, S. P. (2007). Toward modern psychometrics: Application of item response theory models in personality research. In R. W. Robins, R. C. Fraley & R. F. Krueger (Eds.), Handbook of Research Methods in Personality Psychology (pp. 407–423). New York, NY: Guilford Press.
28.
Lindgren, W. (1993). Statistical Theory (4th edn.). New York, NY: Chapman & Hall.
29.
Houts, C. R., & Cai, L. (2013). flexMIRTR user’s manual version 2: Flexible multilevel multidimensional item analysis and test scoring. Chapel Hill, NC: Vector Psychometric Group.
30.
Reckace, M. D. (2009). Multidimensional item response theory. New York, NY: Springer.CrossRef
31.
Forero, C. G., Maydeu-Olivares, A., & Gallardo-Pujol, D. (2009). Factor analysis with ordinal indicators: A Monte Carlo study comparing DWLS and ULS estimation. Structural Equation Modeling, 16, 625–641.CrossRef
32.
Taris, T. W., Bok, I. A., & Meijer, Z. Y. (1998). Assessing stability and change of psychometric properties of multi-item concepts across different situations: A general approach. Journal of Psychology, 132, 301–316.CrossRef
33.
Lambert, M. J., Morton, J. J., Hatfield, D. R., Harmon, C., Hamilton, S., Shimokawa, K., et al. (2004). Administration and scoring manual for the OQ-45.2 (Outcome Questionnaire) (3rd ed.). Wilmington, DE: American Professional Credentialing Services LLC.
34.
35.
Orlando-Edelen, M., Thissen, D., Teresi, J. A., Kleinman, M., & Ocepek-Welikson, K. (2006). Identification of differential item functioning using item response theory and the likelihood based model comparison approach: Applications to the Mini-Mental State Examination. Medical Care, 44, 134–142.CrossRef
36.
González-Betanzos, F., & Abad, F. J. (2012). The effects of purification and the evaluation of differential item functioning with the likelihood ratio test. Methodology, 8, 134–145.CrossRef
37.
McPhail, S., Comans, T., & Haines, T. (2010). Evidence of disagreement between patient-perceived change and conventional longitudinal evaluation of change in health-related quality of life among older adults. Clinical Rehabilitation, 24, 1036–1044.CrossRefPubMed
38.
Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy and research. Journal of Consulting and Clinical Psychology, 59, 12–19.CrossRefPubMed
39.
Meijer, R., & Sijtsma, K. (2001). Methodology review: Evaluating person fit. Applied Psychological Measurement, 25, 107–135.CrossRef
40.
Conijn, J. M., Emons, W. H. M., Van Assen, M. A. L. M., Pedersen, S. S., & Sijtsma, K. (2013). Explanatory, multilevel person-fit analysis of response consistency on the Spielberger State-Trait Anxiety Inventory. Multivariate Behavioral Research, 48, 692–718.CrossRefPubMed
Metadata
Title
Longitudinal measurement invariance of the Dutch Outcome Questionnaire-45 in a clinical sample
Authors
Ruslan Jabrayilov
Wilco H. M. Emons
Kim de Jong
Klaas Sijtsma
Publication date
01-06-2017
Publisher
Springer International Publishing
Published in
Quality of Life Research / Issue 6/2017
Print ISSN: 0962-9343
Electronic ISSN: 1573-2649
DOI
https://doi.org/10.1007/s11136-017-1500-1

Other articles of this Issue 6/2017

Quality of Life Research 6/2017 Go to the issue

OriginalPaper

Establishing the values for patient engagement (PE) in health-related quality of life (HRQoL) research: an international, multiple-stakeholder perspective

OriginalPaper

Impact of caregiver activities and social supports on multidimensional caregiver burden: analyses from nationally-representative surveys of cancer patients and their caregivers

OriginalPaper

Lower educational level and unemployment increase the impact of cardiometabolic conditions on the quality of life: results of a population-based study in South Australia

OriginalPaper

Framing of mobility items: a source of poor agreement between preference-based health-related quality of life instruments in a population of individuals receiving assisted ventilation

OriginalPaper

Quality of life improves in children and adolescents during a community-based overweight and obesity treatment

OriginalPaper

Cultural values: can they explain self-reported health?