Top

Quality of Life Research

Published in:

Open Access 01-03-2024

Power analyses for measurement model misspecification and response shift detection with structural equation modeling

Author: M. G. E. Verdam

Published in: Quality of Life Research | Issue 5/2024

Abstract

Purpose

Statistical power for response shift detection with structural equation modeling (SEM) is currently underreported. The present paper addresses this issue by providing worked-out examples and syntaxes of power calculations relevant for the statistical tests associated with the SEM approach for response shift detection.

Methods

Power calculations and related sample-size requirements are illustrated for two modelling goals: (1) to detect misspecification in the measurement model, and (2) to detect response shift. Power analyses for hypotheses regarding (exact) overall model fit and the presence of response shift are demonstrated in a step-by-step manner. The freely available and user-friendly R-package lavaan and shiny-app ‘power4SEM’ are used for the calculations.

Results

Using the SF-36 as an example, we illustrate the specification of null-hypothesis (H₀) and alternative hypothesis (H₁) models to calculate chi-square based power for the test on overall model fit, the omnibus test on response shift, and the specific test on response shift. For example, we show that a sample size of 506 is needed to reject an incorrectly specified measurement model, when the actual model has two-medium sized cross loadings. We also illustrate power calculation based on the RMSEA index for approximate fit, where H₀ and H₁ are defined in terms of RMSEA-values.

Conclusion

By providing accessible resources to perform power analyses and emphasizing the different power analyses associated with different modeling goals, we hope to facilitate the uptake of power analyses for response shift detection with SEM and thereby enhance the stringency of response shift research.

Available only for authorised users

This is also known as uniform recalibration, where non-uniform recalibration refers to a change in residual factor variances. However, the detection of non-uniform recalibration is not relevant for the investigation of mean change in the common factor [16] and therefore rarely considered in the literature. Therefore, here we focus on uniform recalibration only, and for reasons of conciseness refer to it as simply ‘recalibration’.

Sprangers, M. A. G., & Schwartz, C. E. (1999). Integrating response shift into health-related quality of life research: A theoretical model. Social Science and Medicine, 48, 1507–1515. https://doi.org/10.1016/s0277-9536(99)00045-3CrossRefPubMed

Sprangers, M. A. G., Sajobi, T., Vanier, A., Mayo, N. E., Sawatzky, R., Lix, L. M., Oort, F. J., Sébille, V., & Response Shift – in Sync Working Group. (2021). Response shift results of patient-reported outcome measures: A commentary to The response shift-in sync working group initiative. Quality of Life Research, 30(12), 3299–3308. https://doi.org/10.1007/s11136-020-02747-4CrossRef

Vanier, A., Oort, F. J., McClimans, L., Ow, N., Gulek, B. G., Böhnke, J. R., Sprangers, M. A. G., Sébille, V., Mayo, N., & Response Shift – in Sync Working Group. (2021). Response shift in patient-reported outcomes: Definition, theory, and a revised model. Quality of Life Research, 30(12), 3309–3322. https://doi.org/10.1007/s11136-021-02846-wCrossRef

Sébille, V., Lix, L. M., Ayilara, O. F., Sajobi, T. T., Janssens, C. J. W., Sawatzky, R., Sprangers, M. A. G., Verdam, M. G. E. & the Response Shift – in Sync Working Group. (2021). Critical examination of current response shift methods and proposal for advancing new methods. Quality of Life Research, 30(12), 3325–3342. https://doi.org/10.1007/s11136-020-02755-4CrossRef

Sawatzky, R., Kwon, J.-Y., Barclay, R., Chauhan, C., Frank, L., van den Hout, W. B., Kongsgaard Nielsen, L., Nolte, S., Sprangers, M. A. G., & Response Shift – in Sync Working Group. (2021). Implications of response shift for micro-, meso-, and macro-level healthcare decision-making using results of patient-reported outcome measures. Quality of Life Research, 30(12), 3343–3357. https://doi.org/10.1007/s11136-021-02766-9CrossRef

Sajobi, T. T., Brahmbatt, R., Lix, L. M., Zumbo, B. D., & Sawatzky, R. (2018). Scoping review of response shift methods: Current reporting practices and recommendations. Quality of Life Research, 27(5), 1133–1146. https://doi.org/10.1007/s11136-017-1751-xCrossRefPubMed

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd edition). Erlbaum. https://doi.org/10.4324/9780203771587CrossRef

Boomsma, A. (1982). Robustness of LISREL against small sample sizes in factor analysis models. In K. G. Joreskog & H. Wold (Eds.), Systems under indirection observation: Causality, structure, prediction (Part I) Amsterdam (pp. 149–173). North Holland.

Boomsma, A. (1985). Nonconvergence, improper solutions, and starting values in LISREL maximum likelihood estimation. Psychometrika, 50, 229–242. https://doi.org/10.1007/BF02294248CrossRef

10.

Bentler, P. M., Chou, C.-P., & C. (1987). Practical issues in structural modeling. Sociological Methods and Research, 16, 78–117. https://doi.org/10.1177/0049124187016001004CrossRef

11.

Cattell, R. B. (1978). Conducting a factor analytic research: Strategy and tactics. The scientific use of factor analysis in behavioral and life sciences. Springer.CrossRef

12.

Schwartz, C. E., Ahmed, S., Sawatzky, R., Sajobi, T., Mayo, N., Finkelstein, J., Verdam, M. G. E., Oort, F. J., & Sprangers, M. A. G. (2013). Guidelines for secondary analysis in search of response shift. Quality of Life Research, 22(10), 2663–2673. https://doi.org/10.1007/s11136-013-0402-0CrossRefPubMed

13.

Jak, S., Jorgensen, T. D., Verdam, M. G. E., Oort, F. J., & Elffers, L. (2021). Analytical power calculations for structural equation modeling: A tutorial and shiny app. Behavioral Research Methods, 53, 1385–1406. https://doi.org/10.3758/s13428-020-01479-0CrossRef

14.

Jobst, L. J., Bader, M., & Moshagen, M. (2021). A tutorial on assessing statistical power and determining sample size for structural equation models. Psychological Methods. https://doi.org/10.1037/met0000423CrossRefPubMed

15.

Wang, Y. A., & Rhemtulla, M. (2021). Power analysis for parameter estimation in structural equation modeling: A discussion and tutorial. Advances in Methods and Practices in Psychological Science, 4(1), 1–17. https://doi.org/10.1177/2515245920918253CrossRef

16.

Oort, F. J. (2005). Using structural equation modeling to detect response shift and true change. Quality of Life Research, 14(3), 587–598. https://doi.org/10.1007/s11136-004-0830-yCrossRefPubMed

17.

Kang, H. (2021). Sample size determination and power analysis using the G*Power software. Journal of Educational Evaluation for Health Professions, 18, 17. https://doi.org/10.3352/jeehp.2021.18.17CrossRefPubMedPubMedCentral

18.

Perugini, M., Galucci, M., & Constantini, G. (2018). A practical primer to power analysis for simple experimental designs. International Review of Social Psychology, 31(1), 1–23. https://doi.org/10.5334/irsp.181CrossRef

19.

Ware, J. E., Snow, K. K., Kosinski, M., & Gandek, B. (1993). SF-36 health survey: Manual and interpretation guide. The Health Institute, New England Medical Center.

20.

Sawatzky, R., Sajobi, T., Russel, L., Ademola, A., Anota, A., Awosoga, O. A., Brobbey, Böhnke, J., A., Janssen, C., Lawal, O., Lix, L. M., Sprangers, M. A. G., Sébille, V., Verdam, M. G. E. & the Response Shift – in Sync Working Group (2023). Response shift results of quantitative research using patient-reported outcome measures: A descriptive systematic review. Manuscript submitted for publication.

21.

Rosseel, Y. (2012). Lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48, 1–36. https://doi.org/10.18637/jss.v048.i02CrossRef

22.

Satorra, A., & Saris, W. E. (1985). Power of the likelihood ratio test in covariance structure analysis. Psychometrika, 50(1), 83–90. https://doi.org/10.1007/BF02294150CrossRef

23.

Anagnostopolouos, F., Niakas, D., & Tountas, Y. (2009). Comparison between exploratory factor-analytic and SEM-based approaches to constructing SF-36 summary scores. Quality of Life Research, 18, 53–63. https://doi.org/10.1007/s11136-008-9423-5CrossRef

24.

Oreel, T. H., Nieuwkerk, P. T., Hartog, I. D., Netjes, J. E., Vonk, A. B. A., Lemkes, J., van Laarhoven, H. W. M., Scherer-Rath, M., Henriques, J. P. S., Oort, F. J., Sprangers, M. A. G., & Verdam, M. G. E. (2022). Response shift after coronary revascularization. Quality of Life Research, 31(2), 437–450. https://doi.org/10.1007/s11136-021-02902-5CrossRefPubMed

25.

Bartlett, J.E. (2022, February 1). Introduction to Power Analysis: A Guide to G*Power, jamovi, and Superpower. https://osf.io/zqphw/. https://doi.org/10.17605/OSF.IO/PCFVJ

26.

Fraser, A. R. (2023). Inappropriate use of statistical power. Bone Marrow Transplantation, 58, 474–477. https://doi.org/10.1038/s41409-023-01935-3CrossRefPubMed

27.

Steiger, J. H. and Lind, J. (1980). Statistically-based tests for the number of common factors. Paper presented at the Annual Spring Meeting of the Psychometric Society, Iowa City.

28.

Browne, M. W., & Cudeck, R. (1992). Alternative ways of assessing model fit. Sociological Methods Research, 21, 230–258. https://doi.org/10.1177/0049124192021002005CrossRef

29.

MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 2, 130–149. https://doi.org/10.1037/1082-989X.1.2.130CrossRef

30.

Oort, F. J., Visser, M. R. M., & Sprangers, M. A. G. (2005). An application of structural equation modeling to detect response shifts and true change in quality of life data from cancer patients undergoing invasive surgery. Quality of Life Research, 14, 599–609. https://doi.org/10.1007/s11136-004-0831-xCrossRefPubMed

31.

Gandhi, P. K., Ried, L. D., Huang, I.-C., Kimberlin, C. L., & Kauf, T. L. (2013). Assessment of response shift using two structural equation modeling techniques. Quality of Life Research, 22(3), 461–471. https://doi.org/10.1007/s11136-012-0171-1CrossRefPubMed

32.

Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. https://doi.org/10.3758/bf03193146CrossRefPubMed

33.

Moshagen, M., & Erdfelder, E. (2016). A new strategy for testing structural equation models. Structural Equation Modeling, 23(1), 54–60. https://doi.org/10.1080/10705511.2014.950896CrossRef

34.

Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Allyn & Bacon.

35.

Muthén, L. K., & Muthén, B. O. (2002). How to use a Monte Carlo study to decide on sample size and determine power. Structural Equation Modeling, 9(4), 599–620. https://doi.org/10.1207/S15328007SEM0904_8CrossRef

Title: Power analyses for measurement model misspecification and response shift detection with structural equation modeling
Author: M. G. E. Verdam
Publication date: 01-03-2024
Publisher: Springer International Publishing
Published in: Quality of Life Research / Issue 5/2024
Print ISSN: 0962-9343
Electronic ISSN: 1573-2649
DOI: https://doi.org/10.1007/s11136-024-03605-3

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Power analyses for measurement model misspecification and response shift detection with structural equation modeling

Abstract

Purpose

Methods

Results

Conclusion

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Purpose

Methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 5/2024

Health-related quality of life and associated risk factors in patients with Multiple Osteochondromas: a cross-sectional study

Anchor-based minimal important difference values are often sensitive to the distribution of the change score

Health-related quality of life in patients with extremity bone sarcoma after surgical treatment: a systematic review

Sexual health in Belgian cervical cancer survivors: an exploratory qualitative study

Healthcare consumer acceptability of routine use of the EQ-5D-5L in clinical care: a cross-sectional survey

Reciprocal association between pain and quality of life after newly acquired spinal cord injury