Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2016

Open Access 01-12-2016 | Debate

Improving measurement-invariance assessments: correcting entrenched testing deficiencies

Author: Leslie A. Hayduk

Published in: BMC Medical Research Methodology | Issue 1/2016

Login to get access

Abstract

Background

Factor analysis historically focused on measurement while path analysis employed observed variables as though they were error-free. When factor- and path-analysis merged as structural equation modeling, factor analytic notions dominated measurement discussions – including assessments of measurement invariance across groups. The factor analytic tradition fostered disregard of model testing and consequently entrenched this deficiency in measurement invariance assessments.

Discussion

Applying contemporary model testing requirements to the so-called configural model initiating invariance assessments will improve future assessments but a substantial backlog of deficient assessments remain to be overcome.
This article
  • summarizes the issues,
  • demonstrates the problem using a recent example,
  • illustrates a superior model assessment strategy,
  • and documents disciplinary entrenchment of inadequate testing as exemplified by the journal Organizational Research Methods.

Summary

Employing the few methodologically and theoretically best, rather than precariously-multiple, indicators of latent variables increases the likelihood of achieving properly causally specified structural equation models capable of displaying measurement invariance. Just as evidence of invalidity trumps reliability, evidence of configural model misspecification trumps invariant estimates of misspecified coefficients.
Appendix
Available only for authorised users
Literature
1.
go back to reference Thurstone L. Multiple Factor Analysisl. Chicago: University of Chicago Press; 1947. Thurstone L. Multiple Factor Analysisl. Chicago: University of Chicago Press; 1947.
2.
go back to reference Harmann H. Modern Factor Analysis 2nd edition. Chicago: University of Chicago Press; 1967. Harmann H. Modern Factor Analysis 2nd edition. Chicago: University of Chicago Press; 1967.
3.
go back to reference Lawley DN, Maxwell AE. Factor Analysis as a Statistical Method. 2nd ed. London: Butterworth & Co.; 1971. Lawley DN, Maxwell AE. Factor Analysis as a Statistical Method. 2nd ed. London: Butterworth & Co.; 1971.
4.
go back to reference Wright S. Correlation and causation. J Agric Res. 1921;20:557–85. Wright S. Correlation and causation. J Agric Res. 1921;20:557–85.
5.
go back to reference Wright S. The method of path coefficients. Ann Math Stat. 1934;5:161–215.CrossRef Wright S. The method of path coefficients. Ann Math Stat. 1934;5:161–215.CrossRef
6.
go back to reference Blalock HMJ. Causal Inference in Nonexperimental Research. Chapel Hill: University of North Carolina Press; 1964. Blalock HMJ. Causal Inference in Nonexperimental Research. Chapel Hill: University of North Carolina Press; 1964.
7.
go back to reference Duncan OD. Introduction to Structural Equation Models. New York: Academic; 1975. Duncan OD. Introduction to Structural Equation Models. New York: Academic; 1975.
8.
go back to reference Heise DR. Causal Analysis. New York: Wiley; 1975. Heise DR. Causal Analysis. New York: Wiley; 1975.
9.
go back to reference Sorbom D. Karl Joreskog and LISREL: A personal story. In: Cudeck R, du Toit S, Sorbom D, editors. Structural Equation Modeling: Present and Future. A Festschrift in Honor of karl Joreskog. Lincolnwood: Scientific Software International; 2001. Sorbom D. Karl Joreskog and LISREL: A personal story. In: Cudeck R, du Toit S, Sorbom D, editors. Structural Equation Modeling: Present and Future. A Festschrift in Honor of karl Joreskog. Lincolnwood: Scientific Software International; 2001.
10.
go back to reference Hayduk LA, Glaser DN. Jiving the Four-Step, Waltzing Around Factor Analysis, and Other Serious Fun. Struct Equ Model. 2000;7(1):1–35.CrossRef Hayduk LA, Glaser DN. Jiving the Four-Step, Waltzing Around Factor Analysis, and Other Serious Fun. Struct Equ Model. 2000;7(1):1–35.CrossRef
11.
go back to reference Hayduk LA. Seeing Perfectly Fitting Factor Models That Are Causally Misspecified: Understanding That Close-Fitting Models Can Be Worsethat l. Educ Psychol Meas. 2014;74(6):905–26.CrossRef Hayduk LA. Seeing Perfectly Fitting Factor Models That Are Causally Misspecified: Understanding That Close-Fitting Models Can Be Worsethat l. Educ Psychol Meas. 2014;74(6):905–26.CrossRef
12.
13.
go back to reference Vandenberg RJ, Lance CE. A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organ Res Methods. 2000;3(1):4–70.CrossRef Vandenberg RJ, Lance CE. A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organ Res Methods. 2000;3(1):4–70.CrossRef
14.
go back to reference Hayduk L, Cummings G, Boadu K, Pazderka-Robinson H, Boulianne S. Testing! Testing! One, two, three – Testing the theory in structural equation models. Personal Individ Differ. 2007;42(5):841–50.CrossRef Hayduk L, Cummings G, Boadu K, Pazderka-Robinson H, Boulianne S. Testing! Testing! One, two, three – Testing the theory in structural equation models. Personal Individ Differ. 2007;42(5):841–50.CrossRef
15.
go back to reference Byrne BM, Shavelson RJ, Muthen B. Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychol Bull. 1989;105(3):456–66.CrossRef Byrne BM, Shavelson RJ, Muthen B. Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychol Bull. 1989;105(3):456–66.CrossRef
16.
go back to reference Cheung GW, Rensvold RB. Testing factorial invariance across groups: A reconceptualization and proposed new method. J Manag. 1999;25:1–27. Cheung GW, Rensvold RB. Testing factorial invariance across groups: A reconceptualization and proposed new method. J Manag. 1999;25:1–27.
17.
go back to reference Cheung GW, Lau RS. A Direct Comparison Approach for Testing Measurement Invariance. Organ Res Methods. 2012;15(2):167–98.CrossRef Cheung GW, Lau RS. A Direct Comparison Approach for Testing Measurement Invariance. Organ Res Methods. 2012;15(2):167–98.CrossRef
18.
go back to reference Little TD. Longitudinal Structural Equation Modeling. New York: Guilford Press; 2013. Little TD. Longitudinal Structural Equation Modeling. New York: Guilford Press; 2013.
19.
go back to reference Borsboom D, Mellenbergh GJ, vanHeerden J. The theoretical status of latent variables. Psychol Rev. 2003;110(2):203–19.CrossRefPubMed Borsboom D, Mellenbergh GJ, vanHeerden J. The theoretical status of latent variables. Psychol Rev. 2003;110(2):203–19.CrossRefPubMed
20.
go back to reference Borsboom D, Mellenbergh GJ, vanHeerden J. The concept of validity. Psychol Rev. 2004;111(4):1061–71.CrossRefPubMed Borsboom D, Mellenbergh GJ, vanHeerden J. The concept of validity. Psychol Rev. 2004;111(4):1061–71.CrossRefPubMed
21.
go back to reference ISSP Research Group. International Social Survey Program: Work Orientations I – ISSP 1989. Cologne: GESIS Data Archive, Cologne; 1991. doi:10.4232/1.1840. ZA1840 Data file Version 1.0.0. ISSP Research Group. International Social Survey Program: Work Orientations I – ISSP 1989. Cologne: GESIS Data Archive, Cologne; 1991. doi:10.​4232/​1.​1840. ZA1840 Data file Version 1.0.0.
22.
go back to reference IBM. IBM-SPSS 22. Armonk: International Business Machines Inc; 2013. IBM. IBM-SPSS 22. Armonk: International Business Machines Inc; 2013.
23.
go back to reference Joreskog K, Sorbom D. LISREL 9.1 March 2013. Skokie: Scientific Software International; 2013. Joreskog K, Sorbom D. LISREL 9.1 March 2013. Skokie: Scientific Software International; 2013.
24.
go back to reference Hayduk LA. LISREL Issues, Debates, and Strategies. Baltimore: Johns Hopkins University Press; 1996. Hayduk LA. LISREL Issues, Debates, and Strategies. Baltimore: Johns Hopkins University Press; 1996.
25.
26.
go back to reference Hayduk L. Structural Equation Modeling with LISREL. Baltimore: Johns Hopkins University Press; 1987. Hayduk L. Structural Equation Modeling with LISREL. Baltimore: Johns Hopkins University Press; 1987.
27.
go back to reference Schmitt N, Kuljanin G. Measuring invariance: Review of practice and implications. Hum Resour Manag Rev. 2008;18:210–22.CrossRef Schmitt N, Kuljanin G. Measuring invariance: Review of practice and implications. Hum Resour Manag Rev. 2008;18:210–22.CrossRef
28.
go back to reference Meade AW, Johnson EC, Braddy PW. Power and sensitivity of alternative fit indices in tests of measurement invariance. J Appl Psychol. 2008;93(3):568–92.CrossRefPubMed Meade AW, Johnson EC, Braddy PW. Power and sensitivity of alternative fit indices in tests of measurement invariance. J Appl Psychol. 2008;93(3):568–92.CrossRefPubMed
29.
go back to reference Rensvold RB, Cheung GW. Testing measurement models for factorial invariance: A systematic approach. Educ Psychol Meas. 1998;58(6):1017–34.CrossRef Rensvold RB, Cheung GW. Testing measurement models for factorial invariance: A systematic approach. Educ Psychol Meas. 1998;58(6):1017–34.CrossRef
30.
go back to reference Hayduk LA, Pazderka-Robinson H, Cummings GG, Boadu K, Verbeek ELPTA. The weird world, and equally weird measurement models: Reactive indicators and the validity revolution. Struct Equ Model. 2007;14(2):280–310.CrossRef Hayduk LA, Pazderka-Robinson H, Cummings GG, Boadu K, Verbeek ELPTA. The weird world, and equally weird measurement models: Reactive indicators and the validity revolution. Struct Equ Model. 2007;14(2):280–310.CrossRef
31.
go back to reference Hayduk LA, Pazderka-Robinson H. Fighting to understand the world causally: Three battles connected to the causal implications of structural equation models. In: Outhwaite W, Turner S, editors. Sage Handbook of Social Science Methodology. London: Sage; 2007. p. 147–71. Hayduk LA, Pazderka-Robinson H. Fighting to understand the world causally: Three battles connected to the causal implications of structural equation models. In: Outhwaite W, Turner S, editors. Sage Handbook of Social Science Methodology. London: Sage; 2007. p. 147–71.
32.
go back to reference Borsboom D, Mellenbergh GJ. True scores, latent variables, and constructs: A comment on Schmidt and Hunter. Intelligence. 2002;30:505–14.CrossRef Borsboom D, Mellenbergh GJ. True scores, latent variables, and constructs: A comment on Schmidt and Hunter. Intelligence. 2002;30:505–14.CrossRef
34.
go back to reference Ioannidis JPA. Why most published research findings are false. PLoS Med. 2005;2(8):0696–701.CrossRef Ioannidis JPA. Why most published research findings are false. PLoS Med. 2005;2(8):0696–701.CrossRef
Metadata
Title
Improving measurement-invariance assessments: correcting entrenched testing deficiencies
Author
Leslie A. Hayduk
Publication date
01-12-2016
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2016
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-016-0230-3

Other articles of this Issue 1/2016

BMC Medical Research Methodology 1/2016 Go to the issue