Top

Health and Quality of Life Outcomes

Published in:

Open Access 01-12-2014 | Research

Measurement invariance across chronic conditions: a systematic review and an empirical investigation of the Health Education Impact Questionnaire (heiQ™)

Authors: Michael Schuler, Gunda Musekamp, Jürgen Bengel, Sandra Nolte, Richard H Osborne, Hermann Faller

Published in: Health and Quality of Life Outcomes | Issue 1/2014

Abstract

Background

To examine whether lack of measurement invariance (MI) influences mean comparisons among different disease groups, this paper provides (1) a systematic review of MI in generic constructs across chronic conditions and (2) an empirical analysis of MI in the Health Education Impact Questionnaire (heiQ™).

Methods

(1) We searched for studies of MI among different chronic conditions in online databases. (2) Multigroup confirmatory factor analyses were used to study MI among five chronic conditions (orthopedic condition, rheumatism, asthma, COPD, cancer) in the heiQ™ with N = 1404 rehabilitation inpatients. Impact on latent and composite mean differences was examined.

Results

(1) A total of 30 relevant studies suggested that about one in three items lacked MI. However, only four studies examined impact on latent mean differences. Scale means were only affected in one of these three studies. (2) Across the eight heiQ™ scales, seven scales had items with lack of MI in at least one disease group. However, in only two heiQ™ scales were some latent or composite mean differences affected.

Conclusions

Lack of MI among disease groups is common and may have a relevant influence on mean comparisons when using generic instruments. Therefore, when comparing disease groups, tests of MI should be implemented. More studies of MI and according impact on mean differences in generic questionnaires are needed.

Available only for authorised users

Meredith W: Measurement invariance, factor analysis and factorial invariance. Psychometrika 1993, 58: 525–543. 10.1007/BF02294825CrossRef

Schmitt N, Kuljanin G: Measurement invariance: review of practice and implications. Hum Resour Manag Rev 2008, 18: 210–222. 10.1016/j.hrmr.2008.03.003CrossRef

Chen FF: What happens if we compare chopsticks with forks? The impact of making inappropriate comparisons in cross-cultural research. J Pers Soc Psychol 2008, 95: 1005–1018.CrossRefPubMed

Millsap RE: Statistical approaches to measurement invariance. New York, NY u.a: Psychology Press; 2011.

Steinmetz H: Analyzing observed composite differences across groups is partial measurement invariance enough? Meth Eur J Res Meth Behav Soc Sci 2013, 9: 1–12. 10.1027/1614-2241/a000049

Millsap RE, Kwok O-M: Evaluating the impact of partial factorial invariance on selection in two populations. Psychol Methods 2004, 9: 93–115.CrossRefPubMed

Meredith W, Teresi JA: An essay on measurement and factorial invariance. Med Care 2006, 44: 69–77. 10.1097/01.mlr.0000245438.73837.89CrossRef

Teresi JA: Overview of quantitative measurement methods. Equivalence, invariance, and differential item functioning in health applications. Med Care 2006, 44: S39-S49. 10.1097/01.mlr.0000245452.48613.45CrossRefPubMed

Millsap RE: Comments on methods for the investigation of measurement bias in the Mini-Mental State Examination. Med Care 2006, 44: S171-S175. 10.1097/01.mlr.0000245441.76388.ffCrossRefPubMed

10.

Borsboom D: When does measurement invariance matter? Med Care 2006, 44: S176-S181. 10.1097/01.mlr.0000245143.08679.ccCrossRefPubMed

11.

Gregorich SE: Do self-report instruments allow meaningful comparisons across diverse population groups? Testing measurement invariance using the confirmatory factor analysis framework. Med Care 2006, 44: S78-S94. 10.1097/01.mlr.0000245454.12228.8fPubMedCentralCrossRefPubMed

12.

Teresi JA: Different approaches to differential item functioning in health applications. Advantages, disadvantages and some neglected topics. Med Care 2006, 44: S152-S170. 10.1097/01.mlr.0000245142.74628.abCrossRefPubMed

13.

Schuler M, Jelitte M: Messen wir bei allen Personen das Gleiche? Zur Invarianz von Messungen und response shift in der rehabilitation - Teil 1. Die Rehabilitation 2012, 51: 332–339.CrossRefPubMed

14.

Byrne BM, Shavelson RJ, Muthén B: Testing for the Equivalence of Factor Covariance and Mean Structures - the Issue of Partial Measurement Invariance. Psychol Bull 1989, 105: 456–466.CrossRef

15.

Millsap RE, Meredith W: Factorial invariance: Historical perspectives and new problems. In Factor analysis at 100: Historical developments and future directions. Mahwah, NJ: Lawrence Erlbaum Associates Publishers; US; 2007:131–152.

16.

Schmitt N, Golubovich J, Leong FT: Impact of measurement invariance on construct correlations, mean differences, and relations with external correlates: an illustrative example using big five and RIASEC measures. Assessment 2011, 18: 412–427. 10.1177/1073191110373223CrossRefPubMed

17.

Sass DA: Testing measurement invariance and comparing latent factor means within a confirmatory factor analysis framework. J Psychoeduc Assess 2011, 29: 347–363. 10.1177/0734282911406661CrossRef

18.

Stark S, Chernyshenko OS, Drasgow F: Detecting differential item functioning with confirmatory factor analysis and item response theory: toward a unified strategy. J Appl Psychol 2006, 91: 1292–1306.CrossRefPubMed

19.

Teresi JA, Fleishman JA: Differential item functioning and health assessment. Qual Life Res 2007, 16(Suppl 1):33–42.CrossRefPubMed

20.

Saris WE, Satorra A, van der Veld WM: Testing structural equation models or detection of misspecifications? Struct Equ Model 2009, 16: 561–582. 10.1080/10705510903203433CrossRef

21.

De Beuckelaer A, Swinnen G: Biased latent variable mean comparisons due to measurement noninvariance: A simulation study. In European Association for Methodology series European Association for Methodology series. Edited by: Davidov E, Schmidt P, Billiet J. New York, NY: Routledge/Taylor & Francis Group; 2011:117–147.

22.

Teresi JA, Ramirez M, Lai J-s, Silver S: Occurrences and sources of Differential Item Functioning (DIF) in patient-reported outcome measures: Description of DIF methods, and review of measures of depression, quality of life and general health. Psychol Sci 2008, 50: 538–612.

23.

Vandenberg RJ, Lance CE: A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organ Res Methods 2000, 3: 4–69. 10.1177/109442810031002CrossRef

24.

Bode RK, Lai JS, Cella D, Heinemann AW: Issues in the development of an item bank. Arch Phys Med Rehabil 2003, 84: S52-S60.CrossRefPubMed

25.

Dallmeijer AJ, de Groot V, Roorda LD, Schepers VP, Lindeman E, van den Berg LH, Beelen A, Dekker J: Cross-diagnostic validity of the SF-36 physical functioning scale in patients with stroke, multiple sclerosis and amyotrophic lateral sclerosis: a study using Rasch analysis. J Rehabil Med 2007, 39: 163–169. 10.2340/16501977-0024CrossRefPubMed

26.

Dallmeijer AJ, Dekker J, Roorda LD, Knol DL, van Baalen B, de Groot V, Schepers VP, Lankhorst GJ: Differential item functioning of the functional independence measure in higher performing neurological patients. J Rehabil Med 2005, 37: 346–352. 10.1080/16501970510038284CrossRefPubMed

27.

Lindeboom R, Holman R, Dijkgraaf MG, Sprangers MA, Buskens E, Diederiks JP, De Haan RJ: Scaling the sickness impact profile using item response theory: an exploration of linearity, adaptive use, and patient driven item weights. J Clin Epidemiol 2004, 57: 66–74. 10.1016/S0895-4356(03)00212-9CrossRefPubMed

28.

Steultjens MP, Stolwijk-Swuste J, Roorda LD, Dallmeijer AJ, van Dijk GM, Post B, Dekker J: WOMAC-pf as a measure of physical function in patients with Parkinson’s disease and late-onset sequels of poliomyelitis: unidimensionality and item behaviour. Disabil Rehabil 2012, 34: 1423–1430. 10.3109/09638288.2011.645110CrossRefPubMed

29.

Taylor WJ, McPherson KM: Using Rasch analysis to compare the psychometric properties of the Short Form 36 physical function score and the health assessment questionnaire disability index in patients with psoriatic arthritis and rheumatoid arthritis. Arthritis Rheum 2007, 57: 723–729. 10.1002/art.22770CrossRefPubMed

30.

van Groen MM, ten Klooster PM, Taal E, van de Laar MA, Glas CA: Application of the health assessment questionnaire disability index to various rheumatic diseases. Qual Life Res 2010, 19: 1255–1263. 10.1007/s11136-010-9690-9PubMedCentralCrossRefPubMed

31.

Yu YF, Yu AP, Ahn J: Investigating differential item functioning by chronic diseases in the SF-36 health survey: a latent trait analysis using MIMIC models. Med Care 2007, 45: 851–859. 10.1097/MLR.0b013e318074ce4cCrossRefPubMed

32.

Lundgren-Nilsson A, Tennant A, Grimby G, Sunnerhagen KS: Cross-diagnostic validity in a generic instrument: an example from the functional independence measure in scandinavia. Health Qual Life Out 2006, 4: 55. 10.1186/1477-7525-4-55CrossRef

33.

Hart DL, Mioduski JE, Stratford PW: Simulated computerized adaptive tests for measuring functional status were efficient with good discriminant validity in patients with hip, knee, or foot/ankle impairments. J Clin Epidemiol 2005, 58: 629–638. 10.1016/j.jclinepi.2004.12.004CrossRefPubMed

34.

Pickard AS, Dalal MR, Bushnell DM: A comparison of depressive symptoms in stroke and primary care: applying Rasch models to evaluate the center for epidemiologic studies-depression scale. Value Health 2006, 9: 59–64. 10.1111/j.1524-4733.2006.00082.xCrossRefPubMed

35.

Reilly RE, Bowden SC, Bardenhagen FJ, Cook MJ: Equality of the psychological model underlying depressive symptoms in patients with temporal lobe epilepsy versus heterogeneous neurological disorders. J Clin Exp Neuropsychol 2006, 28: 1257–1271. 10.1080/13803390500376808CrossRefPubMed

36.

Waller NG, Compas BE, Hollon SD, Beckjord E: Measurement of depressive symptoms in women with breast cancer and women with clinical depression: a differential item functioning analysis. J Clin Psychol Med Settings 2005, 12: 127–141. 10.1007/s10880-005-3273-xCrossRef

37.

Wann-Hansson C, Klevsgard R, Hagell P: Cross-diagnostic validity of the Nottingham Health Profile Index of Distress (NHPD). Health Qual Life Out 2008, 6: 47. 10.1186/1477-7525-6-47CrossRef

38.

Atkinson TM, Rosenfeld BD, Sit L, Mendoza TR, Fruscione M, Lavene D, Shaw M, Li Y, Hay J, Cleeland CS, Scher HI, Breitbart WS, Basch E: Using confirmatory factor analysis to evaluate construct validity of the Brief Pain Inventory (BPI). J Pain Symptom Manag 2011, 41: 558–565. 10.1016/j.jpainsymman.2010.05.008CrossRef

39.

Chen CC, Bode RK: Psychometric validation of the Manual Ability Measure-36 (MAM-36) in patients with neurologic and musculoskeletal disorders. Arch Phys Med Rehabil 2010, 91: 414–420. 10.1016/j.apmr.2009.11.012CrossRefPubMed

40.

Coster WJ, Haley SM, Andres PL, Ludlow LH, Bond TL, Ni PS: Refining the conceptual basis for rehabilitation outcome measurement: personal care and instrumental activities domain. Med Care 2004, 42: I62-I72.CrossRefPubMed

41.

Haley SM, Coster WJ, Andres PL, Ludlow LH, Ni P, Bond TL, Sinclair SJ, Jette AM: Activity outcome measurement for postacute care. Med Care 2004, 42: I49-I61.CrossRefPubMed

42.

Weisscher N, Glas CA, Vermeulen M, De Haan RJ: The use of an item response theory-based disability item bank across diseases: accounting for differential item functioning. J Clin Epidemiol 2010, 63: 543–549. 10.1016/j.jclinepi.2009.07.016CrossRefPubMed

43.

Farin E, Fleitz A: The development of an ICF-oriented, adaptive physician assessment instrument of mobility, self-care, and domestic life. Int J Rehabil Res Internationale Zeitschrift fur Rehabilitationsforschung Revue internationale de recherches de readaptation 2009, 32: 98–107.CrossRefPubMed

44.

Yao G, Wu CH: Factorial invariance of the WHOQOL-BREF among disease groups. Qual Life Res 2005, 14: 1881–1888. 10.1007/s11136-005-3867-7CrossRefPubMed

45.

Moorer P, Suurmeije Th P, Foets M, Molenaar IW: Psychometric properties of the RAND-36 among three chronic diseases (multiple sclerosis, rheumatic diseases and COPD) in The Netherlands. Qual Life Res 2001, 10: 637–645. 10.1023/A:1013131617125CrossRefPubMed

46.

Yorke J, Horton M, Jones PW: A critique of Rasch analysis using the Dyspnoea-12 as an illustrative example. J Adv Nurs 2012, 68: 191–198. 10.1111/j.1365-2648.2011.05723.xCrossRefPubMed

47.

Roelofs J, Sluiter JK, Frings-Dresen MH, Goossens M, Thibault P, Boersma K, Vlaeyen JW: Fear of movement and (re)injury in chronic musculoskeletal pain: Evidence for an invariant two-factor model of the Tampa Scale for Kinesiophobia across pain diagnoses and Dutch, Swedish, and Canadian samples. Pain 2007, 131: 181–190. 10.1016/j.pain.2007.01.008CrossRefPubMed

48.

Prieto G, Delgado AR, Perea MV, Ladera V: Differential functioning of mini-mental test items according to disease. Neurologia 2011, 26: 474–480. 10.1016/j.nrl.2011.01.013CrossRefPubMed

49.

Chien TW, Wang WC, Lin SB, Lin CY, Guo HR, Su SB: KIDMAP, a web based system for gathering patients’ feedback on their doctors. BMC Med Res Methodol 2009, 9: 38. 10.1186/1471-2288-9-38PubMedCentralCrossRefPubMed

50.

Given CW, Given B, Stommel M, Collins C, King S, Franklin S: The caregiver reaction assessment (CRA) for caregivers to persons with chronic physical and mental impairments. Res Nurs Health 1992, 15: 271–283. 10.1002/nur.4770150406CrossRefPubMed

51.

Rao D, Choi SW, Victorson D, Bode R, Peterman A, Heinemann A, Cella D: Measuring stigma across neurological conditions: the development of the stigma scale for chronic illness (SSCI). Qual Life Res 2009, 18: 585–595. 10.1007/s11136-009-9475-1PubMedCentralCrossRefPubMed

52.

Wirtz M, Boecker M, Forkmann T, Neumann M: Evaluation of the “Consultation and Relational Empathy” (CARE) measure by means of Rasch-analysis at the example of cancer patients. Patient Educ Couns 2011, 82: 298–306. 10.1016/j.pec.2010.12.009CrossRefPubMed

53.

Wong ST, Nordstokke D, Gregorich S, Perez-Stable EJ: Measurement of social support across women from four ethnic groups: evidence of factorial invariance. J Cross Cult Gerontol 2010, 25: 45–58. 10.1007/s10823-010-9111-0PubMedCentralCrossRefPubMed

54.

Schuler M, Musekamp G, Faller H, Ehlebracht-König I, Gutenbrunner C, Kirchhof R, Bengel J, Nolte S, Osborne RH, Schwarze M: Assessment of proximal outcomes of self-management programs: translation and psychometric evaluation of a German version of the Health Education Impact Questionnaire (heiQ). Qual Life Res 2013, 22: 1391–1403. 10.1007/s11136-012-0268-6CrossRefPubMed

55.

Osborne RH, Elsworth GR, Whitfield K: The Health Education Impact Questionnaire (heiQ): an outcomes and evaluation measure for patient education and self-management interventions for people with chronic conditions. Patient Educ Couns 2007, 66: 192–201. 10.1016/j.pec.2006.12.002CrossRefPubMed

56.

Crotty M, Prendergast J, Battersby MW, Rowett D, Graves SE, Leach G, Giles LC: Self-management and peer support among people with arthritis on a hospital joint replacement waiting list: a randomised controlled trial. Osteoarthr Cartil 2009, 17: 1428–1433. 10.1016/j.joca.2009.05.010CrossRefPubMed

57.

Francis KL, Matthews BL, Van Mechelen W, Bennell KL, Osborne RH: Effectiveness of a community-based osteoporosis education and self-management course: a wait list controlled trial. Osteoporos Int 2009, 20: 1563–1570. 10.1007/s00198-009-0834-0CrossRefPubMed

58.

Nolte S, Elsworth GR, Sinclair AJ, Osborne RH: The extent and breadth of benefits from participating in chronic disease self-management courses: a national patient-reported outcomes survey. Patient Educ Couns 2007, 65: 351–360. 10.1016/j.pec.2006.08.016CrossRefPubMed

59.

Packer TL, Boldy D, Ghahari S, Melling L, Parsons R, Osborne RH: Self-management programs conducted within a practice setting: Who participates, who benefits and what can be learned? Patient Educ Couns 2012, 87: 93–100. 10.1016/j.pec.2011.09.007CrossRefPubMed

60.

Kroon FP, van der Burg LR, Buchbinder R, Osborne RH, Johnston RV, Pitt V: Self-management education programmes for osteoarthritis. Cochrane Database Syst Rev 2014, 1: CD008963.PubMed

61.

Osborne RH, Batterham R, Livingston J: The evaluation of chronic disease self-management support across settings: the international experience of the health education impact questionnaire quality monitoring system. Nurs Clin North Am 2011, 46: 255–270. 10.1016/j.cnur.2011.05.010CrossRefPubMed

62.

Nolte S, Elsworth GR, Sinclair AJ, Osborne RH: Tests of measurement invariance failed to support the application of the “then-test”. J Clin Epidemiol 2009, 62: 1173–1180. 10.1016/j.jclinepi.2009.01.021CrossRefPubMed

63.

Sprangers MA, Schwartz CE: Integrating response shift into health-related quality of life research: a theoretical model. Soc Sci Med 1999, 48: 1507–1515. 10.1016/S0277-9536(99)00045-3CrossRefPubMed

64.

Muthén B: Latent variable modeling in heterogeneous populations. Psychometrika 1989, 54: 557–585. 10.1007/BF02296397CrossRef

65.

Muthén LK, Muthén B: Mplus User’s Guide. Muthén & Muthén: Los Angeles; 2010.

66.

Yoon M, Millsap RE: Detecting violations of factorial invariance using data-based specification searches: A Monte Carlo study. Struct Equ Model 2007, 14: 453–463.CrossRef

67.

L-t H, Bentler PM: Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Model 1999, 6: 1–55. 10.1080/10705519909540118CrossRef

68.

Oberski DJ: Jrule for Mplus. 091st edition. 2009. http://wiki.github.com/daob/JruleMplus/

69.

van der Veld WM, Saris WE: Causes of generalized social trust. In European Association for Methodology series. Edited by: Davidov E, Schmidt P, Billiet J. New York, NY: Routledge/Taylor & Francis Group; 2011:207–247.

70.

Revilla MA: Measurment invariance and quality of composite scores in a face-to-face and web survey. Surv Res Methods 2013, 7: 17–28.

71.

Saris WE, Satorra A, Sörbom D: The detection and correction of specification errors in structural equation models. Sociol Methodol 1987, 17: 105–129.CrossRef

72.

Hancock GR: Effect size, power, and sample size determination for structured means modeling and MIMIC approaches to between-groups hypothesis testing of means on a single latent construct. Psychometrika 2001, 66: 373–388. 10.1007/BF02294440CrossRef

73.

Cohen J: Statistical power analysis for the behavioral sciences. 2. 2print edition. Hillsdale, NJ u.a: Erlbaum; 1988.

74.

Chen FF: Sensitivity of goodness of fit indexes to lack of measurement invariance. Struct Equ Model 2007, 14: 464–504. 10.1080/10705510701301834CrossRef

75.

Meade AW: A taxonomy of effect size measures for the differential functioning of items and scales. J Appl Soc Psychol 2010, 95: 728–743.CrossRef

76.

Nye CD, Drasgow F: Effect size indices for analyses of measurement equivalence: understanding the practical importance of differences between groups. J Appl Soc Psychol 2011, 96: 966–980.CrossRef

77.

Muthén B, Asparouhov T: BSEM measurement invariance analysis. Mplus Webnote 17. 2013. [http://www.statmodel.com/examples/webnotes/webnote17.pdf]

Title: Measurement invariance across chronic conditions: a systematic review and an empirical investigation of the Health Education Impact Questionnaire (heiQ™)
Authors: Michael Schuler
Gunda Musekamp
Jürgen Bengel
Sandra Nolte
Richard H Osborne
Hermann Faller
Publication date: 01-12-2014
Publisher: BioMed Central
Published in: Health and Quality of Life Outcomes / Issue 1/2014
Electronic ISSN: 1477-7525
DOI: https://doi.org/10.1186/1477-7525-12-56

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Measurement invariance across chronic conditions: a systematic review and an empirical investigation of the Health Education Impact Questionnaire (heiQ™)

Abstract

Background

Methods

Results

Conclusions

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2014

Patients' expectations from dental implants: a systematic review of the literature

Mapping of Incontinence Quality of Life (I-QOL) scores to Assessment of Quality of Life 8D (AQoL-8D) utilities in patients with idiopathic overactive bladder

Landless female peasants living in resettlement residential areas in China have poorer quality of life than males: results from a household study in the Yangtze River Delta region

The role of quality of care and attitude towards disability in the relationship between severity of disability and quality of life: findings from a cross-sectional survey among people with physical disability in China

Using item response theory to enrich and expand the PROMIS® pediatric self report banks

Health-related quality of life and related factors of military police officers