Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2011

Open Access 01-12-2011 | Research article

Re-evaluating a vision-related quality of life questionnaire with item response theory (IRT) and differential item functioning (DIF) analyses

Authors: Ruth MA van Nispen, Dirk L Knol, Maaike Langelaan, Ger HMB van Rens

Published in: BMC Medical Research Methodology | Issue 1/2011

Login to get access

Abstract

Background

For the Low Vision Quality Of Life questionnaire (LVQOL) it is unknown whether the psychometric properties are satisfactory when an item response theory (IRT) perspective is considered. This study evaluates some essential psychometric properties of the LVQOL questionnaire in an IRT model, and investigates differential item functioning (DIF).

Methods

Cross-sectional data were used from an observational study among visually-impaired patients (n = 296). Calibration was performed for every dimension of the LVQOL in the graded response model. Item goodness-of-fit was assessed with the S-X2-test. DIF was assessed on relevant background variables (i.e. age, gender, visual acuity, eye condition, rehabilitation type and administration type) with likelihood-ratio tests for DIF. The magnitude of DIF was interpreted by assessing the largest difference in expected scores between subgroups. Measurement precision was assessed by presenting test information curves; reliability with the index of subject separation.

Results

All items of the LVQOL dimensions fitted the model. There was significant DIF on several items. For two items the maximum difference between expected scores exceeded one point, and DIF was found on multiple relevant background variables. Item 1 'Vision in general' from the "Adjustment" dimension and item 24 'Using tools' from the "Reading and fine work" dimension were removed. Test information was highest for the "Reading and fine work" dimension. Indices for subject separation ranged from 0.83 to 0.94.

Conclusions

The items of the LVQOL showed satisfactory item fit to the graded response model; however, two items were removed because of DIF. The adapted LVQOL with 21 items is DIF-free and therefore seems highly appropriate for use in heterogeneous populations of visually impaired patients.
Appendix
Available only for authorised users
Literature
1.
go back to reference Stelmack J: Quality of life of low-vision patients and outcomes of low-vision rehabilitation. Optom Vis Sci. 2001, 78: 335-342. 10.1097/00006324-200105000-00017.CrossRefPubMed Stelmack J: Quality of life of low-vision patients and outcomes of low-vision rehabilitation. Optom Vis Sci. 2001, 78: 335-342. 10.1097/00006324-200105000-00017.CrossRefPubMed
2.
go back to reference Klaver CC, Wolfs RC, Vingerling JR, Hofman A, de Jong PT: Age-specific prevalence and causes of blindness and visual impairment in an older population: the Rotterdam Study. Arch Ophthalmol. 1998, 116: 653-658.CrossRefPubMed Klaver CC, Wolfs RC, Vingerling JR, Hofman A, de Jong PT: Age-specific prevalence and causes of blindness and visual impairment in an older population: the Rotterdam Study. Arch Ophthalmol. 1998, 116: 653-658.CrossRefPubMed
3.
go back to reference McCabe P, Nason F, Demers TP, Friedman D, Seddon JM: Evaluating the effectiveness of a vision rehabilitation intervention using an objective and subjective measure of functional performance. Ophthalmic Epidemiol. 2000, 7: 259-270. 10.1076/opep.7.4.259.4173.CrossRefPubMed McCabe P, Nason F, Demers TP, Friedman D, Seddon JM: Evaluating the effectiveness of a vision rehabilitation intervention using an objective and subjective measure of functional performance. Ophthalmic Epidemiol. 2000, 7: 259-270. 10.1076/opep.7.4.259.4173.CrossRefPubMed
4.
go back to reference de Boer MR, Twisk J, Moll AC, Volker-Dieben HJM, de Vet HCW, van Rens GHMB: Outcomes of low vision services using optometric and multidisciplinary approaches: a non-randomized comparison. Ophthalmic Physiol Opt. 2006, 26: 535-544. 10.1111/j.1475-1313.2006.00424.x.CrossRefPubMed de Boer MR, Twisk J, Moll AC, Volker-Dieben HJM, de Vet HCW, van Rens GHMB: Outcomes of low vision services using optometric and multidisciplinary approaches: a non-randomized comparison. Ophthalmic Physiol Opt. 2006, 26: 535-544. 10.1111/j.1475-1313.2006.00424.x.CrossRefPubMed
5.
go back to reference van Nispen RMA, Knol DL, Langelaan M, de Boer MR, Terwee CB, van Rens GHMB: Applying multilevel item response theory to vision-related quality of life in Dutch visually impaired elderly. Optom Vis Sci. 2007, 84: 710-720. 10.1097/OPX.0b013e31813375b8.CrossRefPubMed van Nispen RMA, Knol DL, Langelaan M, de Boer MR, Terwee CB, van Rens GHMB: Applying multilevel item response theory to vision-related quality of life in Dutch visually impaired elderly. Optom Vis Sci. 2007, 84: 710-720. 10.1097/OPX.0b013e31813375b8.CrossRefPubMed
6.
go back to reference Birk T, Hickl S, Wahl HW, Miller D, Kämmerer A, Holz F, Becker S, Völcker HE: Development and pilot evaluation of a psychosocial intervention program for patients with age-related macular degeneration. Gerontologist. 2004, 44: 836-843. 10.1093/geront/44.6.836.CrossRefPubMed Birk T, Hickl S, Wahl HW, Miller D, Kämmerer A, Holz F, Becker S, Völcker HE: Development and pilot evaluation of a psychosocial intervention program for patients with age-related macular degeneration. Gerontologist. 2004, 44: 836-843. 10.1093/geront/44.6.836.CrossRefPubMed
7.
go back to reference Reeves BC, Harper RA, Russell WB: Enhanced low vision rehabilitation for people with age related macular degeneration: a randomised controlled trial. Br J Ophthalmol. 2004, 88: 1443-1449. 10.1136/bjo.2003.037457.CrossRefPubMedPubMedCentral Reeves BC, Harper RA, Russell WB: Enhanced low vision rehabilitation for people with age related macular degeneration: a randomised controlled trial. Br J Ophthalmol. 2004, 88: 1443-1449. 10.1136/bjo.2003.037457.CrossRefPubMedPubMedCentral
8.
go back to reference Hinds A, Sinclair A, Park J, Suttie A, Paterson H, Macdonald M: Impact of an interdisciplinary low vision service on the quality of life of low vision patients. Br J Ophthalmol. 2003, 87: 1391-1396. 10.1136/bjo.87.11.1391.CrossRefPubMedPubMedCentral Hinds A, Sinclair A, Park J, Suttie A, Paterson H, Macdonald M: Impact of an interdisciplinary low vision service on the quality of life of low vision patients. Br J Ophthalmol. 2003, 87: 1391-1396. 10.1136/bjo.87.11.1391.CrossRefPubMedPubMedCentral
9.
go back to reference Finger R, Fleckenstein M, Holz F, Scholl H: Quality of life in age-related macular degeneration: a review of available vision-specific psychometric tools. Qual Life Res. 2008, 17: 559-574. 10.1007/s11136-008-9327-4.CrossRefPubMed Finger R, Fleckenstein M, Holz F, Scholl H: Quality of life in age-related macular degeneration: a review of available vision-specific psychometric tools. Qual Life Res. 2008, 17: 559-574. 10.1007/s11136-008-9327-4.CrossRefPubMed
10.
go back to reference van Nispen RMA, de Boer MR, van Rens GHMB: Additional psychometric information and vision-specific questionnaires are available for age-related macular degeneration. Qual Life Res. 2009, 18: 65-69. 10.1007/s11136-008-9425-3.CrossRefPubMed van Nispen RMA, de Boer MR, van Rens GHMB: Additional psychometric information and vision-specific questionnaires are available for age-related macular degeneration. Qual Life Res. 2009, 18: 65-69. 10.1007/s11136-008-9425-3.CrossRefPubMed
11.
go back to reference de Boer MR, Moll AC, de Vet HCW, Terwee CB, Volker-Dieben HJM, van Rens GHMB: Psychometric properties of vision-related quality of life questionnaires: a systematic review. Ophthalmic Physiol Opt. 2004, 24: 257-273. 10.1111/j.1475-1313.2004.00187.x.CrossRefPubMed de Boer MR, Moll AC, de Vet HCW, Terwee CB, Volker-Dieben HJM, van Rens GHMB: Psychometric properties of vision-related quality of life questionnaires: a systematic review. Ophthalmic Physiol Opt. 2004, 24: 257-273. 10.1111/j.1475-1313.2004.00187.x.CrossRefPubMed
12.
go back to reference Wolffsohn JS, Cochrane AL: Design of the low vision quality-of-life questionnaire (LVQOL) and measuring the outcome of low-vision rehabilitation. Am J Ophthalmol. 2000, 130: 793-802. 10.1016/S0002-9394(00)00610-3.CrossRefPubMed Wolffsohn JS, Cochrane AL: Design of the low vision quality-of-life questionnaire (LVQOL) and measuring the outcome of low-vision rehabilitation. Am J Ophthalmol. 2000, 130: 793-802. 10.1016/S0002-9394(00)00610-3.CrossRefPubMed
13.
go back to reference de Boer MR, de Vet HCW, Terwee CB, Moll AC, Volker-Dieben HJM, van Rens GHMB: Changes to the subscales of two vision-related quality of life questionnaires are proposed. J Clin Epidemiol. 2005, 58: 1260-1268. 10.1016/j.jclinepi.2005.04.007.CrossRefPubMed de Boer MR, de Vet HCW, Terwee CB, Moll AC, Volker-Dieben HJM, van Rens GHMB: Changes to the subscales of two vision-related quality of life questionnaires are proposed. J Clin Epidemiol. 2005, 58: 1260-1268. 10.1016/j.jclinepi.2005.04.007.CrossRefPubMed
14.
go back to reference de Boer MR, Terwee CB, de Vet HCW, Moll AC, Volker-Dieben HJM, van Rens GHMB: Evaluation of cross-sectional and longitudinal construct validity of two vision-related quality of life questionnaires: the LVQOL and VCM1. Qual Life Res. 2006, 15: 233-248. 10.1007/s11136-005-1524-9.CrossRefPubMed de Boer MR, Terwee CB, de Vet HCW, Moll AC, Volker-Dieben HJM, van Rens GHMB: Evaluation of cross-sectional and longitudinal construct validity of two vision-related quality of life questionnaires: the LVQOL and VCM1. Qual Life Res. 2006, 15: 233-248. 10.1007/s11136-005-1524-9.CrossRefPubMed
15.
go back to reference van Nispen RMA, Knol DL, Neve JJ, van Rens GHMB: A multilevel item response theory model was investigated for longitudinal vision-related quality of life data. J Clin Epidemiol. 2010, 63: 321-330. 10.1016/j.jclinepi.2009.06.012.CrossRefPubMed van Nispen RMA, Knol DL, Neve JJ, van Rens GHMB: A multilevel item response theory model was investigated for longitudinal vision-related quality of life data. J Clin Epidemiol. 2010, 63: 321-330. 10.1016/j.jclinepi.2009.06.012.CrossRefPubMed
16.
go back to reference Reeve BB, Hays RD, Chang C-H, Perfetto EM: Applying item response theory to enhance health outcomes assessment. Qual Life Res. 2007, 16: 1-3.CrossRef Reeve BB, Hays RD, Chang C-H, Perfetto EM: Applying item response theory to enhance health outcomes assessment. Qual Life Res. 2007, 16: 1-3.CrossRef
17.
go back to reference Massof RW: An interval-scaled scoring algorithm for visual function questionnaires. Optom Vis Sci. 2007, 84: 689-704.CrossRef Massof RW: An interval-scaled scoring algorithm for visual function questionnaires. Optom Vis Sci. 2007, 84: 689-704.CrossRef
18.
go back to reference Langelaan M, van Nispen RMA, Knol DL, Moll AC, de Boer MR, Wouters B, van Rens GHMB: Visual Functioning Questionnaire: reevaluation of psychometric properties for a group of working-age adults. Optom Vis Sci. 2007, 84: 775-784. 10.1097/OPX.0b013e3181334b98.CrossRefPubMed Langelaan M, van Nispen RMA, Knol DL, Moll AC, de Boer MR, Wouters B, van Rens GHMB: Visual Functioning Questionnaire: reevaluation of psychometric properties for a group of working-age adults. Optom Vis Sci. 2007, 84: 775-784. 10.1097/OPX.0b013e3181334b98.CrossRefPubMed
19.
go back to reference Lamoureux E, Pesudovs K, Pallant J, Rees G, Hassell JB, Caudle LE, Keeffe JE: An evaluation of the 10-item Vision Core Measure 1 (VCM1) scale (the Core Module of the Vision-related Quality of Life scale) using Rasch analysis. Ophthalmic Epidemiol. 2008, 15: 224-233. 10.1080/09286580802256559.CrossRefPubMed Lamoureux E, Pesudovs K, Pallant J, Rees G, Hassell JB, Caudle LE, Keeffe JE: An evaluation of the 10-item Vision Core Measure 1 (VCM1) scale (the Core Module of the Vision-related Quality of Life scale) using Rasch analysis. Ophthalmic Epidemiol. 2008, 15: 224-233. 10.1080/09286580802256559.CrossRefPubMed
20.
go back to reference Embretson S, Reise S: Item response theory for psychologists. 2000, Mahwah, NJ: Erlbaum Embretson S, Reise S: Item response theory for psychologists. 2000, Mahwah, NJ: Erlbaum
21.
go back to reference Tutz G: Sequential item response models with an ordered response. Brit J Math Stat Psychol. 1990, 43: 39-55. 10.1111/j.2044-8317.1990.tb00925.x.CrossRef Tutz G: Sequential item response models with an ordered response. Brit J Math Stat Psychol. 1990, 43: 39-55. 10.1111/j.2044-8317.1990.tb00925.x.CrossRef
22.
go back to reference van Engelenburg G: On psychometric models for polytomous items with ordered categories within the framework of item response theory. 1997, University of Amsterdam, the Netherlands van Engelenburg G: On psychometric models for polytomous items with ordered categories within the framework of item response theory. 1997, University of Amsterdam, the Netherlands
23.
go back to reference Akkermans LMW: Studies on statistical models for polytomously scored test items. 1998, University of Twente, the Netherlands Akkermans LMW: Studies on statistical models for polytomously scored test items. 1998, University of Twente, the Netherlands
24.
go back to reference Skrondal A, Rabe-Hesketh S: Generalized latent variable modeling: multilevel, longitudinal, and structural equation models. 2004, London, UK: Chapman & Hall, 113-CrossRef Skrondal A, Rabe-Hesketh S: Generalized latent variable modeling: multilevel, longitudinal, and structural equation models. 2004, London, UK: Chapman & Hall, 113-CrossRef
25.
go back to reference Sass DA, Schmitt TA, Walker CM: Estimating non-normal latent trait distributions within item response theory using true and estimated item parameters. Appl Meas Educat. 2008, 21: 65-88. 10.1080/08957340701796415.CrossRef Sass DA, Schmitt TA, Walker CM: Estimating non-normal latent trait distributions within item response theory using true and estimated item parameters. Appl Meas Educat. 2008, 21: 65-88. 10.1080/08957340701796415.CrossRef
26.
go back to reference Orlando Edelen M, Reeve BB: Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement. Qual Life Res. 2007, 16: 5-18. 10.1007/s11136-007-9198-0.CrossRef Orlando Edelen M, Reeve BB: Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement. Qual Life Res. 2007, 16: 5-18. 10.1007/s11136-007-9198-0.CrossRef
27.
go back to reference Crane P, van Belle G, Larson E: Test bias in a cognitive test: differential item functioning in the CASI. Stat Med. 2004, 23: 241-256. 10.1002/sim.1713.CrossRefPubMed Crane P, van Belle G, Larson E: Test bias in a cognitive test: differential item functioning in the CASI. Stat Med. 2004, 23: 241-256. 10.1002/sim.1713.CrossRefPubMed
28.
go back to reference Teresi J, Fleishman J: Differential item functioning and health assessment. Qual Life Res. 2007, 16: 33-42. 10.1007/s11136-007-9184-6.CrossRefPubMed Teresi J, Fleishman J: Differential item functioning and health assessment. Qual Life Res. 2007, 16: 33-42. 10.1007/s11136-007-9184-6.CrossRefPubMed
29.
go back to reference Reeve BB, Hays RD, Bjorner JB, Cook KF, Crane PK, Teresi JA, Thissen D, Revicki DA, Weiss DJ, Hambleton RK, Liu H, Gershon R, Reise SP, Lai J, Cella D: Psychometric evaluation and calibration of health-related quality of life item banks. Plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Med Care. 2007, 45: S22-S31. 10.1097/01.mlr.0000250483.85507.04.CrossRefPubMed Reeve BB, Hays RD, Bjorner JB, Cook KF, Crane PK, Teresi JA, Thissen D, Revicki DA, Weiss DJ, Hambleton RK, Liu H, Gershon R, Reise SP, Lai J, Cella D: Psychometric evaluation and calibration of health-related quality of life item banks. Plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Med Care. 2007, 45: S22-S31. 10.1097/01.mlr.0000250483.85507.04.CrossRefPubMed
30.
go back to reference Samejima F: Estimation of latent ability using response pattern of graded scores. 1969, Psychometric Monograph Supplement No 17: Richmond, VA: William Byrd Press Samejima F: Estimation of latent ability using response pattern of graded scores. 1969, Psychometric Monograph Supplement No 17: Richmond, VA: William Byrd Press
31.
go back to reference Thissen D: MULTILOG™ User's guide. Multiple, categorical item analysis and test scoring using item response theory. 1991, Chicago: Scientific Software Inc. Thissen D: MULTILOG™ User's guide. Multiple, categorical item analysis and test scoring using item response theory. 1991, Chicago: Scientific Software Inc.
33.
go back to reference Orlando M, Thissen D: Likelihood-based item-fit indices for dichotomous item response theory models. Appl Psychol Meas. 2000, 24: 50-64. 10.1177/01466216000241003.CrossRef Orlando M, Thissen D: Likelihood-based item-fit indices for dichotomous item response theory models. Appl Psychol Meas. 2000, 24: 50-64. 10.1177/01466216000241003.CrossRef
34.
go back to reference Orlando M, Thissen D: Further examination of the performance of S-X2, an item fit index for dichotomous item response theory models. Appl Psychol Meas. 2003, 27: 289-298. 10.1177/0146621603027004004.CrossRef Orlando M, Thissen D: Further examination of the performance of S-X2, an item fit index for dichotomous item response theory models. Appl Psychol Meas. 2003, 27: 289-298. 10.1177/0146621603027004004.CrossRef
35.
go back to reference Teresi J, Ocepek-Welikson K, Kleinman M, Cook KF, Crane PK, Gibbons LE, Morales LS, Orlando-Edelen M, Cella D: Evaluating measurement equivalence using item response theory log-likelihood ratio (IRTLR) method to assess differential item functioning (DIF): applications (with illustrations) to measures of physical functioning ability and general distress. Qual Life Res. 2007, 16: 43-68. 10.1007/s11136-007-9186-4.CrossRefPubMed Teresi J, Ocepek-Welikson K, Kleinman M, Cook KF, Crane PK, Gibbons LE, Morales LS, Orlando-Edelen M, Cella D: Evaluating measurement equivalence using item response theory log-likelihood ratio (IRTLR) method to assess differential item functioning (DIF): applications (with illustrations) to measures of physical functioning ability and general distress. Qual Life Res. 2007, 16: 43-68. 10.1007/s11136-007-9186-4.CrossRefPubMed
36.
go back to reference Thissen D: IRTLRDIF v.2.0b: Software for the computation of the statistics involved in item response theory likelihood-ratio tests for differential item functioning. 2001, Chapel Hill, NC: L.L. Thurstone Psychometric Laboratory, University of North Carolina at Chapel Hill Thissen D: IRTLRDIF v.2.0b: Software for the computation of the statistics involved in item response theory likelihood-ratio tests for differential item functioning. 2001, Chapel Hill, NC: L.L. Thurstone Psychometric Laboratory, University of North Carolina at Chapel Hill
38.
go back to reference Langer M, Hill C, Thissen D, Burwinkle T, Varni J, DeWalt D: Item response theory detected differential item functioning between healthy and ill children in quality-of-life measures. J Clin Epidemiol. 2008, 61: 268-276. 10.1016/j.jclinepi.2007.05.002.CrossRefPubMed Langer M, Hill C, Thissen D, Burwinkle T, Varni J, DeWalt D: Item response theory detected differential item functioning between healthy and ill children in quality-of-life measures. J Clin Epidemiol. 2008, 61: 268-276. 10.1016/j.jclinepi.2007.05.002.CrossRefPubMed
39.
go back to reference Gustafsson J: The Rasch model for dichotomous items: Theory, applications and a computer program. 1977, (Internal Rep No. 63) Institute of Education, University of Goteborg Gustafsson J: The Rasch model for dichotomous items: Theory, applications and a computer program. 1977, (Internal Rep No. 63) Institute of Education, University of Goteborg
40.
go back to reference Wolffsohn JS, Cochrane AL, Watt NA: Implementation methods for vision related quality of life questionnaires. Br J Ophthalmol. 2000, 84: 1035-1040. 10.1136/bjo.84.9.1035.CrossRefPubMedPubMedCentral Wolffsohn JS, Cochrane AL, Watt NA: Implementation methods for vision related quality of life questionnaires. Br J Ophthalmol. 2000, 84: 1035-1040. 10.1136/bjo.84.9.1035.CrossRefPubMedPubMedCentral
41.
go back to reference van Nispen RMA, Knol DL, Mokkink LB, Comijs HC, Deeg DJH, van Rens GHMB: Vision-related quality of life Core Measure (VCM1) showed low-impact differential item functioning between groups with different administration modes. J Clin Epidemiol. 2010, 63: 1232-1241. 10.1016/j.jclinepi.2009.12.010.CrossRefPubMed van Nispen RMA, Knol DL, Mokkink LB, Comijs HC, Deeg DJH, van Rens GHMB: Vision-related quality of life Core Measure (VCM1) showed low-impact differential item functioning between groups with different administration modes. J Clin Epidemiol. 2010, 63: 1232-1241. 10.1016/j.jclinepi.2009.12.010.CrossRefPubMed
42.
go back to reference Schwartz N, Strack F, Hippler H, Bishop G: The impact of administration mode on response effects in survey measurement. Appl Cognitive Psychol. 1991, 5: 193-212. 10.1002/acp.2350050304.CrossRef Schwartz N, Strack F, Hippler H, Bishop G: The impact of administration mode on response effects in survey measurement. Appl Cognitive Psychol. 1991, 5: 193-212. 10.1002/acp.2350050304.CrossRef
43.
go back to reference Raju NS, Oshima TC: Two prophecy formulas for assessing the reliability of item response theory-based ability estimates. Educat Psychol Meas. 2005, 65: 361-10.1177/0013164404267289.CrossRef Raju NS, Oshima TC: Two prophecy formulas for assessing the reliability of item response theory-based ability estimates. Educat Psychol Meas. 2005, 65: 361-10.1177/0013164404267289.CrossRef
44.
go back to reference Samejima F: Estimation of reliability coefficients using the test information function and its modifications. Appl Psychol Meas. 1994, 18: 229-10.1177/014662169401800304.CrossRef Samejima F: Estimation of reliability coefficients using the test information function and its modifications. Appl Psychol Meas. 1994, 18: 229-10.1177/014662169401800304.CrossRef
Metadata
Title
Re-evaluating a vision-related quality of life questionnaire with item response theory (IRT) and differential item functioning (DIF) analyses
Authors
Ruth MA van Nispen
Dirk L Knol
Maaike Langelaan
Ger HMB van Rens
Publication date
01-12-2011
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2011
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/1471-2288-11-125

Other articles of this Issue 1/2011

BMC Medical Research Methodology 1/2011 Go to the issue