Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on medication adherence

LIVE: Thursday 27th June 2024, 18:00-19:30 (CEST)
Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.
ADVANCED SEARCH
Log in
Top
Health and Quality of Life Outcomes
Published in: Health and Quality of Life Outcomes 1/2010

Open Access 01-12-2010 | Research

Differential item functioning (DIF) analyses of health-related quality of life instruments using logistic regression

Authors: Neil W Scott, Peter M Fayers, Neil K Aaronson, Andrew Bottomley, Alexander de Graeff, Mogens Groenvold, Chad Gundy, Michael Koller, Morten A Petersen, Mirjam AG Sprangers, the EORTC Quality of Life Group and the Quality of Life Cross-Cultural Meta-Analysis Group

Published in: Health and Quality of Life Outcomes | Issue 1/2010

Login to get access

Abstract

Background

Differential item functioning (DIF) methods can be used to determine whether different subgroups respond differently to particular items within a health-related quality of life (HRQoL) subscale, after allowing for overall subgroup differences in that scale. This article reviews issues that arise when testing for DIF in HRQoL instruments. We focus on logistic regression methods, which are often used because of their efficiency, simplicity and ease of application.

Methods

A review of logistic regression DIF analyses in HRQoL was undertaken. Methodological articles from other fields and using other DIF methods were also included if considered relevant.

Results

There are many competing approaches for the conduct of DIF analyses and many criteria for determining what constitutes significant DIF. DIF in short scales, as commonly found in HRQL instruments, may be more difficult to interpret. Qualitative methods may aid interpretation of such DIF analyses.

Conclusions

A number of methodological choices must be made when applying logistic regression for DIF analyses, and many of these affect the results. We provide recommendations based on reviewing the current evidence. Although the focus is on logistic regression, many of our results should be applicable to DIF analyses in general. There is a need for more empirical and theoretical work in this area.
Literature
1.
Zumbo BD: A handbook on the theory and methods of differential item functioning (DIF): Logistic regression modeling as a unitary framework for binary and Likert-type (ordinal) item scores. Ottowa, ON: Directorate of Human Research and Evaluation, Department of National Defense; 1999.
2.
Crane PK, Gibbons LE, Jolley L, van Belle G: Differential item functioning analysis with ordinal logistic regression techniques: DIFdetect and difwithpar. Med Care 2006, 44: S115-S123. 10.1097/01.mlr.0000245183.28384.edPubMedCrossRef
3.
Gelin MN, Carleton BC, Smith MA, Zumbo BD: The dimensionality and gender differential item functioning of the mini asthma quality of life questionnaire (MINIAQLQ). Soc Indicators Res 2004, 68: 91–105. 10.1023/B:SOCI.0000025580.54702.90CrossRef
4.
Groenvold M, Bjorner JB, Klee MC, Kreiner S: Test for item bias in a quality of life questionnaire. J Clin Epidemiol 1995, 48: 805–816. 10.1016/0895-4356(94)00195-VPubMedCrossRef
5.
Hahn EA, Holzner B, Kemmler G, Sperner-Unterweger B, Hudgens SA, Cella D: Cross-cultural evaluation of health status using item response theory: FACT-B comparisons between Austrian and U.S. patients with breast cancer. Eval Health Prof 2005, 28: 233–259. 10.1177/0163278705275343PubMedCrossRef
6.
Pagano IS, Gotay CC: Ethnic differential item functioning in the assessment of quality of life in cancer patients. Health and Quality of Life Outcomes 2005, 3: 1–10. 10.1186/1477-7525-3-60CrossRef
7.
Petersen MA, Groenvold M, Bjorner JB, Aaronson N, Conroy T, Cull A, Fayers P, Hjermstad M, Sprangers M, Sullivan M, European Organisation for Research and Treatment of Cancer Quality of Life, Group: Use of differential item functioning analysis to assess the equivalence of translations of a questionnaire. Quality of Life Research 2003, 12: 373–385. 10.1023/A:1023488915557PubMedCrossRef
8.
Scott NW, Fayers PM, Bottomley A, Aaronson NK, de Graeff A, Groenvold M, Koller M, Petersen MA, Sprangers MAG: Comparing translations of the EORTC QLQ-C30 using differential item functioning analyses. Quality of Life Research 2006, 15: 1103–1115. 10.1007/s11136-006-0040-xPubMedCrossRef
9.
Scott NW, Fayers PM, Aaronson NK, Bottomley A, de Graeff A, Groenvold M, Koller M, Petersen MA, Sprangers MAG: The use of differential item functioning analyses to identify cultural differences in responses to the EORTC QLQ-C30. Quality of Life Research 2007, 16: 115–129. 10.1007/s11136-006-9120-1PubMedCrossRef
10.
Bjorner JB, Kosinski M, Ware JE: Calibration of an item pool for assessing the burden of headaches: An application of item response theory to the headache impact test (HIT). Quality of Life Research 2003, 12: 913–933. 10.1023/A:1026163113446PubMedCrossRef
11.
Martin M, Blaisdell B, Kwong JW, Bjorner JB: The short-form headache impact test (HIT-6) was psychometrically equivalent in nine languages. J Clin Epidemiol 2004, 57: 1271–1278. 10.1016/j.jclinepi.2004.05.004PubMedCrossRef
12.
Azocar F, Arean P, Miranda J, Munoz RF: Differential item functioning in a Spanish translation of the Beck Depression Inventory. J Clin Psychol 2001, 57: 355–365. 10.1002/jclp.1017PubMedCrossRef
13.
Dancer LS, Anderson AJ, Derlin RL: Use of log-linear models for assessing differential item functioning in a measure of psychological functioning. Journal of Consulting & Clinical Psychology 1994, 62: 710–717.CrossRef
14.
Gelin MN, Zumbo BD: Differential item functioning results may change depending on how an item is scored: An illustration with the Center for Epidemiologic Studies Depression Scale. Educational and Psychological Measurement 2003, 63: 65–74. 10.1177/0013164402239317CrossRef
15.
Iwata N, Turner RJ, Lloyd DA: Race/ethnicity and depressive symptoms in community-dwelling young adults: A differential item functioning analysis. Psychiatry Res 2002, 110: 281–289. 10.1016/S0165-1781(02)00102-6PubMedCrossRef
16.
Iwata N, Buka S: Race/ethnicity and depressive symptoms: A cross-cultural/ethnic comparison among university students in East Asia, North and South America. Soc Sci Med 2002, 55: 2243–2252. 10.1016/S0277-9536(02)00003-5PubMedCrossRef
17.
Zumbo BD, Gelin MN, Hubley AM: Psychometric study of the CES-D: Factor analysis and DIF. Presented at the International Neuropsychological Society Annual Meeting, Chicago; 2001.
18.
Orlando M, Marshall GN: Differential item functioning in a Spanish translation of the PTSD checklist: Detection and evaluation of impact. Psychol Assess 2002, 14: 50–59. 10.1037/1040-3590.14.1.50PubMedCrossRef
19.
Avlund K, Era P, Davidsen M, GauseNilsson I: Item bias in self-reported functional ability among 75-year-old men and women in three Nordic localities. Scand J Soc Med 1996, 24: 206–217.PubMed
20.
Dallmeijer AJ, Dekker J, Roorda LD, Knol DL, van Baalen B, de Groot V, Schepers VPM, Lankhorst GJ: Differential item functioning of the functional independence measure in higher performing neurological patients. J Rehabil Med 2005, 37: 346–352. 10.1080/16501970510038284PubMedCrossRef
21.
Tennant A, Penta M, Tesio L, Grimby G, Thonnard JL, Slade A, Lawton G, Simone A, Carter J, Lundgren-Nilsson A, Tripolski M, Ring H, Biering-Sorensen F, Marincek C, Burger H, Phillips S: Assessing and adjusting for cross-cultural validity of impairment and activity limitation scales through differential item functioning within the framework of the Rasch model: The PRO-ESOR project. Med Care 2004, 42: 37–48. 10.1097/01.mlr.0000103529.63132.77CrossRef
22.
Schmidt S, Mühlan H, Power M: The EUROHIS-QOL 8-item index: Psychometric results of a cross-cultural field study. European Journal of Public Health Advance Access 2006, 16: 420–428. 10.1093/eurpub/cki155CrossRef
23.
Kim M: Detecting DIF across the different language groups in a speaking test. Language Testing 2001, 18: 89–114.CrossRef
24.
Bjorner JB, Kreiner S, Ware JE, Damsgaard MT, Bech P: Differential item functioning in the Danish translation of the SF-36. J Clin Epidemiol 1998, 51: 1189–1202. 10.1016/S0895-4356(98)00111-5PubMedCrossRef
25.
Schmidt S, Debensason D, Mühlan H, Petersen C, Power M, Simeoni MC, Bullinger M: The DISABKIDS generic quality of life instrument showed cross-cultural validity. Journal of Clinical Epidemiology 2006, 59: 587–598. 10.1016/j.jclinepi.2005.09.012PubMedCrossRef
26.
Borsboom D, Mellenbergh GJ, van Heerden J: Different kinds of DIF: A distinction between absolute and relative forms of measurement invariance and bias. Applied Psychological Measurement 2002, 26: 433–450. 10.1177/014662102237798CrossRef
27.
Cole SR, Kawachi I, Maller SJ, Berkman LF: Test of item-response bias in the CES-D scale. Experience from the New Haven EPESE study. J Clin Epidemiol 2000, 53: 285–289. 10.1016/S0895-4356(99)00151-1PubMedCrossRef
28.
Jones RN, Gallo JJ: Education and sex differences in the mini-mental state examination: Effects of differential item functioning. Journals of Gerontology Series B-Psychological Sciences & Social Sciences 2002, 57: P548–58.CrossRef
29.
Mungas D, Reed BR, Crane PK, Haan MN, Gonzalez H: Spanish and English Neuropsychological Assessment Scales (SENAS): Further development and psychometric characteristics. Psychol Assess 2004, 16: 347–359. 10.1037/1040-3590.16.4.347PubMedCrossRef
30.
Niti M, Ng TP, Chiam PC, Kua EH: Item response bias was present in instrumental activity of daily living scale in Asian older adults. Journal of Clinical Epidemiology 2007, 60: 366–374. 10.1016/j.jclinepi.2006.07.012PubMedCrossRef
31.
Jones RN: Racial bias in the assessment of cognitive functioning of older adults. Aging & Mental Health 2003, 7: 83–102.CrossRef
32.
Kristensen TS, Bjorner JB, Christensen KB, Borg V: The distinction between work pace and working hours in the measurement of quantitative demands at work. Work Stress 2004, 18: 305–322. 10.1080/02678370412331314005CrossRef
33.
Holland PW, Wainer H: Differential item functioning. Hillsdale, New Jersey: Lawrence Erlbaum Associates; 1993.
34.
Benson J, Hutchinson SR: The state of the art in bias research in the United States. European Review of Applied Psychology 1997, 47: 281–294.
35.
Clauser BE, Mazor KM: Using statistical procedures to identify differentially functioning test items. Educational Measurement: Issues and Practice 1998, 2: 31–44.
36.
Groenvold M, Petersen MA: The role and use of differential item functioning (DIF) analysis of quality of life data from clinical trials. In Assessing Quality of Life in Clinical Trials. Edited by: Fayers P, Hays R. Oxford: Oxford University Press; 2005:195–208.
37.
Teresi JA: Overview of quantitative measurement methods: Equivalence, invariance, and differential item functioning in health applications. Med Care 2006, 44: S39-S49. 10.1097/01.mlr.0000245452.48613.45PubMedCrossRef
38.
Teresi JA: Different approaches to differential item functioning in health applications: Advantages, disadvantages and some neglected topics. Med Care 2006, 44: S152-S170. 10.1097/01.mlr.0000245142.74628.abPubMedCrossRef
39.
Thissen D, Steinberg L, Wainer H: Detection of differential item functioning using the parameters of item response models. In Differential Item Functioning. Edited by: Holland PW, Wainer H. Hillsdale, New Jersey: Lawrence Erlbaum Associates; 1993:67–114.
40.
Teresi JA, Kleinman M, Ocepek-Welikson K: Modern psychometric methods for detection of differential item functioning: Application to cognitive assessment measures. Stat Med 2000, 19: 1651–1683. 10.1002/(SICI)1097-0258(20000615/30)19:11/12<1651::AID-SIM453>3.0.CO;2-HPubMedCrossRef
41.
Millsap RE: Comments on methods for the investigation of measurement bias in the mini-mental state examination. Med Care 2006, 44: S171-S175. 10.1097/01.mlr.0000245441.76388.ffPubMedCrossRef
42.
Angoff WH: Perspectives on Differential Item Functioning. In Differential Item Functioning Edited by: Holland PW, Wainer H. 1993, 3–24.
43.
Dorans NJ, Holland PW: DIF detection and description: Mantel-Haenszel and standardization. In Differential Item Functioning. Edited by: Holland PW, Wainer H. Hillsdale, New Jersey: Lawrence Erlbaum Associates; 1993:35–66.
44.
Shealy RT, Stout WF: An item response theory model for test bias and differential item functioning. In Differential Item Functioning. Edited by: Holland PW, Wainer H. Hillsdale, New Jersey: Lawrence Erlbaum Associates; 1993:197–240.
45.
46.
Swaminathan H, Rogers HJ: Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement 1990, 27: 361–370. 10.1111/j.1745-3984.1990.tb00754.xCrossRef
47.
Rogers HJ, Swaminathan H: A comparison of logistic regression and Mantel-Haenszel procedures for detecting differential item functioning. Applied Psychological Measurement 1993, 17: 105–116. 10.1177/014662169301700201CrossRef
48.
French AW, Miller TR: Logistic regression and its use in detecting differential functioning in polytomous items. Journal of Educational Measurement 1996, 33: 315–332. 10.1111/j.1745-3984.1996.tb00495.xCrossRef
49.
Jodoin MG, Gierl MJ: Evaluating type I error and power rates using an effect size measure with the logistic regression procedure for DIF detection. Applied Measurement in Education 2001, 14: 329–349. 10.1207/S15324818AME1404_2CrossRef
50.
Scott SC, Goldberg MS, Mayo NE: Statistical assessment of ordinal outcomes in comparative studies. J Clin Epidemiol 1997, 50: 45–55. 10.1016/S0895-4356(96)00312-5PubMedCrossRef
51.
Shimizu Y, Zumbo BD: A logistic regression for differential item functioning primer. Japan Language Testing Association Journal 2005, 7: 110–124.
52.
53.
Millsap RE, Everson HT: Methodology review - statistical approaches for assessing measurement bias. Applied Psychological Measurement 1993, 17: 297–334. 10.1177/014662169301700401CrossRef
54.
Crane PK: Commentary on comparing translations of the EORTC QLQ-C30 using differential item functioning analyses. Quality of life research 2006, 15: 1117–1118. 10.1007/s11136-006-0057-1CrossRef
55.
Lai JS, Teresi J, Gershon R: Procedures for the analysis of differential item functioning (DIF) for small sample sizes. Eval Health Prof 2005, 28: 283–294. 10.1177/0163278705278276PubMedCrossRef
56.
Scott NW, Fayers PM, Aaronson NK, Bottomley A, de Graeff A, Groenvold M, Gundy C, Koller M, Petersen MA, Sprangers MAG: A simulation study provided sample size guidance for differential item functioning (DIF) studies using short scales. J Clin Epidemiol 2009, 62: 288–295. 10.1016/j.jclinepi.2008.06.003PubMedCrossRef
57.
Roussos L, Stout W: A multidimensionality-based DIF analysis paradigm. Applied Psychological Measurement 1996, 20: 355–371. 10.1177/014662169602000404CrossRef
58.
Lewis C: A note on the value of including the studied item in the test score when analyzing test items for DIF. In Differential Item Functioning. Edited by: Holland PW, Wainer H. Hillsdale, New Jersey: Lawrence Erlbaum Associates; 1993:317–320.
59.
Navas-Ara MJ, Gómez-Benito J: Effects of ability scale purification on the identification of DIF. European Journal of Psychological Assessment 2002, 18: 9–15. 10.1027//1015-5759.18.1.9CrossRef
60.
Hidalgo-Montesinos MD, Gómez-Benito J: Test purification and the evaluation of differential item functioning with multinomial logistic regression. European Journal of Psychological Assessment 2003, 19: 1–11. 10.1027//1015-5759.19.1.1CrossRef
61.
Stump TE, Monahan P, McHorney CA: Differential item functioning in the short portable mental status questionnaire. Res Aging 2005, 27: 355–384. 10.1177/0164027504273784CrossRef
62.
Crane PK, van Belle G, Larson EB: Test bias in a cognitive test: Differential item functioning in the CASI. Stat Med 2004, 23: 241–256. 10.1002/sim.1713PubMedCrossRef
63.
Crane PK, Cetin K, Cook KF, Johnson K, Deyo R, Amtmann D: Differential item functioning impact in a modified version of the Roland-Morris disability questionnaire. Quality of Life Research 2007, 16: 981–990. 10.1007/s11136-007-9200-xPubMedCrossRef
64.
Crane PK, Hart DL, Gibbons LE, Cook KF: A 37-item shoulder functional status item pool had negligible differential functioning. J Clin Epidemiol 2006, 59: 478–484. 10.1016/j.jclinepi.2005.10.007PubMedCrossRef
65.
Scott NW, Fayers PM, Aaronson NK, Bottomley A, de Graeff A, Groenvold M, Gundy C, Koller M, Petersen MA, Sprangers MAG: Interpretation of differential item functioning (DIF) analyses using external review. Expert Reviews in Pharmacoeconomics and Outcomes Research 2010, 10: 253–258. 10.1586/erp.10.22CrossRef
66.
Bender R, Lange S: Adjusting for multiple testing - when and how? Journal of Clinical Epidemiology 2001, 54: 343–349. 10.1016/S0895-4356(00)00314-0PubMedCrossRef
67.
Gierl MJ, Khaliq SN: Identifying sources of differential item functioning on translated achievement tests: a confirmatory analysis. Presented at the Annual meeting of the National Council on Measurement in Education, New Orleans; 2000.
68.
Gierl MJ, Rogers WT, Klinger DA: Using statistical and judgmental reviews to identify and interpret translation differential item functioning. Alberta Journal of Educational Research 1999, 45: 353–376.
69.
Hidalgo MD, Lopez-Pina JA: Differential item functioning detection and effect size: A comparison between logistic regression and Mantel-Haenszel procedures. Educational and Psychological Measurement 2004, 64: 903–915. 10.1177/0013164403261769CrossRef
70.
Zieky M: Practical questions in the use of DIF statistics in test development. In Differential Item Functioning. Edited by: Holland PW, Wainer H. Hillsdale, New Jersey: Lawrence Erlbaum Associates; 1993:337–348.
71.
Crane PK, Gibbons LE, Ocepek-Welikson K, Cook K, Cella D, Narasimhalu K, Hays RD, Teresi JA: A comparison of three sets of criteria for determining the presence of differential item functioning. Quality of Life Research 2007, 16: S69-S84. 10.1007/s11136-007-9185-5CrossRef
72.
Borsboom D: When does measurement invariance matter? Med Care 2006, 44: S176-S181. 10.1097/01.mlr.0000245143.08679.ccPubMedCrossRef
73.
Hambleton RK: Good practices for identifying differential item functioning. Med Care 2006, 44: S182-S188. 10.1097/01.mlr.0000245443.86671.c4PubMedCrossRef
74.
Crane PK, Gibbons LE, Narasimhalu K, Lai J-, Cella D: Rapid detection of differential item functioning in assessments of health-related quality of life: The functional assessment of cancer therapy. Quality of Life Research 2007, 16: 101–114. 10.1007/s11136-006-0035-7PubMedCrossRef
75.
Hart DL, Deutscher D, Crane PK, Wang Y: Differential item functioning was negligible in an adaptive test of functional status for patients with knee impairments who spoke English or Hebrew. Quality of Life Research 2009, 18: 1067–1083. 10.1007/s11136-009-9517-8PubMedCrossRef
76.
McHorney CA, Fleishman JA: Assessing and understanding measurement equivalence in health outcome measures. Med Care 2006, 44: S205-S210. 10.1097/01.mlr.0000245451.67862.57PubMedCrossRef
77.
Scott NW, Fayers PM, Aaronson NK, Bottomley A, de Graeff A, Groenvold M, Gundy C, Koller M, Petersen MA, Sprangers MAG: The practical impact of differential item functioning analyses in a health-related quality of life instrument. Quality of Life Research 2009, 18: 1125–1130. 10.1007/s11136-009-9521-zPubMedCrossRef
78.
Langer MM, Hill CD, Thissen D, Burwinkle TM, Varni JW, DeWalt DA: Item response theory detected differential item functioning between healthy and ill children in quality-of-life measures. Journal of Clinical Epidemiology 2008, 61: 268–276. 10.1016/j.jclinepi.2007.05.002PubMedCrossRef
79.
Engelhard G, Davis M, Hansche L: Evaluating the accuracy of judgments obtained from item review committees. Applied Measurement in Education 1999, 12: 199–210. 10.1207/s15324818ame1202_6CrossRef
80.
Ryan KE, Bachman LF: Differential item functioning on two tests of EFL proficiency. Language Testing 1992, 9: 12–29. 10.1177/026553229200900103CrossRef
81.
Allalouf A, Hambleton R, Sireci S: Identifying the causes of translation DIF on verbal items. Journal of Educational Measurement 1999, 36: 185–198. 10.1111/j.1745-3984.1999.tb00553.xCrossRef
82.
Ercikan K: Disentangling sources of differential item functioning in multilanguage assessments. International Journal of Testing 2002, 2: 199–215. 10.1207/S15327574IJT023&4_2CrossRef
83.
Huang CD, Church AT, Katigbak MS: Identifying cultural differences in items and traits - differential item functioning in the NEO personality inventory. Journal of Cross-Cultural Psychology 1997, 28: 192–218. 10.1177/0022022197282004CrossRef
84.
Sireci SG, Berberoglu G: Using bilingual respondents to evaluate translated-adapted items. Applied Measurement in Education 2000, 13: 229–248. 10.1207/S15324818AME1303_1CrossRef
85.
Schmitt AP, Holland PW, Dorans NJ: Evaluating hypotheses about differential item functioning. In Differential Item Functioning. Edited by: Holland PW, Wainer H. Hillsdale, New Jersey: Lawrence Erlbaum Associates; 1993:281–316.
86.
Ramirez M, Teresi JA, Holmes D, Gurrland B, Lantigua R: Differential item functioning (DIF) and the mini-mental state examination (MMSE). Med Care 2006, 44: S95-S106. 10.1097/01.mlr.0000245181.96133.dbPubMedCrossRef
Metadata
Title
Differential item functioning (DIF) analyses of health-related quality of life instruments using logistic regression
Authors
Neil W Scott
Peter M Fayers
Neil K Aaronson
Andrew Bottomley
Alexander de Graeff
Mogens Groenvold
Chad Gundy
Michael Koller
Morten A Petersen
Mirjam AG Sprangers
the EORTC Quality of Life Group and the Quality of Life Cross-Cultural Meta-Analysis Group
Publication date
01-12-2010
Publisher
BioMed Central
Published in
Health and Quality of Life Outcomes / Issue 1/2010
Electronic ISSN: 1477-7525
DOI
https://doi.org/10.1186/1477-7525-8-81

Other articles of this Issue 1/2010

Health and Quality of Life Outcomes 1/2010 Go to the issue

Research

The effect of post-discharge educational intervention on patients in achieving objectives in modifiable risk factors six months after discharge following an episode of acute coronary syndrome, (CAM-2 Project): a randomized controlled trial

Research

Self-reported halitosis and emotional state: impact on oral conditions and treatments

Research

Quality of Life as reported by children and parents: a comparison between students and child psychiatric outpatients

Research

Low Sense of Coherence (SOC) is a mirror of general anxiety and persistent depressive symptoms in adolescent girls - a cross-sectional study of a clinical and a non-clinical cohort

Research

Mapping of the EQ-5D index from clinical outcome measures and demographic variables in patients with coronary heart disease

Research

Uncontrolled asthma: assessing quality of life and productivity of children and their caregivers using a cross-sectional Internet-based survey