Inequalities in the health survey using validation question to filter insufficient effort responding: reducing overestimated effects or creating selection bias?

The presence of insufficient effort responding participants (IERPs) in a survey can produce systematic bias. Validation questions are commonly used to exclude IERPs. Participants were defined as IERPs if responding inconsistently to two matched validation questions, and non-insufficient effort responding participants (non-IERPs) if responding consistently. However, it has not been tested whether validation questions themselves could result in selection bias.

This study was a cross-sectional survey conducted in Guangxi, China. Participants’ intentions to use antibiotics for their children when they have self-limiting diseases, including sore throat, cold, diarrhea, and fever, were measured. The Chi-square tests were used to compare the socio-economic status (SES) between non-IERPs and IERPs. Logistic regression was adopted to test the association between intentions to misuse antibiotics and groups (non-IERPs, IERPs with high SES, and IERPs with low SES).

Data with 3264 non-IERPs and 1543 IERPs were collected. The results showed IERPs had a lower education level (χ2 = 6.100, p = 0.047) and a higher proportion of rural residence (χ2 = 4.750, p = 0.030) compared with non-IERPs. Rural IERPs reported significantly higher rates of intentions to misuse antibiotics when their children have a sore throat (OR = 1.32; 95% CI = 1.11,1.56; p < 0.01), cold (OR = 1.33; 95%CI = 1.13,1.58; p < 0.01), diarrhea (OR = 1.46; 95%CI = 1.20,1.77; p < 0.001), and fever (OR = 1.22; 95% CI = 1.04,1.43; p < 0.05) compared with non-IERPs. IERPs living in urban areas reported significantly lower rates of intentions to use antibiotics when their children have a sore throat (OR = 0.76; 95%CI = 0.62,0.93; p < 0.01) compared with non-IERPs. IERPs with lower levels of education reported significantly higher rates of intentions to use antibiotics when their children have a sore throat (OR = 1.19; 95%CI = 1.02,1.39; p < 0.05), cold (OR = 1.43; 95% CI = 1.23,1.66; p < 0.001), diarrhea (OR = 1.38; 95%CI = 1.15,1.64; p < 0.01), and fever (OR = 1.25; 95% CI = 1.09,1.44; p < 0.01) compared with non-IERPs. IERPs with higher education levels reported significantly lower rates of intentions to use antibiotics when their children have a sore throat (OR = 0.72; 95% CI = 0.56,0.94; p < 0.05), cold (OR = 0.66; 95% CI = 0.51,0.86; p < 0.01), and fever (OR = 0.74; 95% CI = 0.60,0.92; p < 0.01) compared with non-IERPs. IERPs with low-income reported significantly higher rates of intentions to use antibiotics when their children have a cold (OR = 1.36; 95% CI = 1.13,1.64; p < 0.01) and diarrhea (OR = 1.30; 95% CI = 1.05,1.62; p < 0.05) compared with non-IERPs.

Using validation questions to exclude IERPs can result in selection bias in which participants with lower socio-economic standing and poor antibiotic use intentions were disproportionately excluded.