The antibiotic checklist: an observational study of the discrepancy between reported and actually performed checklist items

Checklists are increasingly used to measure quality of care. Recently we implemented an antibiotic checklist in nine Dutch hospitals and showed that use of the checklist resulted in more appropriate antibiotic use. While more appropriate antibiotic use was associated with a reduction in length of stay, use of the checklist in itself was not. In the current study we explored discrepancies between reported and actually performed checklist items at the patient level to test the validity of checklist answers, to evaluate whether discrepancies between reported and actually performed checklist items could explain the lack of effect of checklist use on length of stay, and to identify missed opportunities for performance per checklist item.

Checklist answers represented reported performance. Actual performance was assessed by data from the patients’ medical files. Reported and actually performed checklist items could be ‘both YES’; ‘both NO’; ‘YES reported, NOT actually performed’; or ‘NO reported, YES actually performed’. We determined an overall ‘both YES’ score per checklist, and used mixed models to evaluate whether an association existed between this overall score and patient’s length of hospital stay. Finally, we analysed whether the items that were not actually performed, could have been performed.

Between January and October 2015 physicians filled in 1207 checklists. In total 7881 items were checked. Most items were ‘both YES’ (3392/7881, 43.0%) or ‘both NO’ (2601/7881, 33.0%). The number of ‘YES reported, NOT actually performed’ items was 1628/7881 (20.7%) compared to 260/7881 (3.3%) ‘NO reported, YES actually performed’ items. The level of discrepancy between reported and actually performed items differed per checklist item. The item ‘prescribe antibiotic treatment according to the local guideline’ had the highest percentage of ‘YES reported, NOT actually performed’ items, namely 45.1%. A higher overall ‘both YES’ score of the checklist was significantly associated with a shorter length of hospital stay. Of all checklist items 21.8% were not performed while they could have been performed.

Checklist answers do not accurately assess actual provided care. As actual performance of the antibiotic checklist items is associated with length of stay, efforts to increase actual performance appear to be justified.