An Evaluation of the Veterans Health Administration’s Clinical Reminders System: A National Survey of Generalists

The Veterans Health Administration (VHA) is a leader in developing computerized clinical reminders (CCRs). Primary care physicians’ (PCPs) evaluation of VHA CCRs could influence their future development and use within and outside the VHA.

Survey PCPs about usefulness and usability of VHA CCRs.

In a national survey, VHA PCPs rated on a 7-point scale usefulness and usability of VHA CCRs, and standardized scales (0–100) were constructed. A hierarchical linear mixed (HLM) model predicted physician- and facility-level variables associated with more positive global assessment of CCRs.

Four hundred sixty-one PCPs participated (response rate, 69%). Scale Cronbach’s alpha ranged from 0.62 to 0.82. Perceptions of VHA CCRs were primarily in the midrange, where higher ratings indicate more favorable attitudes (weighted standardized median, IQR): global assessment (50, 28–61), clinical/situational specificity (29, 17–42), integration with workflow/workload (39, 17–50), training (50, 33–67), VHA’s management of CCR use (67, 50–83), design/interface (53, 40–67), perceived role in CCR use (67, 50–83), and self-efficacy (67, 57–78). In a HLM model, design/interface (p < .001), self-efficacy (p < .001), integration with workflow/workload (p < .001), and training (p < .001) were associated with more favorable global assessments of CCRs. Facilities in the west as compared to the south (p = .033), and physicians with academic affiliation (p = .045) had less favorable global assessment of CCRs.

Our systematic assessment of end-users’ perceptions of VHA CCRs suggests that CCRs need to be developed and implemented with a continual focus on improvement based on end-user feedback. Potential target areas include better integration into the primary care clinic workflow/workload.