Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
BMC Medical Informatics and Decision Making
Published in: BMC Medical Informatics and Decision Making 1/2018

Open Access 01-12-2018 | Research article

Evidence-based usability design principles for medication alerting systems

Authors: Romaric Marcilly, Elske Ammenwerth, Erin Roehrer, Julie Niès, Marie-Catherine Beuscart-Zéphir

Published in: BMC Medical Informatics and Decision Making | Issue 1/2018

Login to get access

Abstract

Background

Usability flaws in medication alerting systems may have a negative impact on clinical use and patient safety. In order to prevent the release of alerting systems that contain such flaws, it is necessary to provide designers and evaluators with evidence-based usability design principles. The objective of the present study was to develop a comprehensive, structured list of evidence-based usability design principles for medication alerting systems.

Methods

Nine sets of design principles for medication alerting systems were analyzed, summarized, and structured. We then matched the summarized principles with a list of usability flaws in order to determine the level of underlying evidence.

Results

Fifty-eight principles were summarized from the literature and two additional principles were defined, so that each flaw was matched with a principle. We organized the 60 summarized usability design principles into 6 meta-principles, 38 principles, and 16 sub-principles. Only 15 principles were not matched with a usability flaw. The 6 meta-principles respectively covered the improvement of the signal-to-noise ratio, the support for collaborative working, the fit with a clinician’s workflow, the data display, the transparency of the alerting system, and the actionable tools to be provided within an alert.

Conclusions

It is possible to develop an evidence-based, structured, comprehensive list of usability design principles that are specific to medication alerting systems and are illustrated by the corresponding usability flaws. This list represents an improvement over the current literature. Each principle is now associated with the best available evidence of its violation. This knowledge may help to improve the usability of medication alerting systems and, ultimately, decrease the harmful consequences of the systems’ usability flaws.
Appendix
Available only for authorised users
Literature
1.
Horsky J, Schiff GD, Johnston D, Mercincavage L, Bell D, Middleton B. Interface design principles for usable decision support: a targeted review of best practices for clinical prescribing interventions. J Biomed Inform. 2012;45:1202–16.CrossRefPubMed
2.
Bates DW, Kuperman GJ, Wang S, Gandhi T, Kittler A, Volk L, Spurr C, Khorasani R, Tanasijevic M, Middleton B. Ten commendments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc. 2003;10:523–30.CrossRefPubMedPubMedCentral
3.
Jaspers MW, Smeulers M, Vermeulen H, Peute LW. Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings. J Am Med Inform Assoc. 2011;18:327–34.CrossRefPubMedPubMedCentral
4.
Sittig DF, Wright A, Osheroff JA, Middleton B, Teich JM, Ash JS, Campbell E, Bates DW. Grand challenges in clinical decision support. J Biomed Inform. 2008;41:387–92.CrossRefPubMed
5.
Campbell EM, Guappone KP, Sittig DF, Dykstra RH, Ash JS. Computerized provider order entry adoption: implications for clinical workflow. J Gen Intern Med. 2009;24:21–6.CrossRefPubMed
6.
Marcilly R, Ammenwerth E, Roehrer E, Pelayo S, Vasseur F, Beuscart-Zephir MC. Usability flaws in medication alerting systems: impact on usage and work system. Yearb Med Inform. 2015;10:55–67.CrossRefPubMedPubMedCentral
7.
Horsky J, Kaufman DR, Patel VL. Computer-based drug ordering: evaluation of interaction with a decision-support system. Stud Health Technol Inform. 2004;107:1063–7.PubMed
8.
Ash JS, Sittig DF, Campbell EM, Guappone KP, Dykstra RH. Some unintended consequences of clinical decision support systems. AMIA Annu Symp Proc. 2007:26–30.
9.
Duke JD, Bolchini DA. Successful model and visual design for creating context-aware drug-drug interaction alerts. AMIA Annu Symp Proc. 2011:339–48.
10.
Russ AL, Zillich AJ, McManus MS, Doebbeling BN, Saleem JJ. Prescribers' interactions with medication alerts at the point of prescribing: a multi-method, in situ investigation of the human-computer interaction. Int J Med Inform. 2012;81:232–43.CrossRefPubMed
11.
van der Sijs H, van Gelter T, Vulto A, Berg M, Aarts J. Understanding handling of drug safety alerts: a simulation study. Int J Med Inform. 2010;79:361–9.CrossRefPubMed
12.
Russ AL, Saleem JJ, McManus MS, Zillich AJ, Doebbling BN. Computerized medication alerts and prescriber mental models: observing routine patient care. Proc Human Factors Ergon Soc Annu Meet. 2009;53:655–9.CrossRef
13.
Hartmann Hamilton AR, Anhoj J, Hellebek A, Egebart J, Bjorn B, Lilja B. Computerised physician order entry (CPOE). Stud Health Technol Inform. 2009;148:159–62.PubMed
14.
International Standardization Organization Ergonomics of human system interaction - part 210: human centered design for interactive systems (rep N°9241-210). Geneva: international standardization Organization; 2010.
15.
Marcilly R, Peute L, Beuscart-Zephir MC. From usability engineering to evidence-based usability in health IT. Hat evidence supports the use of computerized alerts and prompts to improve clinicians' prescribing behavior? Stud Health Technol Inform. 2016;222:126–38.PubMed
16.
Marcilly R, Beuscart-Zephir MC, Ammenwerth E, Pelayo S. Seeking evidence to support usability principles for medication-related clinical decision support (CDS) functions. Stud Health Technol Inform. 2013;192:427–31.PubMed
17.
Marcilly R, Ammenwerth E, Vasseur F, Roehrer E, Beuscart-Zephir MC. Usability flaws of medication-related alerting functions: a systematic qualitative review. J Biomed Inform. 2015;55:260–71.CrossRefPubMed
18.
Nielsen J. Enhancing the explanatory power of usability heuristics. CHI '94 proceedings of the SIGCHI conference on human factors in. Comput Syst. 1994:152–8.
19.
Kuperman GJ, Bobb A, Payne TH, Avery AJ, Gandhi TK, Burns G, Classen DC, Bates DW. Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc. 2007;14:29–40.CrossRefPubMedPubMedCentral
20.
Phansalkar S, Edworthy J, Hellier E, Seger DL, Schedlbauer A, Avery AJ, Bates DW. A review of human factors principles for the design and implementation of medication safety alerts in clinical information systems. J Am Med Inform Assoc. 2010;17:493–501.CrossRefPubMedPubMedCentral
21.
Pelayo S, Marcilly R, Bernonville S, Leroy N, Beuscart-Zephir MC. Human factors based recommendations for the design of medication related clinical decision support systems (CDSS). Stud Health Technol Inform. 2011;169:412–6.PubMed
22.
Zachariah M, Phansalkar S, Seidling HM, Neri PM, Cresswell KM, Duke J, Bloomrosen M, Volk LA, Bates DW. Development and preliminary evidence for the validity of an instrument assessing implementation of human-factors principles in medication-related decision-support systems--I-MeDeSA. J Am Med Inform Assoc. 2011;18(Suppl 1):i62–72.CrossRefPubMedPubMedCentral
23.
Horsky J, Phansalkar S, Desai A, Bell D, Middleton B. Design of decision support interventions for medication prescribing. Int J Med Inform. 2013;82:492–503.CrossRefPubMed
24.
Payne TH, Hines LE, Chan RC, Hartman S, Kapusnik-Uner J, Russ AL, et al. Recommendations to improve the usability of drug-drug interaction clinical decision support alerts. J Am Med Inform Assoc. 2015;22:243–1250.CrossRef
25.
Baysari MT, Westbrook JI, Richardson KL, Day RO. The influence of computerized decision support on prescribing during ward-rounds: are the decision-makers targeted? J Am Med Inform Assoc. 2015;18:754–9.CrossRef
26.
Kortteisto T, Komulainen J, Makela M, Kunnamo I, Kaila M. Clinical decision support must be useful, functional is not enough: a qualitative study of computer-based clinical decision support in primary care. BMC Health Serv Res. 2012;12:349–57.CrossRefPubMedPubMedCentral
27.
Russ AL, Zillich AJ, McManus MS, Doebbeling BN, Saleem JJA. Human factors investigation of medication alerts: barriers to prescriber decision-making and clinical workflow. AMIA Annu Symp Proc. 2009:548–52.
28.
Saleem JJ, Patterson ES, Militello L, Render ML, Orshansky G, Asch SM. Exploring barriers and facilitators to the use of computerized clinical reminders. J Am Med Inform Assoc. 2005;12:438–47.CrossRefPubMedPubMedCentral
29.
van der Sijs H, Aarts J, van Gelter T, Berg M, Vulto A. Turning off frequently overridden drug alerts: limited opportunities for doing it safely. J Am Med Inform Assoc. 2008;15:439–48.CrossRefPubMedPubMedCentral
30.
Feldstein A, Simon SR, Schneider J, Krall M, Laferriere D, Smith DH, Sittig DF, Soumerai SB. How to design computerized alerts to safe prescribing practices. Jt Comm J Qual Saf. 2004;30:602–13.CrossRefPubMed
31.
Krall MA, Sittig DF. Clinician's assessments of outpatient electronic medical record alert and reminder usability and usefulness requirements. Proc AMIA Symp. 2002:400–4.
32.
Weingart SN, Massagli M, Cyrulik A, Isaac T, Morway L, Sands DZ, Weissman JS. Assessing the value of electronic prescribing in ambulatory care: a focus group study. Int J Med Inform. 2009;78:571–8.CrossRefPubMed
33.
Russ AL, Saleem JJ, McManus MS, Frankel RM, Zillich AJ. The workflow of computerized medication ordering in primary care is not prescriptive. Proc Human Factors Ergon Soc Annu Meet. 2010;54:840–4.CrossRef
34.
Wipfli R, Betrancourt M, Guardia A, Lovis CA. Qualitative analysis of prescription activity and alert usage in a computerized physician order entry system. Stud Health Technol Inform. 2011;169:940–4.PubMed
35.
Khajouei R, Peek N, Wierenga PC, Kersten MJ, Jaspers MW. Effect of predefined order sets and usability problems on efficiency of computerized medication ordering. Int J Med Inform. 2010;79:690–8.CrossRefPubMed
36.
Patterson ES, Nguyen AD, Halloran JP, Asch SM. Human factors barriers to the effective use of ten HIV clinical reminders. J Am Med Inform Assoc. 2004;11:50–9.CrossRefPubMedPubMedCentral
37.
Chused AE, Kuperman GJ, Stetson PD. Alert override reasons: a failure to communicate. AMIA Annu Symp Proc. 2008:111–5.
38.
Koppel R, Metlay JP, Cohen A, Abaluck B, Localio AR, Kimmel SE, Strom BL. Role of computerized physician order entry systems in facilitating medication errors. JAMA. 2005;293:1197–203.CrossRefPubMed
39.
Khajouei R, de Jongh D, Jaspers MW. Usability evaluation of a computerized physician order entry for medication ordering. Stud Health Technol Inform. 2009;150:532–6.PubMed
40.
Saleem JJ, Patterson ES, Militello L, Anders S, Falciglia M, Wissman JA, Roth EM, Asch SM. Impact of clinical reminder redesign on learnability, efficiency, usability, and workload for ambulatory clinic nurses. J Am Med Inform Assoc. 2007;14:632–40.CrossRefPubMedPubMedCentral
41.
Trafton J, Martins S, Michel M, Lewis E, Wang D, Combs A, Scates N, Tu S, Goldstein MK. Evaluation of the acceptability and usability of a decision support system to encourage safe and effective use of opioid therapy for chronic, noncancer pain by primary care providers. Pain Med. 2010;11:575–85.CrossRefPubMed
42.
Chan J, Shojania KG, Easty AC, Etchells EE. Usability evaluation of order sets in a computerised provider order entry system. BMJ Qual Saf. 2011;20:932–40.CrossRefPubMed
43.
Nielsen J. Usability Engineering Boston: Academic Press; 1993.
44.
Scapin DL, Bastien JMC. Ergonomic criteria for evaluating the ergonomic quality of interactive systems. Behav Inf Technol. 1997;6:220–31.CrossRef
45.
Norman DA. The Design of Everyday Things. New-York: Basic Book; 1988.
46.
47.
Náfrádi L, Nakamoto K, Schulz PJ. Is patient empowerment the key to promote adherence? A systematic review of the relationship between self-efficacy, health locus of control and medication adherence. PLoS One. 2017;12
48.
Marcilly R, Monkman H, Villumsen S, Kaufman D, Beuscart-Zéphir M-C. How to present evidence-based usability design principles dedicated to medication-related alerting systems to designers and evaluators? Results from a workshop. Stud Health Technol Inform. 2016;228:609–13.PubMed
49.
Vicente KJ. Cognitive work analysis. Mahwah, NJ: Lawrence Erlbaum Associates; 1999.
50.
Riedmann D, Jung M, Hackl WO, Stuhlinger W, van der SH, Ammenwerth E. Development of a context model to prioritize drug safety alerts in CPOE systems. BMC Med Inform Decis Mak. 2011;11:35.CrossRefPubMedPubMedCentral
Metadata
Title
Evidence-based usability design principles for medication alerting systems
Authors
Romaric Marcilly
Elske Ammenwerth
Erin Roehrer
Julie Niès
Marie-Catherine Beuscart-Zéphir
Publication date
01-12-2018
Publisher
BioMed Central
Published in
BMC Medical Informatics and Decision Making / Issue 1/2018
Electronic ISSN: 1472-6947
DOI
https://doi.org/10.1186/s12911-018-0615-9

Other articles of this Issue 1/2018

BMC Medical Informatics and Decision Making 1/2018 Go to the issue

Research article

Development and validation of a model for the adoption of structured and standardised data recording among healthcare professionals

Research article

Combination of conditional random field with a rule based method in the extraction of PICO elements

Technical advance

Development and evaluating multimarker models for guiding treatment decisions

Research article

Decision makers’ experience of participatory dynamic simulation modelling: methods for public health policy

Research article

A pre-post study testing a lung cancer screening decision aid in primary care

Research article

Informing evaluation of a smartphone application for people with acquired brain injury: a stakeholder engagement study