Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on sleep in brain health

LIVE: Thursday 2nd May 2024, 18:00-19:30 (CEST)

Quality sleep is essential for health. But what happens to our brains when sleep patterns are disturbed? Join our experts to explore the interplay between sleep disruption and neurological diseases, and the questions that you need to be asking your patients to help you prevent the harmful effects of sleep deprivation.
ADVANCED SEARCH
Log in
Top
Critical Care
Published in: Critical Care 1/2016

Open Access 01-12-2016 | Research

The usability of ventilators: a comparative evaluation of use safety and user experience

Authors: Plinio P. Morita, Peter B. Weinstein, Christopher J. Flewwelling, Carleene A. Bañez, Tabitha A. Chiu, Mario Iannuzzi, Aastha H. Patel, Ashleigh P. Shier, Joseph A. Cafazzo

Published in: Critical Care | Issue 1/2016

Login to get access

Abstract

Background

The design complexity of critical care ventilators (CCVs) can lead to use errors and patient harm. In this study, we present the results of a comparison of four CCVs from market leaders, using a rigorous methodology for the evaluation of use safety and user experience of medical devices.

Methods

We carried out a comparative usability study of four CCVs: Hamilton G5, Puritan Bennett 980, Maquet SERVO-U, and Dräger Evita V500. Forty-eight critical care respiratory therapists participated in this fully counterbalanced, repeated measures study. Participants completed seven clinical scenarios composed of 16 tasks on each ventilator.
Use safety was measured by percentage of tasks with use errors or close calls (UE/CCs). User experience was measured by system usability and workload metrics, using the Post-Study System Usability Questionnaire (PSSUQ) and the National Aeronautics and Space Administration Task Load Index (NASA-TLX).

Results

Nine of 18 post hoc contrasts between pairs of ventilators were significant after Bonferroni correction, with effect sizes between 0.4 and 1.09 (Cohen’s d). There were significantly fewer UE/CCs with SERVO-U when compared to G5 (p = 0.044) and V500 (p = 0.020). Participants reported higher system usability for G5 when compared to PB980 (p = 0.035) and higher system usability for SERVO-U when compared to G5 (p < 0.001), PB980 (p < 0.001), and V500 (p < 0.001). Participants reported lower workload for G5 when compared to PB980 (p < 0.001) and lower workload for SERVO-U when compared to PB980 (p < 0.001) and V500 (p < 0.001). G5 scored better on two of nine possible comparisons; SERVO-U scored better on seven of nine possible comparisons. Aspects influencing participants’ performance and perception include the low sensitivity of G5’s touchscreen and the positive effect from the quality of SERVO-U’s user interface design.

Conclusions

This study provides empirical evidence of how four ventilators from market leaders compare and highlights the importance of medical technology design. Within the boundaries of this study, we can infer that SERVO-U demonstrated the highest levels of use safety and user experience, followed by G5. Based on qualitative data, differences in outcomes could be explained by interaction design, quality of hardware components used in manufacturing, and influence of consumer product technology on users’ expectations.
Appendix
Available only for authorised users
Literature
1.
Needham DM, Bronskill SE, Calinawan JR, Sibbald WJ, Pronovost PJ, Laupacis A. Projected incidence of mechanical ventilation in Ontario to 2026: preparing for the aging baby boomers. Crit Care Med. 2005;33:574–9.CrossRefPubMed
2.
Edwards JR, Peterson KD, Andrus ML, Tolson JS, Goulding JS, Dudeck MA, et al. National Healthcare Safety Network (NHSN) Report, data summary for 2006, issued June 2007. Am J Infect Control. 2007;35:290–301.CrossRefPubMed
3.
Wunsch H, Wagner J, Herlim M, Chong D, Kramer A, Halpern SD. ICU occupancy and mechanical ventilator use in the United States. Crit Care Med. 2013;41:2712–9.CrossRefPubMed
4.
Esteban A, Anzueto A, Alía I, Gordo F, Apezteguía C, Pálizas F, et al. How is mechanical ventilation employed in the intensive care unit? An international utilization review. Am J Respir Crit Care Med. 2000;161:1450–8.CrossRefPubMed
5.
Metnitz PGH, Metnitz B, Moreno RP, Bauer P, Sorbo LD, Hoermann C, et al. Epidemiology of mechanical ventilation: analysis of the SAPS 3 database. Intensive Care Med. 2009;35:816–25.CrossRefPubMed
6.
7.
8.
Gilstrap D, Macintyre N. Patient ventilator interactions: implications for clinical management. Am J Respir Crit Care Med. 2013;188:1058–68.CrossRefPubMed
9.
Chastre J, Fagon J-Y. Ventilator-associated pneumonia. Am J Respir Crit Care Med. 2002;165:867–903.CrossRefPubMed
10.
Chatwin M, Heather S, Hanak A, Polkey MI, Simonds AK. Analysis of home support and ventilator malfunction in 1,211 ventilator-dependent patients. Eur Respir J. 2010;35:310–6.CrossRefPubMed
11.
Gravenstein JS. How does human error affect safety in anesthesia? Surg Oncol Clin N Am. 2000;9:81–95.PubMed
12.
Auriant I, Reignier J, Pibarot ML, Bachat S, Tenaillon A, Raphael JC. Critical incidents related to invasive mechanical ventilation in the ICU: preliminary descriptive study. Intensive Care Med. 2002;28:452–8.CrossRefPubMed
13.
14.
15.
Gajic O, Frutos-Vivar F, Esteban A, Hubmayr RD, Anzueto A. Ventilator settings as a risk factor for acute respiratory distress syndrome in mechanically ventilated patients. Intensive Care Med. 2005;31:922–6.CrossRefPubMed
16.
Donchin Y, Seagull FJ. The hostile environment of the intensive care unit. Curr Opin Crit Care. 2002;8:316–20.CrossRefPubMed
17.
18.
Wiklund ME, Kendler J, Strochlic AY. Usability testing of medical devices. Boca Raton: CRC Press; 2011.
19.
Nielsen J. Usability engineering. Burlington: Morgan Kaufmann; 1993.
20.
21.
Ginsburg G. Human factors engineering: a tool for medical device evaluation in hospital procurement decision-making. J Biomed Inform. 2005;38:213–9.CrossRefPubMed
22.
Rubin J, Chisnell D. Handbook of usability testing: how to plan, design and conduct effective tests. Hoboken: Wiley; 2008.
23.
Templier F, Miroux P, Dolveck F, Descatha A, Goddet N-S, Jeleff C, et al. Evaluation of the ventilator-user interface of 2 new advanced compact transport ventilators. Respir Care. 2007;52:1701–9.PubMed
24.
McCrory B, Lowndes BR, LaGrange CA, Miller EE, Hallbeck MS. Comparative usability testing of conventional and single incision laparoscopic surgery devices. Hum Factors. 2012;55:619–31.CrossRef
25.
Diggory P, Fernandez C, Humphrey A, Jones V, Murphy M. Comparison of elderly people’s technique in using two dry powder inhalers to deliver zanamivir: randomised controlled trial. Br Med J. 2001;322:577–9.CrossRef
26.
Gonzalez-Bermejo J, Laplanche V, Husseini FE, Duguet A, Derenne J-P, Similowski T. Evaluation of the user-friendliness of 11 home mechanical ventilators. Eur Respir J. 2006;27:1236–43.CrossRefPubMed
27.
Vignaux L, Tassaux D, Jolliet P. Evaluation of the user-friendliness of seven new generation intensive care ventilators. Intensive Care Med. 2009;35:1687–91.CrossRefPubMed
28.
29.
Cassano-Piché A, Cafazzo JA, Chagpar A, Easty AC. How human factors methods can help in the procurement process. Biomed Instrum Technol. 2010;44:49–56.CrossRef
30.
Kutner MH, Nachtsheim CJ, Neter J. Applied linear statistical models. New York: McGraw-Hill/Irwin; 2005.
31.
Gravetter FJ, Wallnau LB. Statistics for the behavioral sciences. 7th ed. Belmont: Wadsworth Publishing; 2006.
32.
van Teijlingen ER, Hundley V. The importance of pilot studies. Soc Res Update. 2001;35:1–4.
33.
Cohen J. Statistical power analysis for the behavioral sciences. London: Routledge; 1988.
34.
35.
36.
37.
Richard JCM, Richard M, Kacmarek RM. ICU mechanical ventilators, technological advances vs. user friendliness: the right picture is worth a thousand numbers. Intensive Care Med. 2009;35:1662–3.CrossRefPubMed
38.
Kamouri AL, Kamouri J, Smith KH. Training by exploration: facilitating the transfer of procedural knowledge through analogical reasoning. Int J Man Mach Stud. 1986;24:171–92.CrossRef
39.
Carayon P, Wetterneck TB, Hundt AS, Ozkaynak M, Ram P, DeSilvey J, et al. Observing nurse interaction with infusion pump technologies. Adv Pat Saf. 2005;2:349–64.
40.
Zhang J, Johnson TR, Patel VL, Paige DL, Kubose T. Using usability heuristics to evaluate patient safety of medical devices. J Biomed Inform. 2003;36:23–30.CrossRefPubMed
41.
Middleton B, Bloomrosen M, Dente MA, Hashmat B, Koppel R, Overhage JM, et al. Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. J Am Med Informatics Assoc. 2013;20:e2–8.CrossRef
42.
43.
Morita PP, Cafazzo JA. Challenges and paradoxes of human factors in health technology design. JMIR J Hum Factors. 2015;3:1–7.
44.
Nielsen J, Levy J. Measuring usability: preference vs. performance. Commun ACM. 1994;37:66–75.CrossRef
45.
Lewis JR. Psychometric evaluation of the PSSUQ using data from five years of usability studies. Int J Hum Comput Interact. 2002;14:463–88.CrossRef
46.
Dosani M, Hunc K, Dumont GA, Dunsmuir D, Barralon P, Schwarz SKW, et al. A vibro-tactile display for clinical monitoring. Anesth Analg. 2012;115:588–94.PubMed
47.
Görges M, Winton P, Koval V, Lim J, Stinson J, Choi PT, et al. An evaluation of an expert system for detecting critical events during anesthesia in a human patient simulator. Anesth Analg. 2013;117:380–91.CrossRefPubMed
48.
Schutte JL, Gales S, Filipponi A, Saptono A, Parmanto B, McCue M. Evaluation of a telerehabilitation system for community-based rehabilitation system. Int J Telerehabilitation. 2012;4:15–24.CrossRef
49.
Hart SG, Staveland LE. Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. Adv Psychol. 1988;1:139–83.CrossRef
50.
Hart SG. Nasa-Task Load Index (NASA-TLX); 20 years later. Proc Hum Factors Ergon Soc Annu Meet. 2006;50:904–8. SAGE Publications.
51.
Wachter SB, Johnson K, Albert R, Syroid N, Drews F, Westenskow D. The evaluation of a pulmonary display to detect adverse respiratory events using high resolution human simulator. J Am Med Inform Assoc. 2006;13:635–42.CrossRefPubMedPubMedCentral
52.
Görges M, Staggers N. Evaluations of physiological monitoring displays: a systematic review. J Clin Monit Comput. 2008;22:45–66.CrossRefPubMed
53.
54.
Sankoh AJ, Huque MF, Dubey SD. Some comments on frequently used multiple endpoint adjustment methods in clinical trials. Stat Med. 1997;16:2529–42.CrossRefPubMed
55.
Fairbanks RJ, Caplan S. Poor interface design and lack of usability testing facilitate medical error. Jt Comm J Qual Saf. 2004;30:579–84.CrossRefPubMed
56.
Hodge E, Blunt M, Young P. Capnography activation is improved by better ventilator interface ergonomics. Crit Care. 2011;15:P139.CrossRefPubMedCentral
57.
58.
Pickering BW, Litell JM, Herasevich V, Gajic O. Clinical review: the hospital of the future - building intelligent environments to facilitate safe and effective acute care delivery. Crit Care. 2012;16:220.CrossRefPubMedPubMedCentral
59.
Colpaert K, Claus B, Somers A, Vandewoude K, Robays H, Decruyenaere J. Impact of computerized physician order entry on medication prescription errors in the intensive care unit: a controlled cross-sectional trial. Crit Care. 2006;10:R21.CrossRefPubMedPubMedCentral
60.
Lin L, Vicente K, Doyle D. Patient safety, potential adverse drug events, and medical device design: a human factors engineering approach. J Biomed Inform. 2001;34:274–84.CrossRefPubMed
61.
Graham TAD, Kushniruk AW, Bullard MJ, Holroyd BR, Meurer DP, Rowe BH. How usability of a web-based clinical decision support system has the potential to contribute to adverse medical events. AMIA Annu Symp Proc. 2008;257–61.
62.
Stadlbauer V, Wallner S, Sojakovic T, Smolle KH. Comparison of three different multi-analyte point-of-care devices during clinical routine on a medical ICU. Crit Care. 2011;433:e1–11.
63.
64.
Hugonnet S, Uckay I, Pittet D. Staffing level: a determinant of late-onset ventilator-associated pneumonia. Crit Care. 2007;11:1–7.CrossRef
65.
Sevdalis N, Brett SJ. Improving care by understanding the way we work: human factors and behavioural science in the context of intensive care. Crit Care. 2009;13:139.CrossRefPubMedPubMedCentral
Metadata
Title
The usability of ventilators: a comparative evaluation of use safety and user experience
Authors
Plinio P. Morita
Peter B. Weinstein
Christopher J. Flewwelling
Carleene A. Bañez
Tabitha A. Chiu
Mario Iannuzzi
Aastha H. Patel
Ashleigh P. Shier
Joseph A. Cafazzo
Publication date
01-12-2016
Publisher
BioMed Central
Published in
Critical Care / Issue 1/2016
Electronic ISSN: 1364-8535
DOI
https://doi.org/10.1186/s13054-016-1431-1

Other articles of this Issue 1/2016

Critical Care 1/2016 Go to the issue

Commentary

Genomics and pharmacogenomics of sepsis: so close and yet so far

Review

Individualizing endpoints in randomized clinical trials to better inform individual patient care: the TARGET proposal

Review

Should we treat pyrexia? And how do we do it?

Review

The neglected role of abdominal compliance in organ-organ interactions

Review

Current evidence for the effectiveness of heated and humidified high flow nasal cannula supportive therapy in adult patients with respiratory failure

Letter

Stress ulcer prophylaxis in adult neurocritical care patients—no firm evidence for benefit or harm