http://orcid.org/0000-0002-1952-0124
Figures
Abstract
Background
Excessive workload may impair patient safety. However, little is known about emergency care providers’ workload during the treatment of life-threatening cases including cardiopulmonary resuscitation (CPR). Therefore, we tested the hypothesis that subjective and physiological indicators of workload are associated with the patient’s initial NACA score and that workload is particularly high during CPR.
Methods
NASA task load index (NASA-tlx) and alarm codes were obtained for 216 sorties of pre-hospital emergency medical care. Furthermore, initial NACA scores of 140 patients were extracted from the physicians’ protocols. The physiological workload indicators mean heart rate (HR) and permutation entropy (PeEn) were calculated for 51 sorties of primary care. General linear mixed models were used to analyze the association of NACA scores with subjective (NASA-tlx) and physiological (mean HR, PeEn) measures of workload.
Results
In contrast to the physiological variables PeEn (p = 0.10) and HR (p = 0.19), the mental (p<0.001) and temporal demands (p<0.001) as well as the effort (p<0.001) and frustration (p = 0.04) subscale of the NASA-tlx were significantly associated with initial NACA scores. Compared to NACA = I, an initial NACA score of VI (representing CPR) increased workload by a mean of 389.5% (p = 0.001) in the mental and 345.9% (p<0.001) in the temporal demands, effort by a mean of 446,8% (p = 0.002) and frustration by 190.0% (p = 0.03). In line with the increase in NASA-tlx, PeEn increased by 20.6% (p = 0.01) and HR by 6.4% (p = 0.57).
Citation: Schneider F, Martin J, Schneider G, Schulz CM (2018) The impact of the patient’s initial NACA score on subjective and physiological indicators of workload during pre-hospital emergency care. PLoS ONE 13(8): e0202215. https://doi.org/10.1371/journal.pone.0202215
Editor: Ashish K. Khanna, Cleveland Clinic Lerner College of Medicine of Case Western Reserve University, UNITED STATES
Received: March 24, 2018; Accepted: July 29, 2018; Published: August 9, 2018
Copyright: © 2018 Schneider et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The treatment of emergency patients can lead to occupational stress and job strain.[1] As in emergency medicine, anesthesia is a field in which intense focus and quick decisions are essential. Hence, various instruments to measure workload have been extensively studied in the environment of anesthesia. In a review by Leedal and Smith, different methodological approaches to indicators of anesthetists’ subjective and physiological workload in the operating theatre were described.[2] Leedal and Smith emphasized the importance of the NASA task load index (NASA-tlx) as an instrument for subjective workload measurements.[2] The NASA-tlx is a multi-dimensional questionnaire that has originally been designed by Hart and Staveland for the National Aeronautics and Space Agency (NASA) to assess pilots’ workload during flight simulations;[3] it has been used in various medical environments[4–6] including anesthesia.[7] Recently, Parsons and co-workers used the NASA-tlx to assess workload during pediatric resuscitation.[8]
Besides subjective workload evaluation, physiological indicators have proven to be valuable correlates of anesthetists’ workload in the operation theatre.[9] In addition to heart rate itself (mean HR),[9–11] heart rate variability (HRV) metrics have been shown to quantify workload in experimental settings and under real-life circumstances.[9, 12–15] Among these, the non-linear HRV metric Permutation Entropy (PeEn) has been found to be a sensible correlate of workload in anesthetists providing general anesthesia.[9] Our recent work indicated that non-linear HRV metrics can also distinguish workload levels during pre-hospital emergency care.[15] However, though excessive workload may impair patient safety,[16–18] data on workload during pre-hospital care including cardiopulmonary resuscitation (CPR) are still scarce.
An additional concern is a potential effect of the alarm code on the physician’s strain prior to the arrival at the scene. Usually, getting valid information about the patient’s situation or the emergency scene previous to arrival is hard to achieve in pre-hospital emergency care.[19] Nevertheless, the intention of briefing alarm-codes is to provide the emergency physician with some information. As this information itself (being valid or not) itself may cause strain, we speculate that the identification of specific alarm codes causing particular strain may help to define training targets.
In order to provide further evidence for the validity of workload measures in pre-hospital emergency care we tested the hypothesis that subjective and physiological indicators of workload are associated with the patient’s initial NACA scores and that workload is particularly high during CPR. Furthermore, we analyzed the physician’s strain related to specific alarm-codes.
Methods
Study design
This observational, exploratory study took part during a period of 5 months. 20 physicians completed queries after each sortie in order to evaluate their subjective workload during their shifts as physicians providing pre-hospital emergency care. Furthermore, a chest-belt (Zephyr BioHarnessTM 3, Zephyr Technology Corp., Annapolis, MD, USA) was used to gather electrocardiograms (ECG) during their shifts. All physicians were specialized in emergency medicine; a supplementary qualification that every physician in Germany may obtain after two years of clinical work including at least 6 months in intensive care medicine or 12 months in anesthesia or the emergency department.[20]
The local Ethics Committee at Klinikum rechts der Isar, Technische Universität München approved the study (N° 5771/13; May 11th, 2015). Written informed consent was obtained from all participants and all data concerning the participating physicians as well as their patients were raised anonymously. No limitations regarding food intake, activity, sleep or medication were given. During the shift changes no study personnel was present; the emergency physicians filled in the query and put on the chest-belt themselves. Parts of the results, specifically the results regarding the validity of linear and non-linear HRV metrics, have been published earlier.[15]
Data collection
In Germany, a system of pre-hospital physician-based emergency care is established. In cases where a life-threatening injury or severe sickness (according to a catalogue of certain keywords) is likely, the rescue coordination center sends an emergency physician response vehicle to the scene.[21] This vehicle is staffed with an emergency physician and a paramedic.
From July to November 2015 the emergency physicians were asked to fill in a query after each sortie. The query contained the NASA-tlx,[3] alarm time and date and the alarm code. Pursuant to the physicians’ emergency protocols, the timestamps for alarm, arrival at the patient and handover of the patient at the admitting emergency ward as well as initial NACA scores for each patient were obtained. The sorties took place in a suburban environment in Southern Germany.
After the study period, all participating physicians were asked to complete a survey on their strain related to the alarm codes that they were confronted with during their duties as emergency physicians. A detailed description on the processing of the ECG raw data gathered from the chest-belt and the computation of HRV metrics is provided elsewhere.[15]
Use of the NACA scoring-system for patient characteristics
The National Advisory Committee for Aeronautics (NACA, a precursor institution of today’s NASA) score is a scoring system that assesses an emergency patient’s severity of injury or illness by eight levels.[22, 23] After it has been originally introduced as a scoring system for trauma patients 24h after admission to a hospital, the modified NACA score presented by Tryba and colleagues in 1980 can be used for pre-hospital injury severity assessment as well. [23, 24] The grades of the NACA score and their verbal descriptions (used in their German translation for this study) are provided in Table 1. As the category NACA 0 (‘No injury or disease’) is not part of the emergency physician’s protocols, it has not been included. Also, cases with an initial NACA score of VII (death of the patient) were excluded. The NACA score well predicts the mortality and the need for ventilation therapy of patients and has substantial inter-rater reliability.[22–24]
Evaluation of subjective workload via NASA task load index
In line with prior work in the field of anesthesia,[9] a six-dimensional version of the NASA-tlx was used for the evaluation of subjective workload; it included the categories mental demands, physical demands, temporal demands, performance, effort, and frustration.[25] Each rating scale ranges from 0 to 20; the steps do not have an additional description. For the study a German transcription of the simplified paper and pencil version of the NASA-tlx provided on the online appearance of the NASA was used.[26] An English translation of the query as well as the German version used for the study are provided in the supplementary material (S1 Text).
Physiological workload indicators
Based on the times gathered from the physician’s protocols, the timespan from the physician’s arrival at the patient to the handover of the patient at the admitting emergency ward was identified. Due to their high potential to differentiate workload in emergency medicine[15] as well as in anesthetists providing general anesthesia,[9] PeEn and mean HR were included in the analysis[9, 27] of these ECG segments.
Evaluation of physician pre-sortie anxiety related to the alarm code
The physicians were asked to complete a survey on their strain when reading a certain alarm code. Therefore, a query presented all alarm codes that occurred during the study period (N = 99) in a randomized order. The physicians were asked to rate each alarm code. The Likert scale ranged from ‘not nervous at all’ (1) to ‘very nervous’ (5).
Statistical analysis
To evaluate the NASA-tlx, descriptive statistics for the NASA-tlx scores and the physician’s physiological workload correlates according to NACA scores were computed. It was accounted for repeated measurements within subjects. Thus, general linear mixed models (GLMM) were used to explore differences among the subjective and physiological workload correlates. In line with our own prior publications,[9, 15] as well as the results of other researchers in the field,[28, 29] the analysis was not adjusted for experience, sex and age. Statistical analysis was performed using the software package SPSS Statistics 24.0.0.0 (IBM Corp. Armonk, NY, USA) and statistical significance was defined as p < 0.05.
Results
In total, twenty emergency physicians took part in this study (16 male, 4 female). NASA-tlx was obtained during 216 sorties by 17 physicians. Due to missing data in the protocols, NACA scores of only 140 cases were included. The age of the emergency physicians ranged from 30 to 52 years (median 38 years). All physicians were experienced in pre-hospital emergency care; that means they worked as emergency physicians once or twice a month for a median of 6.8 years (minimum 0.9 years; maximum 15.7 years).
Association of the emergency physician’s subjective and physiological workload indicators with patient’s initial NACA scores
Table 2 provides descriptive statistics for the NASA-tlx, Table 3 for mean HR and PeEn, respectively, according to NACA scores. For the sections temporal demands and effort three queries had to be excluded due to missing marks in the questionnaire. In the GLMM the mental (p<0.001) and temporal demands (p<0.001) as well as the effort (p<0.001) and frustration (p = 0.04) subscale of the NASA-tlx were significantly associated with the patient’s initial NACA score (Table 4). Furthermore, for both subjective and physiological indicators of workload, coefficients were highest for the treatment of patients with NACA VI (CPR).
Mean code related strain
99 different alarm codes occurred during the 216 sorties. For all of them, subjective strain ratings were obtained from all 20 physicians. The median strain for all 99 alarm codes was 1.85 (interquartile range 1.65–2.45). Table 5 provides the top ten alarm codes that caused the highest and lowest subjective anxiety level, respectively. Mean strain, interquartile range, and frequency of all alarm codes are provided in supporting information file S1 Table.
Discussion
The aim of this study was to investigate the association among patient’s initial NACA scores and subjective as well as physiological indicators of workload during pre-hospital emergency care. As hypothesized, subjective workload in the subscales mental and temporal demands as well as in effort and frustration of the NASA-tlx was significantly associated with the patient’s initial NACA score.
CPR (NACA VI) was significantly linked to workload in temporal demands (p<0.001), mental demands (p = 0.001), effort (p = 0.002), and frustration (p = 0.03). Although not significant in the ANOVA of the GLMM but in line with what we expected, CPR had the highest coefficients for physical demands and performance as well as the physiological workload indicators PeEn and mean HR, respectively. The increase in workload during sorties that included CPR seems to be important as lower performance during CPR has been associated with individual stress and the social dynamics of the resuscitation situation.[30–33] Moreover, the peak of workload indicators during CPR in our study is in line with results reported by Havel and co-workers who found perceived exertion (Borg’s Rating of Perceived Exertion, RPE) and systolic blood pressure to increase constantly during CPR.[34] Correspondingly, CPR is frequently declared a very stressful situation for all team members[32, 33] and survival rates after CPR were related to the time of day.[35]
Remarkably, the dimension performance was not associated with the patient’s initial NACA score. Consistently, in our research on anesthetists’ workload in the operation theatre NASA-tlx performance measures showed a low area under the curve (AUC, 0.508) regarding their ability to separate levels of workload between the phases of induction and maintenance of anesthesia.[9] However, Leedal and Smith stated that performance in the anesthetic environment is difficult to measure because the outcome is influenced by external factors and individuals might compensate disadvantages in performance by increased effort.[2]
The indicators of physiological workload, PeEn (p = 0.10) and mean HR (p = 0.19), did not reach statistical significance according to the ANOVA of the GLMM. However, in pair-wise comparisons, mean PeEn differed between NACA scores (Table 2). This is in line with our previous work, where HRV metrics provided construct validity as correlates of workload in the operation theatre[9] as well as during pre-hospital emergency care.[15] Among them, especially PeEn provided a high AUC (0.993) to separate workload levels before the alarm and during primary patient care.[15] Facing the little number of NACA VI cases, the lack of statistical significance for PeEn in the present analysis might be caused by a type II error.
Mean HR, in contrast, although highest during CPR, did not provide consistent changes in the pair-wise comparisons of NACA scores (Table 3). Accordingly, during pre-hospital emergency care, mean HR showed an AUC of only 0.558 when comparing different workload levels.[15] Likewise, the physician’s mean HR seemed to be independent of the influence of NACA scores in this study. This was surprising since the AUC of mean HR to discriminate the induction of general anesthesia from maintenance in the operation theatre setting was 0.826.[9] As discussed in detail elsewhere, the poor performance of mean HR during pre-hospital emergency care might be due to high variations among the physician’s baseline heart rate.[15] Nevertheless, a variety of studies indicates a correlation between workload and heart rate.[9, 36] Notably, the findings of the present study are in contrast to the results of Schoenenberger and co-workers who found physical demands to be the factor that is most influenced by patient characteristics which–in our study–should be reflected by the NACA score.[16] According to Leedal and Smith’s review, however, physiological workload indicators do not necessarily correlate with workload, since their use is based on the assumption that changes in workload causes changes to the body.[2]
NASA-tlx analyses have been used to quantify and analyze workload in a variety of medical settings, e.g. analysis of workload differences among radiation oncology professional subgroups,[4] commanding a joint emergency exercise,[5] analysis of workload caused by noise in the operating theatre.[6] Recently, Parson and colleagues demonstrated the validity of the NASA-tlx scale for workload measurement during pediatric resuscitation.[8] However, to our knowledge, this analysis is the first to evaluate the NASA-tlx in pre-hospital emergency care.
The analysis of alarm codes unveiled that sorties which included ‘child’ as part of the briefing-alarm code caused the highest strain within the participants (Table 5). Surprisingly, CPR and ongoing CPR on the scene were not among the ten alarm codes provoking the highest strain. This might reflect that CPR per se is a well-trained scenario that does not cause considerable strain within the physician a priori, but the particular CPR in an unknown environment during pre-hospital emergency care does cause high amounts of workload. Moreover, there seems to be further need for training of pediatric emergencies embedded in the emergency physician’s curriculum since they cause high levels of strain. [16–18] Nevertheless, it has to be pointed out that CPR is an independent category in the NACA scoring system, whereas the medical condition of children during emergency care may range from NACA I to NACA VII. Hence, further research on the correlation of briefing-alarm codes and NACA scores as well as physicians’ workload is needed.
NACA scores have been commonly used to characterize patients;[23, 24, 37–40] with good inter-rater reliability.[22] Still, the NACA scoring-system has been criticized for its lack of objectivity,[39] its dependency on the time of classification[37] and its association with the emergency physician’s experience.[40]
This study has some limitations. First, due to the low absolute number of CPRs, the number of cases with a NACA score of 6 was low (N = 7). This increases the risk for both type I and type II errors. Additionally, physiological correlates of workload were available for a smaller number (N = 13) of emergency physicians, only. As this study was exploratory, an a priori calculation of power size was not possible. Second, as already mentioned above, the patient’s initial NACA score itself is influenced by experience, time and other subjective parameters and might therefore also function as a surrogate of subjective workload. Third, another important consideration is the heterogeneity of the study group, which in our opinion is both strength and limitation of the study. Not all participants worked at the department of anesthesiology and the IQR for experience in emergency medicine ranged from 0.9 to 15.7 years. Nevertheless, as we intended to investigate the workload measures in pre-hospital care, the heterogeneity might also improve reproducibility as 25% of the emergency physicians in Germany are residents and an overall of only 50% of the pre-hospital care providers works in anesthesia. Furthermore, Byrne and co-workers found larger differences between subjects of the same rather than different experience[29] and DeAnda and colleagues were not able to find differences between second year residents and experienced anesthesiologists in their response to simulated critical incidents.[28] Last, the anxiety regarding sorties that include the treatment of children might be biased by the fact that the participating physicians work at a university hospital, where the pediatric department is not located on the main campus and therefore critically ill children are not part of their daily clinical work.
Conclusion
We conclude that this study provides further evidence for the validity of the NASA-tlx in the assessment of emergency physicians’ subjective workload during pre-hospital emergency care. Specifically, the dimensions mental demand, temporal demand, effort and frustration were associated with the patient’s NACA score. As indicated by all parameters including PeEn and HR, workload was highest during CPR. The analysis of briefing-code related strain showed that the participants’ strain was especially high when children were involved.
Supporting information
S1 Table. HRV, NACA scores and alarm codes.
This table contains the data used for statistical analysis.
https://doi.org/10.1371/journal.pone.0202215.s001
(XLSX)
S1 Text. NASA task load query.
This file contains the German query used for acquisition of the NASA task load index as well as an English translation.
https://doi.org/10.1371/journal.pone.0202215.s002
(DOCX)
References
- 1. Estryn-Behar M, Doppia MA, Guetarni K, Fry C, Machet G, Pelloux P, et al. Emergency physicians accumulate more stress factors than other physicians-results from the French SESMAT study. Emergency medicine journal: EMJ. 2011;28(5):397–410. Epub 2010/12/03. pmid:21123828.
- 2. Leedal JM, Smith AF. Methodological approaches to anaesthetists' workload in the operating theatre. British journal of anaesthesia. 2005;94(6):702–9. Epub 2005/04/09. pmid:15817711.
- 3.
Hart SG, Staveland LE. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In: Peter AH, Najmedin M, editors. Advances in Psychology. Volume 52: North-Holland; 1988. p. 139–83.
- 4. Mazur LM, Mosaly PR, Jackson M, Chang SX, Burkhardt KD, Adams RD, et al. Quantitative Assessment of Workload and Stressors in Clinical Radiation Oncology. International Journal of Radiation Oncology*Biology*Physics. 2012;83(5):e571–e6. https://doi.org/10.1016/j.ijrobp.2012.01.063.
- 5. Prytz EG, Rybing J, Jonson CO. Workload differences across command levels and emergency response organizations during a major joint training exercise. Journal of emergency management (Weston, Mass). 2016;14(4):289–97. Epub 2016/08/31. pmid:27575644.
- 6. Wheelock A, Suliman A, Wharton R, Babu ED, Hull L, Vincent C, et al. The Impact of Operating Room Distractions on Stress, Workload, and Teamwork. Annals of surgery. 2015;261(6):1079–84. Epub 2015/08/21. pmid:26291954.
- 7. McNeer RR, Bennett CL, Dudaryk R. Intraoperative Noise Increases Perceived Task Load and Fatigue in Anesthesiology Residents: A Simulation-Based Study. Anesth Analg. 2016;122(2):512–25. Epub 2016/01/23. pmid:26797555.
- 8. Parsons SE, Carter EA, Waterhouse LJ, Sarcevic A, O'Connell KJ, Burd RS. Assessment of workload during pediatric trauma resuscitation. The journal of trauma and acute care surgery. 2012;73(5):1267–72. Epub 2012/11/03. pmid:23117383.
- 9. Martin J, Schneider F, Kowalewskij A, Jordan D, Hapfelmeier A, Kochs EF, et al. Linear and non-linear heart rate metrics for the assessment of anaesthetists' workload during general anaesthesia. British journal of anaesthesia. 2016;117(6):767–74. pmid:27956675
- 10. Weinger MB, Herndon OW, Zornow MH, Paulus MP, Gaba DM, Dallen LT. An objective methodology for task analysis and workload assessment in anesthesia providers. Anesthesiology. 1994;80(1):77–92. pmid:8291734
- 11. Schulz CM, Schneider E, Fritz L, Vockeroth J, Hapfelmeier A, Wasmeier M, et al. Eye-Tracking for assessment of workload: A pilot study in an anaesthesia simulator environment. Br J Anaesth. 2011;106(1):44–50. pmid:21037266
- 12. Henelius A, Hirvonen K, Holm A, Korpela J, Muller K. Mental workload classification using heart rate metrics. Conf Proc IEEE Eng Med Biol Soc. 2009;2009:1836–9. pmid:19963519.
- 13. Cinaz B, Arnrich B, La Marca R, Troster G. Monitoring of mental workload levels during an everyday life office-work scenario. Pers Ubiquit Comput. 2013;17(2):229–39. PubMed PMID: WOS:000314089200003.
- 14. Karhula K, Henelius A, Harma M, Sallinen M, Lindholm H, Kivimaki M, et al. Job strain and vagal recovery during sleep in shift working health care professionals. Chronobiol Int. 2014;31(10):1179–89. PubMed PMID: WOS:000345237200012. pmid:25216071
- 15. Schneider F, Martin J, Hapfelmeier A, Jordan D, Schneider G, Schulz CM. The validity of linear and non-linear heart rate metrics as workload indicators of emergency physicians. PloS one. 2017;12(11):e0188635. pmid:29190808
- 16. Schoenenberger S, Moulin P, Brangier E, Gilibert D. Patients' characteristics and healthcare providers' perceived workload in French hospital emergency wards. International journal of occupational safety and ergonomics: JOSE. 2014;20(4):551–9. Epub 2014/12/17. pmid:25513791.
- 17. Çalişkan Tür F, Toker İ, Şaşmaz CT, Hacar S, Türe B. Occupational stress experienced by residents and faculty physicians on night shifts. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine. 2016;24:34. PubMed PMID: PMC4802727. pmid:27000300
- 18. Gaba DM, Howard SK, Jump B. Production pressure in the work environment. California anesthesiologists' attitudes and experiences. Anesthesiology. 1994;81(2):488–500. Epub 1994/08/01. pmid:8053599.
- 19.
Bernhardt M, Technicians NAoEM. Präklinisches Traumamanagement: Elsevier, Urban & Fischer; 2012.
- 20.
Ärztetag B. Weiterbildungsordnung für die Ärzte Bayerns vom 24. April 2004 Mühlbaurstraße 16, 81677 München (GERMANY): Bayerische Landesärztekammer (BLÄK); 2017 [14.02.2018]. Available from: http://www.blaek.de/weiterbildung/WBO_2004/download/WO%202004_2017.pdf.
- 21. Huber S, Cronlein M, von Matthey F, Hanschen M, Seidl F, Kirchhoff C, et al. Effect of private versus emergency medical systems transportation in trauma patients in a mostly physician based system- a retrospective multicenter study based on the TraumaRegister DGU(R). Scand J Trauma Resusc Emerg Med. 2016;24:60. Epub 2016/04/29. pmid:27121607; PubMed Central PMCID: PMCPMC4849091.
- 22. Raatiniemi L, Liisanantti J, Tommila M, Moilanen S, Ohtonen P, Martikainen M, et al. Evaluating helicopter emergency medical missions: a reliability study of the HEMS benefit and NACA scores. Acta anaesthesiologica Scandinavica. 2017;61(5):557–65. Epub 2017/03/21. pmid:28317095.
- 23. Raatiniemi L, Mikkelsen K, Fredriksen K, Wisborg T. Do pre-hospital anaesthesiologists reliably predict mortality using the NACA severity score? A retrospective cohort study. Acta anaesthesiologica Scandinavica. 2013;57(10):1253–9. PubMed PMID: PMC4287201. pmid:24134443
- 24. Weiss M, Bernoulli L, Zollinger A. [The NACA scale. Construct and predictive validity of the NACA scale for prehospital severity rating in trauma patients]. Der Anaesthesist. 2001;50(3):150–4. Epub 2001/04/24. pmid:11315486.
- 25.
Hart SG SL. Development of the NASA-TLX (Task Load Index): results of empirical and theoretical research. In: PA H, editor. Human Mental Workload. Amsterdam: Elsevier1988. p. 139–83.
- 26. Hart SG, Staveland LE. NASA-TLX: Paper and Pencil Version. National Aeronautics and Space Administration, Human Systems Integration Division; 1988.
- 27. Rieger A, Stoll R, Kreuzfeld S, Behrens K, Weippert M. Heart rate and heart rate variability as indirect markers of surgeons' intraoperative stress. Int Arch Occup Environ Health. 2014;87(2):165–74. Epub 2013/02/02. pmid:23370764.
- 28. DeAnda A, Gaba DM. Role of experience in the response to simulated critical incidents. Anesthesia and analgesia. 1991;72(3):308–15. Epub 1991/03/01. pmid:1994759.
- 29. Byrne AJ, Murphy A, McIntyre O, Tweed N. The relationship between experience and mental workload in anaesthetic practice: an observational study. Anaesthesia. 2013;68(12):1266–72. pmid:24164559.
- 30. Bjørkli CA, Øvergård KI, Bjørshol CA, Myklebust H, Hoff T. Effects of socio-emotional stressors on ventilation rate and subjective workload during simulated CPR by lay rescuers. Applied Ergonomics. 2012;43(4):799–802. pmid:22177598
- 31. Abella BS, Alvarado JP, Myklebust H, Edelson DP, Barry A, O'Hearn N, et al. Quality of cardiopulmonary resuscitation during in-hospital cardiac arrest. Jama. 2005;293(3):305–10. Epub 2005/01/20. pmid:15657323.
- 32. Pitts S, Kellermann AL. Hyperventilation during cardiac arrest. The Lancet. 2004;364(9431):313–5. https://doi.org/10.1016/S0140-6736(04)16740-8.
- 33. Leary M, Abella BS. The challenge of CPR quality: Improvement in the real world. Resuscitation. 2008;77(1):1–3. pmid:18325434
- 34. Havel C, Herkner H, Haugk M, Richling N, Riedmuller E, Trimmel H, et al. Physical strain on advanced life support providers in different out of hospital environments. Resuscitation. 2008;77(1):81–6. pmid:18068888
- 35. Wallace SK, Abella BS, Shofer FS, Leary M, Agarwal AK, Mechem CC, et al. Effect of Time of Day on Prehospital Care and Outcomes After Out-of-Hospital Cardiac Arrest. Circulation. 2013;127(15):1591–6. pmid:23509060
- 36. Weinger MB, Reddy SB, Slagle JM. Multiple measures of anesthesia workload during teaching and nonteaching cases. Anesth Analg. 2004;98(5):1419–25, table of contents. Epub 2004/04/24. pmid:15105224.
- 37. Alessandrini H, Oberladstätter D, Trimmel H, Jahn B, Baubin M. NACA-Scoringsystem. Notfall + Rettungsmedizin. 2012;15(1):42–50.
- 38. Bonatti J, Goschl O, Larcher P, Wodlinger R, Flora G. Predictors of short-term survival after helicopter rescue. Resuscitation. 1995;30(2):133–40. Epub 1995/10/01. pmid:8560102.
- 39. Schlechtriemen T, Burghofer K, Lackner CK, Altemeyer KH. Validierung des NACA-Score anhand objektivierbarer Parameter. Notfall & Rettungsmedizin. 2005;8(2):96–108.
- 40. Knapp J, Bernhard M, Hainer C, Sikinger M, Brenner T, Schlechtriemen T, et al. Besteht ein Zusammenhang zwischen der Einschätzung der Vitalgefährdung und der notfallmedizinischen Erfahrung des Notarztes? Der Anaesthesist. 2008;57(11):1069. pmid:18839122