Top

Published in:

Open Access 01-12-2016 | Research

Development of an algorithm to aid triage decisions for intensive care unit admission: a clinical vignette and retrospective cohort study

Authors: Joao Gabriel Rosa Ramos, Beatriz Perondi, Roger Daglius Dias, Leandro Costa Miranda, Claudio Cohen, Carlos Roberto Ribeiro Carvalho, Irineu Tadeu Velasco, Daniel Neves Forte

Published in: Critical Care | Issue 1/2016

Abstract

Background

Intensive care unit (ICU) admission triage is performed routinely and is often based solely on clinical judgment, which could mask biases. A computerized algorithm to aid ICU triage decisions was developed to classify patients into the Society of Critical Care Medicine’s prioritization system. In this study, we sought to evaluate the reliability and validity of this algorithm.

Methods

Nine senior physicians evaluated forty clinical vignettes based on real patients. The reference standard was defined as the priorities ascribed by two investigators with full access to patients’ records. Agreement of algorithm-based priorities with the reference standard and with intuitive priorities provided by the physicians were evaluated. Correlations between algorithm prioritization and physicians’ judgment of the appropriateness of ICU admissions in scarcity and nonscarcity settings were also evaluated. Validity was further assessed by retrospectively applying this algorithm to 603 patients with requests for ICU admission for association with clinical outcomes.

Results

Agreement between algorithm-based priorities and the reference standard was substantial, with a median κ of 0.72 (interquartile range [IQR] 0.52–0.77). Algorithm-based priorities demonstrated higher interrater reliability (overall κ 0.61, 95 % confidence interval [CI] 0.57–0.65; median percentage agreement 0.64, IQR 0.59–0.70) than physicians’ intuitive prioritization (overall κ 0.51, 95 % CI 0.47–0.55; median percentage agreement 0.49, IQR 0.44–0.56) (p = 0.001). Algorithm-based priorities were also associated with physicians’ judgment of appropriateness of ICU admission (priorities 1, 2, 3, and 4 vignettes would be admitted to the last ICU bed in 83.7 %, 61.2 %, 45.2 %, and 16.8 % of the scenarios, respectively; p < 0.001) and with actual ICU admission, palliative care consultation, and hospital mortality in the retrospective cohort.

Conclusions

This ICU admission triage algorithm demonstrated good reliability and validity. However, more studies are needed to evaluate a difference in benefit of ICU admission justifying the admission of one priority stratum over the others.

Available only for authorised users

Truog RD, Brock DW, Cook DJ, Danis M, Luce JM, Rubenfeld GD, et al. Rationing in the intensive care unit. Crit Care Med. 2006;34:958–63. quiz 971.CrossRefPubMed

Sinuff T, Kahnamoui K, Cook DJ, Luce JM, Levy MM. Rationing critical care beds: a systematic review. Crit Care Med. 2004;32:1588–97.CrossRefPubMed

Azoulay E, Pochard F, Chevret S, Vinsonneau C, Garrouste M, Cohen Y, et al. Compliance with triage to intensive care recommendations. Crit Care Med. 2001;29:2132–6.CrossRefPubMed

Garrouste-Orgeas M, Montuclard L, Timsit JF, Reignier J, Desmettre T, Karoubi P, et al. Predictors of intensive care unit refusal in French intensive care units: a multiple-center study. Crit Care Med. 2005;33:750–5.CrossRefPubMed

Escher M, Perneger TV, Chevrolet JC. National questionnaire survey on what influences doctors’ decisions about admission to intensive care. BMJ. 2004;329:425. doi:10.1136/bmj.329.7463.425.CrossRefPubMedPubMedCentral

Christian MD, Fowler R, Muller MP, Gomersall C, Sprung CL, Hupert N, et al. Critical care resource allocation: trying to PREEDICCT outcomes without a crystal ball. Crit Care. 2013;17:107.CrossRefPubMedPubMedCentral

Sprung CL, Danis M, Iapichino G, Artigas A, Kesecioglu J, Moreno R, et al. Triage of intensive care patients: identifying agreement and controversy. Intensive Care Med. 2013;39:1916–24. doi:10.1007/s00134-013-3033-6.CrossRefPubMed

Task Force of the American College of Critical Care Medicine, Society of Critical Care Medicine. Guidelines for intensive care unit admission, discharge, and triage. Crit Care Med. 1999;27(3):633–8.CrossRef

Twomey M, Wallis LA, Myers JE. Limitations in validating emergency department triage scales. Emerg Med J. 2007;24:477–9. doi:10.1136/emj.2007.046383.CrossRefPubMedPubMedCentral

10.

Streiner DL, Norman GR. “Precision” and “accuracy”: two terms that are neither. J Clin Epidemiol. 2006;59:327–30. doi:10.1016/j.jclinepi.2005.09.005.CrossRefPubMed

11.

Angus DC, Seymour CW, Coopersmith CM, Deutschman CS, Klompas M, Levy MM, et al. A framework for the development and interpretation of different sepsis definitions and clinical criteria. Crit Care Med. 2016;44:e113–21. doi:10.1097/CCM.0000000000001730.CrossRefPubMed

12.

Tavares De Carvalho R, Parsons HA. Manual de cuidados paliativos ANCP: ampliado e atualizado. 2nd ed. Sao Paulo, Brazi: Academia Nacional de Cuidados Paliativos (ANCP); 2012.

13.

Katz S, Ford AB, Moskowitz RW, Jackson BA, Jaffe MW. Studies of illness in the aged. The index of ADL: a standardized measure of biological and psychosocial function. JAMA. 1963;185:914–9.CrossRefPubMed

14.

Lowry R. VassarStats: website for statistical computation. http://vassarstats.net/. Accessed 14 March 2016.

15.

Walter SD, Eliasziw M, Donner A. Sample size and optimal designs for reliability studies. Stat Med. 1998;17:101–10.CrossRefPubMed

16.

Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.CrossRefPubMed

17.

Caldeira VM, Silva Júnior JM, Oliveira AM, Rezende S, Araújo LA, Santana MR, et al. Criteria for patient admission to an intensive care unit and related mortality rates. Rev Assoc Med Bras. 2010;56:528–34.CrossRefPubMed

18.

Christian MD, Hawryluck L, Wax RS, Cook T, Lazar NM, Herridge MS, et al. Development of a triage protocol for critical care during an influenza pandemic. CMAJ. 2006;175:1377–81. doi:10.1503/cmaj.060911.CrossRefPubMedPubMedCentral

19.

Cheung WK, Myburgh J, Seppelt IM, Parr MJ, Blackwell N, Demonte S, et al. A multicentre evaluation of two intensive care unit triage protocols for use in an influenza pandemic. Med J Aust. 2012;197:178–81.CrossRefPubMed

20.

Shum HP, Chan KC, Lau CW, Leung AK, Chan KW, Yan WW. Triage decisions and outcomes for patients with Triage Priority 3 on the Society of Critical Care Medicine scale. Crit Care Resusc. 2010;12:42–9.PubMed

21.

Knaus WA, Harrell Jr FE, Lynn J, Goldman L, Phillips RS, Connors Jr AF, et al. The SUPPORT prognostic model: objective estimates of survival for seriously ill hospitalized adults. Ann Intern Med. 1995;122:191–203.CrossRefPubMed

22.

Maghsoudi B, Tabei SH, Zand F, Tabatabaee H, Akbarzadeh A. A model for decision making for intensive care unit admission in source limited hospitals. Iran Red Crescent Med J. 2014;16:e15497. doi:10.5812/ircmj.15497.CrossRefPubMedPubMedCentral

23.

Sinuff T, Adhikari NK, Cook DJ, Schunemann HJ, Griffith LE, Rocker G, et al. Mortality predictions in the intensive care unit: comparing physicians with scoring systems. Crit Care Med. 2006;34:878–85. doi:10.1097/01.CCM.0000201881.58644.41.CrossRefPubMed

24.

Parenti N, Reggiani ML, Iannone P, Percudani D, Dowding D. A systematic review on the validity and reliability of an emergency department triage scale, the Manchester Triage System. Int J Nurs Stud. 2014;51:1062–9. doi:10.1016/j.ijnurstu.2014.01.013.CrossRefPubMed

25.

Christ M, Grossmann F, Winter D, Bingisser R, Platz E. Modern triage in the emergency department. Dtsch Arztebl Int. 2010;107:892–8.PubMedPubMedCentral

26.

Farrohknia N, Castren M, Ehrenberg A, Lind L, Oredsson S, Jonsson H, et al. Emergency department triage scales and their components: a systematic review of the scientific evidence. Scand J Trauma Resusc Emerg Med. 2011;19:42. doi:10.1186/1757-7241-19-42.CrossRefPubMedPubMedCentral

27.

Piers RD, Azoulay E, Ricou B, Dekeyser Ganz F, Decruyenaere J, Max A, et al. Perceptions of appropriateness of care among European and Israeli intensive care unit nurses and physicians. JAMA. 2011;306:2694–703. doi:10.1001/jama.2011.1888.CrossRefPubMed

28.

McHugh ML. Interrater reliability: the κ statistic. Biochem Med (Zagreb). 2012;22:276–82.CrossRef

29.

Mohan D, Fischhoff B, Farris C, Switzer GE, Rosengart MR, Yealy DM, et al. Validating a vignette-based instrument to study physician decision making in trauma triage. Med Decis Making. 2014;34:242–52. doi:10.1177/0272989X13508007.CrossRefPubMedPubMedCentral

30.

Graff I, Goldschmidt B, Glien P, Bogdanow M, Fimmers R, Hoeft A, et al. The German version of the Manchester Triage System and its quality criteria – first assessment of validity and reliability. PLoS One. 2014;9:e88995. doi:10.1371/journal.pone.0088995.CrossRefPubMedPubMedCentral

31.

Mirhaghi A, Heydari A, Mazlom R, Ebrahimi M. The reliability of the Canadian Triage and Acuity Scale: meta-analysis. N Am J Med Sci. 2015;7:299–305. doi:10.4103/1947-2714.161243.CrossRefPubMedPubMedCentral

32.

Ebrahimi M, Heydari A, Mazlom R, Mirhaghi A. The reliability of the Australasian Triage Scale: a meta-analysis. World J Emerg Med. 2015;6:94–9. doi:10.5847/wjem.j.1920-8642.2015.02.002.CrossRefPubMedPubMedCentral

33.

Reith FC, Van den Brande R, Synnot A, Gruen R, Maas AI. The reliability of the Glasgow Coma Scale: a systematic review. Intensive Care Med. 2016;42:3–15. doi:10.1007/s00134-015-4124-3.CrossRefPubMed

34.

Riley R, Holman C, Fletcher D. Inter-rater reliability of the ASA physical status classification in a sample of anaesthetists in Western Australia. Anaesth Intensive Care. 2014;42:614–8.PubMed

35.

Gomersall CD, Joynt GM. What is the benefit in triage? Crit Care Med. 2011;39:911–2.CrossRefPubMed

36.

Mohan D, Angus DC, Ricketts D, Farris C, Fischhoff B, Rosengart MR, et al. Assessing the validity of using serious game technology to analyze physician decision making. PLoS One. 2014;9:e105445. doi:10.1371/journal.pone.0105445.CrossRefPubMedPubMedCentral

37.

van der Wulp I, van Baar ME, Schrijvers AJ. Reliability and validity of the Manchester Triage System in a general emergency department patient population in the Netherlands: results of a simulation study. Emerg Med J. 2008;25:431–4. doi:10.1136/emj.2007.055228.CrossRefPubMed

38.

Grouse AI, Bishop RO, Bannon AM. The Manchester Triage System provides good reliability in an Australian emergency department. Emerg Med J. 2009;26:484–6. doi:10.1136/emj.2008.065508.CrossRefPubMed

39.

Redelmeier DA, Shafir E. Medical decision making in situations that offer multiple alternatives. JAMA. 1995;273:302–5.CrossRefPubMed

40.

Green MJ, Farber NJ, Ubel PA, Mauger DT, Aboff BM, Sosman JM, et al. Lying to each other: when internal medicine residents use deception with their colleagues. Arch Intern Med. 2000;160:2317–23.CrossRefPubMed

41.

Aberegg SK, Haponik EF, Terry PB. Omission bias and decision making in pulmonary and critical care medicine. Chest. 2005;128:1497–505. doi:10.1378/chest.128.3.1497.CrossRefPubMed

42.

Cook DJ. Determinants in Canadian health care workers of the decision to withdraw life support from the critically ill. JAMA. 1995;273:703–8. doi:10.1001/jama.273.9.703.CrossRefPubMed

43.

O’Brien Jr JM, Aberegg SK, Ali NA, Diette GB, Lemeshow S. Results from the National Sepsis Practice Survey: predictions about mortality and morbidity and recommendations for limitation of care orders. Crit Care. 2009;13:R96. doi:10.1186/cc7926.CrossRefPubMedPubMedCentral

44.

Dahine J, Mardini L, Jayaraman D. The perceived likelihood of outcome of critical care patients and its impact on triage decisions: a case-based survey of intensivists and internists in a Canadian, quaternary care hospital network. PLoS One. 2016;11:e0149196. doi:10.1371/journal.pone.0149196.CrossRefPubMedPubMedCentral

Title: Development of an algorithm to aid triage decisions for intensive care unit admission: a clinical vignette and retrospective cohort study
Authors: Joao Gabriel Rosa Ramos
Beatriz Perondi
Roger Daglius Dias
Leandro Costa Miranda
Claudio Cohen
Carlos Roberto Ribeiro Carvalho
Irineu Tadeu Velasco
Daniel Neves Forte
Publication date: 01-12-2016
Publisher: BioMed Central
Published in: Critical Care / Issue 1/2016
Electronic ISSN: 1364-8535
DOI: https://doi.org/10.1186/s13054-016-1262-0

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Development of an algorithm to aid triage decisions for intensive care unit admission: a clinical vignette and retrospective cohort study

Abstract

Background

Methods

Results

Conclusions

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2016

Echocardiography in shock management

Do we need new prokinetics to reduce enteral feeding intolerance during critical illness?

Outcomes of patients admitted to intensive care units for acute manifestation of small-vessel vasculitis: a multicenter, retrospective study

Successful treatment with fecal microbiota transplantation in patients with multiple organ dysfunction syndrome and diarrhea following severe sepsis

The intensive care infection score – a novel marker for the prediction of infection and its severity

Early-phase cumulative hypotension duration and severe-stage progression in oliguric acute kidney injury with and without sepsis: an observational study