Top

European Journal of Trauma and Emergency Surgery

Published in:

Open Access 16-08-2022 | Central Nervous System Trauma | Original Article

Inter-rater reliability of the Abbreviated Injury Scale scores in patients with severe head injury shows good inter-rater agreement but variability between countries. An inter-country comparison study

Authors: Amy C. Gunning, Menco J. S. Niemeyer, Mark van Heijl, Karlijn J. P. van Wessem, Ronald V. Maier, Zsolt J. Balogh, Luke P. H. Leenen

Published in: European Journal of Trauma and Emergency Surgery | Issue 3/2023

Abstract

Introduction

Substantial difference in mortality following severe traumatic brain injury (TBI) across international trauma centers has previously been demonstrated. This could be partly attributed to variability in the severity coding of the injuries. This study evaluated the inter-rater and intra-rater reliability of Abbreviated Injury Scale (AIS) scores of patients with severe TBI across three international level I trauma centers.

Methods

A total 150 patients (50 per center) were randomly selected from each respective trauma registry: University Medical Center Utrecht (UMCU), the Netherlands; John Hunter Hospital (JHH), Australia; and Harborview Medical Center (HMC), the United States. Reliability between coders and trauma centers was measured with the intraclass correlation coefficient (ICC).

Results

The reliability between the coders and the original trauma registry scores was 0.50, 0.50, and 0.41 in, respectively, UMCU, JHH, and HMC. The AIS coders at UMCU scored the most AIS codes of ≥ 4. Reliability within the trauma centers was substantial in UMCU (ICC = 0.62) and HMC (ICC = 0.78) and almost perfect in JHH (ICC = 0.85). Reliability between trauma centers was 0.70 between UMCU and JHH, 0.70 between JHH and HMC, and 0.59 between UMCU and HMC.

Conclusion

The results of this study demonstrated a substantial and almost perfect reliability of the AIS coders within the same trauma center, but variability across trauma centers. This indicates a need to improve inter-rater reliability in AIS coders and quality assessments of trauma registry data, specifically for patients with head injuries. Future research should study the effect of differences in AIS scoring on outcome predictions.

Champion HR, Panebianco NL, De Waele JJ, et al. American College of surgeons resources document. In: Vincent J-L, Hall JB, editors., et al., Encyclopedia of intensive care medicine. Springer Berlin Heidelberg; 2012. p. 155–60.

Gunning AC, Lansink KW, van Wessem KJ, et al. Demographic patterns and outcomes of patients in level I trauma centers in three international trauma systems. World J Surg. 2015;39(11):2677–84. https://doi.org/10.1007/s00268-015-3162-x.CrossRefPubMedPubMedCentral

Copes WS, Lawnick M, Champion HR, Sacco WJ. A comparison of Abbreviated Injury Scale 1980 and 1985 versions. J Trauma. 1988;28(1):78–86. https://doi.org/10.1097/00005373-198801000-00011.CrossRefPubMed

Association for the Advancement of Automotive Medicine. http://www.aaam.org/about-ais.html. Accessed 24 Feb 2021.

Boyd CR, Tolson MA, Copes WS. Evaluating trauma care: the TRISS method. Trauma Score and the Injury Severity Score. J Trauma. 1987;27(4):370–8.CrossRefPubMed

Lefering R. Development and validation of the revised injury severity classification score for severely injured patients. Eur J Trauma Emerg Surg. 2009;35(5):437–47. https://doi.org/10.1007/s00068-009-9122-0.CrossRefPubMed

Kimura A, Chadbunchachai W, Nakahara S. Modification of the Trauma and Injury Severity Score (TRISS) method provides better survival prediction in Asian blunt trauma victims. World J Surg. 2012;36(4):813–8. https://doi.org/10.1007/s00268-012-1498-z.CrossRefPubMed

Reiter A, Mauritz W, Jordan B, et al. Improving risk adjustment in critically ill trauma patients: the TRISS-SAPS Score. J Trauma. 2004;57(2):375–80. https://doi.org/10.1097/01.ta.0000104016.78539.94.CrossRefPubMed

Moore L, Hanley JA, Turgeon AF, Lavoie A, Eric B. A new method for evaluating trauma centre outcome performance: TRAM-adjusted mortality estimates. Ann Surg. 2010;251(5):952–8. https://doi.org/10.1097/SLA.0b013e3181d97589.CrossRefPubMed

10.

Joosse P, de Jongh MA, van Delft-Schreurs CC, Verhofstad MH, Goslings JC. Improving performance and agreement in injury coding using the Abbreviated Injury Scale: a training course helps. Health Inf Manag. 2014;43(2):17–22. https://doi.org/10.1177/183335831404300203.CrossRefPubMed

11.

MacKenzie EJ, Shapiro S, Eastham JN. The Abbreviated Injury Scale and Injury Severity Score. Levels of inter- and intrarater reliability. Med Care. 1985;23(6):823–35. https://doi.org/10.1097/00005650-198506000-00008.CrossRefPubMed

12.

Read-Allsopp C. Establishing inter-rater reliability scoring in a state trauma system. J Trauma Nurs. 2004;11(1):35–9. https://doi.org/10.1097/00043860-200411010-00006.CrossRefPubMed

13.

Palmer CS, Lang J, Russell G, et al. Mapping Abbreviated Injury Scale data from 1990 to 1998 versions: a stepping-stone in the contemporary evaluation of trauma. Injury. 2013;44(11):1437–42. https://doi.org/10.1016/j.injury.2012.08.033.CrossRefPubMed

14.

Stewart KE, Cowan LD, Thompson DM. Changing to AIS 2005 and agreement of injury severity scores in a trauma registry with scores based on manual chart review. Injury. 2011;42(9):934–9. https://doi.org/10.1016/j.injury.2010.05.033.CrossRefPubMed

15.

Long WB, Sacco WJ, Copes WS, Lawnick MM, Proctor SM, Sacco JB. An evaluation of expert human and automated Abbreviated Injury Scale and ICD-9-CM injury coding. J Trauma. 1994;36(4):499–503. https://doi.org/10.1097/00005373-199404000-00007.CrossRefPubMed

16.

Neale R, Rokkas P, McClure RJ. Interrater reliability of injury coding in the Queensland Trauma Registry. Emerg Med (Fremantle). 2003;15(1):38–41. https://doi.org/10.1046/j.1442-2026.2003.00406.x.CrossRefPubMed

17.

Olthof DC, Luitse JS, de Groot FM, Goslings JC. A Dutch regional trauma registry: quality check of the registered data. BMJ Qual Saf. 2013;22(9):752–8. https://doi.org/10.1136/bmjqs-2013-001888.CrossRefPubMed

18.

Kottner J, Audige L, Brorson S, et al. Guidelines for reporting reliability and agreement studies (GRRAS) were proposed. J Clin Epidemiol. 2011;64(1):96–106. https://doi.org/10.1016/j.jclinepi.2010.03.002.CrossRefPubMed

19.

The Declaration of Helsinki. http://www.wma.net/en/30publications/10policies/b3/. Accessed 24 Feb 2021.

20.

The Good Clinical Practice Guidelines http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500002874.pdf Accessed 24 Feb 2021.

21.

Norman GR, Streiner DL. Biostatistics: the bare essentials. Shelton, CT: People's Medical Publishing House-USA; 2008.

22.

Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155–63. https://doi.org/10.1016/j.jcm.2016.02.012.CrossRefPubMedPubMedCentral

23.

Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74. https://doi.org/10.2307/2529310.CrossRefPubMed

24.

Ringdal KG, Hestnes M, Palmer CS. Differences and discrepancies between 2005 and 2008 Abbreviated Injury Scale versions-time to standardise. Scand J Trauma Resusc Emerg Med. 2012;20:11. https://doi.org/10.1186/1757-7241-20-11.CrossRefPubMedPubMedCentral

25.

Maduz R, Kugelmeier P, Meili S, Doring R, Meier C, Wahl P. Major influence of interobserver reliability on polytrauma identification with the Injury Severity Score (ISS): time for a centralised coding in trauma registries? Injury. 2017;48(4):885–9. https://doi.org/10.1016/j.injury.2017.02.015.CrossRefPubMed

26.

Gundersen HJ, Osterby R. Optimizing sampling efficiency of stereological studies in biology: or 'do more less well!'. J Microsc. 1981;121(Pt 1):65–73. https://doi.org/10.1111/j.1365-2818.1981.tb01199.x.CrossRefPubMed

Title: Inter-rater reliability of the Abbreviated Injury Scale scores in patients with severe head injury shows good inter-rater agreement but variability between countries. An inter-country comparison study
Authors: Amy C. Gunning
Menco J. S. Niemeyer
Mark van Heijl
Karlijn J. P. van Wessem
Ronald V. Maier
Zsolt J. Balogh
Luke P. H. Leenen
Publication date: 16-08-2022
Publisher: Springer Berlin Heidelberg
Keyword: Central Nervous System Trauma
Published in: European Journal of Trauma and Emergency Surgery / Issue 3/2023
Print ISSN: 1863-9933
Electronic ISSN: 1863-9941
DOI: https://doi.org/10.1007/s00068-022-02059-x

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Inter-rater reliability of the Abbreviated Injury Scale scores in patients with severe head injury shows good inter-rater agreement but variability between countries. An inter-country comparison study

Abstract

Introduction

Methods

Results

Conclusion

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Introduction

Methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 3/2023

Acute abdominal pain at referral emergency departments: an analysis of performance of three time-dependent quality indicators

A systematic review of the timing of intubation in patients with traumatic brain injury: pre-hospital versus in-hospital intubation

Burn injury characteristics, referral pattern, treatment (costs), and outcome in burn patients admitted to a hospital with or without a specialized Burn Centre (BURN-Pro)

Indications for prehospital intubation among severely injured children and the prevalence of significant traumatic brain injury among those intubated due to impaired level of consciousness

Pseudoaneurysms after high-grade blunt solid organ injury and the utility of delayed computed tomography angiography

Tranexamic acid in hip hemiarthroplasty surgery: a systematic review and meta-analysis