Skip to main content
Top
Published in: Implementation Science 1/2018

Open Access 01-12-2018 | Research

Economic evaluation of the NET intervention versus guideline dissemination for management of mild head injury in hospital emergency departments

Authors: Duncan Mortimer, Marije Bosch, Joanne E. Mckenzie, Simon Turner, Marisa Chau, Jennie L. Ponsford, Jonathan C. Knott, Russell L. Gruen, Sally E. Green

Published in: Implementation Science | Issue 1/2018

Login to get access

Abstract

Background

Evidence-based guidelines for the management of mild traumatic brain injury (mTBI) in the emergency department (ED) are now widely available, and yet, clinical practice remains inconsistent with the guidelines. The Neurotrauma Evidence Translation (NET) intervention was developed to increase the uptake of guideline recommendations and improve the management of minor head injury in Australian emergency departments (EDs). However, the adoption of this type of intervention typically entails an upfront investment that may or may not be fully offset by improvements in clinical practice, health outcomes and/or reductions in health service utilisation. The present study estimates the cost and cost-effectiveness of the NET intervention, as compared to the passive dissemination of the guideline, to evaluate whether any improvements in clinical practice or health outcomes due to the NET intervention can be obtained at an acceptable cost.

Methods and findings

Study setting: The NET cluster randomised controlled trial [ACTRN12612001286831]. Study sample: Seventeen EDs were randomised to the control condition and 14 to the intervention. One thousand nine hundred forty-three patients were included in the analysis of clinical practice outcomes (NET sample). A total of 343 patients from 14 control and 10 intervention EDs participated in follow-up interviews and were included in the analysis of patient-reported health outcomes (NET-Plus sample). Outcome measures: Appropriate post-traumatic amnesia (PTA) screening in the ED (primary outcome). Secondary clinical practice outcomes: provision of written information on discharge (INFO) and safe discharge (defined as CT scan appropriately provided plus PTA plus INFO). Secondary patient-reported, post-discharge health outcomes: anxiety (Hospital Anxiety and Depression Scale), post-concussive symptoms (Rivermead), and preference-based health-related quality of life (SF6D). Methods: Trial-based economic evaluations from a health sector perspective, with time horizons set to coincide with the final follow-up for the NET sample (2 months post-intervention) and to 1-month post-discharge for the NET-Plus sample. Results: Intervention and control groups were not significantly different in health service utilisation received in the ED/inpatient ward following the initial mTBI presentation (adjusted mean difference $23.86 per patient; 95%CI − $106, $153; p = 0.719) or over the longer follow-up in the NET-plus sample (adjusted mean difference $341.78 per patient; 95%CI − $58, $742; p = 0.094). Savings from lower health service utilisation are therefore unlikely to offset the significantly higher upfront cost of the intervention (mean difference $138.20 per patient; 95%CI $135, $141; p < 0.000). Estimates of the net effect of the intervention on total cost (intervention cost net of health service utilisation) suggest that the intervention entails significantly higher costs than the control condition (adjusted mean difference $169.89 per patient; 95%CI $43, $297, p = 0.009). This effect is larger in absolute magnitude over the longer follow-up in the NET-plus sample (adjusted mean difference $505.06; 95%CI $96, $915; p = 0.016), mostly due to additional health service utilisation. For the primary outcome, the NET intervention is more costly and more effective than passive dissemination; entailing an additional cost of $1246 per additional patient appropriately screened for PTA ($169.89/0.1363; Fieller’s 95%CI $525, $2055). For NET to be considered cost-effective with 95% confidence, decision-makers would need to be willing to trade one quality-adjusted life year (QALY) for 25 additional patients appropriately screened for PTA. While these results reflect our best estimate of cost-effectiveness given the data, it is possible that a NET intervention that has been scaled and streamlined ready for wider roll-out may be more or less cost-effective than the NET intervention as delivered in the trial.

Conclusions

While the NET intervention does improve the management of mTBI in the ED, it also entails a significant increase in cost and—as delivered in the trial—is unlikely to be cost-effective at currently accepted funding thresholds. There may be a scope for a scaled-up and streamlined NET intervention to achieve a better balance between costs and outcomes.

Trial registration

Australian New Zealand Clinical Trials Registry ACTRN12612001286​831, date registered 12 December 2012.
Appendix
Available only for authorised users
Footnotes
1
Preference-based HRQoL weights from the SF12-based SF6D must be combined with a period of time to permit estimation of intervention effects in QALY terms (and incremental cost-effectiveness in cost per QALY terms). For the NET study, SF12 data was available for a single time point, some 7 months post-discharge (on average). In the absence of repeated observation on HRQoL, we will treat measures of HRQoL as a summary measure of the effect of the intervention with respect to anxiety, sleep, functioning, and adverse effects at a single point in time and we make no attempt to combine SF6D index scores with time for the purposes of calculating QALYs.
 
2
The protocol specified a time horizon for the NET-Plus sample to coincide with the final follow-up for the NET-Plus sample, originally planned to take place at 3 months post-injury. Due to delays in recruitment, follow-up for NET-Plus patients occurred much later and over a much longer period than planned (mean = 210 days post discharge; SD 38.5 days; IQR 181–239; min = 130, max = 321). In such circumstances, using patient self-report on a 4-week window of health service utilisation to calculate per diems and then multiplying out over the entire period from discharge to follow-up would amplify any measurement errors due to recall bias or timing. We therefore treat patient self-report on health service utilisation as representative of a 4-week period, post-discharge from their initial mTBI presentation and take this period as the time horizon for economic evaluations in the NET-Plus sample.
 
3
Recommendations to fund a new intervention or change clinical practice are only consistent with maximising population health if they deliver a unit of the outcome of interest at a better price than is on offer elsewhere (i.e. better than interventions that are not or will not be funded if the new intervention or policy change is recommended). Decision-makers therefore need to know the price at which they can buy a unit of health outcome via the evaluated intervention or policy change (captured by the incremental cost effectiveness ratio) and the price at which alternative interventions could deliver that same health outcome (captured by the funding threshold, also known as the cost-effectiveness threshold or reference ICER). As Edney et al. [34] put it, “funding of a new technology with an ICER greater than the reference ICER results in a net decrease in population health, as represented by net QALY losses” (p248).
 
4
Using a new approach, Edney et al. [34] estimated the reference ICER for the Australian health care system at AUD28033 (95%CI AUD20758–AUD37667). This is broadly consistent with estimates for Australia (95%CI AUD28904, AUD51470 at current exchange rates) reported by Woods et al. [37] but much lower than the ballpark figure of AUD50000 per QALY that has been suggested as a rule-of-thumb for decision-makers [38] and much lower than previous estimates based on stated and revealed preference methods. For example, George et al. [39] estimated the threshold or reference ICER at between AUD42000 and AUD76000 using past funding decisions to intuit the revealed preferences of decision-makers. Shiroiwa et al. [40] estimated the funding threshold using a stated preference approach (maximum social willingness to pay per QALY) at between AUD64000 and AUD89000 (with higher/lower estimates for different sub-populations).
 
5
Calculated by dividing the funding threshold expressed in cost per QALY terms by the upper confidence limit on the cost-effectiveness ratio for the outcome of interest. For cost per additional patient appropriately screened for PTA, this calculation is $50,000/$2055 for the parametric 95%CI and $50,000/$2066 for the non-parametric 95%CI (see Table 5 and accompanying discussion). This calculation gives us 24.33 additional patients appropriately screened for PTA per QALY (or 0.04 QALYs per patient appropriately screened for PTA).
 
6
NET is the most cost-effective alternative for over 80% of the distribution if the funding threshold exceeds $34,000 per point improvement in SF6D index scores (where a point improvement is a full 0 (death) to 1 (full health) improvement at a single point in time). The ratio of cost per point improvement in SF6D index scores and cost per QALY ($34,000/$50,000) gives us the duration (fraction of a year) over which the SF6D index scores observed in the NET-Plus sample would have to persist in order for $34,000 per point improvement in SF6D index scores to be equivalent to a threshold of $50,000 per QALY. This calculation reflects the fact that QALYs are the product of duration (in years) and SF6D index scores, with one QALY being equivalent to 1 year with an SF6D score equal to one.
 
7
Only 60% of surveyed clinicians in intervention group EDs claimed to have attended or watched a PTA session, only 30% attended or watched CT session, and only 55% attended or watched session on handing out patient information.
 
8
Doubling the cost of attendance time, travel time and direct travel costs, and the cost of consumables for local opinion leaders (to accommodate participation from 28 rather than 14 EDs) but leaving other costs associated with delivery of the TTT workshops unchanged would see the total cost of delivering the TTT workshops increase to around $108,000 per 28 EDs, far less than double the cost of delivery to 14 EDs. This would equate to a decrease in the cost per ED from $5004 per ED (for 14 participating EDs) to $3870 per ED (for 28 participating EDs).
 
Literature
1.
go back to reference Tavender EJ, Bosch M, Green S, O'Connor D, Pitt V, Phillips K, et al. Quality and consistency of guidelines for the management of mild traumatic brain injury in the emergency department. Acad Emerg Med. 2011;18:880–9.CrossRefPubMed Tavender EJ, Bosch M, Green S, O'Connor D, Pitt V, Phillips K, et al. Quality and consistency of guidelines for the management of mild traumatic brain injury in the emergency department. Acad Emerg Med. 2011;18:880–9.CrossRefPubMed
2.
go back to reference Bosch M, McKenzie JE, Mortimer D, Tavender EJ, Francis JJ, Brennan SE, et al. Implementing evidence-based recommended practices for the management of patients with mTBI in Australian EDs: study protocol for a CRCT. Trials. 2014;15(281):1–20.CrossRefPubMedPubMedCentral Bosch M, McKenzie JE, Mortimer D, Tavender EJ, Francis JJ, Brennan SE, et al. Implementing evidence-based recommended practices for the management of patients with mTBI in Australian EDs: study protocol for a CRCT. Trials. 2014;15(281):1–20.CrossRefPubMedPubMedCentral
3.
go back to reference Holmes MW, Goodacre S, Stevenson MD, Pandor A, Pickering A. The cost-effectiveness of diagnostic management strategies for adults with minor head injury. Injury. 2012;43:1423–31.CrossRefPubMed Holmes MW, Goodacre S, Stevenson MD, Pandor A, Pickering A. The cost-effectiveness of diagnostic management strategies for adults with minor head injury. Injury. 2012;43:1423–31.CrossRefPubMed
4.
go back to reference Smits M, Dippel DW, Nederkoorn PJ, Dekker HM, Vos PE, Kool DR, et al. Minor head injury: CT-based strategies for management–a cost-effectiveness analysis. Radiology. 2010;254:532–40.CrossRefPubMed Smits M, Dippel DW, Nederkoorn PJ, Dekker HM, Vos PE, Kool DR, et al. Minor head injury: CT-based strategies for management–a cost-effectiveness analysis. Radiology. 2010;254:532–40.CrossRefPubMed
5.
go back to reference Stein SC, Burnett MG, Glick HA. Indications for CT scanning in mild traumatic brain injury: a cost-effectiveness study. J Trauma. 2006;61:558–66.CrossRefPubMed Stein SC, Burnett MG, Glick HA. Indications for CT scanning in mild traumatic brain injury: a cost-effectiveness study. J Trauma. 2006;61:558–66.CrossRefPubMed
6.
go back to reference Pandor A, Harnan S, Goodacre S, Pickering A, Fitzgerald P, Rees A. Diagnostic accuracy of clinical characteristics for identifying CT abnormality following minor head injury: a systematic review and meta-analysis. J Neurotrauma. 2012;29:707–18.CrossRefPubMed Pandor A, Harnan S, Goodacre S, Pickering A, Fitzgerald P, Rees A. Diagnostic accuracy of clinical characteristics for identifying CT abnormality following minor head injury: a systematic review and meta-analysis. J Neurotrauma. 2012;29:707–18.CrossRefPubMed
7.
go back to reference Grimshaw J, Eccles MP, Thomas R, MacLennan G, Ramsay C, Fraser C, et al. Toward evidence-based quality improvement: evidence (and its limitations) of the effectiveness of guideline dissemination and implementation strategies 1966-1998. J Gen Med Intern Med. 2006;21:S14–20. Grimshaw J, Eccles MP, Thomas R, MacLennan G, Ramsay C, Fraser C, et al. Toward evidence-based quality improvement: evidence (and its limitations) of the effectiveness of guideline dissemination and implementation strategies 1966-1998. J Gen Med Intern Med. 2006;21:S14–20.
9.
go back to reference Vale L, Thomas R, MacLennan G, Grimshaw J, Vale L, Thomas R, et al. Systematic review of economic evaluations and cost analyses of guideline implementation strategies. Eur J Health Econ. 2007;8(2):111–21 PubMed PMID: 17347844.CrossRefPubMed Vale L, Thomas R, MacLennan G, Grimshaw J, Vale L, Thomas R, et al. Systematic review of economic evaluations and cost analyses of guideline implementation strategies. Eur J Health Econ. 2007;8(2):111–21 PubMed PMID: 17347844.CrossRefPubMed
10.
go back to reference Tavender EJ, Bosch M, Gruen RL, Green SE, Michie S, Brennan SE, et al. Developing a targeted, theory-informed implementation intervention using two theoretical frameworks to address health professional and organisational factors: a case study to improve the management of mild traumatic brain injury in the emergency department. Implement Sci. 2015;10:74.CrossRefPubMedPubMedCentral Tavender EJ, Bosch M, Gruen RL, Green SE, Michie S, Brennan SE, et al. Developing a targeted, theory-informed implementation intervention using two theoretical frameworks to address health professional and organisational factors: a case study to improve the management of mild traumatic brain injury in the emergency department. Implement Sci. 2015;10:74.CrossRefPubMedPubMedCentral
13.
go back to reference Bosch M, Tavender EJ, Brennan SE, Knott J, Gruen RLG, S.E. The many organisational factors relevant to planning change in emergency care departments: a qualitative study to inform a cluster randomised controlled trial aiming to improve the management of patients with mild traumatic brain injuries. PLoS One. 2016;11:e0148091.CrossRefPubMedPubMedCentral Bosch M, Tavender EJ, Brennan SE, Knott J, Gruen RLG, S.E. The many organisational factors relevant to planning change in emergency care departments: a qualitative study to inform a cluster randomised controlled trial aiming to improve the management of patients with mild traumatic brain injuries. PLoS One. 2016;11:e0148091.CrossRefPubMedPubMedCentral
14.
go back to reference Tavender EJ, Bosch M, Gruen RLG, Green SE, Knott J, Francis JJ, Michie S, et al. Understanding practice: the factors that influence management of mild traumatic brain injury in the emergency department--a qualitative study using the theoretical domains framework. Implement Sci. 2014;9:8.CrossRefPubMedPubMedCentral Tavender EJ, Bosch M, Gruen RLG, Green SE, Knott J, Francis JJ, Michie S, et al. Understanding practice: the factors that influence management of mild traumatic brain injury in the emergency department--a qualitative study using the theoretical domains framework. Implement Sci. 2014;9:8.CrossRefPubMedPubMedCentral
15.
go back to reference NSW_Institute_of_Trauma_and_Injury_Management. Initial management of closed head injury in adults. 2nd ed. Australia: NSW Ministry of Health; 2011. NSW_Institute_of_Trauma_and_Injury_Management. Initial management of closed head injury in adults. 2nd ed. Australia: NSW Ministry of Health; 2011.
16.
go back to reference Scott I, Duckett S. In search of professional consensus in defining and reducing low-value care. Med J Aust. 2015;203:179–81.CrossRefPubMed Scott I, Duckett S. In search of professional consensus in defining and reducing low-value care. Med J Aust. 2015;203:179–81.CrossRefPubMed
18.
go back to reference Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, et al. Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2010;(3):CD005470-CD. doi: https://doi.org/10.1002/14651858.CD005470.pub2. PubMed PMID: PMC4164371. Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, et al. Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2010;(3):CD005470-CD. doi: https://​doi.​org/​10.​1002/​14651858.​CD005470.​pub2. PubMed PMID: PMC4164371.
20.
go back to reference Mortimer D, French S, McKenzie J, O’Connor D, Green S. Economic evaluation of active implementation versus guideline dissemination for evidence-based care of acute low-back pain in a general practice setting. PLoS One. 2013;8(10):e75647.CrossRefPubMedPubMedCentral Mortimer D, French S, McKenzie J, O’Connor D, Green S. Economic evaluation of active implementation versus guideline dissemination for evidence-based care of acute low-back pain in a general practice setting. PLoS One. 2013;8(10):e75647.CrossRefPubMedPubMedCentral
21.
go back to reference NSW Institute of Trauma and Injury Management. Initial management of closed head injury in adults, second edition. Sydney: NSW Ministry of Health; 2011. NSW Institute of Trauma and Injury Management. Initial management of closed head injury in adults, second edition. Sydney: NSW Ministry of Health; 2011.
22.
go back to reference French SD, Green SE, O’Connor D, McKenzie JE, Francis JJ, Michie S, et al. Developing theory-informed behaviour change interventions to implement evidence into practice: a systematic approach using the Theoretical Domains Framework. Implement Sci. 2012; 2012 in press, accepted 23 December 2011. French SD, Green SE, O’Connor D, McKenzie JE, Francis JJ, Michie S, et al. Developing theory-informed behaviour change interventions to implement evidence into practice: a systematic approach using the Theoretical Domains Framework. Implement Sci. 2012; 2012 in press, accepted 23 December 2011.
23.
go back to reference Whelan-Goodinson R, Ponsford J, Schonberger M. Validity of the Hospital Anxiety and Depression Scale to assess depression and anxiety following traumatic brain injury as compared with the Structured Clinical Interview for DSM-IV. J Affect Disord. 2009;114:94–102.CrossRefPubMed Whelan-Goodinson R, Ponsford J, Schonberger M. Validity of the Hospital Anxiety and Depression Scale to assess depression and anxiety following traumatic brain injury as compared with the Structured Clinical Interview for DSM-IV. J Affect Disord. 2009;114:94–102.CrossRefPubMed
24.
go back to reference Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand. 1983;67:361–70.CrossRefPubMed Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand. 1983;67:361–70.CrossRefPubMed
25.
go back to reference King NS, Crawford S, Wenden FJ, Moss NE, Wade DT. The Rivermead Post Concussion Symptoms Questionnaire: a measure of symptoms commonly experienced after head injury and its reliability. J Neurol. 1995;242:587–92.CrossRefPubMed King NS, Crawford S, Wenden FJ, Moss NE, Wade DT. The Rivermead Post Concussion Symptoms Questionnaire: a measure of symptoms commonly experienced after head injury and its reliability. J Neurol. 1995;242:587–92.CrossRefPubMed
26.
go back to reference Brazier JE, Roberts JR. The estimation of a preference-based index from the SF-12. Med Care. 2004;42:851–9.CrossRefPubMed Brazier JE, Roberts JR. The estimation of a preference-based index from the SF-12. Med Care. 2004;42:851–9.CrossRefPubMed
27.
go back to reference Brazier JE, Rowen D, Hanmer J. Revised SF-6D scoring programmes: a summary of improvements. PRO Newsletter. 2008;40:14–5. Brazier JE, Rowen D, Hanmer J. Revised SF-6D scoring programmes: a summary of improvements. PRO Newsletter. 2008;40:14–5.
28.
go back to reference Ware J, Kosinski M, Keller SD. A 12-item short-form health survey: construction of scales and preliminary tests of reliability and validity. Med Care. 1996;34:220–33.CrossRefPubMed Ware J, Kosinski M, Keller SD. A 12-item short-form health survey: construction of scales and preliminary tests of reliability and validity. Med Care. 1996;34:220–33.CrossRefPubMed
29.
go back to reference Buntin MB, Zaslavsky AM. Too much ado about two-part models and transformation? Comparing methods of modeling Medicare expenditures. J Health Econ. 2004;23:525–42.CrossRefPubMed Buntin MB, Zaslavsky AM. Too much ado about two-part models and transformation? Comparing methods of modeling Medicare expenditures. J Health Econ. 2004;23:525–42.CrossRefPubMed
30.
go back to reference ABS. 6401.0 - consumer Price index, Australia, March 2016. Canberra: Australian Bureau of Statistics; 2016. ABS. 6401.0 - consumer Price index, Australia, March 2016. Canberra: Australian Bureau of Statistics; 2016.
31.
go back to reference Glick H, Doshi J, Sonnad S, Polsky D. Economic evaluation in clinical trials. New York: Oxford University Press; 2007. Glick H, Doshi J, Sonnad S, Polsky D. Economic evaluation in clinical trials. New York: Oxford University Press; 2007.
32.
go back to reference Luo N, Johnson J. Using instrument-defined health state transitions to estimate minimally important differences for four preference-based health-related quality of life instruments. Med Care. 2010;48:365–71.CrossRefPubMed Luo N, Johnson J. Using instrument-defined health state transitions to estimate minimally important differences for four preference-based health-related quality of life instruments. Med Care. 2010;48:365–71.CrossRefPubMed
33.
go back to reference Walters S, Brazier J. Comparison of the minimally important difference for two health state utility measures: EQ-5D and SF-6D. Qual Life Res. 2005;14:1523–32.CrossRefPubMed Walters S, Brazier J. Comparison of the minimally important difference for two health state utility measures: EQ-5D and SF-6D. Qual Life Res. 2005;14:1523–32.CrossRefPubMed
34.
go back to reference Edney L, Afzali H, Cheng T, Karnon J. Estimating the reference incremental cost-effectiveness ratio for the Australian health system. PharmacoEconomics. 2018;36:239–52.CrossRefPubMed Edney L, Afzali H, Cheng T, Karnon J. Estimating the reference incremental cost-effectiveness ratio for the Australian health system. PharmacoEconomics. 2018;36:239–52.CrossRefPubMed
35.
go back to reference Vallejo-Torres L, García-Lorenzo B, Castilla I, Valcárcel-Nazco C, García-Pérez L, Linertová R, et al. On the estimation of the cost-effectiveness threshold: why, what, how? Value Health. 2016;19:558–66.CrossRefPubMed Vallejo-Torres L, García-Lorenzo B, Castilla I, Valcárcel-Nazco C, García-Pérez L, Linertová R, et al. On the estimation of the cost-effectiveness threshold: why, what, how? Value Health. 2016;19:558–66.CrossRefPubMed
36.
go back to reference Te Ao B, Tobias M, Ameratunga S, McPherson K, Theadom A, Dowell A, et al. Burden of traumatic brain injury in New Zealand: incidence, prevalence and disability-adjusted life years. Neuroepidemiology. 2015;44(4):255–61.CrossRefPubMed Te Ao B, Tobias M, Ameratunga S, McPherson K, Theadom A, Dowell A, et al. Burden of traumatic brain injury in New Zealand: incidence, prevalence and disability-adjusted life years. Neuroepidemiology. 2015;44(4):255–61.CrossRefPubMed
Metadata
Title
Economic evaluation of the NET intervention versus guideline dissemination for management of mild head injury in hospital emergency departments
Authors
Duncan Mortimer
Marije Bosch
Joanne E. Mckenzie
Simon Turner
Marisa Chau
Jennie L. Ponsford
Jonathan C. Knott
Russell L. Gruen
Sally E. Green
Publication date
01-12-2018
Publisher
BioMed Central
Published in
Implementation Science / Issue 1/2018
Electronic ISSN: 1748-5908
DOI
https://doi.org/10.1186/s13012-018-0834-6

Other articles of this Issue 1/2018

Implementation Science 1/2018 Go to the issue