Skip to main content
Top
Published in: BMC Psychiatry 1/2018

Open Access 01-12-2018 | Research article

Accuracy of risk scales for predicting repeat self-harm and suicide: a multicentre, population-level cohort study using routine clinical data

Authors: Sarah Steeg, Leah Quinlivan, Rebecca Nowland, Robert Carroll, Deborah Casey, Caroline Clements, Jayne Cooper, Linda Davies, Duleeka Knipe, Jennifer Ness, Rory C. O’Connor, Keith Hawton, David Gunnell, Nav Kapur

Published in: BMC Psychiatry | Issue 1/2018

Login to get access

Abstract

Background

Risk scales are used widely in the management of patients presenting to hospital following self-harm. However, there is evidence that their diagnostic accuracy in predicting repeat self-harm is limited. Their predictive accuracy in population settings, and in identifying those at highest risk of suicide is not known.

Method

We compared the predictive accuracy of the Manchester Self-Harm Rule (MSHR), ReACT Self-Harm Rule (ReACT), SAD PERSONS Scale (SPS) and Modified SAD PERSONS Scale (MSPS) in an unselected sample of patients attending hospital following self-harm. Data on 4000 episodes of self-harm presenting to Emergency Departments (ED) between 2010 and 2012 were obtained from four established monitoring systems in England. Episodes were assigned a risk category for each scale and followed up for 6 months.

Results

The episode-based repeat rate was 28% (1133/4000) and the incidence of suicide was 0.5% (18/3962). The MSHR and ReACT performed with high sensitivity (98% and 94% respectively) and low specificity (15% and 23%). The SPS and the MSPS performed with relatively low sensitivity (24–29% and 9–12% respectively) and high specificity (76–77% and 90%). The area under the curve was 71% for both MSHR and ReACT, 51% for SPS and 49% for MSPS. Differences in predictive accuracy by subgroup were small. The scales were less accurate at predicting suicide than repeat self-harm.

Conclusions

The scales failed to accurately predict repeat self-harm and suicide. The findings support existing clinical guidance not to use risk classification scales alone to determine treatment or predict future risk.
Appendix
Available only for authorised users
Literature
1.
go back to reference Hawton K, Bergen H, Casey D, Simkin S, Palmer B, Cooper J, Kapur N, Horrocks J, House A, Lilley R, et al. Self-harm in England: a tale of three cities - multicentre study of self-harm. Soc Psychiatry Psychiatr Epidemiol. 2007;42(7):513–21.CrossRefPubMed Hawton K, Bergen H, Casey D, Simkin S, Palmer B, Cooper J, Kapur N, Horrocks J, House A, Lilley R, et al. Self-harm in England: a tale of three cities - multicentre study of self-harm. Soc Psychiatry Psychiatr Epidemiol. 2007;42(7):513–21.CrossRefPubMed
2.
go back to reference Carroll R, Metcalfe C, Gunnell D. Hospital presenting self-harm and risk of fatal and non-fatal repetition: systematic review and meta-analysis. PLoS One. 2014;9(2):e89944.CrossRefPubMedPubMedCentral Carroll R, Metcalfe C, Gunnell D. Hospital presenting self-harm and risk of fatal and non-fatal repetition: systematic review and meta-analysis. PLoS One. 2014;9(2):e89944.CrossRefPubMedPubMedCentral
3.
go back to reference Bergen H, Hawton K, Waters K, Ness J, Cooper J, Steeg S, Kapur N. Premature death after self-harm: a multicentre cohort study. Lancet. 2012;380(9853):1568–74.CrossRefPubMed Bergen H, Hawton K, Waters K, Ness J, Cooper J, Steeg S, Kapur N. Premature death after self-harm: a multicentre cohort study. Lancet. 2012;380(9853):1568–74.CrossRefPubMed
4.
go back to reference Ness J, Hawton K, Bergen H, Cooper J, Steeg S, Kapur N, Clarke M, Waters K. Alcohol use and misuse, self-harm and subsequent mortality: an epidemiological and longitudinal study from the multicentre study of self-harm in England. Emerg Med J. 2015;32(10):793–9.CrossRefPubMed Ness J, Hawton K, Bergen H, Cooper J, Steeg S, Kapur N, Clarke M, Waters K. Alcohol use and misuse, self-harm and subsequent mortality: an epidemiological and longitudinal study from the multicentre study of self-harm in England. Emerg Med J. 2015;32(10):793–9.CrossRefPubMed
5.
go back to reference NICE. The long term care and treatment of self-harm. Clinical guideline 133. London: National Institute of Health and Care Excellence; 2011. NICE. The long term care and treatment of self-harm. Clinical guideline 133. London: National Institute of Health and Care Excellence; 2011.
6.
go back to reference Kapur N, Steeg S, Webb R, Haigh M, Bergen H, Hawton K, Ness J, Waters K, Cooper J. Does clinical management improve outcomes following self-harm? Results from the multicentre study of self-harm in England. PLoS One. 2013;8(8):e70434.CrossRefPubMedPubMedCentral Kapur N, Steeg S, Webb R, Haigh M, Bergen H, Hawton K, Ness J, Waters K, Cooper J. Does clinical management improve outcomes following self-harm? Results from the multicentre study of self-harm in England. PLoS One. 2013;8(8):e70434.CrossRefPubMedPubMedCentral
7.
go back to reference Steeg S, Emsley R, Carr M, Cooper J, Kapur N. Routine hospital management of self-harm and risk of further self-harm: propensity score analysis using record-based cohort data. Psychol Med. 2017;48(2):1–12. Steeg S, Emsley R, Carr M, Cooper J, Kapur N. Routine hospital management of self-harm and risk of further self-harm: propensity score analysis using record-based cohort data. Psychol Med. 2017;48(2):1–12.
8.
go back to reference Quinlivan L, Cooper J, Steeg S, Davies L, Hawton K, Gunnell D, Kapur N. Scales for predicting risk following self-harm: an observational study in 32 hospitals in England. BMJ Open. 2014;4(5):e004732.CrossRefPubMedPubMedCentral Quinlivan L, Cooper J, Steeg S, Davies L, Hawton K, Gunnell D, Kapur N. Scales for predicting risk following self-harm: an observational study in 32 hospitals in England. BMJ Open. 2014;4(5):e004732.CrossRefPubMedPubMedCentral
9.
go back to reference Quinlivan L, Cooper J, Davies L, Hawton K, Gunnell D, Kapur N. Which are the most useful scales for predicting repeat self-harm? A systematic review evaluating risk scales using measures of diagnostic accuracy. BMJ Open. 2016;6(2):e009297.CrossRefPubMedPubMedCentral Quinlivan L, Cooper J, Davies L, Hawton K, Gunnell D, Kapur N. Which are the most useful scales for predicting repeat self-harm? A systematic review evaluating risk scales using measures of diagnostic accuracy. BMJ Open. 2016;6(2):e009297.CrossRefPubMedPubMedCentral
10.
go back to reference Carter G, Milner A, McGill K, Pirkis J, Kapur N, Spittal MJ. Predicting suicidal behaviours using clinical instruments: systematic review and meta-analysis of positive predictive values for risk scales. Br J Psychiatry. 2017;210(6):387-+.CrossRefPubMed Carter G, Milner A, McGill K, Pirkis J, Kapur N, Spittal MJ. Predicting suicidal behaviours using clinical instruments: systematic review and meta-analysis of positive predictive values for risk scales. Br J Psychiatry. 2017;210(6):387-+.CrossRefPubMed
11.
go back to reference Katz C, Randall JR, Sareen J, Chateau D, Walld R, Leslie WD, Wang J, Bolton JM. Predicting suicide with the SAD PERSONS scale. Depress Anxiety. 2017;34(9):809–16.CrossRefPubMed Katz C, Randall JR, Sareen J, Chateau D, Walld R, Leslie WD, Wang J, Bolton JM. Predicting suicide with the SAD PERSONS scale. Depress Anxiety. 2017;34(9):809–16.CrossRefPubMed
12.
go back to reference Quinlivan L, Cooper J, Meehan D, Longson D, Potokar J, Hulme T, Marsden J, Brand F, Lange K, Riseborough E, et al. Predictive accuracy of risk scales following self-harm: multicentre, prospective cohort study. Br J Psychiatry. 2017;210(6):429-+.CrossRefPubMedPubMedCentral Quinlivan L, Cooper J, Meehan D, Longson D, Potokar J, Hulme T, Marsden J, Brand F, Lange K, Riseborough E, et al. Predictive accuracy of risk scales following self-harm: multicentre, prospective cohort study. Br J Psychiatry. 2017;210(6):429-+.CrossRefPubMedPubMedCentral
13.
go back to reference Cooper J, Steeg S, Bennewith O, Lowe M, Gunnell D, House A, Hawton K, Kapur N. Are hospital services for self-harm getting better? An observational study examining management, service provision and temporal trends in England. BMJ Open. 2013;3(11):e003444.CrossRefPubMedPubMedCentral Cooper J, Steeg S, Bennewith O, Lowe M, Gunnell D, House A, Hawton K, Kapur N. Are hospital services for self-harm getting better? An observational study examining management, service provision and temporal trends in England. BMJ Open. 2013;3(11):e003444.CrossRefPubMedPubMedCentral
14.
go back to reference Cooper J, Kapur N, Dunning J, Guthrie E, Appleby L, Mackway-Jones K. A clinical tool for assessing risk after self-harm. Ann Emerg Med. 2006;48(4):459–66.CrossRefPubMed Cooper J, Kapur N, Dunning J, Guthrie E, Appleby L, Mackway-Jones K. A clinical tool for assessing risk after self-harm. Ann Emerg Med. 2006;48(4):459–66.CrossRefPubMed
15.
go back to reference Steeg S, Kapur N, Webb R, Applegate E, Stewart SLK, Hawton K, Bergen H, Waters K, Cooper J. The development of a population-level clinical screening tool for self-harm repetition and suicide: the ReACT Self-Harm Rule. Psychol Med. 2012;42(11):2383–94.CrossRefPubMed Steeg S, Kapur N, Webb R, Applegate E, Stewart SLK, Hawton K, Bergen H, Waters K, Cooper J. The development of a population-level clinical screening tool for self-harm repetition and suicide: the ReACT Self-Harm Rule. Psychol Med. 2012;42(11):2383–94.CrossRefPubMed
16.
go back to reference Patterson WM, Dohn HH, Bird J, Patterson GA. Evaluation of suicidal patients - the sad persons scale. Psychosomatics. 1983;24(4):343-&.CrossRef Patterson WM, Dohn HH, Bird J, Patterson GA. Evaluation of suicidal patients - the sad persons scale. Psychosomatics. 1983;24(4):343-&.CrossRef
17.
go back to reference Bolton J, Rae Spiwak R, Sareen J. Predicting suicide attempts with the sad persons scale: a longitudinal analysis. J Clin Psychiatry. 2012;73(6):e735–41.CrossRefPubMed Bolton J, Rae Spiwak R, Sareen J. Predicting suicide attempts with the sad persons scale: a longitudinal analysis. J Clin Psychiatry. 2012;73(6):e735–41.CrossRefPubMed
18.
go back to reference Beck RW, Morris JB, Beck AT. Cross-validation of suicidal-intent-scale. Psychol Rep. 1974;34(2):445–6.CrossRefPubMed Beck RW, Morris JB, Beck AT. Cross-validation of suicidal-intent-scale. Psychol Rep. 1974;34(2):445–6.CrossRefPubMed
19.
go back to reference Carroll R, Metcalfe C, Gunnell D. Hospital management of self-harm patients and risk of repetition: systematic review and meta-analysis. J Affect Disord. 2014;168:476–83.CrossRefPubMed Carroll R, Metcalfe C, Gunnell D. Hospital management of self-harm patients and risk of repetition: systematic review and meta-analysis. J Affect Disord. 2014;168:476–83.CrossRefPubMed
20.
go back to reference Kapur N, Cooper J, King-Hele S, Webb R, Lawlor M, Rodway C, Appleby L. The repetition of suicidal behavior: a multicenter cohort study. J Clin Psychiatry. 2006;67(10):1599–609.CrossRefPubMed Kapur N, Cooper J, King-Hele S, Webb R, Lawlor M, Rodway C, Appleby L. The repetition of suicidal behavior: a multicenter cohort study. J Clin Psychiatry. 2006;67(10):1599–609.CrossRefPubMed
22.
go back to reference Royston P, White IR. Multiple Imputation by Chained Equations (MICE): implementation in Stata. J Stat Softw. 2011;45(4):1–20.CrossRef Royston P, White IR. Multiple Imputation by Chained Equations (MICE): implementation in Stata. J Stat Softw. 2011;45(4):1–20.CrossRef
23.
go back to reference Rubin DB. Multiple imputation for non-response in surveys. New York: John Wiley & Sons; 1987.CrossRef Rubin DB. Multiple imputation for non-response in surveys. New York: John Wiley & Sons; 1987.CrossRef
24.
go back to reference Altman DG, Bland JM. Diagnostic-tests-3 - receiver operating characteristic plots .7. Br Med J. 1994;309(6948):188.CrossRef Altman DG, Bland JM. Diagnostic-tests-3 - receiver operating characteristic plots .7. Br Med J. 1994;309(6948):188.CrossRef
25.
go back to reference Kapur N, Murphy E, Cooper J, Bergen H, Hawton K, Simkin S, Casey D, Horrocks J, Lilley R, Noble R, et al. Psychosocial assessment following self-harm: results from the multi-centre monitoring of self-harm project. J Affect Disord. 2008;106(3):285–93.CrossRefPubMed Kapur N, Murphy E, Cooper J, Bergen H, Hawton K, Simkin S, Casey D, Horrocks J, Lilley R, Noble R, et al. Psychosocial assessment following self-harm: results from the multi-centre monitoring of self-harm project. J Affect Disord. 2008;106(3):285–93.CrossRefPubMed
26.
go back to reference Hawes M, Yaseen Z, Briggs J, Galynker I. The Modular Assessment of Risk for Imminent Suicide (MARIS): a proof of concept for a multi-informant tool for evaluation of short-term. Compr Psychiatry. 2017;72:88–96.CrossRefPubMed Hawes M, Yaseen Z, Briggs J, Galynker I. The Modular Assessment of Risk for Imminent Suicide (MARIS): a proof of concept for a multi-informant tool for evaluation of short-term. Compr Psychiatry. 2017;72:88–96.CrossRefPubMed
27.
go back to reference Walsh CG, Ribeiro JD, Franklin JC. Predicting risk of suicide attempts over time through machine learning. Clin Psychol Sci. 2017;5(3):457–69.CrossRef Walsh CG, Ribeiro JD, Franklin JC. Predicting risk of suicide attempts over time through machine learning. Clin Psychol Sci. 2017;5(3):457–69.CrossRef
29.
go back to reference Hancock M, Kent P. Interpretation of dichotomous outcomes: risk, odds, risk ratios, odds ratios and number needed to treat. J Physiother. 2016;62(3):172–4.CrossRefPubMed Hancock M, Kent P. Interpretation of dichotomous outcomes: risk, odds, risk ratios, odds ratios and number needed to treat. J Physiother. 2016;62(3):172–4.CrossRefPubMed
30.
go back to reference Runeson B, Odeberg J, Pettersson A, Edbom T, Adamsson IJ, Waern M. Instruments for the assessment of suicide risk: a systematic review evaluating the certainty of the evidence. PLoS One. 2017;12(7):e0180292.CrossRefPubMedPubMedCentral Runeson B, Odeberg J, Pettersson A, Edbom T, Adamsson IJ, Waern M. Instruments for the assessment of suicide risk: a systematic review evaluating the certainty of the evidence. PLoS One. 2017;12(7):e0180292.CrossRefPubMedPubMedCentral
31.
32.
go back to reference Hunter C, Chantler K, Kapur N, Cooper J. Service user perspectives on psychosocial assessment following self-ham and its impact on further help-seeking: a qualitative study. J Affect Disord. 2013;145(3):315–23.CrossRefPubMed Hunter C, Chantler K, Kapur N, Cooper J. Service user perspectives on psychosocial assessment following self-ham and its impact on further help-seeking: a qualitative study. J Affect Disord. 2013;145(3):315–23.CrossRefPubMed
33.
go back to reference Miller IW, Camargo CA Jr, Arias SA, Sullivan AF, Allen MH, Goldstein AB, Manton AP, Espinola JA, Jones R, Hasegawa K, et al. Suicide prevention in an emergency department population the ED-SAFE study. Jama Psychiatry. 2017;74(6):563–70.CrossRefPubMedPubMedCentral Miller IW, Camargo CA Jr, Arias SA, Sullivan AF, Allen MH, Goldstein AB, Manton AP, Espinola JA, Jones R, Hasegawa K, et al. Suicide prevention in an emergency department population the ED-SAFE study. Jama Psychiatry. 2017;74(6):563–70.CrossRefPubMedPubMedCentral
Metadata
Title
Accuracy of risk scales for predicting repeat self-harm and suicide: a multicentre, population-level cohort study using routine clinical data
Authors
Sarah Steeg
Leah Quinlivan
Rebecca Nowland
Robert Carroll
Deborah Casey
Caroline Clements
Jayne Cooper
Linda Davies
Duleeka Knipe
Jennifer Ness
Rory C. O’Connor
Keith Hawton
David Gunnell
Nav Kapur
Publication date
01-12-2018
Publisher
BioMed Central
Published in
BMC Psychiatry / Issue 1/2018
Electronic ISSN: 1471-244X
DOI
https://doi.org/10.1186/s12888-018-1693-z

Other articles of this Issue 1/2018

BMC Psychiatry 1/2018 Go to the issue