Top

BMC Medical Informatics and Decision Making

Published in:

Open Access 01-12-2018 | Research article

Detecting change in comparison to peers in NHS prescribing data: a novel application of cumulative sum methodology

Authors: Alex J. Walker, Seb Bacon, Richard Croker, Ben Goldacre

Published in: BMC Medical Informatics and Decision Making | Issue 1/2018

Abstract

Background

The widely used OpenPrescribing.net service provides standard measures which compare prescribing of Clinical Commissioning Groups (CCGs) and English General Practices against that of their peers. Detecting changes in prescribing behaviour compared with peers can help identify missed opportunities for medicines optimisation. Automating the process of detecting these changes is necessary due to the volume of data, but challenging due to variation in prescribing volume for different measures and locations. We set out to develop and implement a method of detecting change on all individual prescribing measures, in order to notify CCGs and practices of such changes in a timely manner.

Methods

We used the statistical process control method CUSUM to detect prescribing behaviour changes in relation to population trends for the individual standard measures on OpenPrescribing. Increases and decreases in percentile were detected separately, using a multiple of standard deviation as the threshold for detecting change. The algorithm was modified to continue re-triggering when trajectory persists. It was deployed, user-tested, and summary statistics generated on the number of alerts by CCG and practice.

Results

The algorithm detected changes in prescribing for 32 prespecified measures, across a wide range of CCG and practice sizes. Across the 209 English CCGs, a mean of 2.5 increase and 2.4 decrease alerts were triggered per CCG, per month. For the 7578 practices, a mean of 1.3 increase and 1.4 decrease alerts were triggered per practice, per month.

Conclusions

The CUSUM method appears to effectively discriminate between random noise and sustained change in prescribing behaviour. This method aims to allow practices and CCGs to be informed of important changes quickly, with a view to improve their prescribing behaviour. The number of alerts triggered for CCGs and practices appears to be appropriate. Prescribing behaviour after users are alerted to changes will be monitored in order to assess the impact of these alerts.

Available only for authorised users

Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, O'Brien MA, Johansen M, Grimshaw J, Oxman AD. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;(6):CD000259.

Dreischulte T, Donnan P, Grant A, Hapca A, McCowan C, Guthrie B. Safer Prescribing--A Trial of Education, Informatics, and Financial Incentives. N. Engl. J. Med. 2016;374:1053–64.CrossRefPubMed

Guthrie B, Kavanagh K, Robertson C, Barnett K, Treweek S, Petrie D, et al. Data feedback and behavioural change intervention to improve primary care prescribing safety (EFIPPS): multicentre, three arm, cluster randomised controlled trial. BMJ. 2016;354:i4079.CrossRefPubMedPubMedCentral

Elouafkaoui P, Young L, Newlands R, Duncan EM, Elders A, Clarkson JE, et al. An Audit and Feedback Intervention for Reducing Antibiotic Prescribing in General Dental Practice: The RAPiD Cluster Randomised Controlled Trial. PLoS Med. 2016;13:e1002115.CrossRefPubMedPubMedCentral

OpenPrescribing –homepage [Internet]. [cited 2017 Mar 13]. Available from: https://openprescribing.net/

NHS Digital prescribing data [Internet]. [cited 2017 Mar 14]. Available from: http://content.digital.nhs.uk/gpprescribingdata

Shewhart WA.. Economic Control of Quality Of Manufactured Product. Martino Fine Books; 1931 (Reprint 2015).

Thor J, Lundberg J, Ask J, Olsson J, Carli C, Härenstam KP, et al. Application of statistical process control in healthcare improvement: systematic review. Qual. Saf. Health Care. 2007;16:387–99.CrossRefPubMedPubMedCentral

Mohammed MA. Using statistical process control to improve the quality of health care. Qual. Saf. Health Care. 2004;13:243–5. BMJ Publishing Group LtdCrossRefPubMedPubMedCentral

10.

Perla RJ, Provost LP, Murray SK. The run chart: a simple analytical tool for learning from variation in healthcare processes. BMJ Qual. Saf. 2011;20:46–51.CrossRefPubMed

11.

Fretheim A, Tomic O. Statistical process control and interrupted time series: a golden opportunity for impact evaluation in quality improvement. BMJ Qual Saf. 2015;24:748–52.

12.

NHS Business Service Authority Information Portal [Internet]. [cited 2017 Mar 14]. Available from: https://apps.nhsbsa.nhs.uk/infosystems/welcome

13.

Openprescribing –Direct Oral Anticoagulants (DOACs) prescribing [Internet]. [cited 2017 Mar 13]. Available from: https://openprescribing.net/measure/doacs/

14.

OpenPrescribing –Prescribing of pregabalin [Internet]. [cited 2017 Mar 13]. Available from: https://openprescribing.net/measure/pregabalin/

15.

OpenPrescribing –Cerazette vs. desogestrel measure [Internet]. [cited 2017 Mar 13]. Available from: https://openprescribing.net/measure/desogestrel/

16.

Montgomery DC. Introduction to Statistical Quality Control. New Jersey: Wiley; 2009.

17.

OpenPrescribing API [Internet]. [cited 2017 Mar 14]. Available from: https://openprescribing.net/api/

18.

OpenPrescribing –NHS WALSALL CCG (05Y) [Internet]. [cited 2017 Mar 13]. Available from: https://openprescribing.net/ccg/05Y/

19.

OpenPrescribing –ALBION STREET GROUP PRACTICE (G85138) [Internet]. [cited 2017 Mar 13]. Available from: https://openprescribing.net/practice/G85138/

20.

Detecting change in prescribing data using CUSUM.ipynb [Internet]. [cited 13/032017]. Available from: https://gist.github.com/alexwalkercebm/7a917b4c456c718e89d8412b9ac61745

21.

Alerts from previous 12 months.xlsx [Internet]. [cited 2017 Mar 14]. Available from: https://figshare.com/s/1a3db51d3529134b3fc5

22.

Wood J. Lambert MF. CUSUM charts for monitoring clinical practice quality using primary care prescribing data: a case study of an initiative to encourage generic prescribing of proton pump inhibitors. J. Clin. Pharm. Ther. 2011;36:680–6.CrossRefPubMed

23.

Leandro G, Rolando N, Gallus G, Rolles K, Burroughs AK. Monitoring surgical and medical outcomes: the Bernoulli cumulative SUM chart. A novel application to assess clinical interventions. Postgrad. Med. J. 2005;81:647–52.CrossRefPubMedPubMedCentral

24.

Novick RJ, Fox SA, Stitt LW, Forbes TL, Steiner S. Direct comparison of risk-adjusted and non–risk-adjusted CUSUM analyses of coronary artery bypass surgery outcomes. J Thorac Cardiovasc Surg. 2006/8;132:386–91.

25.

Steiner SH, Cook RJ, Farewell VT, Treasure T. Monitoring surgical performance using risk-adjusted cumulative sum charts. Biostatistics. 2000;1:441–52.CrossRefPubMed

26.

Chen T-T, Chung K-P, Hu F-C, Fan C-M, Yang M-C. The use of statistical process control (risk-adjusted CUSUM, risk-adjusted RSPRT and CRAM with prediction limits) for monitoring the outcomes of out-of-hospital cardiac arrest patients rescued by the EMS system. J. Eval. Clin. Pract. 2011;17:71–7.CrossRefPubMed

27.

Cornélissen G, Halberg F, Hawkins D, Otsuka K, Henke W. Individual assessment of antihypertensive response by self-starting cumulative sums. J. Med. Eng. Technol. 1997;21:111–20.CrossRefPubMed

28.

Sibanda T, Sibanda N. The CUSUM chart method as a tool for continuous monitoring of clinical outcomes using routinely collected data. BMC Med. Res. Methodol. 2007;7:46.CrossRefPubMedPubMedCentral

29.

Sasikumar R, Bangusha DS. Cumulative sum charts and its healthcare applications; a systematic review. Sri Lankan J App Stats. 2014;15.

30.

Woodall WH, Adams BM. The statistical design of CUSUM charts. Qual. Eng. 1993;5:559–70.CrossRef

31.

Dexter F, Ledolter J, Hindman BJ. Bernoulli Cumulative Sum (CUSUM) control charts for monitoring of anesthesiologists’ performance in supervising anesthesia residents and nurse anesthetists. Anesth. Analg. 2014;119:679–85.CrossRefPubMed

Title: Detecting change in comparison to peers in NHS prescribing data: a novel application of cumulative sum methodology
Authors: Alex J. Walker
Seb Bacon
Richard Croker
Ben Goldacre
Publication date: 01-12-2018
Publisher: BioMed Central
Published in: BMC Medical Informatics and Decision Making / Issue 1/2018
Electronic ISSN: 1472-6947
DOI: https://doi.org/10.1186/s12911-018-0642-6

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Detecting change in comparison to peers in NHS prescribing data: a novel application of cumulative sum methodology

Abstract

Background

Methods

Results

Conclusions

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2018

Development of a personalized diagnostic model for kidney stone disease tailored to acute care by integrating large clinical, demographics and laboratory data: the diagnostic acute care algorithm - kidney stones (DACA-KS)

CogStack - experiences of deploying integrated information retrieval and extraction services in a large National Health Service Foundation Trust hospital

Information sharing across generations and environments (InfoSAGE): study design and methodology protocol

Effect of clinical decision rules, patient cost and malpractice information on clinician brain CT image ordering: a randomized controlled trial

Correction to: A pattern learning-based method for temporal expression extraction and normalization from multi-lingual heterogeneous clinical texts

Multicriteria decision analysis (MCDA) in health care: a systematic review of the main characteristics and methodological steps