Top

Published in:

01-07-2020 | Original Research Article

R and Shiny for Cost-Effectiveness Analyses: Why and When? A Hypothetical Case Study

Authors: Rose Hart, Darren Burns, Bram Ramaekers, Shijie Ren, Daniel Gladwell, Will Sullivan, Niall Davison, Owain Saunders, Indeg Sly, Theresa Cain, Dawn Lee

Published in: PharmacoEconomics | Issue 7/2020

Abstract

Introduction

Health economics models are typically built in Microsoft Excel^® owing to its wide familiarity, accessibility and perceived transparency. However, given the increasingly rapid and analytically complex decision-making needs of both the pharmaceutical industry and the field of health economics and outcomes research (HEOR), the demands of cost-effectiveness analyses may be better met by the programming language R.

Objective

This case study provides an explicit comparison between Excel and R for contemporary cost-effectiveness analysis.

Methods

We constructed duplicate cost-effectiveness models using Excel and R (with a user interface built using the Shiny package) to address a hypothetical case study typical of contemporary health technology assessment.

Results

We compared R and Excel versions of the same model design to determine the advantages and limitations of the modelling platforms in terms of (i) analytical capability, (ii) data safety, (iii) building considerations, (iv) usability for technical and non-technical users and (v) model adaptability.

Conclusions

The findings of this explicit comparison are used to produce recommendations for when R might be more suitable than Excel in contemporary cost-effectiveness analyses. We conclude that selection of appropriate modelling software needs to consider case-by-case modelling requirements, particularly (i) intended audience, (ii) complexity of analysis, (iii) nature and frequency of updates and (iv) anticipated model run time.

Available only for authorised users

Microsoft R Application Network. Introduction to R. 2019. https://mran.microsoft.com/documents/what-is-r. Accessed 6 Feb 2019.

R Core Team. R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2018. https://www.R-project.org/. Accessed 10 May 2019.

Woolacott N, Hawkins N, Mason A, Kainth A, Khadjesari Z. Etanercept and efalizumab for the treatment of psoriasis: a systematic review. Health Technol Assess. 2006;10(46):1–233.CrossRef

National Institute for Health and Care Excellence (NICE). Rituximab for the first-line treatment of stage III–IV follicular lymphoma (TA243). 2012. https://www.nice.org.uk/guidance/ta243. Accessed 11 June 2019.

Institute for Clinical and Economic Review (ICER). Extended-release opioid agonists and antagonist medications for addiction treatment (MAT) in patients with opioid use disorder: effectiveness and value. 2018. https://icer-review.org/wp-content/uploads/2018/04/ICER_OUD_Final_Evidence_Report_120318.pdf. Accessed 11 June 2019.

Baio G. R for trial and model-based cost-effectiveness analysis. 2019. http://www.statistica.it/gianluca/teaching/r-hta-workshop/. Accessed 30 Jan 2019.

Incerti D, Thom H, Baio G, Jansen JP. R you still using excel? The advantages of modern software tools for health technology assessment. Value Health. 2019;22(5):575–9. https://doi.org/10.1016/j.jval.2019.01.003.CrossRefPubMed

Krijkamp EM, Alarid-Escudero F, Enns EA, Jalal HJ, Hunink MGM, Pechlivanoglou P. Microsimulation modeling for health decision sciences using R: a tutorial. Med Decis Making. 2018;38(3):400–22. https://doi.org/10.1177/0272989X18754513.CrossRefPubMedPubMedCentral

Hollman C, Paulden M, Pechlivanoglou P, McCabe C. A comparison of four software programs for implementing decision analytic cost-effectiveness models. Pharmacoeconomics. 2017;35(8):817–30. https://doi.org/10.1007/s40273-017-0510-8.CrossRefPubMed

10.

Williams C, Lewsey JD, Briggs AH, Mackay DF. Cost-effectiveness analysis in R using a multi-state modeling survival analysis framework: a tutorial. Med Decis Making. 2017;37(4):340–52. https://doi.org/10.1177/0272989X16651869.CrossRefPubMed

11.

Jalal H, Pechlivanoglou P, Krijkamp E, Alarid-Escudero F, Enns E, Hunink MGM. An overview of R in health decision sciences. Med Decis Making. 2017;37(7):735–46. https://doi.org/10.1177/0272989X16686559.CrossRefPubMed

12.

Baio G, Heath A. When simple becomes complicated: why excel should lose its place at the top table. Glob Reg Health Technol Assess. 2017;4(1):e3–6.

13.

Sampson CJ, Arnold R, Bryan S, Clarke P, Ekins S, Hatswell A, et al. Transparency in decision modelling: what, why, who and how? Pharmacoeconomics. 2019;37(11):1355–69. https://doi.org/10.1007/s40273-019-00819-z.CrossRefPubMed

14.

Incerti D, Curtis JR, Shafrin J, Lakdawalla DN, Jansen JP. A flexible open-source decision model for value assessment of biologic treatment for rheumatoid arthritis. Pharmacoeconomics. 2019;37(6):829–43. https://doi.org/10.1007/s40273-018-00765-2.CrossRefPubMed

15.

Alarid-Escudero F, Krijkamp EM, Pechlivanoglou P, Jalal H, Kao SZ, Yang A, et al. A need for change! A coding framework for improving transparency in decision modeling. Pharmacoeconomics. 2019;37(11):1329–39. https://doi.org/10.1007/s40273-019-00837-x.CrossRefPubMed

16.

Eddy DM, Hollingworth W, Caro JJ, Tsevat J, McDonald KM, Wong JB, et al. Model transparency and validation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7. Value Health. 2012;15(6):843–50. https://doi.org/10.1016/j.jval.2012.04.012.CrossRefPubMed

17.

Tappenden P, Caro JJ. Improving transparency in decision models: current issues and potential solutions. Pharmacoeconomics. 2019;37(11):1303–4. https://doi.org/10.1007/s40273-019-00850-0.CrossRefPubMed

18.

Allaire J, Xie Y, McPherson J, Luraschi J, Ushey K, Atkins A et al. rmarkdown: dynamic documents for R. R package version 1.13. 2019. https://rmarkdown.rstudio.com. Accessed 3 Dec 2019.

19.

RStudio. Version control with git and SVN. 2019. https://support.rstudio.com/hc/en-us/articles/200532077-Version-Control-with-Git-and-SVN. Accessed 3 Dec 2019.

20.

National Health Care Institute Zorginstituut Nederland (ZIN). Template Pharmacoeconomic dossier. 2016. https://www.zorginstituutnederland.nl/binaries/zinl/documenten/publicatie/2016/07/25/template-farmaco-economisch-dossier/Template+Pharmacoeconomic+dossier.doc. Accessed 12 Nov 2019.

21.

Pharmaceutical Management Agency (PHARMAC). Prescription for pharmacoeconomic analysis—methods for cost-utility analysis. 2015. https://www.pharmac.govt.nz/assets/pfpa-2-2.pdf. Accessed 12 Nov 2019.

22.

Canadian Agency for Drugs and Technologies in Health (CADTH). Procedure and submission guidelines for the CADTH common drug review. 2019. https://cadth.ca/sites/default/files/cdr/process/Procedure_and_Guidelines_for_CADTH_CDR.pdf. Accessed 12 Nov 2019.

23.

Healtheconomics.com. ICER using new economic modeling platform for value assessment. 2018. https://www.healtheconomics.com/industry-news/icer-using-new-economic-modeling-platform-for-value-assessment. Accessed 12 Nov 2019.

24.

RStudio. Shiny. 2019. https://shiny.rstudio.com/. Accessed 2 Dec 2019.

25.

Chang W, Cheng J, Allaire J, Xie Y, McPherson J. Shiny: web application framework for R. R package version 1.3.2. 2019. https://CRAN.R-project.org/package=shiny. Accessed 2 July 2019.

26.

Innovation and value initiative. open-source value platform. 2019. https://www.thevalueinitiative.org/open-source-value-project/. Accessed 6 Feb 2019.

27.

Food and Drug Administration (FDA). RR-Drug. 2019. https://openfda.shinyapps.io/RR_D/. Accessed 12 Nov 2019.

28.

(CHOICE). TUoW-TCHOPaEI. PriMER platform: diffusion of precision medicine technologies. 2019. https://uwchoice.shinyapps.io/primer/. Accessed 12 Nov 2019.

29.

Center for Evaluation of Value and Risk in Health (CEVR). Global health cost effectiveness analysis registry. 2019. http://healtheconomics.tuftsmedicalcenter.org/ghcearegistry/. Accessed 12 Nov 2019.

30.

Eichler HG, Oye K, Baird LG, Abadie E, Brown J, Drum CL, et al. Adaptive licensing: taking the next step in the evolution of drug approval. Clin Pharmacol Ther. 2012;91(3):426–37. https://doi.org/10.1038/clpt.2011.345.CrossRefPubMed

31.

Food and Drug Administration (FDA). Accelerated Approval Program. 2018. https://www.fda.gov/drugs/resourcesforyou/healthprofessionals/ucm313768.htm. Accessed 2 July 2019.

32.

European Medicines Agency. Conditional marketing authorisation: report on ten years of experience at the european medicines agency. 2016. https://www.ema.europa.eu/en/documents/report/conditional-marketing-authorisation-report-ten-years-experience-european-medicines-agency_en.pdf. Accessed 14 May 2019.

33.

Hatswell AJ, Baio G, Berlin JA, Irs A, Freemantle N. Regulatory approval of pharmaceuticals without a randomised controlled study: analysis of EMA and FDA approvals 1999–2014. BMJ Open. 2016;6(6):e011666. https://doi.org/10.1136/bmjopen-2016-011666.CrossRefPubMedPubMedCentral

34.

European Medicines Agency. Yescarta. 2018. https://www.ema.europa.eu/en/medicines/human/EPAR/yescarta. Accessed 9 Apr 2019.

35.

European Medicines Agency. Kymriah. 2018. https://www.ema.europa.eu/en/medicines/human/EPAR/kymriah. Accessed 9 Apr 2019.

36.

Food and Drug Administration (FDA). Yescarta (axicabtagene ciloleucel). 2018. https://www.fda.gov/vaccines-blood-biologics/cellular-gene-therapy-products/yescarta-axicabtagene-ciloleucel. Accessed 7 June 2019.

37.

Food and Drug Administration (FDA). Kymriah (tisagenlecleucel). 2019. https://www.fda.gov/vaccines-blood-biologics/cellular-gene-therapy-products/kymriah-tisagenlecleucel. Accessed 6 June 2019.

38.

Hettle R, Corbett M, Hinde S, Hodgson R, Jones-Diette J, Woolacott N et al. Exploring the assessment and appraisal of regenerative medicines and cell therapy products. 2015. https://www.nice.org.uk/Media/Default/About/what-we-do/Science%20policy%20and%20research/final-york-report-march-16.pdf. Accessed 2 July 2019.

39.

Picanco-Castro V, Goncalves Pereira C, Swiech K, Ribeiro Malmegrim KC, Tadeu Covas D, Silveira Porto G. Emerging CAR T cell therapies: clinical landscape and patent technological routes. Hum Vaccin Immunother. 2019. https://doi.org/10.1080/21645515.2019.1689744.CrossRefPubMed

40.

Abreu TR, Fonseca NA, Goncalves N, Moreira JN. Current challenges and emerging opportunities of CAR-T cell therapies. J Control Release. 2020;319:246–61. https://doi.org/10.1016/j.jconrel.2019.12.047.CrossRefPubMed

41.

National Institute for Health and Care Excellence (NICE). Tisagenlecleucel for treating relapsed or refractory B-cell acute lymphoblastic leukaemia in people aged up to 25 years [TA554]. 2018. https://www.nice.org.uk/guidance/ta554/documents/final-appraisal-determination-document. Accessed 17 Apr 2019.

42.

National Institute for Health and Care Excellence (NICE). Axicabtagene ciloleucel for treating diffuse large B-cell lymphoma and primary mediastinal large B-cell lymphoma after 2 or more systemic therapies [TA559]. 2019. https://www.nice.org.uk/guidance/ta559/documents/final-appraisal-determination-document. Accessed 23 Jan 2019.

43.

National Institute for Health and Care Excellence (NICE). Tisagenlecleucel for treating relapsed or refractory diffuse large B-cell lymphoma after 2 or more systemic therapies (TA567). 2019. https://www.nice.org.uk/guidance/ta567/documents/final-appraisal-determination-document. Accessed 05 July 2019.

44.

Woods B, Sideris E, Palmer S, Latimer N, Soares M. NICE DSU Technical Support Document 19. Partitioned survival analysis for decision modelling in health care: a critical review. 2017. http://scharr.dept.shef.ac.uk/nicedsu/wp-content/uploads/sites/7/2017/06/Partitioned-Survival-Analysis-final-report.pdf. Accessed 2 May 2019.

45.

National Institute for Health and Care Excellence (NICE). Guide to the methods of technology appraisal. 2013. https://www.nice.org.uk/guidance/pmg9/resources/guide-to-the-methods-of-technology-appraisal-2013-pdf-2007975843781. Accessed 2 May 2019.

46.

Sousa D, Ferreira F, Félix F, Lopesa M. Acute lymphoblastic leukemia in children and adolescents: prognostic factors and analysis of survival. Rev Bras Hematol Hemoter. 2015;37(4):223–9.CrossRef

47.

Fuster JL. Current approach to relapsed acute lymphoblastic leukemia in children. World J Hematol. 2014;3(3):49–70. https://doi.org/10.5315/wjh.v3.i3.49.CrossRef

48.

National Institute for Health and Care Excellence. Single technology appraisal axicabtagene ciloleucel for treating diffuse large B-cell lymphoma and primary mediastinal B-cell lymphoma after 2 or more systemic therapies (TA559): Committee Papers. 2019. https://www.nice.org.uk/guidance/ta559/evidence/appraisal-consultation-committee-papers-pdf-6661404973. Accessed 11 Apr 2019.

49.

Faria R, Hernandez Alava M, Manca A, Wailoo AJ. NICE DSU Technical Support Document 17: The use of observational data to inform estimates of treatment effectiveness for Technology Appraisal: methods for comparative individual patient data. 2015. http://scharr.dept.shef.ac.uk/nicedsu/wp-content/uploads/sites/7/2016/03/TSD17-DSU-Observational-data-FINAL.pdf. Accessed 10 May 2019.

50.

Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70(1):41–55.CrossRef

51.

Stuart EA. Matching methods for causal inference: a review and a look forward. Stat Sci. 2010;25(1):1–21. https://doi.org/10.1214/09-STS313.CrossRefPubMedPubMedCentral

52.

Latimer N. NICE DSU technical support document 14: undertaking survival analysis for economic evaluations alongside clinical trials—extrapolation with patient-level data. 2011. http://nicedsu.org.uk/wp-content/uploads/2016/03/NICE-DSU-TSD-Survival-analysis.updated-March-2013.v2.pdf. Accessed 2 May 2019.

53.

Lambert PC, Thompson JR, Weston CL, Dickman PW. Estimating and modeling the cure fraction in population-based cancer survival analysis. Biostatistics. 2007;8(3):576–94. https://doi.org/10.1093/biostatistics/kxl030.CrossRefPubMed

54.

Othus M, Barlogie B, Leblanc ML, Crowley JJ. Cure models as a useful statistical tool for analyzing survival. Clin Cancer Res. 2012;18(14):3731–6. https://doi.org/10.1158/1078-0432.CCR-11-2859.CrossRefPubMedPubMedCentral

55.

Lambert PC. Modeling of the cure fraction in survival studies. Stata J. 2007;7(3):351–75.CrossRef

56.

Grant TS, Burns D, Kiff C, Lee D. A case study examining the usefulness of cure modelling for the prediction of survival based on data maturity. Pharmacoeconomics. 2020;38(4):385–95. https://doi.org/10.1007/s40273-019-00867-5.CrossRefPubMed

57.

RStudio. RStudio Connect. 2019. https://rstudio.com/products/connect/. Accessed 3 Dec 2019.

58.

Ghabri S, Stevenson M, Moller J, Caro JJ. Trusting the results of model-based economic analyses: is there a pragmatic validation solution? Pharmacoeconomics. 2019;37(1):1–6. https://doi.org/10.1007/s40273-018-0711-9.CrossRefPubMed

59.

National Institute for Health and Care Excellence (NICE). Guide to the processes of technology appraisal. 2014. https://www.nice.org.uk/process/pmg19. Accessed 11 June 2019.

60.

Canadian Agency for Drugs and Technologies in Health (CADTH). CADTH common drug review (CDR). 2019. https://www.cadth.ca/about-cadth/what-we-do/products-services/cdr. Accessed 11 June 2019.

61.

National Centre for Pharmacoeconomics (NCPE). HTA Guidelines. 2019. http://www.ncpe.ie/submission-process/hta-guidelines/. Accessed 11 June 2019.

62.

Pharmaceutical Benefits Advisory Committee (PBAC). The pharmaceutical benefits advisory committee guidlines. 2016. https://pbac.pbs.gov.au/information/printable-version-of-guidelines.html. Accessed 11 June 2019.

63.

Moller J, Davis S, Stevenson M, Caro JJ. Validation of a DICE simulation against a discrete event simulation implemented entirely in code. Pharmacoeconomics. 2017;35(10):1103–9. https://doi.org/10.1007/s40273-017-0534-0.CrossRefPubMed

64.

Philips Z, Ginnelly L, Sculpher M, Claxton K, Golder S, Riemsma R, et al. Review of guidelines for good practice in decision-analytic modelling in health technology assessment. Health Technol Assess. 2004;8(36):1–158. https://doi.org/10.3310/hta8360.CrossRefPubMed

Title: R and Shiny for Cost-Effectiveness Analyses: Why and When? A Hypothetical Case Study
Authors: Rose Hart
Darren Burns
Bram Ramaekers
Shijie Ren
Daniel Gladwell
Will Sullivan
Niall Davison
Owain Saunders
Indeg Sly
Theresa Cain
Dawn Lee
Publication date: 01-07-2020
Publisher: Springer International Publishing
Published in: PharmacoEconomics / Issue 7/2020
Print ISSN: 1170-7690
Electronic ISSN: 1179-2027
DOI: https://doi.org/10.1007/s40273-020-00903-9

Keynote webinar | Spotlight on medication adherence

Springer Medicine

R and Shiny for Cost-Effectiveness Analyses: Why and When? A Hypothetical Case Study

Abstract

Introduction

Objective

Methods

Results

Conclusions

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Introduction

Objective

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 7/2020

The Accuracy and Usefulness of the National Comprehensive Cancer Network Evidence Blocks Affordability Rating

Health Economic Models for Metastatic Colorectal Cancer: A Methodological Review

Economic Costs of Myasthenia Gravis: A Systematic Review

International Valuation Protocol for the EQ-5D-Y-3L

Conducting Value for Money Analyses for Non-randomised Interventional Studies Including Service Evaluations: An Educational Review with Recommendations

Cost-Effectiveness Analysis of Patiromer in Combination with Renin–Angiotensin–Aldosterone System Inhibitors for Chronic Kidney Disease in Sweden