Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2012

Open Access 01-12-2012 | Research article

Comparing multiple competing interventions in the absence of randomized trials using clinical risk-benefit analysis

Authors: Alejandro Lazo-Langner, Marc A Rodger, Nicholas J Barrowman, Tim Ramsay, Philip S Wells, Douglas A Coyle

Published in: BMC Medical Research Methodology | Issue 1/2012

Login to get access

Abstract

Background

To demonstrate the use of risk-benefit analysis for comparing multiple competing interventions in the absence of randomized trials, we applied this approach to the evaluation of five anticoagulants to prevent thrombosis in patients undergoing orthopedic surgery.

Methods

Using a cost-effectiveness approach from a clinical perspective (i.e. risk benefit analysis) we compared thromboprophylaxis with warfarin, low molecular weight heparin, unfractionated heparin, fondaparinux or ximelagatran in patients undergoing major orthopedic surgery, with sub-analyses according to surgery type. Proportions and variances of events defining risk (major bleeding) and benefit (thrombosis averted) were obtained through a meta-analysis and used to define beta distributions. Monte Carlo simulations were conducted and used to calculate incremental risks, benefits, and risk-benefit ratios. Finally, net clinical benefit was calculated for all replications across a range of risk-benefit acceptability thresholds, with a reference range obtained by estimating the case fatality rate - ratio of thrombosis to bleeding.

Results

The analysis showed that compared to placebo ximelagatran was superior to other options but final results were influenced by type of surgery, since ximelagatran was superior in total knee replacement but not in total hip replacement.

Conclusions

Using simulation and economic techniques we demonstrate a method that allows comparing multiple competing interventions in the absence of randomized trials with multiple arms by determining the option with the best risk-benefit profile. It can be helpful in clinical decision making since it incorporates risk, benefit, and personal risk acceptance.
Appendix
Available only for authorised users
Literature
1.
go back to reference Eddy DM: Clinical decision making: from theory to practice. Anatomy of a decision. JAMA. 1990, 263: 441-443. 10.1001/jama.1990.03440030128037.CrossRefPubMed Eddy DM: Clinical decision making: from theory to practice. Anatomy of a decision. JAMA. 1990, 263: 441-443. 10.1001/jama.1990.03440030128037.CrossRefPubMed
2.
go back to reference Lynd LD, O'Brien BJ: Advances in risk-benefit evaluation using probabilistic simulation methods: an application to the prophylaxis of deep vein thrombosis. J Clin Epidemiol. 2004, 57: 795-803. 10.1016/j.jclinepi.2003.12.012.CrossRefPubMed Lynd LD, O'Brien BJ: Advances in risk-benefit evaluation using probabilistic simulation methods: an application to the prophylaxis of deep vein thrombosis. J Clin Epidemiol. 2004, 57: 795-803. 10.1016/j.jclinepi.2003.12.012.CrossRefPubMed
3.
go back to reference Holden WL, Juhaeri J, Dai W: Benefit-risk analysis: a proposal using quantitative methods. Pharmacoepidemiol Drug Saf. 2003, 12: 611-616. 10.1002/pds.887.CrossRefPubMed Holden WL, Juhaeri J, Dai W: Benefit-risk analysis: a proposal using quantitative methods. Pharmacoepidemiol Drug Saf. 2003, 12: 611-616. 10.1002/pds.887.CrossRefPubMed
4.
go back to reference Bender R: Calculating confidence intervals for the number needed to treat. Control Clin Trials. 2001, 22: 102-110. 10.1016/S0197-2456(00)00134-3.CrossRefPubMed Bender R: Calculating confidence intervals for the number needed to treat. Control Clin Trials. 2001, 22: 102-110. 10.1016/S0197-2456(00)00134-3.CrossRefPubMed
5.
go back to reference Lesaffre E, Pledger G: A note on the number needed to treat. Control Clin Trials. 1999, 20: 439-447. 10.1016/S0197-2456(99)00018-5.CrossRefPubMed Lesaffre E, Pledger G: A note on the number needed to treat. Control Clin Trials. 1999, 20: 439-447. 10.1016/S0197-2456(99)00018-5.CrossRefPubMed
6.
go back to reference Holden WL, Juhaeri J, Dai W: Benefit-risk analysis: examples using quantitative methods. Pharmacoepidemiol Drug Saf. 2003, 12: 693-697. 10.1002/pds.794.CrossRefPubMed Holden WL, Juhaeri J, Dai W: Benefit-risk analysis: examples using quantitative methods. Pharmacoepidemiol Drug Saf. 2003, 12: 693-697. 10.1002/pds.794.CrossRefPubMed
7.
go back to reference Shakespeare TP, Gebski VJ, Veness MJ, Simes J: Improving interpretation of clinical studies by use of confidence levels, clinical significance curves, and risk-benefit contours. Lancet. 2001, 357: 1349-1353. 10.1016/S0140-6736(00)04522-0.CrossRefPubMed Shakespeare TP, Gebski VJ, Veness MJ, Simes J: Improving interpretation of clinical studies by use of confidence levels, clinical significance curves, and risk-benefit contours. Lancet. 2001, 357: 1349-1353. 10.1016/S0140-6736(00)04522-0.CrossRefPubMed
8.
go back to reference Willan AR, O'Brien BJ, Cook DJ: Benefit-risk ratios in the assessment of the clinical evidence of a new therapy. Control Clin Trials. 1997, 18: 121-130. 10.1016/S0197-2456(96)00092-X.CrossRefPubMed Willan AR, O'Brien BJ, Cook DJ: Benefit-risk ratios in the assessment of the clinical evidence of a new therapy. Control Clin Trials. 1997, 18: 121-130. 10.1016/S0197-2456(96)00092-X.CrossRefPubMed
9.
go back to reference Stinnett AA, Mullahy J: Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Med Decis Making. 1998, 18: S68-S80. 10.1177/0272989X9801800209.CrossRefPubMed Stinnett AA, Mullahy J: Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Med Decis Making. 1998, 18: S68-S80. 10.1177/0272989X9801800209.CrossRefPubMed
10.
go back to reference Briggs AH, Mooney CZ, Wonderling DE: Constructing confidence intervals for cost-effectiveness ratios: an evaluation of parametric and non-parametric techniques using Monte Carlo simulation. Stat Med. 1999, 18: 3245-3262. 10.1002/(SICI)1097-0258(19991215)18:23<3245::AID-SIM314>3.0.CO;2-2.CrossRefPubMed Briggs AH, Mooney CZ, Wonderling DE: Constructing confidence intervals for cost-effectiveness ratios: an evaluation of parametric and non-parametric techniques using Monte Carlo simulation. Stat Med. 1999, 18: 3245-3262. 10.1002/(SICI)1097-0258(19991215)18:23<3245::AID-SIM314>3.0.CO;2-2.CrossRefPubMed
11.
go back to reference Shaffer M, Watterberg K: Joint distribution approaches to simultaneously quantifying benefit and risk. BMC Med Res Methodol. 2006, 6: 48-10.1186/1471-2288-6-48.CrossRefPubMedPubMedCentral Shaffer M, Watterberg K: Joint distribution approaches to simultaneously quantifying benefit and risk. BMC Med Res Methodol. 2006, 6: 48-10.1186/1471-2288-6-48.CrossRefPubMedPubMedCentral
12.
go back to reference Bucher HC, Guyatt GH, Griffith LE, Walter SD: The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. J Clin Epidemiol. 1997, 50: 683-691. 10.1016/S0895-4356(97)00049-8.CrossRefPubMed Bucher HC, Guyatt GH, Griffith LE, Walter SD: The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. J Clin Epidemiol. 1997, 50: 683-691. 10.1016/S0895-4356(97)00049-8.CrossRefPubMed
13.
go back to reference Song F, Altman DG, Glenny AM, Deeks JJ: Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses. Br Med J. 2003, 326: 472-10.1136/bmj.326.7387.472.CrossRef Song F, Altman DG, Glenny AM, Deeks JJ: Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses. Br Med J. 2003, 326: 472-10.1136/bmj.326.7387.472.CrossRef
14.
go back to reference Brennan A, Akehurst R: Modelling in health economic evaluation. What is its place? What is its value?. Pharmacoeconomics. 2000, 17: 445-459. 10.2165/00019053-200017050-00004.CrossRefPubMed Brennan A, Akehurst R: Modelling in health economic evaluation. What is its place? What is its value?. Pharmacoeconomics. 2000, 17: 445-459. 10.2165/00019053-200017050-00004.CrossRefPubMed
15.
go back to reference Doubilet P, Begg CB, Weinstein MC, Braun P, McNeil BJ: Probabilistic sensitivity analysis using Monte Carlo simulation. A practical approach. Med Decis Making. 1985, 5: 157-177. 10.1177/0272989X8500500205.CrossRefPubMed Doubilet P, Begg CB, Weinstein MC, Braun P, McNeil BJ: Probabilistic sensitivity analysis using Monte Carlo simulation. A practical approach. Med Decis Making. 1985, 5: 157-177. 10.1177/0272989X8500500205.CrossRefPubMed
16.
go back to reference Drummond MF, O'Brien B, Stoddart GL, Torrance GW: Collection and analysis of data. Methods for the economic evaluation of health programmes. Edited by: Drummond MF, O'Brien B, Stoddart GL, Torrance GW. 1997, Oxford UK: Oxford University Press, 232-264. Drummond MF, O'Brien B, Stoddart GL, Torrance GW: Collection and analysis of data. Methods for the economic evaluation of health programmes. Edited by: Drummond MF, O'Brien B, Stoddart GL, Torrance GW. 1997, Oxford UK: Oxford University Press, 232-264.
17.
go back to reference Drummond MF, Sculpher MJ, Torrance GW, O'Brien BJ, Stoddart GL: Methods for the economic evaluation of health care programmes. 2005, Oxford, UK: Oxford University Press, 3 Drummond MF, Sculpher MJ, Torrance GW, O'Brien BJ, Stoddart GL: Methods for the economic evaluation of health care programmes. 2005, Oxford, UK: Oxford University Press, 3
18.
go back to reference Black WC: The CE plane: a graphic representation of cost-effectiveness. Med Decis Making. 1990, 10: 212-214. 10.1177/0272989X9001000308.CrossRefPubMed Black WC: The CE plane: a graphic representation of cost-effectiveness. Med Decis Making. 1990, 10: 212-214. 10.1177/0272989X9001000308.CrossRefPubMed
19.
go back to reference Briggs AH: Handling uncertainty in cost-effectiveness models. Pharmacoeconomics. 2000, 17: 479-500. 10.2165/00019053-200017050-00006.CrossRefPubMed Briggs AH: Handling uncertainty in cost-effectiveness models. Pharmacoeconomics. 2000, 17: 479-500. 10.2165/00019053-200017050-00006.CrossRefPubMed
20.
go back to reference Geerts WH, Bergqvist D, Pineo GF, Heit JA, Samama CM, Lassen MR, et al: Prevention of Venous Thromboembolism: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines (8th Edition). Chest. 2008, 133: 381S-453. 10.1378/chest.08-0656.CrossRefPubMed Geerts WH, Bergqvist D, Pineo GF, Heit JA, Samama CM, Lassen MR, et al: Prevention of Venous Thromboembolism: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines (8th Edition). Chest. 2008, 133: 381S-453. 10.1378/chest.08-0656.CrossRefPubMed
21.
go back to reference Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF: Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses. Lancet. 1999, 354: 1896-1900. 10.1016/S0140-6736(99)04149-5.CrossRefPubMed Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF: Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses. Lancet. 1999, 354: 1896-1900. 10.1016/S0140-6736(99)04149-5.CrossRefPubMed
22.
go back to reference Lazo-Langner A, Coyle D, Barrowman NJ, Ramsay T, Wells PS, Scarvelis D, et al: Clinical outcomes in patients receiving venous thromboembolism (VTE) prophylaxis after orthopedic surgery (OS). A systematic review and meta analysis of proportions. Blood. 2006, 108: Abstract 626-CrossRef Lazo-Langner A, Coyle D, Barrowman NJ, Ramsay T, Wells PS, Scarvelis D, et al: Clinical outcomes in patients receiving venous thromboembolism (VTE) prophylaxis after orthopedic surgery (OS). A systematic review and meta analysis of proportions. Blood. 2006, 108: Abstract 626-CrossRef
23.
go back to reference The Columbus Investigators: Low-molecular-weight heparin in the treatment of patients with venous thromboembolism. N Engl J Med. 1997, 337: 657-662.CrossRef The Columbus Investigators: Low-molecular-weight heparin in the treatment of patients with venous thromboembolism. N Engl J Med. 1997, 337: 657-662.CrossRef
24.
go back to reference Schulman S, Kearon C: Definition of major bleeding in clinical investigations of antihemostatic medicinal products in non-surgical patients. J Thromb Haemost. 2005, 3: 692-694. 10.1111/j.1538-7836.2005.01204.x.CrossRefPubMed Schulman S, Kearon C: Definition of major bleeding in clinical investigations of antihemostatic medicinal products in non-surgical patients. J Thromb Haemost. 2005, 3: 692-694. 10.1111/j.1538-7836.2005.01204.x.CrossRefPubMed
25.
go back to reference Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, et al: Assessing the quality of reports of randomized clinical trials: Is blinding necessary?. Control Clin Trials. 1996, 17: 1-12. 10.1016/0197-2456(95)00134-4.CrossRefPubMed Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, et al: Assessing the quality of reports of randomized clinical trials: Is blinding necessary?. Control Clin Trials. 1996, 17: 1-12. 10.1016/0197-2456(95)00134-4.CrossRefPubMed
26.
go back to reference Schulz KF, Grimes DA: Allocation concealment in randomised trials: defending against deciphering. Lancet. 2002, 359: 614-618. 10.1016/S0140-6736(02)07750-4.CrossRefPubMed Schulz KF, Grimes DA: Allocation concealment in randomised trials: defending against deciphering. Lancet. 2002, 359: 614-618. 10.1016/S0140-6736(02)07750-4.CrossRefPubMed
27.
go back to reference Farnum NR, Stanton LW: Some Results Concerning the Estimation of Beta Distribution Parameters in PERT. J Oper Res Soc. 1987, 38: 287-290.CrossRef Farnum NR, Stanton LW: Some Results Concerning the Estimation of Beta Distribution Parameters in PERT. J Oper Res Soc. 1987, 38: 287-290.CrossRef
28.
go back to reference Indurkhya A, Mitra N, Schrag D: Using propensity scores to estimate the cost-effectiveness of medical therapies. Stat Med. 2006, 25: 1561-1576. 10.1002/sim.2267.CrossRefPubMed Indurkhya A, Mitra N, Schrag D: Using propensity scores to estimate the cost-effectiveness of medical therapies. Stat Med. 2006, 25: 1561-1576. 10.1002/sim.2267.CrossRefPubMed
29.
go back to reference Macario A, Chow JL, Dexter F: A Markov computer simulation model of the economics of neuromuscular blockade in patients with acute respiratory distress syndrome. BMC Med Inform Decis Mak. 2006, 6: 15-10.1186/1472-6947-6-15.CrossRefPubMedPubMedCentral Macario A, Chow JL, Dexter F: A Markov computer simulation model of the economics of neuromuscular blockade in patients with acute respiratory distress syndrome. BMC Med Inform Decis Mak. 2006, 6: 15-10.1186/1472-6947-6-15.CrossRefPubMedPubMedCentral
30.
go back to reference Moher D, Schultz KF, Altman DG: The CONSORT statement: revised recommendation for improving the quality of reports of parallel-group randomised trials. Lancet. 2001, 357: 1191-1194. 10.1016/S0140-6736(00)04337-3.CrossRefPubMed Moher D, Schultz KF, Altman DG: The CONSORT statement: revised recommendation for improving the quality of reports of parallel-group randomised trials. Lancet. 2001, 357: 1191-1194. 10.1016/S0140-6736(00)04337-3.CrossRefPubMed
31.
go back to reference Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S: Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials. 1995, 16: 62-73. 10.1016/0197-2456(94)00031-W.CrossRefPubMed Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S: Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials. 1995, 16: 62-73. 10.1016/0197-2456(94)00031-W.CrossRefPubMed
32.
go back to reference Garg AX, Suri RS, Barrowman N, Rehman F, Matsell D, Rosas-Arellano MP, et al: Long-term renal prognosis of diarrhea-associated hemolytic uremic syndrome: a systematic review, meta-analysis, and meta-regression. JAMA. 2003, 290: 1360-1370. 10.1001/jama.290.10.1360.CrossRefPubMed Garg AX, Suri RS, Barrowman N, Rehman F, Matsell D, Rosas-Arellano MP, et al: Long-term renal prognosis of diarrhea-associated hemolytic uremic syndrome: a systematic review, meta-analysis, and meta-regression. JAMA. 2003, 290: 1360-1370. 10.1001/jama.290.10.1360.CrossRefPubMed
33.
go back to reference Laird NM, Mosteller F: Some statistical methods for combining experimental results. Int J Technol Assess Health Care. 1990, 6: 5-30. 10.1017/S0266462300008916.CrossRefPubMed Laird NM, Mosteller F: Some statistical methods for combining experimental results. Int J Technol Assess Health Care. 1990, 6: 5-30. 10.1017/S0266462300008916.CrossRefPubMed
34.
go back to reference Jakubczyk M, Kaminski B: Cost-effectiveness acceptability curves - caveats quantified. Health Econ. 2010, 19: 955-963.CrossRefPubMed Jakubczyk M, Kaminski B: Cost-effectiveness acceptability curves - caveats quantified. Health Econ. 2010, 19: 955-963.CrossRefPubMed
35.
go back to reference Moreno E, Giron FJ, Vazquez-Polo FJ, Negrin MA: Optimal healthcare decisions: Comparing medical treatments on a cost-effectiveness basis. European Journal of Operational Research. 2010, 204: 180-187. 10.1016/j.ejor.2009.10.012.CrossRef Moreno E, Giron FJ, Vazquez-Polo FJ, Negrin MA: Optimal healthcare decisions: Comparing medical treatments on a cost-effectiveness basis. European Journal of Operational Research. 2010, 204: 180-187. 10.1016/j.ejor.2009.10.012.CrossRef
36.
go back to reference Newcombe RG: Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat Med. 1998, 17: 857-872. 10.1002/(SICI)1097-0258(19980430)17:8<857::AID-SIM777>3.0.CO;2-E.CrossRefPubMed Newcombe RG: Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat Med. 1998, 17: 857-872. 10.1002/(SICI)1097-0258(19980430)17:8<857::AID-SIM777>3.0.CO;2-E.CrossRefPubMed
Metadata
Title
Comparing multiple competing interventions in the absence of randomized trials using clinical risk-benefit analysis
Authors
Alejandro Lazo-Langner
Marc A Rodger
Nicholas J Barrowman
Tim Ramsay
Philip S Wells
Douglas A Coyle
Publication date
01-12-2012
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2012
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/1471-2288-12-3

Other articles of this Issue 1/2012

BMC Medical Research Methodology 1/2012 Go to the issue