Skip to main content
Top
Published in: BMC Medical Informatics and Decision Making 1/2014

Open Access 01-12-2014 | Research article

Concordance between decision analysis and matching systematic review of randomized controlled trials in assessment of treatment comparisons: a systematic review

Authors: Rahul S Mhaskar, Hesborn Wao, Helen Mahony, Ambuj Kumar, Benjamin Djulbegovic

Published in: BMC Medical Informatics and Decision Making | Issue 1/2014

Login to get access

Abstract

Background

Systematic review (SR) of randomized controlled trials (RCT) is the gold standard for informing treatment choice. Decision analyses (DA) also play an important role in informing health care decisions. It is unknown how often the results of DA and matching SR of RCTs are in concordance. We assessed whether the results of DA are in concordance with SR of RCTs matched on patient population, intervention, control, and outcomes.

Methods

We searched PubMed up to 2008 for DAs comparing at least two interventions followed by matching SRs of RCTs. Data were extracted on patient population, intervention, control, and outcomes from DAs and matching SRs of RCTs. Data extraction from DAs was done by one reviewer and from SR of RCTs by two independent reviewers.

Results

We identified 28 DAs representing 37 comparisons for which we found matching SR of RCTs. Results of the DAs and SRs of RCTs were in concordance in 73% (27/37) of cases. The sensitivity analyses conducted in either DA or SR of RCTs did not impact the concordance. Use of single (4/37) versus multiple data source (33/37) in design of DA model was statistically significantly associated with concordance between DA and SR of RCTs.

Conclusions

Our findings illustrate the high concordance of current DA models compared with SR of RCTs. It is shown previously that there is 50% concordance between DA and matching single RCT. Our study showing the concordance of 73% between DA and matching SR of RCTs underlines the importance of totality of evidence (i.e. SR of RCTs) in the design of DA models and in general medical decision-making.
Appendix
Available only for authorised users
Literature
1.
go back to reference Salmond SS: Randomized controlled trials: methodological concepts and critique. Orthop Nurs. 2008, 27: 116-122. 10.1097/01.NOR.0000315626.44137.94. quiz 123–114CrossRefPubMed Salmond SS: Randomized controlled trials: methodological concepts and critique. Orthop Nurs. 2008, 27: 116-122. 10.1097/01.NOR.0000315626.44137.94. quiz 123–114CrossRefPubMed
2.
go back to reference Deeks JJ: Systematic reviews of published evidence: miracles or minefields?. Ann Oncol. 1998, 9: 703-709. 10.1023/A:1008335706631.CrossRefPubMed Deeks JJ: Systematic reviews of published evidence: miracles or minefields?. Ann Oncol. 1998, 9: 703-709. 10.1023/A:1008335706631.CrossRefPubMed
3.
go back to reference Goldhaber-Fiebert JD: Accounting for biases when linking empirical studies and simulation models. Med Decis Making. 2012, 32: 397-399. 10.1177/0272989X12441398.CrossRefPubMedPubMedCentral Goldhaber-Fiebert JD: Accounting for biases when linking empirical studies and simulation models. Med Decis Making. 2012, 32: 397-399. 10.1177/0272989X12441398.CrossRefPubMedPubMedCentral
4.
5.
go back to reference Pauker SG, Kassirer JP: Decision analysis. N Engl J Med. 1987, 316: 250-258. 10.1056/NEJM198701293160505.CrossRefPubMed Pauker SG, Kassirer JP: Decision analysis. N Engl J Med. 1987, 316: 250-258. 10.1056/NEJM198701293160505.CrossRefPubMed
6.
go back to reference Pauker SG, Kassirer JP: Clinical application of decision analysis: a detailed illustration. Semin Nucl Med. 1978, 8: 324-335. 10.1016/S0001-2998(78)80018-X.CrossRefPubMed Pauker SG, Kassirer JP: Clinical application of decision analysis: a detailed illustration. Semin Nucl Med. 1978, 8: 324-335. 10.1016/S0001-2998(78)80018-X.CrossRefPubMed
7.
go back to reference Kassirer JP, Moskowitz AJ, Lau J, Pauker SG: Decision analysis: a progress report. Ann Intern Med. 1987, 106: 275-291. 10.7326/0003-4819-106-2-275.CrossRefPubMed Kassirer JP, Moskowitz AJ, Lau J, Pauker SG: Decision analysis: a progress report. Ann Intern Med. 1987, 106: 275-291. 10.7326/0003-4819-106-2-275.CrossRefPubMed
8.
go back to reference Sears ED, Chung KC: Decision analysis in plastic surgery: a primer. Plastic Reconstruct Surg. 2010, 126 (4): 1373-1380. 10.1097/PRS.0b013e3181ead10a.CrossRef Sears ED, Chung KC: Decision analysis in plastic surgery: a primer. Plastic Reconstruct Surg. 2010, 126 (4): 1373-1380. 10.1097/PRS.0b013e3181ead10a.CrossRef
9.
go back to reference Richardson WS, Detsky AS: Users’ guides to the medical literature. VII. How to use a clinical decision analysis. B. What are the results and will they help me in caring for my patients? Evidence Based Medicine Working Group. JAMA. 1995, 273: 1610-1613. 10.1001/jama.1995.03520440064038.CrossRefPubMed Richardson WS, Detsky AS: Users’ guides to the medical literature. VII. How to use a clinical decision analysis. B. What are the results and will they help me in caring for my patients? Evidence Based Medicine Working Group. JAMA. 1995, 273: 1610-1613. 10.1001/jama.1995.03520440064038.CrossRefPubMed
10.
go back to reference Richardson WS, Detsky AS: Users’ guides to the medical literature. VII. How to use a clinical decision analysis. A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA. 1995, 273: 1292-1295. 10.1001/jama.1995.03520400062046.CrossRefPubMed Richardson WS, Detsky AS: Users’ guides to the medical literature. VII. How to use a clinical decision analysis. A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA. 1995, 273: 1292-1295. 10.1001/jama.1995.03520400062046.CrossRefPubMed
11.
go back to reference Bress JN, Hulgan T, Lyon JA, Johnston CP, Lehmann H, Sterling TR: Agreement of decision analyses and subsequent clinical studies in infectious diseases. Am J Med. 2007, 120 (461): e461-e469. Bress JN, Hulgan T, Lyon JA, Johnston CP, Lehmann H, Sterling TR: Agreement of decision analyses and subsequent clinical studies in infectious diseases. Am J Med. 2007, 120 (461): e461-e469.
12.
go back to reference Mhaskar RS, Kumar A, Djulbegovic B: Agreement of Decision Analyses and Matching Randomized Controlled Trials in Assessment of Treatment Comparisons: A Systematic Review. 2008, Pennsylvania: Society for Medical Decision Making Mhaskar RS, Kumar A, Djulbegovic B: Agreement of Decision Analyses and Matching Randomized Controlled Trials in Assessment of Treatment Comparisons: A Systematic Review. 2008, Pennsylvania: Society for Medical Decision Making
13.
go back to reference Wilczynski NL, McKibbon KA, Haynes RB: Sensitive Clinical Queries retrieved relevant systematic reviews as well as primary studies: an analytic survey. J Clin Epidemiol. 2011, 64: 1341-1349. 10.1016/j.jclinepi.2011.04.007.CrossRefPubMed Wilczynski NL, McKibbon KA, Haynes RB: Sensitive Clinical Queries retrieved relevant systematic reviews as well as primary studies: an analytic survey. J Clin Epidemiol. 2011, 64: 1341-1349. 10.1016/j.jclinepi.2011.04.007.CrossRefPubMed
14.
go back to reference Elkin EB, Weinstein MC, Kuntz KM, Bunnell CA, Weeks JC: Adjuvant ovarian suppression versus chemotherapy for premenopausal, hormone-responsive breast cancer: quality of life and efficacy tradeoffs. Breast Cancer Res Treat. 2005, 93: 25-34. 10.1007/s10549-005-3380-2.CrossRefPubMed Elkin EB, Weinstein MC, Kuntz KM, Bunnell CA, Weeks JC: Adjuvant ovarian suppression versus chemotherapy for premenopausal, hormone-responsive breast cancer: quality of life and efficacy tradeoffs. Breast Cancer Res Treat. 2005, 93: 25-34. 10.1007/s10549-005-3380-2.CrossRefPubMed
15.
go back to reference Cuzick J, Ambroisine L, Davidson N, Jakesz R, Kaufmann M, Regan M, Sainsbury R, LHRH-agonists in Early Breast Cancer Overview group: Use of luteinising-hormone-releasing hormone agonists as adjuvant treatment in premenopausal patients with hormone-receptor-positive breast cancer: a meta-analysis of individual patient data from randomised adjuvant trials. Lancet. 2007, 369: 1711-1723.CrossRefPubMed Cuzick J, Ambroisine L, Davidson N, Jakesz R, Kaufmann M, Regan M, Sainsbury R, LHRH-agonists in Early Breast Cancer Overview group: Use of luteinising-hormone-releasing hormone agonists as adjuvant treatment in premenopausal patients with hormone-receptor-positive breast cancer: a meta-analysis of individual patient data from randomised adjuvant trials. Lancet. 2007, 369: 1711-1723.CrossRefPubMed
16.
go back to reference Verhoef LC, Stalpers LJ, Verbeek AL, Wobbes T, van Daal WA: Breast-conserving treatment or mastectomy in early breast cancer: a clinical decision analysis with special reference to the risk of local recurrence. Eur J Cancer. 1991, 27: 1132-1137. 10.1016/0277-5379(91)90310-A.CrossRefPubMed Verhoef LC, Stalpers LJ, Verbeek AL, Wobbes T, van Daal WA: Breast-conserving treatment or mastectomy in early breast cancer: a clinical decision analysis with special reference to the risk of local recurrence. Eur J Cancer. 1991, 27: 1132-1137. 10.1016/0277-5379(91)90310-A.CrossRefPubMed
17.
go back to reference Morris AD, Morris RD, Wilson JF, White J, Steinberg S, Okunieff P, Arriagada R, Le MG, Blichert-Toft M, van Dongen JA: Breast-conserving therapy vs mastectomy in early-stage breast cancer: a meta-analysis of 10-year survival. Cancer J Sci Am. 1997, 3: 6-12.PubMed Morris AD, Morris RD, Wilson JF, White J, Steinberg S, Okunieff P, Arriagada R, Le MG, Blichert-Toft M, van Dongen JA: Breast-conserving therapy vs mastectomy in early-stage breast cancer: a meta-analysis of 10-year survival. Cancer J Sci Am. 1997, 3: 6-12.PubMed
18.
go back to reference Boughey JC, Cormier JN, Xing Y, Hunt KK, Meric-Bernstam F, Babiera GV, Ross MI, Kuerer HM, Singletary SE, Bedrosian I: Decision analysis to assess the efficacy of routine sentinel lymphadenectomy in patients undergoing prophylactic mastectomy. Cancer. 2007, 110: 2542-2550. 10.1002/cncr.23067.CrossRefPubMed Boughey JC, Cormier JN, Xing Y, Hunt KK, Meric-Bernstam F, Babiera GV, Ross MI, Kuerer HM, Singletary SE, Bedrosian I: Decision analysis to assess the efficacy of routine sentinel lymphadenectomy in patients undergoing prophylactic mastectomy. Cancer. 2007, 110: 2542-2550. 10.1002/cncr.23067.CrossRefPubMed
19.
go back to reference Kell MR, Burke JP, Barry M, Morrow M: Outcome of axillary staging in early breast cancer: a meta-analysis. Breast Cancer Res Treat. 2010, 120: 441-447. 10.1007/s10549-009-0705-6.CrossRefPubMed Kell MR, Burke JP, Barry M, Morrow M: Outcome of axillary staging in early breast cancer: a meta-analysis. Breast Cancer Res Treat. 2010, 120: 441-447. 10.1007/s10549-009-0705-6.CrossRefPubMed
20.
go back to reference Higgins KM, Shah MD, Ogaick MJ, Enepekides D: Treatment of early-stage glottic cancer: meta-analysis comparison of laser excision versus radiotherapy. J Otolaryngol Head Neck Surg. 2009, 38: 603-612.PubMed Higgins KM, Shah MD, Ogaick MJ, Enepekides D: Treatment of early-stage glottic cancer: meta-analysis comparison of laser excision versus radiotherapy. J Otolaryngol Head Neck Surg. 2009, 38: 603-612.PubMed
21.
go back to reference Wong JB, Sonnenberg FA, Salem DN, Pauker SG: Myocardial revascularization for chronic stable angina. Analysis of the role of percutaneous transluminal coronary angioplasty based on data available in 1989. Ann Intern Med. 1990, 113: 852-871. 10.7326/0003-4819-113-11-852.CrossRefPubMed Wong JB, Sonnenberg FA, Salem DN, Pauker SG: Myocardial revascularization for chronic stable angina. Analysis of the role of percutaneous transluminal coronary angioplasty based on data available in 1989. Ann Intern Med. 1990, 113: 852-871. 10.7326/0003-4819-113-11-852.CrossRefPubMed
22.
go back to reference Jeremias A, Kaul S, Rosengart TK, Gruberg L, Brown DL: The impact of revascularization on mortality in patients with nonacute coronary artery disease. Am J Med. 2009, 122: 152-161. 10.1016/j.amjmed.2008.07.027.CrossRefPubMed Jeremias A, Kaul S, Rosengart TK, Gruberg L, Brown DL: The impact of revascularization on mortality in patients with nonacute coronary artery disease. Am J Med. 2009, 122: 152-161. 10.1016/j.amjmed.2008.07.027.CrossRefPubMed
23.
go back to reference Stalpers LJ, Verbeek AL, van Daal WA: Radiotherapy or surgery for T2N0M0 glottic carcinoma? A decision-analytic approach. Radiother Oncol. 1989, 14: 209-217. 10.1016/0167-8140(89)90169-2.CrossRefPubMed Stalpers LJ, Verbeek AL, van Daal WA: Radiotherapy or surgery for T2N0M0 glottic carcinoma? A decision-analytic approach. Radiother Oncol. 1989, 14: 209-217. 10.1016/0167-8140(89)90169-2.CrossRefPubMed
24.
go back to reference Nease RF, Ross JM: The decision to enter a randomized trial of tamoxifen for the prevention of breast cancer in healthy women: an analysis of the tradeoffs. Am J Med. 1995, 99: 180-189. 10.1016/S0002-9343(99)80138-7.CrossRefPubMed Nease RF, Ross JM: The decision to enter a randomized trial of tamoxifen for the prevention of breast cancer in healthy women: an analysis of the tradeoffs. Am J Med. 1995, 99: 180-189. 10.1016/S0002-9343(99)80138-7.CrossRefPubMed
25.
go back to reference Cuzick J, Powles T, Veronesi U, Forbes J, Edwards R, Ashley S, Boyle P: Overview of the main outcomes in breast-cancer prevention trials. Lancet. 2003, 361: 296-300. 10.1016/S0140-6736(03)12342-2.CrossRefPubMed Cuzick J, Powles T, Veronesi U, Forbes J, Edwards R, Ashley S, Boyle P: Overview of the main outcomes in breast-cancer prevention trials. Lancet. 2003, 361: 296-300. 10.1016/S0140-6736(03)12342-2.CrossRefPubMed
26.
go back to reference Practical Nonparametric Statistics. Edited by: Conover WJ. 1999, John Wiley & Sons, 3 Practical Nonparametric Statistics. Edited by: Conover WJ. 1999, John Wiley & Sons, 3
27.
go back to reference Col NF, Eckman MH, Karas RH, Pauker SG, Goldberg RJ, Ross EM, Orr RK, Wong JB: Patient-specific decisions about hormone replacement therapy in postmenopausal women. JAMA. 1997, 277: 1140-1147. 10.1001/jama.1997.03540380054031.CrossRefPubMed Col NF, Eckman MH, Karas RH, Pauker SG, Goldberg RJ, Ross EM, Orr RK, Wong JB: Patient-specific decisions about hormone replacement therapy in postmenopausal women. JAMA. 1997, 277: 1140-1147. 10.1001/jama.1997.03540380054031.CrossRefPubMed
28.
go back to reference Dolan JG: Can decision analysis adequately represent clinical problems?. J Clin Epidemiol. 1990, 43: 277-284. 10.1016/0895-4356(90)90008-D.CrossRefPubMed Dolan JG: Can decision analysis adequately represent clinical problems?. J Clin Epidemiol. 1990, 43: 277-284. 10.1016/0895-4356(90)90008-D.CrossRefPubMed
30.
go back to reference Dolan JG: Multi-criteria clinical decision support: a primer on the use of multiple criteria decision making methods to promote evidence-based, patient-centered healthcare. Patient. 2010, 3: 229-248. 10.2165/11539470-000000000-00000.CrossRefPubMedPubMedCentral Dolan JG: Multi-criteria clinical decision support: a primer on the use of multiple criteria decision making methods to promote evidence-based, patient-centered healthcare. Patient. 2010, 3: 229-248. 10.2165/11539470-000000000-00000.CrossRefPubMedPubMedCentral
31.
go back to reference Weinstein MC, O’Brien B, Hornberger J, Jackson J, Johannesson M, McCabe C, Luce BR, ISPOR Task Force on Good Research Practices--Modeling Studies: Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR Task Force on Good Research Practices–Modeling Studies. Value Health. 2003, 6: 9-17. 10.1046/j.1524-4733.2003.00234.x.CrossRefPubMed Weinstein MC, O’Brien B, Hornberger J, Jackson J, Johannesson M, McCabe C, Luce BR, ISPOR Task Force on Good Research Practices--Modeling Studies: Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR Task Force on Good Research Practices–Modeling Studies. Value Health. 2003, 6: 9-17. 10.1046/j.1524-4733.2003.00234.x.CrossRefPubMed
32.
go back to reference Philips Z, Bojke L, Sculpher M, Claxton K, Golder S: Good practice guidelines for decision-analytic modelling in health technology assessment: a review and consolidation of quality assessment. Pharmacoeconomics. 2006, 24: 355-371. 10.2165/00019053-200624040-00006.CrossRefPubMed Philips Z, Bojke L, Sculpher M, Claxton K, Golder S: Good practice guidelines for decision-analytic modelling in health technology assessment: a review and consolidation of quality assessment. Pharmacoeconomics. 2006, 24: 355-371. 10.2165/00019053-200624040-00006.CrossRefPubMed
33.
go back to reference Philips Z, Ginnelly L, Sculpher M, Claxton K, Golder S, Riemsma R, Woolacoot N, Glanville J: Review of guidelines for good practice in decision-analytic modelling in health technology assessment. Health Technol Assess. 2004, 8 (iii-iv, ix-xi): 1-158.PubMed Philips Z, Ginnelly L, Sculpher M, Claxton K, Golder S, Riemsma R, Woolacoot N, Glanville J: Review of guidelines for good practice in decision-analytic modelling in health technology assessment. Health Technol Assess. 2004, 8 (iii-iv, ix-xi): 1-158.PubMed
Metadata
Title
Concordance between decision analysis and matching systematic review of randomized controlled trials in assessment of treatment comparisons: a systematic review
Authors
Rahul S Mhaskar
Hesborn Wao
Helen Mahony
Ambuj Kumar
Benjamin Djulbegovic
Publication date
01-12-2014
Publisher
BioMed Central
Published in
BMC Medical Informatics and Decision Making / Issue 1/2014
Electronic ISSN: 1472-6947
DOI
https://doi.org/10.1186/1472-6947-14-57

Other articles of this Issue 1/2014

BMC Medical Informatics and Decision Making 1/2014 Go to the issue