Skip to main content
Top
Published in: BMC Medical Informatics and Decision Making 1/2007

Open Access 01-12-2007 | Research article

Medicine in words and numbers: a cross-sectional survey comparing probability assessment scales

Authors: Cilia LM Witteman, Silja Renooij, Pieter Koele

Published in: BMC Medical Informatics and Decision Making | Issue 1/2007

Login to get access

Abstract

Background

In the complex domain of medical decision making, reasoning under uncertainty can benefit from supporting tools. Automated decision support tools often build upon mathematical models, such as Bayesian networks. These networks require probabilities which often have to be assessed by experts in the domain of application. Probability response scales can be used to support the assessment process. We compare assessments obtained with different types of response scale.

Methods

General practitioners (GPs) gave assessments on and preferences for three different probability response scales: a numerical scale, a scale with only verbal labels, and a combined verbal-numerical scale we had designed ourselves. Standard analyses of variance were performed.

Results

No differences in assessments over the three response scales were found. Preferences for type of scale differed: the less experienced GPs preferred the verbal scale, the most experienced preferred the numerical scale, with the groups in between having a preference for the combined verbal-numerical scale.

Conclusion

We conclude that all three response scales are equally suitable for supporting probability assessment. The combined verbal-numerical scale is a good choice for aiding the process, since it offers numerical labels to those who prefer numbers and verbal labels to those who prefer words, and accommodates both more and less experienced professionals.
Appendix
Available only for authorised users
Literature
1.
go back to reference Pauker SG, Kassirer JP: The threshold approach to clinical decision making. New England Journal of Medicine. 1980, 302: 1109-1117.CrossRefPubMed Pauker SG, Kassirer JP: The threshold approach to clinical decision making. New England Journal of Medicine. 1980, 302: 1109-1117.CrossRefPubMed
2.
go back to reference Tversky A, Kahneman D: Judgment under Uncertainty: Heuristics and Biases. Science. 1974, 185: 1124-1131.CrossRefPubMed Tversky A, Kahneman D: Judgment under Uncertainty: Heuristics and Biases. Science. 1974, 185: 1124-1131.CrossRefPubMed
3.
go back to reference Kuipers B, Moskowitz AJ, Kassirer JP: Critical decisions under uncertainty: representation and structure. Cognitive Science. 1988, 12: 177-210.CrossRef Kuipers B, Moskowitz AJ, Kassirer JP: Critical decisions under uncertainty: representation and structure. Cognitive Science. 1988, 12: 177-210.CrossRef
4.
go back to reference Timmermans D, Kievit J, Van Bockel H: How do surgeons' probability estimates of operative mortality compare with a decision analytic model?. Acta Psychologica. 1996, 93: 107-120.CrossRefPubMed Timmermans D, Kievit J, Van Bockel H: How do surgeons' probability estimates of operative mortality compare with a decision analytic model?. Acta Psychologica. 1996, 93: 107-120.CrossRefPubMed
5.
go back to reference Tversky A, Koehler DJ: Support Theory: A nonextensional representation of subjective probability. Psychological Review. 1994, 101: 547-567.CrossRef Tversky A, Koehler DJ: Support Theory: A nonextensional representation of subjective probability. Psychological Review. 1994, 101: 547-567.CrossRef
6.
go back to reference Redelmeier DA, Koehler DJ, Liberman V, Tversky A: Probability judgment in medicine: Discounting unspecified possibilities. Medical Decision Making. 1995, 15: 227-230.CrossRefPubMed Redelmeier DA, Koehler DJ, Liberman V, Tversky A: Probability judgment in medicine: Discounting unspecified possibilities. Medical Decision Making. 1995, 15: 227-230.CrossRefPubMed
7.
go back to reference Cahan A, Gilon D, Manor O, Paltiel O: Probabilistic reasoning and clinical decision-making: do doctors overestimate diagnostic probabilities?. QJM: An International Journal of Medicine. 2003, 96: 763-769.CrossRef Cahan A, Gilon D, Manor O, Paltiel O: Probabilistic reasoning and clinical decision-making: do doctors overestimate diagnostic probabilities?. QJM: An International Journal of Medicine. 2003, 96: 763-769.CrossRef
8.
go back to reference Spetzler CS, Stael von Holstein CAS: Probability encoding in decision analysis. Management Science. 1975, 22: 340-358.CrossRef Spetzler CS, Stael von Holstein CAS: Probability encoding in decision analysis. Management Science. 1975, 22: 340-358.CrossRef
9.
go back to reference Renooij S: Probability elicitation for belief networks: Issues to consider. Knowledge Engineering Review. 2001, 16: 255-269.CrossRef Renooij S: Probability elicitation for belief networks: Issues to consider. Knowledge Engineering Review. 2001, 16: 255-269.CrossRef
10.
go back to reference O'Hagan A, Buck CE, Daneshkhah A, Eiser JR, Garthwaite PH, Jenkinson DJ, Oakley JE, Rakow T: Uncertain Judgements: Eliciting experts' probabilities. 2006, Chichester: John WileyCrossRef O'Hagan A, Buck CE, Daneshkhah A, Eiser JR, Garthwaite PH, Jenkinson DJ, Oakley JE, Rakow T: Uncertain Judgements: Eliciting experts' probabilities. 2006, Chichester: John WileyCrossRef
11.
go back to reference Van der Gaag LC, Renooij S, Witteman CLM, Aleman BMP, Taal BG: Probabilities for a probabilistic network: A case study in oesophageal cancer. Artificial Intelligence in Medicine. 2002, 25: 123-148.CrossRefPubMed Van der Gaag LC, Renooij S, Witteman CLM, Aleman BMP, Taal BG: Probabilities for a probabilistic network: A case study in oesophageal cancer. Artificial Intelligence in Medicine. 2002, 25: 123-148.CrossRefPubMed
12.
go back to reference Woloshin S, Schwartz LM, Byram S, Fischhoff B, Welch G: A new scale for assessing perceptions of chance: A validation study. Medical Decision Making. 2000, 20: 298-307.CrossRefPubMed Woloshin S, Schwartz LM, Byram S, Fischhoff B, Welch G: A new scale for assessing perceptions of chance: A validation study. Medical Decision Making. 2000, 20: 298-307.CrossRefPubMed
13.
go back to reference Charitos Th, Van der Gaag LC, Visscher S, Schurink K, Lucas PJF: A dynamic Bayesian network for diagnosing ventilator-associated pneumonia in ICU patients. Proceedings of the 10th Intelligent Data Analysis in Medicine and Pharmacology Workshop. Edited by: Holmes JH, Peek NB. 2005, 32-37. Charitos Th, Van der Gaag LC, Visscher S, Schurink K, Lucas PJF: A dynamic Bayesian network for diagnosing ventilator-associated pneumonia in ICU patients. Proceedings of the 10th Intelligent Data Analysis in Medicine and Pharmacology Workshop. Edited by: Holmes JH, Peek NB. 2005, 32-37.
14.
go back to reference Vicari RM, Flores CD, Silvestre AM, Seixas LJ, Ladeira M, Coelho H: A multi-agent intelligent environment for medical knowledge. Artificial Intelligence in Medicine. 2003, 27: 335-366.CrossRefPubMed Vicari RM, Flores CD, Silvestre AM, Seixas LJ, Ladeira M, Coelho H: A multi-agent intelligent environment for medical knowledge. Artificial Intelligence in Medicine. 2003, 27: 335-366.CrossRefPubMed
15.
go back to reference Geenen PL, Van der Gaag LC: Developing a Bayesian network for clinical diagnosis in veterinary medicine: from the individual to the herd. Proceedings of the Third Bayesian Modelling Applications Workshop; Edinburgh. 2005 Geenen PL, Van der Gaag LC: Developing a Bayesian network for clinical diagnosis in veterinary medicine: from the individual to the herd. Proceedings of the Third Bayesian Modelling Applications Workshop; Edinburgh. 2005
16.
go back to reference Populaire S, Ginestet P, Blanc J, Denoeux T: Fusion of expert knowledge with data using belief functions: a case study in waste-water treatment. Proceedings of the Fifth International Conference on Information Fusion. 2002, 1613-1618. Populaire S, Ginestet P, Blanc J, Denoeux T: Fusion of expert knowledge with data using belief functions: a case study in waste-water treatment. Proceedings of the Fifth International Conference on Information Fusion. 2002, 1613-1618.
17.
go back to reference Witteman CLM, Renooij S: Evaluation of a verbal-numerical probability scale. International Journal of Approximate Reasoning. 2003, 33: 117-131.CrossRef Witteman CLM, Renooij S: Evaluation of a verbal-numerical probability scale. International Journal of Approximate Reasoning. 2003, 33: 117-131.CrossRef
18.
go back to reference Pearl J: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. 1988, Palo Alto: Morgan Kauffman Publishers Pearl J: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. 1988, Palo Alto: Morgan Kauffman Publishers
19.
go back to reference Andreassen S, Woldbye M, Falck B, Andersen SK: MUNIN. A causal probabilistic network for interpretation of electromyographic findings. Proceedings of the 10th International Joint Conference on AI. 1987, 366-372. Andreassen S, Woldbye M, Falck B, Andersen SK: MUNIN. A causal probabilistic network for interpretation of electromyographic findings. Proceedings of the 10th International Joint Conference on AI. 1987, 366-372.
20.
go back to reference Korver M, Lucas PJF: Converting a rule-based expert system into a belief network. Medical Informatics. 1993, 18: 219-241.CrossRefPubMed Korver M, Lucas PJF: Converting a rule-based expert system into a belief network. Medical Informatics. 1993, 18: 219-241.CrossRefPubMed
21.
go back to reference Dietz FJ, Mira J, Iturralde E, Zubillage Z: DIAVAL, a Bayesian expert system for echocardiography. AI in Medicine. 1997, 10: 59-73. Dietz FJ, Mira J, Iturralde E, Zubillage Z: DIAVAL, a Bayesian expert system for echocardiography. AI in Medicine. 1997, 10: 59-73.
22.
go back to reference Lucas PJF, Boot H, Taal BG: Computer-based decision-support in the management of primary gastric non-Hodgkin lymphoma. Methods of Information in Medicine. 1998, 37: 206-219.PubMed Lucas PJF, Boot H, Taal BG: Computer-based decision-support in the management of primary gastric non-Hodgkin lymphoma. Methods of Information in Medicine. 1998, 37: 206-219.PubMed
23.
go back to reference Druzdzel MJ, Van der Gaag LC: Building probabilistic networks: where do the numbers come from?. IEEE Transactions in Knowledge and Data Engineering. 2000, 15: 1-18. Druzdzel MJ, Van der Gaag LC: Building probabilistic networks: where do the numbers come from?. IEEE Transactions in Knowledge and Data Engineering. 2000, 15: 1-18.
24.
go back to reference Henrion M, Pradhan M, Del Favero B, Huang K, Provan G, O'Rorke P: Why is diagnosis using belief networks insensitive to imprecision in probabilities?. Proceedings of the 12th Conference on Uncertainty in AI. 1996, 307-314. Henrion M, Pradhan M, Del Favero B, Huang K, Provan G, O'Rorke P: Why is diagnosis using belief networks insensitive to imprecision in probabilities?. Proceedings of the 12th Conference on Uncertainty in AI. 1996, 307-314.
25.
go back to reference Brun W, Teigen KH: Verbal probabilities: ambiguous, context-dependent, or both?. Organizational Behavior and Human Decision Processes. 1988, 41: 390-404.CrossRef Brun W, Teigen KH: Verbal probabilities: ambiguous, context-dependent, or both?. Organizational Behavior and Human Decision Processes. 1988, 41: 390-404.CrossRef
26.
go back to reference Merz JF, Druzdzel MJ, Mazur DJ: Verbal expressions of probability in informed consent litigation. Medical Decision Making. 1991, 11: 273-281.CrossRefPubMed Merz JF, Druzdzel MJ, Mazur DJ: Verbal expressions of probability in informed consent litigation. Medical Decision Making. 1991, 11: 273-281.CrossRefPubMed
27.
go back to reference Erev I, Cohen BL: Verbal versus numerical probabilities: efficiency, biases, and the preference paradox. Organizational Behavior and Human Decision Processes. 1990, 45: 1-18.CrossRef Erev I, Cohen BL: Verbal versus numerical probabilities: efficiency, biases, and the preference paradox. Organizational Behavior and Human Decision Processes. 1990, 45: 1-18.CrossRef
28.
go back to reference Renooij S, Witteman CLM: Talking probabilities: communicating probabilistic information with words and numbers. International Journal of Approximate Reasoning. 1999, 22: 169-194.CrossRef Renooij S, Witteman CLM: Talking probabilities: communicating probabilistic information with words and numbers. International Journal of Approximate Reasoning. 1999, 22: 169-194.CrossRef
29.
go back to reference Teigen KH, Brun W: Verbal expressions of uncertainty and probability. Thinking: Psychological perspectives on reasoning, judgment and decision making. Edited by: Hardman D, Macchi L. 2003, Chichester: Wiley, 125-145. Teigen KH, Brun W: Verbal expressions of uncertainty and probability. Thinking: Psychological perspectives on reasoning, judgment and decision making. Edited by: Hardman D, Macchi L. 2003, Chichester: Wiley, 125-145.
30.
go back to reference Coupé VMH, Van der Gaag LC, Habbema JDF: Sensitivity analysis: an aid for probability elicitation. Knowledge Engineering Review. 2000, 15: 1-18.CrossRef Coupé VMH, Van der Gaag LC, Habbema JDF: Sensitivity analysis: an aid for probability elicitation. Knowledge Engineering Review. 2000, 15: 1-18.CrossRef
31.
go back to reference Sloman S, Rottenstreich Y, Wisniewski E, Hadjichristidis C, Fox CR: Typical versus atypical unpacking and superadditive probability judgment. Journal of Experimental Psychology: Learning, Memory and Cognition. 2004, 30: 573-582. Sloman S, Rottenstreich Y, Wisniewski E, Hadjichristidis C, Fox CR: Typical versus atypical unpacking and superadditive probability judgment. Journal of Experimental Psychology: Learning, Memory and Cognition. 2004, 30: 573-582.
32.
go back to reference Fischhoff B, Slovic P, Lichtenstein S: Fault trees: Sensitivity of estimated failure probabilities to problem representation. Journal of Experimental Psychology: Human Perception and Performance. 1978, 4: 330-344. Fischhoff B, Slovic P, Lichtenstein S: Fault trees: Sensitivity of estimated failure probabilities to problem representation. Journal of Experimental Psychology: Human Perception and Performance. 1978, 4: 330-344.
33.
go back to reference Wallsten TS, Budescu DV, Zwick R, Kemp SM: Preferences and reasons for communicating probabilistic information in verbal or numerical terms. Bulletin of the Psychonomic Society. 1993, 31: 135-138.CrossRef Wallsten TS, Budescu DV, Zwick R, Kemp SM: Preferences and reasons for communicating probabilistic information in verbal or numerical terms. Bulletin of the Psychonomic Society. 1993, 31: 135-138.CrossRef
34.
go back to reference Wallsten TS, Fillenbaum S, Cox JA: Base rate effects on the interpretations of probability and frequency expressions. Journal of Memory and Language. 1986, 25: 571-587.CrossRef Wallsten TS, Fillenbaum S, Cox JA: Base rate effects on the interpretations of probability and frequency expressions. Journal of Memory and Language. 1986, 25: 571-587.CrossRef
Metadata
Title
Medicine in words and numbers: a cross-sectional survey comparing probability assessment scales
Authors
Cilia LM Witteman
Silja Renooij
Pieter Koele
Publication date
01-12-2007
Publisher
BioMed Central
Published in
BMC Medical Informatics and Decision Making / Issue 1/2007
Electronic ISSN: 1472-6947
DOI
https://doi.org/10.1186/1472-6947-7-13

Other articles of this Issue 1/2007

BMC Medical Informatics and Decision Making 1/2007 Go to the issue