Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2013

Open Access 01-12-2013 | Technical advance

The agreement chart

Authors: Shrikant I Bangdiwala, Viswanathan Shankar

Published in: BMC Medical Research Methodology | Issue 1/2013

Login to get access

Abstract

Background

When assessing the concordance between two methods of measurement of ordinal categorical data, summary measures such as Cohen’s (1960) kappa or Bangdiwala’s (1985) B-statistic are used. However, a picture conveys more information than a single summary measure.

Methods

We describe how to construct and interpret Bangdiwala’s (1985) agreement chart and illustrate its use in visually assessing concordance in several example clinical applications.

Results

The agreement charts provide a visual impression that no summary statistic can convey, and summary statistics reduce the information to a single characteristic of the data. However, the visual impression is personal and subjective, and not usually reproducible from one reader to another.

Conclusions

The agreement chart should be used to complement the summary kappa or B-statistics, not to replace them. The graphs can be very helpful to researchers as an early step to understand relationships in their data when assessing concordance.
Appendix
Available only for authorised users
Literature
1.
go back to reference Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics. 1977, 33 (1): 159-174. 10.2307/2529310.CrossRefPubMed Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics. 1977, 33 (1): 159-174. 10.2307/2529310.CrossRefPubMed
2.
go back to reference Westlund KB, Kurland LT: Studies on multiple sclerosis in Winnipeg, Manitoba, and New Orleans, Louisiana. II. A controlled investigation of factors in the life history of the Winnipeg patients. Am J Hyg. 1953, 57 (3): 397-407.PubMed Westlund KB, Kurland LT: Studies on multiple sclerosis in Winnipeg, Manitoba, and New Orleans, Louisiana. II. A controlled investigation of factors in the life history of the Winnipeg patients. Am J Hyg. 1953, 57 (3): 397-407.PubMed
3.
go back to reference Cohen J: A coefficient for agreement for nominal scales. Educ Psychol Meas. 1960, 20: 37-46. 10.1177/001316446002000104.CrossRef Cohen J: A coefficient for agreement for nominal scales. Educ Psychol Meas. 1960, 20: 37-46. 10.1177/001316446002000104.CrossRef
4.
go back to reference Bangdiwala SI: International statistical institute centenary session 1985. A graphical test for observer agreement. 1985, Amsterdam: International Statistical Institute, 307-308. Bangdiwala SI: International statistical institute centenary session 1985. A graphical test for observer agreement. 1985, Amsterdam: International Statistical Institute, 307-308.
5.
go back to reference Munoz SR, Bangdiwala SI: Interpretation of Kappa and B statistics measures of agreement. J Appl Stat. 1997, 24 (1): 105-112. 10.1080/02664769723918.CrossRef Munoz SR, Bangdiwala SI: Interpretation of Kappa and B statistics measures of agreement. J Appl Stat. 1997, 24 (1): 105-112. 10.1080/02664769723918.CrossRef
6.
go back to reference Shankar V, Bangdiwala SI: Behavior of agreement measures in the presence of zero cells and biased marginal distributions. J Appl Stat. 2008, 35 (4): 445-464. 10.1080/02664760701835052.CrossRef Shankar V, Bangdiwala SI: Behavior of agreement measures in the presence of zero cells and biased marginal distributions. J Appl Stat. 2008, 35 (4): 445-464. 10.1080/02664760701835052.CrossRef
7.
go back to reference Bangdiwala SI, Cohn R, Hazard C, Davis CE, Prineas RJ: Comparisons of cause of death verification methods and costs in the lipid research clinics program mortality follow-up study. Control Clin Trials. 1989, 10 (2): 167-187. 10.1016/0197-2456(89)90029-9.CrossRefPubMed Bangdiwala SI, Cohn R, Hazard C, Davis CE, Prineas RJ: Comparisons of cause of death verification methods and costs in the lipid research clinics program mortality follow-up study. Control Clin Trials. 1989, 10 (2): 167-187. 10.1016/0197-2456(89)90029-9.CrossRefPubMed
8.
go back to reference Michel CE, Solomon AW, Magbanua JP, Massae PA, Huang L, Mosha J, West SK, Nadala EC, Bailey R, Wisniewski C, et al: Field evaluation of a rapid point-of-care assay for targeting antibiotic treatment for trachoma control: a comparative study. Lancet. 2006, 367 (9522): 1585-1590. 10.1016/S0140-6736(06)68695-9.CrossRefPubMed Michel CE, Solomon AW, Magbanua JP, Massae PA, Huang L, Mosha J, West SK, Nadala EC, Bailey R, Wisniewski C, et al: Field evaluation of a rapid point-of-care assay for targeting antibiotic treatment for trachoma control: a comparative study. Lancet. 2006, 367 (9522): 1585-1590. 10.1016/S0140-6736(06)68695-9.CrossRefPubMed
9.
go back to reference Meyer D, Zeileis A, Hornik K: vcd: Visualizing Categorical Data: R package version 1.2-13. 2012 Meyer D, Zeileis A, Hornik K: vcd: Visualizing Categorical Data: R package version 1.2-13. 2012
10.
go back to reference Tufte ER: The visual display of quantitative information. 1983, Cheshire, CT: Graphics Press Tufte ER: The visual display of quantitative information. 1983, Cheshire, CT: Graphics Press
Metadata
Title
The agreement chart
Authors
Shrikant I Bangdiwala
Viswanathan Shankar
Publication date
01-12-2013
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2013
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/1471-2288-13-97

Other articles of this Issue 1/2013

BMC Medical Research Methodology 1/2013 Go to the issue