Top

Health Services and Outcomes Research Methodology

Published in:

01-06-2017

Intraclass correlation coefficients: clearing the air, extending some cautions, and making some requests

Author: Robert Trevethan

Published in: Health Services and Outcomes Research Methodology | Issue 2/2017

Abstract

Intraclass correlation coefficients (ICCs) are frequently employed in health science research, often to assess intrarater and interrater reliability. In many cases, insufficient details are provided about these ICCs and there seem to be misunderstandings about their selection and how they should be interpreted. This paper is intended primarily to provide a clear, accessible description of ICCs, including how they should be selected, interpreted, and reported. Emphasis is given to areas where researchers seem to encounter the greatest conceptual difficulties and to exhibit the greatest misconceptions. Two extended examples are used to support the points being made. Major additional aims of this paper are to raise the awareness of authors, reviewers, and editors concerning the importance of using appropriate ICCs, and to encourage them to ensure that complete and accurate information about ICCs is reported in journal articles. Failure to do so perpetuates a risk that incorrect decisions might be made about matters that are of crucial importance for people’s health.

Brozek, J., Alexander, H.: Components of variance and the consistency of repeated measurements. Res. Q. 18, 152–166 (1947)PubMed

Carrasco, J.L., Jover, L.: Estimating the generalized concordance correlation coefficient through variance components. Biometrics 59, 849e58 (2003)CrossRef

Fleiss, J.: The Design and Analysis of Clinical Experiments. Wiley, New York (1986)

Franco, A., Malhotra, N., Simonovits, G.: Publication bias in the social sciences: unlocking the file drawer. Science 345, 1502–1505 (2014). doi:10.1126/science.1255484 CrossRefPubMed

Hallgren, K.A.: Computing inter-rater reliability for observational data: an overview and tutorial. Tutor. Quant. Methods Psychol. 8(1), 23–34 (2012)CrossRefPubMedPubMedCentral

Hayen, A., Dennis, R., Finch, C.: Determining the intra- and inter-observer reliability of screening tools used in sports injury research. J. Sci. Med. Sport 10, 201–210 (2007). doi:10.1016/j.jsams.2006.09.002 CrossRefPubMed

Kottner, J., Audigé, L., Brorson, S., Donner, A., Gajewski, B.J., Hróbjartsson, A., Roberts, C., Shoukri, M., Streiner, D.L.: Guidelines for reporting reliability and agreement studies (GRRAS) were proposed. J. Clin. Epidemiol. 64, 96–106 (2011). doi:10.1016/j.jclinepi.2010.03.002 CrossRefPubMed

Krebs, D.E.: Declare your ICC type. Phys. Ther. 66, 1431 (1986)CrossRefPubMed

Landers, R.N.: Computing intraclass correlations (ICC) as estimates of interrater reliability in SPSS. The Winnower 2, e143518.81744 (2015). doi:10.15200/winn.143518.81744

Lee, K.M., Lee, J., Chung, C.Y., Ahn, S., Sung, K.H., Kim, T.W., Lee, H.J., Park, M.S.: Pitfalls and important issues in testing reliability using intraclass correlation coefficients in orthopaedic research. Clin. Orthop. Surg. 4, 149–155 (2012). doi:10.4055/cios.2012.4.2.149 CrossRefPubMedPubMedCentral

McAra, S.J.: Glygeryl trinitrate and toe-brachial indexes in pedal ischaemia. Unpublished Ph.D. thesis. Charles Sturt University, Albury, Australia (2015). http://primo.unilinc.edu.au/CSU:CSU_ALL:dtl_csu80101

McGraw, K.O., Wong, S.P.: Forming inferences about some intraclass correlation coefficients. Psychol. Methods 1, 30–46 (1996)CrossRef

Müller, R., Büttner, P.: A critical discussion of intraclass correlation coefficients. Stat. Med. 13, 2465–2476 (1994)CrossRefPubMed

Nunnally, J.C., Bernstein, I.H.: Psychometric Theory, 3rd edn. McGraw Hill, New York (1994)

Portney, L.G., Watkins, M.P.: Foundations of clinical research: applications to practice, 3rd edn. Pearson Education, Upper Saddle River (2009)

Rankin, G., Stokes, M.: Reliability of assessment tools in rehabilitation: an illustration of appropriate statistical analyses. Clin. Rehabil. 12, 187–199 (1998)CrossRefPubMed

Romanos, M.T., Raspovic, A., Perrin, B.M.: The reliability of toe systolic pressure and the toe brachial index in patients with diabetes. J. Foot Ankle Res. 3, 31 (2010). doi:10.1186/1757-1146-3-31 CrossRefPubMedPubMedCentral

Raykov, T., Dimitrov, D.M., von Eye, A., Marcoulides, G.A.: Interrater agreement evaluation: a latent variable modeling approach. Educ. Psychol. Meas. 73, 512–531 (2013). doi:10.1177/0013164412449016 CrossRef

Scanlon, C., Park, K., Mapletoft, D., Begg, L., Burns, J.: Interrater and intrarater reliability of photoplethysmography for measuring toe blood pressure and toe-brachial index in people with diabetes mellitus. J. Foot Ankle Res. 5, 13 (2012). doi:10.1186/1757-1146-5-13 CrossRefPubMedPubMedCentral

Shrout, P.E., Fleiss, J.L.: Intraclass correlations: uses in assessing rater reliability. Psychol. Bull. 86, 420–428 (1979)CrossRefPubMed

Sonter, J., Chuter, V., Casey, S.: Intratester and intertester reliability of toe pressure measurements in people with and without diabetes performed by podiatric physicians. J. Am. Podiatr. Med. Assoc. 105, 201–208 (2015a). doi:10.7547/0003-0538-105.3.201 CrossRefPubMed

Sonter, J., Sadler, S., Chuter, V.: Inter-rater reliability of automated devices for measurement of toe systolic blood pressure and the toe brachial index. Blood Press. Monit. 20, 47–51 (2015b). doi:10.1097/MBP.0000000000000083 CrossRefPubMed

Tehan, P.E., Bray, A., Chuter, V.H.: Non-invasive vascular assessment in the foot with diabetes: sensitivity and specificity of the ankle brachial index, toe brachial index and continuous wave Doppler for detecting peripheral arterial disease. J. Diabetes Complicat. 30, 155–160 (2015). doi:10.1016/j.jdiacomp.2015.07.019 CrossRefPubMed

Wrobel, J.S., Armstrong, D.G.: Reliability and validity of current physical examination techniques of the foot and ankle. J. Am. Podiatr. Med. Assoc. 98, 197–206 (2008)CrossRefPubMed

Title: Intraclass correlation coefficients: clearing the air, extending some cautions, and making some requests
Author: Robert Trevethan
Publication date: 01-06-2017
Publisher: Springer US
Published in: Health Services and Outcomes Research Methodology / Issue 2/2017
Print ISSN: 1387-3741
Electronic ISSN: 1572-9400
DOI: https://doi.org/10.1007/s10742-016-0156-6

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Intraclass correlation coefficients: clearing the air, extending some cautions, and making some requests

Abstract

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Please log in to get access to this content