Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on medication adherence

LIVE: Thursday 27th June 2024, 18:00-19:30 (CEST)
Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.
ADVANCED SEARCH
Log in
Top
Knee Surgery, Sports Traumatology, Arthroscopy
Published in: Knee Surgery, Sports Traumatology, Arthroscopy 12/2023

Open Access 30-10-2023 | REVIEW

Improving the reliability of measurements in orthopaedics and sports medicine

Authors: Aleksandra Królikowska, Paweł Reichert, Jon Karlsson, Caroline Mouton, Roland Becker, Robert Prill

Published in: Knee Surgery, Sports Traumatology, Arthroscopy | Issue 12/2023

Login to get access

Abstract

A large space still exists for improving the measurements used in orthopaedics and sports medicine, especially as we face rapid technological progress in devices used for diagnostic or patient monitoring purposes. For a specific measure to be valuable and applicable in clinical practice, its reliability must be established. Reliability refers to the extent to which measurements can be replicated, and three types of reliability can be distinguished: inter-rater, intra-rater, and test–retest. The present article aims to provide insights into reliability as one of the most important and relevant properties of measurement tools. It covers essential knowledge about the methods used in orthopaedics and sports medicine for reliability studies. From design to interpretation, this article guides readers through the reliability study process. It addresses crucial issues such as the number of raters needed, sample size calculation, and breaks between particular trials. Different statistical methods and tests are presented for determining reliability depending on the type of gathered data, with particular attention to the commonly used intraclass correlation coefficient.
Literature
1.
Bartko JJ (1966) The intraclass correlation coefficient as a measure of reliability. Psychol Rep 19(1):3–11CrossRefPubMed
2.
Bartko JJ (1976) On various intraclass correlation reliability coefficients. Psychol Bull 83(5):762–765CrossRef
3.
Bland JM, Altman DG (1990) A note on the use of the intraclass correlation coefficient in the evaluation of agreement between two methods of measurement. Comput Biol Med 20(5):337–340CrossRefPubMed
4.
Bland JM, Altman DG (1986) Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1(8476):307–310CrossRefPubMed
5.
Brown BW Jr, Lucero RJ, Foss AB (1962) A situation where the Pearson correlation coefficient leads to erroneous assessment of reliability. J Clin Psychol 18:95–97CrossRefPubMed
6.
Bruton A, Conway JH, Holgate ST (2000) Reliability: what is it, and how is it measured? Physiotherapy 86(2):94–99CrossRef
7.
Bujang MA, Baharum N (2016) Sample size guideline for correlation analysis. World J Soc Sci Res 3(1):37–46CrossRef
8.
Bujang MA, Baharum N (2017) A simplified guide to determination of sample size requirements for estimating the value of intraclass correlation coefficient: a review. Arch Orofac Sci 12(1):1–11
9.
Cicchetti DV, Sparrow SA (1981) Developing criteria for establishing interrater reliability of specific items: applications to assessment of adaptive behavior. Am J Ment Defic 86(2):127–137PubMed
10.
Czamara A, Królikowska A, Szuba Ł, Widuchowski W, Kentel M (2015) Single- vs. double-bundle anterior cruciate ligament reconstruction: a new aspect of knee assessment during activities involving dynamic knee rotation. J Strength Cond Res 29(2):489–499CrossRefPubMed
11.
Daly L, Bourke GJ (2008) Interpretation and uses of medical statistics, Fifth Edition. Wiley-Blackwell
12.
Giraudeau B, Mary JY (2001) Planning a reproducibility study: how many subjects and how many replicates per subject for an expected width of the 95 per cent confidence interval of the intraclass correlation coefficient. Stat Med 20(21):3205–3214CrossRefPubMed
13.
Hays WL (1973) Statistics for the social sciences, 2nd Edition. Holt, Rinehart & Winston of Canada, Vancouver
14.
15.
Hunt RJ (1986) Percent agreement, Pearson’s correlation, and kappa as measures of inter-examiner reliability. J Dent Res 65(2):128–130CrossRefPubMed
16.
Karanicolas PJ, Bhandari M, Kreder H, Moroni A, Richardson M, Walter SD et al (2009) Evaluating agreement: conducting a reliability study. J Bone Joint Surg Am 91(Suppl 3):99–106PubMed
17.
Koo TK, Li MY (2016) A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 15(2):155–163CrossRefPubMedPubMedCentral
18.
Królikowska A, Czamara A, Szuba Ł, Reichert P (2018) The effect of longer versus shorter duration of supervised physiotherapy after ACL reconstruction on the vertical jump landing limb symmetry. Biomed Res Int 2018:7519467CrossRefPubMedPubMedCentral
19.
Królikowska A, Maj A, Dejnek M, Prill R, Skotowska-Machaj A, Kołcz A (2023) Wrist motion assessment using Microsoft Azure Kinect DK: a reliability study in healthy individuals. Adv Clin Exp Med 32(2):203–209PubMed
20.
Królikowska A, Mika A, Plaskota B, Daszkiewicz M, Kentel M, Kołcz A et al (2022) Reliability and validity of the athletic shoulder (ASH) test performed using portable isometric-based strength training device. Biology (Basel) 11(4):577PubMed
21.
Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33(1):159–174CrossRefPubMed
22.
Lee KM, Lee J, Chung CY, Ahn S, Sung KH, Kim TW et al (2012) Pitfalls and important issues in testing reliability using intraclass correlation coefficients in orthopaedic research. Clin Orthop Surg 4(2):149–155CrossRefPubMedPubMedCentral
23.
Lienert GA, Raatz U (1998) Testaufbau und Testanalyse. PsychologieVerlagsUnion. Beltz, Weinheim
24.
McGraw K, Wong SP (1996) Forming inferences about some intraclass correlation coefficients. Psychol Methods 1(1):30–46CrossRef
25.
Razali N, Wah Y (2011) Power comparisons of Shapiro–Wilk, Kolmogorov–Smirnov, Lilliefors and Anderson–Darling tests. J Stat Model Anal 2(1):21–33
26.
Peacock J, Peacock P (2011) Oxford handbook of medical statistics. Oxford University Press, Oxford
27.
Peacock JL, Kerry SM, Balise RR (2017) Presenting medical statistics: from proposal to publication. Oxford University Press, OxfordCrossRef
28.
Portney LG, Watkins MP (2009) Foundations of clinical research: applications to practice, 3rd edn. Prentice Hall, Hoboken
29.
Prill R, Królikowska A, Becker R, Karlsson J (2023) Why there is a need to improve evaluation standards for clinical studies in orthopaedic and sports medicine. Knee Surg Sports Traumatol Arthrosc 31(1):4–5CrossRefPubMed
30.
Prill R, Królikowska A, de Girolamo L, Becker R, Karlsson J (2023) Checklists, risk of bias tools, and reporting guidelines for research in orthopedics, sports medicine, and rehabilitation. Knee Surg Sports Traumatol Arthrosc 31(8):3029–3033CrossRefPubMed
31.
Schlüter IM, Prill R, Królikowska A, Cruysen C, Becker R (2022) A pilot study on the reliability of ultrasound-based assessment of patella diameter and sulcus angle. Diagnost (Basel) 12(12):3164CrossRef
32.
Shrout PE, Fleiss JL (1979) Intraclass correlations: uses in assessing rater reliability. Psychol Bull 86(2):420–428CrossRefPubMed
33.
Sim J, Wright CC (2005) The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther 85(3):257–268CrossRefPubMed
34.
Stengel D, Bhandari M, Hanson B (2009) Statistics and data management. AOTrauma—statistics and data management. Georg Thieme, Stuttgart
35.
Walter SD, Eliasziw M, Donner A (1998) Sample size and optimal designs for reliability studies. Stat Med 17(1):101–110CrossRefPubMed
36.
Wise KL, Kelly BJ, Knudsen ML, Macalena JA (2019) Reliability studies and surveys. In: Musahl V, Karlsson J, Hirschmann MT, Ayeni OR, Marx RG, Koh JL, Nakamura N (eds) Basic methods handbook for clinical orthopaedic research: a practical guide and case based research approach. Springer, Berlin, pp 343–358CrossRef
Metadata
Title
Improving the reliability of measurements in orthopaedics and sports medicine
Authors
Aleksandra Królikowska
Paweł Reichert
Jon Karlsson
Caroline Mouton
Roland Becker
Robert Prill
Publication date
30-10-2023
Publisher
Springer Berlin Heidelberg
Published in
Knee Surgery, Sports Traumatology, Arthroscopy / Issue 12/2023
Print ISSN: 0942-2056
Electronic ISSN: 1433-7347
DOI
https://doi.org/10.1007/s00167-023-07635-1

Other articles of this Issue 12/2023

Knee Surgery, Sports Traumatology, Arthroscopy 12/2023 Go to the issue

KNEE

Near complete quadriceps tendon healing 2 years following harvest in anterior cruciate ligament reconstruction

Correction

Correction to: Disruption of joint arthroplasty services in Europe during the COVID-19 pandemic: an online survey within the European Hip Society (EHS) and the European Knee Associates (EKA)

KNEE

The apex of the deep cartilage is a stable landmark to position the femoral tunnel during remnant-preserving anterior cruciate ligament reconstruction

KNEE

Graft failure within 2 years of isolated anterior cruciate ligament reconstruction is associated with increased risk of secondary meniscus tears

HIP

The impact of surgical randomised controlled trials on the management of FAI syndrome: a citation analysis

KNEE

Distal femoral torsion is correlated with higher-grade trochlear dysplasia and shorter anterior condyles in patients with patellar dislocation and increased femoral torsion