Skip to main content
Top
Published in: BMC Musculoskeletal Disorders 1/2008

Open Access 01-12-2008 | Research article

Agreement, reliability and validity in 3 shoulder questionnaires in patients with rotator cuff disease

Authors: Ole M Ekeberg, Erik Bautz-Holter, Einar K Tveitå, Anne Keller, Niels G Juel, Jens I Brox

Published in: BMC Musculoskeletal Disorders | Issue 1/2008

Login to get access

Abstract

Background

Self-report questionnaires play an important role as outcome measures in shoulder research. Having an estimate of the measurement error of these questionnaires is of importance when assessing follow-up results after treatment and when planning intervention studies. The aim of this study was to cross-culturally adapt the Norwegian version of the OSS and WORC questionnaire and examine and compare agreement, reliability and construct validity of the disease-specific shoulder questionnaire WORC with two commonly used shoulder questionnaires, SPADI and OSS, in patients with rotator cuff disease.

Methods

74 patients with rotator cuff disease were recruited from the outpatient clinic of the Physical Medicine and Rehabilitation Department at Ullevaal University Hospital in Oslo, Norway. A test-retest design was used, and the questionnaires were filled out by the patients at the clinic, with a one week interval between test administrations. Agreement (repeatability coefficient), reliability (ICC) and construct validity were examined and compared for WORC, SPADI and OSS.

Results

Reliability analysis was restricted to the 55 patients (51 ± 10 yrs) who reported no change between test administrations according to scoring on a global scale. The agreement, reliability and construct validity was moderate for all three questionnaires with ICC ranging from 0.83 to 0.85, repeatability coefficient from 16.1 to 19.7 and Spearman rank correlations between total scores from r = 0.57 to 0.69. There was a lower degree of floor and ceiling effects in SPADI compared to WORC and OSS.

Conclusion

We conclude that the agreement and reliability of the three shoulder questionnaires examined, WORC index, SPADI and OSS are acceptable and that differences between scores were small. The Norwegian version of the questionnaires is acceptable for assessing Norwegian-speaking patients with rotator cuff disease. The moderate agreement and construct validity should be taken into consideration when assessing follow-up results after treatment and in the planning of prospective studies.
Appendix
Available only for authorised users
Literature
1.
go back to reference Green S, Buchbinder R, Glazier R, Forbes A: Interventions for shoulder pain. Cochrane Database Syst Rev. 2000, CD001156-2 Green S, Buchbinder R, Glazier R, Forbes A: Interventions for shoulder pain. Cochrane Database Syst Rev. 2000, CD001156-2
2.
go back to reference Bot SD, Terwee CB, Windt van der DA, Bouter LM, Dekker J, de Vet HC: Clinimetric evaluation of shoulder disability questionnaires: a systematic review of the literature. Annals of the Rheumatic Diseases. 2004, 63 (4): 335-341. 10.1136/ard.2003.007724.CrossRefPubMedPubMedCentral Bot SD, Terwee CB, Windt van der DA, Bouter LM, Dekker J, de Vet HC: Clinimetric evaluation of shoulder disability questionnaires: a systematic review of the literature. Annals of the Rheumatic Diseases. 2004, 63 (4): 335-341. 10.1136/ard.2003.007724.CrossRefPubMedPubMedCentral
3.
go back to reference Kirkley A, Alvarez C, Griffin S: The development and evaluation of a disease-specific quality-of-life questionnaire for disorders of the rotator cuff: The Western Ontario Rotator Cuff Index. Clin J Sport Med. 2003, 13 (2): 84-92. 10.1097/00042752-200303000-00004.CrossRefPubMed Kirkley A, Alvarez C, Griffin S: The development and evaluation of a disease-specific quality-of-life questionnaire for disorders of the rotator cuff: The Western Ontario Rotator Cuff Index. Clin J Sport Med. 2003, 13 (2): 84-92. 10.1097/00042752-200303000-00004.CrossRefPubMed
4.
go back to reference Patrick DL, Deyo RA: Generic and disease-specific measures in assessing health status and quality of life. Medical Care. 1989, 27 (3 Suppl): S217-S232. 10.1097/00005650-198903001-00018.CrossRefPubMed Patrick DL, Deyo RA: Generic and disease-specific measures in assessing health status and quality of life. Medical Care. 1989, 27 (3 Suppl): S217-S232. 10.1097/00005650-198903001-00018.CrossRefPubMed
5.
go back to reference Hollinshead RM, Mohtadi NG, Guchte Vande RA, Wadey VM: Two 6-year follow-up studies of large and massive rotator cuff tears: comparison of outcome measures. J Shoulder Elbow Surg. 2000, 9 (5): 373-381. 10.1067/mse.2000.108389.CrossRefPubMed Hollinshead RM, Mohtadi NG, Guchte Vande RA, Wadey VM: Two 6-year follow-up studies of large and massive rotator cuff tears: comparison of outcome measures. J Shoulder Elbow Surg. 2000, 9 (5): 373-381. 10.1067/mse.2000.108389.CrossRefPubMed
6.
go back to reference Holtby R, Razmjou H: Measurement properties of the Western Ontario rotator cuff outcome measure: a preliminary report. J Shoulder Elbow Surg. 2005, 14 (5): 506-510. 10.1016/j.jse.2005.02.017.CrossRefPubMed Holtby R, Razmjou H: Measurement properties of the Western Ontario rotator cuff outcome measure: a preliminary report. J Shoulder Elbow Surg. 2005, 14 (5): 506-510. 10.1016/j.jse.2005.02.017.CrossRefPubMed
7.
go back to reference Getahun TY, MacDermid JC, Patterson SD: Concurrent validity of patient rating scales in assessment of outcome after rotator cuff repair. Journal of Musculoskeletal Research. 2000, 4 (2): 119-127. 10.1142/S021895770000015X.CrossRef Getahun TY, MacDermid JC, Patterson SD: Concurrent validity of patient rating scales in assessment of outcome after rotator cuff repair. Journal of Musculoskeletal Research. 2000, 4 (2): 119-127. 10.1142/S021895770000015X.CrossRef
8.
go back to reference MacDermid JC, Drosdowech D, Faber K: Responsiveness of self-report scales in patients recovering from rotator cuff surgery. J Shoulder Elbow Surg. 2006, 15 (4): 407-414. 10.1016/j.jse.2005.09.005.CrossRefPubMed MacDermid JC, Drosdowech D, Faber K: Responsiveness of self-report scales in patients recovering from rotator cuff surgery. J Shoulder Elbow Surg. 2006, 15 (4): 407-414. 10.1016/j.jse.2005.09.005.CrossRefPubMed
9.
go back to reference Razmjou H, Bean A, van OV, MacDermid JC, Holtby R: Cross-sectional and longitudinal construct validity of two rotator cuff disease-specific outcome measures. BMC Musculoskelet Disord. 2006, 7: 26-10.1186/1471-2474-7-26.CrossRefPubMedPubMedCentral Razmjou H, Bean A, van OV, MacDermid JC, Holtby R: Cross-sectional and longitudinal construct validity of two rotator cuff disease-specific outcome measures. BMC Musculoskelet Disord. 2006, 7: 26-10.1186/1471-2474-7-26.CrossRefPubMedPubMedCentral
10.
go back to reference Diehr P, Chen L, Patrick D, Feng Z, Yasui Y: Reliability, effect size, and responsiveness of health status measures in the design of randomized and cluster-randomized trials. Contemp Clin Trials. 2005, 26 (1): 45-58. 10.1016/j.cct.2004.11.014.CrossRefPubMed Diehr P, Chen L, Patrick D, Feng Z, Yasui Y: Reliability, effect size, and responsiveness of health status measures in the design of randomized and cluster-randomized trials. Contemp Clin Trials. 2005, 26 (1): 45-58. 10.1016/j.cct.2004.11.014.CrossRefPubMed
11.
go back to reference Lachin JM: The role of measurement reliability in clinical trials. Clin Trials. 2004, 1 (6): 553-566. 10.1191/1740774504cn057oa.CrossRefPubMed Lachin JM: The role of measurement reliability in clinical trials. Clin Trials. 2004, 1 (6): 553-566. 10.1191/1740774504cn057oa.CrossRefPubMed
12.
go back to reference Streiner DL, Norman GR: Health measurement scales. 2003, Oxford Medical Publication Streiner DL, Norman GR: Health measurement scales. 2003, Oxford Medical Publication
13.
go back to reference Weir JP: Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. J Strength Cond Res. 2005, 19 (1): 231-240. 10.1519/15184.1.PubMed Weir JP: Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. J Strength Cond Res. 2005, 19 (1): 231-240. 10.1519/15184.1.PubMed
14.
go back to reference de Vet HCW, Terwee CB, Knol DL, Bouter LM: When to use agreement versus reliability measures. Journal of Clinical Epidemiology. 2006, 59 (10): 1033-1039. 10.1016/j.jclinepi.2005.10.015.CrossRefPubMed de Vet HCW, Terwee CB, Knol DL, Bouter LM: When to use agreement versus reliability measures. Journal of Clinical Epidemiology. 2006, 59 (10): 1033-1039. 10.1016/j.jclinepi.2005.10.015.CrossRefPubMed
15.
go back to reference Beaton DE, Hogg-Johnson S, Bombardier C: Evaluating changes in health status: reliability and responsiveness of five generic health status measures in workers with musculoskeletal disorders. J Clin Epidemiol. 1997, 50 (1): 79-93. 10.1016/S0895-4356(96)00296-X.CrossRefPubMed Beaton DE, Hogg-Johnson S, Bombardier C: Evaluating changes in health status: reliability and responsiveness of five generic health status measures in workers with musculoskeletal disorders. J Clin Epidemiol. 1997, 50 (1): 79-93. 10.1016/S0895-4356(96)00296-X.CrossRefPubMed
16.
go back to reference Terwee CB, Bot SD, de Boer MR, Windt van der DA, Knol DL, Dekker J, Bouter LM, de Vet HC: Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007, 60 (1): 34-42. 10.1016/j.jclinepi.2006.03.012.CrossRefPubMed Terwee CB, Bot SD, de Boer MR, Windt van der DA, Knol DL, Dekker J, Bouter LM, de Vet HC: Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007, 60 (1): 34-42. 10.1016/j.jclinepi.2006.03.012.CrossRefPubMed
17.
go back to reference McHorney CA, Tarlov AR: Individual-patient monitoring in clinical practice: are available health status surveys adequate?. Qual Life Res. 1995, 4 (4): 293-307. 10.1007/BF01593882.CrossRefPubMed McHorney CA, Tarlov AR: Individual-patient monitoring in clinical practice: are available health status surveys adequate?. Qual Life Res. 1995, 4 (4): 293-307. 10.1007/BF01593882.CrossRefPubMed
18.
go back to reference Kirshner B, Guyatt G: A methodological framework for assessing health indices. Journal of chronic diseases. 1985, 38 (1): 27-36. 10.1016/0021-9681(85)90005-0.CrossRefPubMed Kirshner B, Guyatt G: A methodological framework for assessing health indices. Journal of chronic diseases. 1985, 38 (1): 27-36. 10.1016/0021-9681(85)90005-0.CrossRefPubMed
19.
go back to reference Dawson J, Fitzpatrick R, Carr A: A self-administered questionnaire for assessment of symptoms and function of the shoulder. J Bone Joint Surg Am. 1998, 80 (5): 766-767. 10.1302/0301-620X.80B5.8823.CrossRefPubMed Dawson J, Fitzpatrick R, Carr A: A self-administered questionnaire for assessment of symptoms and function of the shoulder. J Bone Joint Surg Am. 1998, 80 (5): 766-767. 10.1302/0301-620X.80B5.8823.CrossRefPubMed
20.
go back to reference Roach KE, Budiman-Mak E, Songsiridej N, Lertratanakul Y: Development of a shoulder pain and disability index. Arthritis Care Res. 1991, 4 (4): 143-149. 10.1002/art.1790040403.CrossRefPubMed Roach KE, Budiman-Mak E, Songsiridej N, Lertratanakul Y: Development of a shoulder pain and disability index. Arthritis Care Res. 1991, 4 (4): 143-149. 10.1002/art.1790040403.CrossRefPubMed
21.
go back to reference Guillemin F, Bombardier C, Beaton D: Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993, 46 (12): 1417-1432. 10.1016/0895-4356(93)90142-N.CrossRefPubMed Guillemin F, Bombardier C, Beaton D: Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993, 46 (12): 1417-1432. 10.1016/0895-4356(93)90142-N.CrossRefPubMed
22.
go back to reference Beaton DE, Bombardier C, Guillemin F, Ferraz MB: Guidelines for the process of cross-cultural adaptation of self-report measures. Spine. 2000, 25 (24): 3186-3191. 10.1097/00007632-200012150-00014.CrossRefPubMed Beaton DE, Bombardier C, Guillemin F, Ferraz MB: Guidelines for the process of cross-cultural adaptation of self-report measures. Spine. 2000, 25 (24): 3186-3191. 10.1097/00007632-200012150-00014.CrossRefPubMed
23.
go back to reference Hawkins RJ, Matsen FA: Clinical evaluation of shoulder problems. The Shoulder. 1998, WB Saunders, 188-189. Hawkins RJ, Matsen FA: Clinical evaluation of shoulder problems. The Shoulder. 1998, WB Saunders, 188-189.
24.
go back to reference Brox JI, Gjengedal E, Uppheim G, Bohmer AS, Brevik JI, Ljunggren AE, Staff PH: Arthroscopic surgery versus supervised exercises in patients with rotator cuff disease (stage II impingement syndrome): a prospective, randomized, controlled study in 125 patients with a 2 1/2-year follow-up. J Shoulder Elbow Surg. 1999, 8 (2): 102-111. 10.1016/S1058-2746(99)90001-0.CrossRefPubMed Brox JI, Gjengedal E, Uppheim G, Bohmer AS, Brevik JI, Ljunggren AE, Staff PH: Arthroscopic surgery versus supervised exercises in patients with rotator cuff disease (stage II impingement syndrome): a prospective, randomized, controlled study in 125 patients with a 2 1/2-year follow-up. J Shoulder Elbow Surg. 1999, 8 (2): 102-111. 10.1016/S1058-2746(99)90001-0.CrossRefPubMed
25.
go back to reference Altman DG: Practical statistics for medical research. 1999, Chapman & Hall Altman DG: Practical statistics for medical research. 1999, Chapman & Hall
26.
go back to reference Shrout PE, Fleiss JL: Intraclass Correlations – Uses in Assessing Rater Reliability. Psychological Bulletin. 1979, 86 (2): 420-428. 10.1037/0033-2909.86.2.420.CrossRefPubMed Shrout PE, Fleiss JL: Intraclass Correlations – Uses in Assessing Rater Reliability. Psychological Bulletin. 1979, 86 (2): 420-428. 10.1037/0033-2909.86.2.420.CrossRefPubMed
27.
go back to reference Fleiss JL, Cohen J: Equivalence of Weighted Kappa and Intraclass Correlation Coefficient As Measures of Reliability. Educational and Psychological Measurement. 1973, 33 (3): 613-619. 10.1177/001316447303300309.CrossRef Fleiss JL, Cohen J: Equivalence of Weighted Kappa and Intraclass Correlation Coefficient As Measures of Reliability. Educational and Psychological Measurement. 1973, 33 (3): 613-619. 10.1177/001316447303300309.CrossRef
29.
go back to reference Bland JM, Altman DG: Measuring agreement in method comparison studies. Stat Methods Med Res. 1999, 8 (2): 135-160. 10.1191/096228099673819272.CrossRefPubMed Bland JM, Altman DG: Measuring agreement in method comparison studies. Stat Methods Med Res. 1999, 8 (2): 135-160. 10.1191/096228099673819272.CrossRefPubMed
30.
go back to reference Schmitt J, Di Fabio RP: The validity of prospective and retrospective global change criterion measures. Arch Phys Med Rehabil. 2005, 86 (12): 2270-2276. 10.1016/j.apmr.2005.07.290.CrossRefPubMed Schmitt J, Di Fabio RP: The validity of prospective and retrospective global change criterion measures. Arch Phys Med Rehabil. 2005, 86 (12): 2270-2276. 10.1016/j.apmr.2005.07.290.CrossRefPubMed
31.
go back to reference Cook KF, Gartsman GM, Roddey TS, Olson SL: The measurement level and trait-specific reliability of 4 scales of shoulder functioning: an empiric investigation. Arch Phys Med Rehabil. 2001, 82 (11): 1558-1565. 10.1053/apmr.2001.26622.CrossRefPubMed Cook KF, Gartsman GM, Roddey TS, Olson SL: The measurement level and trait-specific reliability of 4 scales of shoulder functioning: an empiric investigation. Arch Phys Med Rehabil. 2001, 82 (11): 1558-1565. 10.1053/apmr.2001.26622.CrossRefPubMed
32.
go back to reference Merbitz C, Morris J, Grip JC: Ordinal scales and foundations of misinference. Arch Phys Med Rehabil. 1989, 70 (4): 308-312.PubMed Merbitz C, Morris J, Grip JC: Ordinal scales and foundations of misinference. Arch Phys Med Rehabil. 1989, 70 (4): 308-312.PubMed
33.
go back to reference Schmitt JS, Di Fabio RP: Reliable change and minimum important difference (MID) proportions facilitated group responsiveness comparisons using individual threshold criteria. J Clin Epidemiol. 2004, 57 (10): 1008-1018. 10.1016/j.jclinepi.2004.02.007.CrossRefPubMed Schmitt JS, Di Fabio RP: Reliable change and minimum important difference (MID) proportions facilitated group responsiveness comparisons using individual threshold criteria. J Clin Epidemiol. 2004, 57 (10): 1008-1018. 10.1016/j.jclinepi.2004.02.007.CrossRefPubMed
34.
go back to reference Angst F, Goldhahn J, Pap G, Mannion AF, Roach KE, Siebertz D, Drerup S, Schwyzer HK, Simmen BR: Cross-cultural adaptation, reliability and validity of the German Shoulder Pain and Disability Index (SPADI). Rheumatology (Oxford, England). 2007, 46 (1): 87-92. 10.1093/rheumatology/kel040.CrossRef Angst F, Goldhahn J, Pap G, Mannion AF, Roach KE, Siebertz D, Drerup S, Schwyzer HK, Simmen BR: Cross-cultural adaptation, reliability and validity of the German Shoulder Pain and Disability Index (SPADI). Rheumatology (Oxford, England). 2007, 46 (1): 87-92. 10.1093/rheumatology/kel040.CrossRef
35.
go back to reference Cloke DJ, Lynn SE, Watson H, Steen IN, Purdy S, Williams JR: A comparison of functional, patient-based scores in subacromial impingement. J Shoulder Elbow Surg. 2005, 14 (4): 380-384. 10.1016/j.jse.2004.08.008.CrossRefPubMed Cloke DJ, Lynn SE, Watson H, Steen IN, Purdy S, Williams JR: A comparison of functional, patient-based scores in subacromial impingement. J Shoulder Elbow Surg. 2005, 14 (4): 380-384. 10.1016/j.jse.2004.08.008.CrossRefPubMed
36.
go back to reference Beaton D, Richards RR: Assessing the reliability and responsiveness of 5 shoulder questionnaires. J Shoulder Elbow Surg. 1998, 7 (6): 565-572. 10.1016/S1058-2746(98)90002-7.CrossRefPubMed Beaton D, Richards RR: Assessing the reliability and responsiveness of 5 shoulder questionnaires. J Shoulder Elbow Surg. 1998, 7 (6): 565-572. 10.1016/S1058-2746(98)90002-7.CrossRefPubMed
Metadata
Title
Agreement, reliability and validity in 3 shoulder questionnaires in patients with rotator cuff disease
Authors
Ole M Ekeberg
Erik Bautz-Holter
Einar K Tveitå
Anne Keller
Niels G Juel
Jens I Brox
Publication date
01-12-2008
Publisher
BioMed Central
Published in
BMC Musculoskeletal Disorders / Issue 1/2008
Electronic ISSN: 1471-2474
DOI
https://doi.org/10.1186/1471-2474-9-68

Other articles of this Issue 1/2008

BMC Musculoskeletal Disorders 1/2008 Go to the issue