Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
Clinical Orthopaedics and Related Research®
Published in: Clinical Orthopaedics and Related Research® 1/2016

01-01-2016 | Basic Research

Are Patient-reported Outcome Measures in Orthopaedics Easily Read by Patients?

Authors: Ibraheim El-Daly, MBBS, MRCS, PgCert, Hajir Ibraheim, MBBS, BSc, MRCP, Karthig Rajakulendran, MBBS, BSc, MRCS, Paul Culpan, MBBS, BSc, MRCS, FRCS (Tr & Orth), Peter Bates, MBBS, BSc, MRCS, FRCS (Tr & Orth)

Published in: Clinical Orthopaedics and Related Research® | Issue 1/2016

Login to get access

Abstract

Background

Patient-reported outcome measures (PROMs) are commonly used by healthcare providers as means of assessing health-related quality of life and function at any given time. The complexity of PROMs can differ and when combined with varying degrees of adult literacy, error can be introduced if patients fail to understand questions. With an average adult literacy level of 11-year-old students in the United Kingdom, it is unclear to what degree PROMs can be read and understood by most patients (readability); to our knowledge, this has not been evaluated.

Questions/purposes

We wished to determine the readability of commonly used PROMs in orthopaedic surgery, as assessed by a validated tool that measures the complexity of the language in these surveys.

Methods

We performed a MEDLINE search to identify the most-commonly reported PROMs in orthopaedic research. One hundred twenty-one PROMs were identified and reviewed by 19 attending orthopaedic surgeons at our institution. Fifty-nine were selected as the most commonly used in our department. Of these, 52 (78%) were disease specific and included: 12 (20%) knee, 10 (17%) shoulder, seven (12%) spine, six (10%) hip, five (8%) foot and ankle, four (7%) elbow, three (5%) pelvis, three (5%) hand and wrist, and two (3%) lower limb. The remaining seven (12%) PROMs were general health questionnaires. The Flesch Reading Ease Score is a validated readability tool measuring average sentence length and syllables per word. It is expressed on a scale from 0 to 100 with higher scores indicating easier reading. We extracted the text from each PROM and inserted it in the same online Flesch Reading Ease Score calculator to generate a score.

Results

The mean readability score was 55 (range, 0–93), corresponding to text best understood by 16- to 18-year-old students (11th–12th grades). Twenty-nine PROMs (49%) scored less than 60, classifying them as at least fairly difficult to read. Eight (14%) scored less than 30, best understood by university graduates. Only seven of 59 PROMs analyzed scored greater than 79, corresponding to text that can be understood by the average UK adult.

Conclusions

The majority of PROMs analyzed are written at a level that is incomprehensible to the average UK adult.

Clinical Relevance

This issue needs to be addressed if we are to continue basing our research conclusions on outcome scores. The information obtained is useful for patients to understand their musculoskeletal health, governmental agencies allocating healthcare resources, provision of management guidelines, and as a link to other data sets, such as hospital episodes statistics. Accurate and reliable data can be obtained only if patients who complete these evaluations are able to read and understand the questions asked.
Appendix
Available only for authorised users
Literature
1.
Agel J, LaPrade RF. Assessment of differences between the modified Cincinnati and International Knee Documentation Committee patient outcome scores: a prospective study. Am J Sports Med. 2009;2151–2157.
2.
Albright J, de Guzman C, Acebo, P, Paiva D, Faulkner M, Swanson J. Readability of patient education materials: implications for clinical practice. Appl Nurs Res. 1996;9:139–143.PubMedCrossRef
3.
Alvey J, Palmer S, Otter S. A comparison of the readability of two patient-reported outcome measures used to evaluate foot surgery. J Foot Ankle Surg. 2012;51:412–414.PubMedCrossRef
4.
Amadio PC, Berquist TH, Smith DK, Ilstrup DM, Cooney WP 3rd, Linscheid RL. Scaphoid malunion. J Hand Surg Am. 1989;14:679–687.PubMedCrossRef
5.
American Medical Association. Outcomes Research Resource Guide 1997: A Survey of Current Activities. Chicago, IL: American Medical Association; 1997.
6.
7.
Baker DW, Wolf MS, Feinglass J, Thompson JA, Gazmararian JA, Huang J. Health literacy and mortality among elderly persons. Arch Intern Med. 2007;167:1503–1509.PubMedCrossRef
8.
Beaton DE, Katz JN, Fossel AH, Wright JG, Tarasuk V. Measuring the whole or the parts? Validity, reliability, and responsiveness of the disabilities of the arm, shoulder and hand outcome measure in different regions of the upper extremity. J Hand Ther. 2001;14:128–146.PubMedCrossRef
9.
Beaton DE, Wright JG, Katz JN. Development of the Quick-DASH: comparison of three item-reduction approaches. J Bone Joint Surg Am. 2005;87:1038–1046.PubMedCrossRef
10.
Beckman HB, Markakis KM, Suchman AL, Frankel RM. The doctor-patient relationship and malpractice: lessons from plaintiff depositions. Arch Intern Med. 1994;154:1365–1370.PubMedCrossRef
11.
Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Crotty K. Low health literacy and health outcomes: an updated systematic review. Ann Intern Med. 2011;155:97–107.PubMedCrossRef
12.
Binkley JM, Stratford PW, Lott SA, Riddle DL. The Lower Extremity Functional Scale (LEFS): scale development, measurement properties, and clinical application. North American Orthopaedic Rehabilitation Research Network. Phys Ther. 1999;79:371–383.PubMed
13.
Bolton JE, Breen AC. The Bournemouth Questionnaire: a short-form comprehensive outcome measure: I. Psychometric properties in back pain patients. J Manipulative Physiol Ther. 1999;22:503–510.PubMedCrossRef
14.
Bolton JE, Humphreys BK. The Bournemouth Questionnaire: a short-form comprehensive outcome measure: II. Psychometric properties in neck pain patients. J Manipulative Physiol Ther. 2002;25:141–148.PubMedCrossRef
15.
Budiman-Mak E, Conrad KJ, Roach KE. The Foot Function Index: a measure of foot pain and disability. J Clin Epidemiol. 1991;44:561–570.PubMedCrossRef
16.
17.
18.
Chew LD, Bradley KA, Boyko EJ. Brief questions to identify patients with inadequate health literacy. Fam Med. 2004;36:588–594.PubMed
19.
Chung KC, Pillsbury MS, Walters MR, Hayward RA. Reliability and validity testing of the Michigan Hand Outcomes Questionnaire. J Hand Surg Am. 1998;575–587.
20.
Clancy C, Collins FS. Patient-Centered Outcomes Research Institute: the intersection of science and health care. Sci Transl Med. 2010;2:37cm18.
21.
Cotugna N, Vickery CE, Carpenter-Haefele KM. Evaluation of literacy level of patient education pages in health-related journals. J Commun Health. 2005;30:213–219.CrossRef
22.
Davis AM, Perruccio AV, Canizares M, Tennant A, Hawker GA, Conaghan PG, Roos EM, Jordan JM, Maillefert JF, Dougados M, Lohmander LS. The development of a short measure of physical function for hip OA HOOS-Physical Function Shortform (HOOS-PS): an OARSI/OMERACT initiative. Osteoarthritis Cartilage. 2008;16:551–559.PubMedCrossRef
23.
Davis TC, Mayeaux EJ, Frederickson D, Bocchini JA Jr, Jackson RH, Murphy PW. Reading ability of parents compared with reading level of pediatric patient education materials. Pediatrics. 1994;93:460–468.PubMed
24.
Dawson J, Doll H, Boller I, Fitzpatrick R, Little C, Rees J, Jenkinson C, Carr AJ. The development and validation of a patient-reported questionnaire to assess outcomes of elbow surgery. J Bone Joint Surg Br. 2008;90:466–473.PubMedCrossRef
25.
Dawson J, Fitzpatrick R, Carr A. Questionnaire on the perceptions of patients about shoulder surgery. J Bone Joint Surg Br. 1996;78:593–600.PubMed
26.
Dawson J, Fitzpatrick R, Carr A. The assessment of shoulder instability. J Bone Joint Surg Br. 1999;81:420–426.PubMedCrossRef
27.
Dawson J, Fitzpatrick R, Carr A, Murray D. Questionnaire on the perceptions of patients about total hip replacement. J Bone Joint Surg Br. 1996;78:185–190.PubMed
28.
Dawson J, Fitzpatrick R, Murray D, Carr A. Questionnaire on the perceptions of patients about total knee replacement. J Bone Joint Surg Br. 1998;80:63–69.PubMedCrossRef
29.
30.
31.
32.
Eltorai AE, Sharma P, Wang J, Daniels AH. Most American Academy of Orthopaedic Surgeons’ online patient education material exceeds average patient reading level. Clin Orthop Relat Res. 2015;473:1181–1186.PubMedCrossRef
33.
Estrada CA, Hryniewicz MM, Higgs VB, Collins C, Byrd JC. Anticoagulant patient information material is written at high readability levels. Stroke. 2000;31:2966–2970.PubMedCrossRef
34.
EuroQol Group. EuroQol: a new facility for the measurement of health-related quality-of-life. Health Policy. 1990;16:199–208.CrossRef
35.
Fairbank JC, Couper J, Davies JB, O’Brien JP. The Oswestry low back pain disability questionnaire. Physiotherapy. 1980;66:271–273.PubMed
36.
Flandry F, Hunt JP, Terry GC, Hughston JC. Analysis of subjective knee complaints using visual analog scales. Am J Sports Med. 1991;19:112–118.PubMedCrossRef
37.
38.
Friedman DB, Hoffman-Goetz L. A systematic review of readability and comprehension instruments used for print and web-based cancer information. Health Educ Behav. 2006;33:352–373.PubMedCrossRef
39.
Glazer HR, Kirk LM, Bosler FE. Patient education pamphlets about prevention, detection, and treatment of breast cancer for low literacy women. Patient Educ Couns. 1996;27:185–189.PubMedCrossRef
40.
41.
Griffin J, McKenna K, Tooth L. Discrepancy between older clients’ ability to read and comprehend and the reading level of written educational materials used by occupational therapists. Am J Occup Ther. 2006;60:70–80.PubMedCrossRef
42.
Hale S, Hertel J. Reliability and sensitivity of the Foot and Ankle Disability Index in subjects with chronic ankle instability. J Athl Train. 2005;40:35–40.PubMedPubMedCentral
43.
44.
Higgins LD, Taylor MK, Park D, Ghodadra N, Marchant M, Pietrobon R, Cook C; International Knee Documentation Committee. Reliability and validity of the International Knee Documentation Committee (IKDC) Subjective Knee Form. Joint Bone Spine. 2007;74:594–599.PubMedCrossRef
45.
46.
Hochhauser M. The informed consent form: document development and evaluation. Drug Information Journal. 2000;34:1309–1317.
47.
Jester A, Harth A, Germann G. Measuring levels of upper-extremity disability in employed adults using the DASH Questionnaire. J Hand Surg Am. 2005;30:1074.e1–1074.e10.
48.
Johanson NA, Liang MH, Daltroy L, Rudicel S, Richmond J. American Academy of Orthopaedic Surgeons lower limb outcomes assessment instruments: reliability, validity, and sensitivity to change. J Bone Joint Surg Am. 2004;86:902–909.PubMed
49.
Jordan A, Manniche C, Mosdal C, Hindsberger C. The Copenhagen Neck Functional Disability Scale: a study of reliability and validity. J Manipulative Physiol Ther. 1998;21:520–527.PubMed
50.
Kaikkonen A, Kannus P, Jarvinen M. A performance test protocol and scoring scale for the evaluation of ankle injuries. Am J Sports Med. 1994;22:462–469.PubMedCrossRef
51.
Kane RL. Introduction. Understanding Health Care Outcomes Research. 2nd ed. Sudbury, MA: Jones and Bartlett Publishers; 2006:3–22.
52.
Katz MG, Jacobson TA, Veledar E, Kripalani S. Patient literacy and question-asking behavior in the medical encounter: a mixed-methods analysis. J Gen Intern Med. 2007;22:782–786.PubMedPubMedCentralCrossRef
53.
54.
King GJ, Richards RR, Zuckerman JD, Blasier R, Dillman C, Friedman RJ, Gartsman GM, Iannotti JP, Murnahan JP, Mow VC, Woo SL. A standardized method for assessment of elbow function: Research Committee, American Shoulder and Elbow Surgeons. J Shoulder Elbow Surg. 1999;8:351–354.PubMedCrossRef
55.
Kirkley A, Griffin S, Alvarez C. The development and evaluation of a disease-specific quality of life measurement tool for rotator cuff disease: The Western Ontario Rotator Cuff Index (WORC). Clin J Sport Med. 2003;13:84–92.PubMedCrossRef
56.
Kirkley A, Griffin S, McLintock H, Ng L. The development and evaluation of a disease-specific quality of life measurement tool for shoulder instability: The Western Ontario Shoulder instability Index (WOSI). Am J Sports Med. 1998;26:764–772.PubMed
57.
Kirkley A, Griffin S, Whelan D. The development and validation of a quality of life-measurement tool for patients with meniscal pathology: the Western Ontario Meniscal Evaluation Tool (WOMET). Clin J Sport Med. 2007;17:349–356.PubMedCrossRef
58.
Kirshner B, Guyatt G. A methodological framework for assessing health indices. J Chron Dis. 1985;38:27–36.PubMedCrossRef
59.
Klassbo M, Larsson E, Mannevik E. Hip disability and osteoarthritis outcome score: an extension of the Western Ontario and McMaster Universities Osteoarthritis Index. Scand J Rheumatol. 2003;32:46–51.PubMedCrossRef
60.
Kopec JA, Esdaile JM, Abrahamowicz M, Abenhaim L, Wood-Dauphinee S, Lamping DL, Williams JI. The Quebec Back Pain Disability Scale: measurement properties. Spine (Phila Pa 1976). 1995;20:341–352.
61.
Kuhn JE. The assessment of outcomes for the treatment of the overhead athlete. In: Krishnan SG, Hawkins RJ, Warren RF, eds. The Shoulder and the Overhead Athlete. Philadelphia, PA: Lippincott-Williams & Wilkins; 2004;33–48.
62.
Kujala UM, Jaakkola LH, Koskinen SK, Taimela S, Hurme M, Nelimarkka O. Scoring of patellofemoral disorders. Arthroscopy. 1993;9:159–163.PubMedCrossRef
63.
64.
Lawlis GF, Cuencas R, Selby D, McCoy CE. The development of Dallas pain questionnaire: an assessment of the impact of spinal pain on behavior. Spine (Phila Pa 1976). 1989;14:515–516.
65.
Lequesne MG, Mery C, Samson M, Gerard P. Indexes of severity for osteoarthritis of the hip and knee: validation: value in comparison with other assessment tests. Scand J Rheumatol Suppl. 1987;65:85–89.PubMedCrossRef
66.
Lysholm J, Gillquist J. Evaluation of knee ligament surgery results with special emphasis on use of a scoring scale. Am J Sports Med. 1982;10:150–154.PubMedCrossRef
67.
MacDermid JC. Development of a scale for patient rating of wrist pain and disability. J Hand Ther. 1996;9:178–183.PubMedCrossRef
68.
Majeed SA. Grading the outcome of pelvic fractures. J Bone Joint Surg Br. 1989;71: 304–306.PubMed
69.
70.
McLaughlin GH. SMOG grading: a new readability formula. J Reading. 1969;12:639–646.
71.
Melzack R. The McGill Pain Questionnaire: major properties and scoring methods. Pain. 1975;1:277–299.PubMedCrossRef
72.
73.
Mohtadi N. Development and validation of the quality of life outcome measure (questionnaire) for chronic anterior cruciate ligament deficiency. Am J Sports Med. 1998;26:350–359.PubMed
74.
Morrey BF, Sanchez-Sotelo J, eds. The Elbow and its Disorders. 2nd ed. Philadelphia, PA: WB Saunders; 1993.
75.
Morris NS, MacLean CD, Chew LD, Littenberg B. The Single Item Literacy Screener: evaluation of a brief instrument to identify limited reading ability. BMC Fam Prac. 2006;7:21.CrossRef
76.
77.
78.
79.
Nielsen-Bohlman L, Panzer AM, Kindig DA; Committee on Health Literacy. What is health literacy. Health Literacy: A Prescription to End Confusion. Washington, DC: National Academies Press; 2004:31–58.
80.
Parikh NS, Parker RM, Nurss JR, Baker DW, Williams MV. Shame and health literacy: the unspoken connection. Patient Educ Couns. 1996;27:33-39.PubMedCrossRef
81.
Patsopoulos NA, Analatos AA, Ioannidis JP. Relative citation impact of various study designs in the health sciences. JAMA. 2005;293:2362–2366.PubMedCrossRef
82.
Pincus T, Summey JA, Soraci SA Jr, Wallston KA, Hummon NP. Assessment of patient satisfaction in activities of daily living using a modified Stanford Health Assessment Questionnaire. Arthritis Rheum. 1983;26:1346–53.PubMedCrossRef
83.
84.
85.
Roach KE, Budiman-Mak E, Sangsiridej N, Lertrantanakul Y. Development of a shoulder pain and disability index. Arthritis Care Res. 1991;4:143–149.PubMedCrossRef
86.
Roddey TS, Olson SL, Cook KF, Gartsman GM, Hanten W. Comparisons of the University of California-Los Angeles Shoulder Scale and the Simple Shoulder Test with the shoulder pain and disability index: single-administration reliability and validity. Phys Ther. 2000;80:759–768.PubMed
87.
Roland M, Morris R. A study of the natural history of back pain. Part I: development of a reliable and sensitive measure of disability in low-back pain. Spine (Phila Pa 1976). 1983;8:141–144.
88.
Roos EM, Brandsson S, Karlsson J. Validation of the foot and ankle outcome score for ankle ligament reconstruction. Foot Ankle Int. 2001;22:788–794.PubMed
89.
Roos EM, Roos HP, Lohmander LS, Ekdahl C, Beynnon BD. Knee Injury and Osteoarthritis Outcome Score (KOOS): development of a self-administered outcome measure. J Orthop Sports Phys Ther. 1998;28:88–96.PubMedCrossRef
90.
Roos EM, Toksvig-Larsen S. Knee injury and Osteoarthritis Outcome Score (KOOS): validation and comparison to the WOMAC in total knee replacement. Health Qual Life Outcomes. 2003;1:17.PubMedPubMedCentralCrossRef
91.
Stockmeyer NO. Using Microsoft Word’s readability program. Michigan Bar J. 2009;88:46–47.
92.
Streiner DL, Norman GR, eds. Health Measurement Scales: A Practical Guide to Their Development and Use. New York, NY: Oxford University Press; 1995.
93.
Sudore RL, Yaffe K, Satterfield S, Harris TB, Mehta KM, Simonsick EM, Newman AB, Rosano C, Rooks R, Rubin SM, Ayonayon HN, Schillinger D. Limited literacy and mortality in the elderly: the health, aging, and body composition study. J Gen Intern Med. 2006;21:806–812.PubMedPubMedCentralCrossRef
94.
Suk M, Hanson BP, Norvell DC, Helfet DL. Musculoskeletal Outcomes Measures and Instruments. New York, NY: Thieme Medical Publishers; 2005.
95.
Swiontkowski MF, Engelberg R, Martin DP, Agel J. Short musculoskeletal function assessment questionnaire: validity, reliability, and responsiveness. J Bone Joint Surg Am. 1999;81:1245–1260.PubMedCrossRef
96.
Tegner Y, Lysholm J. Rating systems in the evaluation of knee ligament injuries. Clin Orthop Relat Res. 1985;198:43–49.PubMed
97.
Templeman D, Goulet J, Duwelius PJ, Olson S, Davidson M. Internal fixation of displaced fractures of the sacrum. Clin Orthop Relat Res. 1996;329:180–185.PubMedCrossRef
98.
Thorborg K, Hölmich P, Christensen R, Petersen J, Roos EM. The Copenhagen Hip and Groin Outcome Score (HAGOS): development and validation according to the COSMIN checklist. Br J Sports Med. 2011;45:478–491.PubMedCrossRef
99.
Tinker MA. Introduction. Legibility of Print. Ames, IA: Iowa State University Press; 1963:1–8.
100.
101.
Vernon H, Mior S. The Neck Disability Index: a study of reliability and validity. J Manipulative Physiol Ther. 1991;14:409–415.PubMed
102.
Wallace LS, Rogers E, Turner L, Keenum A, Weiss BD. Suitability of written supplemental materials available on the Internet for non-prescription medications. Am J Health System Pharm. 2006;63:71–78.CrossRef
103.
Wallace LS, Rogers ES, Roskos SE, Holiday DB, Weiss BD. Brief report: screening items to identify patients with limited health literacy skills. J Gen Intern Med. 2006;21:874–877.PubMedPubMedCentralCrossRef
104.
Wallace LS, Roskos SE, Weiss BD. Readability characteristics of consumer medication information for asthma inhalation devices. J Asthma. 2006;43:375–378.PubMedCrossRef
105.
Wallace LS, Turner LW, Ballard JE, Keenum AJ, Weiss BD. Evaluation of web-based osteoporosis educational materials. J Womens Health (Larchmt). 2005;14:936–945.CrossRef
106.
Walsh TM, Volsko TA. Readability assessment of internet-based consumer health information. Respir Care. 2008;53:1310–1315.PubMed
107.
Ware J Jr, Kosinski M, Keller SD. A 12-item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med Care. 1996;34:220–233.PubMedCrossRef
108.
Ware JE Jr, Sherbourne CD. The MOS 36-item short-form health survey (SF-36): I. Conceptual framework and item selection. Med Care. 1992;30:473–483.PubMedCrossRef
109.
Washburn RA, Smith KW, Jette AM, Janney CA. The Physical Activity Scale for the Elderly (PASE): development and evaluation. J Clin Epidemiol. 1993;46:153–162.PubMedCrossRef
110.
111.
Zahiri CA, Schmalzried TP, Szuszczewicz ES, Amstutz HC. Assessing activity in joint replacement patients. J Arthroplasty. 1998;13:890–895.PubMedCrossRef
Metadata
Title
Are Patient-reported Outcome Measures in Orthopaedics Easily Read by Patients?
Authors
Ibraheim El-Daly, MBBS, MRCS, PgCert
Hajir Ibraheim, MBBS, BSc, MRCP
Karthig Rajakulendran, MBBS, BSc, MRCS
Paul Culpan, MBBS, BSc, MRCS, FRCS (Tr & Orth)
Peter Bates, MBBS, BSc, MRCS, FRCS (Tr & Orth)
Publication date
01-01-2016
Publisher
Springer US
Published in
Clinical Orthopaedics and Related Research® / Issue 1/2016
Print ISSN: 0009-921X
Electronic ISSN: 1528-1132
DOI
https://doi.org/10.1007/s11999-015-4595-0

Other articles of this Issue 1/2016

Clinical Orthopaedics and Related Research® 1/2016 Go to the issue

From Bench to Bedside

From Bench to Bedside: A Perfect Fit? Osseointegration Can Improve Function for Patients with Amputations

Symposium: 2015 Knee Society Proceedings

Time-driven Activity-based Costing More Accurately Reflects Costs in Arthroplasty Surgery

CORR Insights

CORR Insights®: Surface Damage Is Not Reduced With Highly Crosslinked Polyethylene Tibial Inserts at Short-term

Clinical Research

Is Embolization an Effective Treatment for Recurrent Hemorrhage After Hip or Knee Arthroplasty?

Symposium: 2015 Knee Society Proceedings

The Knee Society Short Form Reduces Respondent Burden in the Assessment of Patient-reported Outcomes

CORR Insights

CORR Insights®: Time-driven Activity-based Costing More Accurately Reflects Costs in Arthroplasty Surgery