Skip to main content
Top
Published in: Clinical Orthopaedics and Related Research® 6/2011

01-06-2011 | Clinical Research

Can the Toronto Extremity Salvage Score Produce Reliable Results When Used Online?

Authors: Mark Clayer, MD, Aileen Davis, PhD

Published in: Clinical Orthopaedics and Related Research® | Issue 6/2011

Login to get access

Abstract

Background

Web-based questionnaires have become popular, however, access to the Internet can be biased regarding age, gender, and education, among other factors. Therefore, it is unknown whether this is a reasonable avenue to administer a questionnaire to patients or whether Web-based can be a reliable alternative to paper-based.

Questions/purposes

We determined whether the Internet version of the Toronto Extremity Salvage Score is reproducible compared with the paper-based version and the compliance and completion rates.

Patients and Methods

The study population consisted of 81 adults who had had surgery for a musculoskeletal tumor of the lower extremity more than 12 months earlier. The Toronto Extremity Salvage Score was administered by paper at a baseline interview and then readministered via Internet 7 to 14 days later to those with access.

Results

Sixty of the 81 patients (74%) were able to use the Internet. Increasing age and lower education levels were correlated with a lower likelihood of using the Internet. Questionnaires were done online and on paper by 56 patients but 10 were excluded because of self-reported change in circumstances. The mean TESS was 85.7 (range, 41.1–100; SD, 17.26) for the paper-based questionnaire and 85.2 (range, 42.5–100; SD, 17.47) for the Internet-based questionnaire. The intraclass correlation coefficient was 0.97.

Conclusions

The questionnaire can be transferred successfully to the Internet and can be used reliably instead of a paper-based instrument. Recruitment to use an Internet-based questionnaire is limited only by the percentage of patients able to access and use the Internet.
Literature
1.
go back to reference Athale N, Sturley A, Skoczen S, Kavanaugh A, Lenert L. A web-compatible instrument for measuring self-reported disease activity in arthritis. J Rheumatol. 2004;31:223–228.PubMed Athale N, Sturley A, Skoczen S, Kavanaugh A, Lenert L. A web-compatible instrument for measuring self-reported disease activity in arthritis. J Rheumatol. 2004;31:223–228.PubMed
2.
go back to reference Bälter KA, Bälter O, Fondell E, Lagerros YT. Web-based and mailed questionnaires: a comparison of response rates and compliance. Epidemiology. 2005;16:577–579.PubMedCrossRef Bälter KA, Bälter O, Fondell E, Lagerros YT. Web-based and mailed questionnaires: a comparison of response rates and compliance. Epidemiology. 2005;16:577–579.PubMedCrossRef
3.
go back to reference Bliven BD, Kaufman SE, Spertus JA. Electronic collection of health-related quality of life data: validity, time benefits, and patient preference. Qual Life Res. 2001;10:15–22.PubMedCrossRef Bliven BD, Kaufman SE, Spertus JA. Electronic collection of health-related quality of life data: validity, time benefits, and patient preference. Qual Life Res. 2001;10:15–22.PubMedCrossRef
4.
go back to reference Buchanan T, Smith JL. Using the Internet for psychological research: personality testing on the World Wide Web. Br J Psychol. 1999;90:125–144.PubMedCrossRef Buchanan T, Smith JL. Using the Internet for psychological research: personality testing on the World Wide Web. Br J Psychol. 1999;90:125–144.PubMedCrossRef
5.
go back to reference Bush N, Donaldson G, Moinpour C, Haberman M, Milliken D, Markle V, Lauson J. Development, feasibility and compliance of a web-based system for very frequent QOL and symptom home self-assessment after hematopoietic stem cell transplantation. Qual Life Res. 2005;14:77–93.PubMedCrossRef Bush N, Donaldson G, Moinpour C, Haberman M, Milliken D, Markle V, Lauson J. Development, feasibility and compliance of a web-based system for very frequent QOL and symptom home self-assessment after hematopoietic stem cell transplantation. Qual Life Res. 2005;14:77–93.PubMedCrossRef
6.
go back to reference Bushnell DM, Martin ML, Parasuraman B. Electronic versus paper questionnaires: a further comparison in persons with asthma. J Asthma. 2003;40:751–762.PubMedCrossRef Bushnell DM, Martin ML, Parasuraman B. Electronic versus paper questionnaires: a further comparison in persons with asthma. J Asthma. 2003;40:751–762.PubMedCrossRef
7.
go back to reference Cals JW, Boumans D, Lardinois RJ, Gonzales R, Hopstaken RM, Butler CC, Dinant GJ. Public beliefs on antibiotics and respiratory tract infections: an internet-based questionnaire study. Br J Gen Pract. 2007;57:942–947.PubMedCrossRef Cals JW, Boumans D, Lardinois RJ, Gonzales R, Hopstaken RM, Butler CC, Dinant GJ. Public beliefs on antibiotics and respiratory tract infections: an internet-based questionnaire study. Br J Gen Pract. 2007;57:942–947.PubMedCrossRef
8.
go back to reference Cantrell MA, Lupinacci P. Methodological issues in online data collection. J Adv Nurs. 2007;60:544–549.PubMedCrossRef Cantrell MA, Lupinacci P. Methodological issues in online data collection. J Adv Nurs. 2007;60:544–549.PubMedCrossRef
9.
go back to reference Davis AM, Bell RS, Badley EM, Yoshida K, Williams JI. Evaluating functional outcome in patients with lower extremity sarcoma. Clin Orthop Relat Res. 1999;358:90–100.PubMedCrossRef Davis AM, Bell RS, Badley EM, Yoshida K, Williams JI. Evaluating functional outcome in patients with lower extremity sarcoma. Clin Orthop Relat Res. 1999;358:90–100.PubMedCrossRef
10.
go back to reference Davis AM, Sennik S, Griffin AM, Wunder JS, O’Sullivan B, Catton CN, Bell RS. Predictors of functional outcomes following limb salvage surgery for lower-extremity soft tissue sarcoma. J Surg Oncol. 2000;73:206–211.PubMedCrossRef Davis AM, Sennik S, Griffin AM, Wunder JS, O’Sullivan B, Catton CN, Bell RS. Predictors of functional outcomes following limb salvage surgery for lower-extremity soft tissue sarcoma. J Surg Oncol. 2000;73:206–211.PubMedCrossRef
11.
go back to reference Davis AM, Wright JG, Williams JI, Bombardier C, Griffin A, Bell RS. Development of a measure of physical function for patients with bone and soft tissue sarcoma. Qual Life Res. 1996;5:508–516.PubMedCrossRef Davis AM, Wright JG, Williams JI, Bombardier C, Griffin A, Bell RS. Development of a measure of physical function for patients with bone and soft tissue sarcoma. Qual Life Res. 1996;5:508–516.PubMedCrossRef
12.
go back to reference Davis RN. Web-based administration of a personality questionnaire: comparison with traditional methods. Behav Res Methods Instrum Comput. 1999;31:572–577.PubMedCrossRef Davis RN. Web-based administration of a personality questionnaire: comparison with traditional methods. Behav Res Methods Instrum Comput. 1999;31:572–577.PubMedCrossRef
13.
go back to reference Edwards P, Roberts I, Clarke M, DiGuiseppi C, Pratap S, Wentz R, Kwan I. Increasing response rates to postal questionnaires: systematic review. BMJ. 2002;324:1183–1191.PubMedCrossRef Edwards P, Roberts I, Clarke M, DiGuiseppi C, Pratap S, Wentz R, Kwan I. Increasing response rates to postal questionnaires: systematic review. BMJ. 2002;324:1183–1191.PubMedCrossRef
14.
go back to reference Ekman A, Dickman PW, Klint A, Weiderpass E, Litton JE. Feasibility of using Web-based questionnaires in large population-based epidemiological studies. Eur J Epidemiol. 2006;21:103–111.PubMedCrossRef Ekman A, Dickman PW, Klint A, Weiderpass E, Litton JE. Feasibility of using Web-based questionnaires in large population-based epidemiological studies. Eur J Epidemiol. 2006;21:103–111.PubMedCrossRef
15.
go back to reference Ekman A, Klint A, Dickman PW, Adami HO, Litton JE. Optimizing the design of Web-based questionnaires: experience from a population-based study among 50,000 women. Eur J Epidemiol. 2007;22:293–300.PubMedCrossRef Ekman A, Klint A, Dickman PW, Adami HO, Litton JE. Optimizing the design of Web-based questionnaires: experience from a population-based study among 50,000 women. Eur J Epidemiol. 2007;22:293–300.PubMedCrossRef
16.
go back to reference Enneking W. Modification of the system for functional evaluation in the surgical management of musculoskeletal tumours. In: Enneking W, ed. Limb Salvage in Musculoskeletal Oncology. New York, NY: Churchill Livingstone; 1987;626–639. Enneking W. Modification of the system for functional evaluation in the surgical management of musculoskeletal tumours. In: Enneking W, ed. Limb Salvage in Musculoskeletal Oncology. New York, NY: Churchill Livingstone; 1987;626–639.
17.
go back to reference Enneking WF, Dunham W, Gebhardt MC, Malawar M, Pritchard DJ. A system for the functional evaluation of reconstructive procedures after surgical treatment of tumors of the musculoskeletal system. Clin Orthop Relat Res. 1993;286:241-246.PubMed Enneking WF, Dunham W, Gebhardt MC, Malawar M, Pritchard DJ. A system for the functional evaluation of reconstructive procedures after surgical treatment of tumors of the musculoskeletal system. Clin Orthop Relat Res. 1993;286:241-246.PubMed
18.
go back to reference Gerrand CH, Wunder JS, Kandel RA, O’Sullivan B, Catton CN, Bell RS, Griffin AM, Davis AM. The influence of anatomic location on functional outcome in lower-extremity soft-tissue sarcoma. Ann Surg Oncol. 2004;11:476–482.PubMedCrossRef Gerrand CH, Wunder JS, Kandel RA, O’Sullivan B, Catton CN, Bell RS, Griffin AM, Davis AM. The influence of anatomic location on functional outcome in lower-extremity soft-tissue sarcoma. Ann Surg Oncol. 2004;11:476–482.PubMedCrossRef
19.
go back to reference Gilmour JA, Scott SD, Huntington N. Nurses and Internet health information: a questionnaire survey. J Adv Nurs. 2008;61:19–28.PubMed Gilmour JA, Scott SD, Huntington N. Nurses and Internet health information: a questionnaire survey. J Adv Nurs. 2008;61:19–28.PubMed
20.
go back to reference Hanscom B, Lurie JD, Homa K, Weinstein JN. Computerized questionnaires and the quality of survey data. Spine (Phila Pa 1976). 2002;27:1797–1801. Hanscom B, Lurie JD, Homa K, Weinstein JN. Computerized questionnaires and the quality of survey data. Spine (Phila Pa 1976). 2002;27:1797–1801.
21.
go back to reference Harewood GC, Yacavone RF, Locke GR 3rd, Wiersema MJ. Prospective comparison of endoscopy patient satisfaction surveys: e-mail versus standard mail versus telephone. Am J Gastroenterol. 2001;96:3312–3317.PubMedCrossRef Harewood GC, Yacavone RF, Locke GR 3rd, Wiersema MJ. Prospective comparison of endoscopy patient satisfaction surveys: e-mail versus standard mail versus telephone. Am J Gastroenterol. 2001;96:3312–3317.PubMedCrossRef
22.
go back to reference Joinson A. Social desirability, anonymity, and Internet-based questionnaires. Behav Res Methods Instrum Comput. 1999;31:433–438.PubMedCrossRef Joinson A. Social desirability, anonymity, and Internet-based questionnaires. Behav Res Methods Instrum Comput. 1999;31:433–438.PubMedCrossRef
23.
go back to reference Jones JB, Snyder CF, Wu AW. Issues in the design of Internet-based systems for collecting patient-reported outcomes. Qual Life Res. 2007;16:1407–1417.PubMedCrossRef Jones JB, Snyder CF, Wu AW. Issues in the design of Internet-based systems for collecting patient-reported outcomes. Qual Life Res. 2007;16:1407–1417.PubMedCrossRef
24.
go back to reference Jones R, Pitt N. Health surveys in the workplace: comparison of postal, email and World Wide Web methods. Occup Med (Lond). 1999;49:556–558.CrossRef Jones R, Pitt N. Health surveys in the workplace: comparison of postal, email and World Wide Web methods. Occup Med (Lond). 1999;49:556–558.CrossRef
25.
go back to reference Khalily C, Behnke S, Seligson D. Treatment of closed tibia shaft fractures: a survey from the 1997 Orthopaedic Trauma Association and Osteosynthesis International-Gerhard Kuntscher Kreis meeting. J Orthop Trauma. 2000;14:577–581.PubMedCrossRef Khalily C, Behnke S, Seligson D. Treatment of closed tibia shaft fractures: a survey from the 1997 Orthopaedic Trauma Association and Osteosynthesis International-Gerhard Kuntscher Kreis meeting. J Orthop Trauma. 2000;14:577–581.PubMedCrossRef
26.
go back to reference Kim HL, Gerber GS, Patel RV, Hollowell CM, Bales GT. Practice patterns in the treatment of female urinary incontinence: a postal and internet survey. Urology. 2001;57:45–48.PubMedCrossRef Kim HL, Gerber GS, Patel RV, Hollowell CM, Bales GT. Practice patterns in the treatment of female urinary incontinence: a postal and internet survey. Urology. 2001;57:45–48.PubMedCrossRef
27.
go back to reference Kongsved SM, Basnov M, Holm-Christensen K, Hjollund NH. Response rate and completeness of questionnaires: a randomized study of internet versus paper-and-pencil versions. J Med Internet Res. 2007;9:e25.PubMedCrossRef Kongsved SM, Basnov M, Holm-Christensen K, Hjollund NH. Response rate and completeness of questionnaires: a randomized study of internet versus paper-and-pencil versions. J Med Internet Res. 2007;9:e25.PubMedCrossRef
28.
go back to reference Kumle M, Weiderpass E, Braaten T, Persson I, Adami H, Lund E. Use of oral contraceptives and breast cancer risk: The Norwegian-Swedish Women’s Lifestyle and Health Cohort Study. Cancer Epidemiol Biomarkers Prev. 2002;11:1375–1381.PubMed Kumle M, Weiderpass E, Braaten T, Persson I, Adami H, Lund E. Use of oral contraceptives and breast cancer risk: The Norwegian-Swedish Women’s Lifestyle and Health Cohort Study. Cancer Epidemiol Biomarkers Prev. 2002;11:1375–1381.PubMed
29.
go back to reference Lampert MH, Gerber LH, Glatstein E, Rosenberg SA, Danoff JV. Soft tissue sarcoma: functional outcome after wide local excision and radiation therapy. Arch Phys Med Rehabil. 1984;65:477–480.PubMed Lampert MH, Gerber LH, Glatstein E, Rosenberg SA, Danoff JV. Soft tissue sarcoma: functional outcome after wide local excision and radiation therapy. Arch Phys Med Rehabil. 1984;65:477–480.PubMed
30.
go back to reference Leece P, Bhandari M, Sprague S, Swiontkowski MF, Schemitsch EH, Tornetta P, Devereaux PJ, Guyatt GH. Internet versus mailed questionnaires: a controlled comparison (2). J Med Internet Res. 2004;6:e39.PubMedCrossRef Leece P, Bhandari M, Sprague S, Swiontkowski MF, Schemitsch EH, Tornetta P, Devereaux PJ, Guyatt GH. Internet versus mailed questionnaires: a controlled comparison (2). J Med Internet Res. 2004;6:e39.PubMedCrossRef
31.
go back to reference Matarasso A, Elkwood A, Rankin M, Elkowitz M. National plastic surgery survey: face lift techniques and complications. Plastic Reconstr Surg. 2000;106:1185–1195; discussion 1196. Matarasso A, Elkwood A, Rankin M, Elkowitz M. National plastic surgery survey: face lift techniques and complications. Plastic Reconstr Surg. 2000;106:1185–1195; discussion 1196.
32.
go back to reference Meropol NJ, Egleston BL, Buzaglo JS, Benson AB 3rd, Cegala DJ, Diefenbach MA, Fleisher L, Miller SM, Sulmasy DP, Weinfurt KP; CONNECT Study Research Group. Cancer patients’ preferences for quality and length of life. Cancer. 2008;113:3459–3466.PubMedCrossRef Meropol NJ, Egleston BL, Buzaglo JS, Benson AB 3rd, Cegala DJ, Diefenbach MA, Fleisher L, Miller SM, Sulmasy DP, Weinfurt KP; CONNECT Study Research Group. Cancer patients’ preferences for quality and length of life. Cancer. 2008;113:3459–3466.PubMedCrossRef
33.
go back to reference O’Leary TE, Diller L, Recklitis CJ. The effects of response bias on self-reported quality of life among childhood cancer survivors. Qual Life Res. 2007;16:1211–1220.PubMedCrossRef O’Leary TE, Diller L, Recklitis CJ. The effects of response bias on self-reported quality of life among childhood cancer survivors. Qual Life Res. 2007;16:1211–1220.PubMedCrossRef
34.
go back to reference Ritter P, Lorig K, Laurent D, Matthews K. Internet versus mailed questionnaires: a randomized comparison. J Med Internet Res. 2004;6:e29.PubMedCrossRef Ritter P, Lorig K, Laurent D, Matthews K. Internet versus mailed questionnaires: a randomized comparison. J Med Internet Res. 2004;6:e29.PubMedCrossRef
35.
go back to reference Robinson MH, Spruce L, Eeles R, Fryatt I, Harmer CL, Thomas JM, Westbury G. Limb function following conservation treatment of adult soft tissue sarcoma. Eur J Cancer. 1991;27:1567–1574.PubMedCrossRef Robinson MH, Spruce L, Eeles R, Fryatt I, Harmer CL, Thomas JM, Westbury G. Limb function following conservation treatment of adult soft tissue sarcoma. Eur J Cancer. 1991;27:1567–1574.PubMedCrossRef
36.
go back to reference Saleh KJ, Radosevich DM, Kassim RA, Moussa M, Dykes D, Bottolfson H, Gioe TJ, Robinson H. Comparison of commonly used orthopaedic outcome measures using palm-top computers and paper surveys. J Orthop Res. 2002;20:1146–1151.PubMedCrossRef Saleh KJ, Radosevich DM, Kassim RA, Moussa M, Dykes D, Bottolfson H, Gioe TJ, Robinson H. Comparison of commonly used orthopaedic outcome measures using palm-top computers and paper surveys. J Orthop Res. 2002;20:1146–1151.PubMedCrossRef
37.
go back to reference Schreiber D, Bell RS, Wunder JS, O’Sullivan B, Turcotte R, Masri BA, Davis AM. Evaluating function and health related quality of life in patients treated for extremity soft tissue sarcoma. Qual Life Res. 2006;15:1439–1446.PubMedCrossRef Schreiber D, Bell RS, Wunder JS, O’Sullivan B, Turcotte R, Masri BA, Davis AM. Evaluating function and health related quality of life in patients treated for extremity soft tissue sarcoma. Qual Life Res. 2006;15:1439–1446.PubMedCrossRef
38.
go back to reference Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psych Bull. 1979;86:420–428.CrossRef Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psych Bull. 1979;86:420–428.CrossRef
39.
go back to reference Smith B, Smith TC, Gray GC, Ryan MA; Millennium Cohort Study Team. When epidemiology meets the Internet: Web-based surveys in the Millennium Cohort Study. Am J Epidemiol. 2007;166:1345–1354.PubMedCrossRef Smith B, Smith TC, Gray GC, Ryan MA; Millennium Cohort Study Team. When epidemiology meets the Internet: Web-based surveys in the Millennium Cohort Study. Am J Epidemiol. 2007;166:1345–1354.PubMedCrossRef
40.
go back to reference Sugarbaker PH, Barofsky I, Rosenberg SA, Gianola FJ. Quality of life assessment of patients in extremity sarcoma clinical trials. Surgery. 1982;91:17–23.PubMed Sugarbaker PH, Barofsky I, Rosenberg SA, Gianola FJ. Quality of life assessment of patients in extremity sarcoma clinical trials. Surgery. 1982;91:17–23.PubMed
42.
go back to reference World Health Organization. International Classification of Impairments, Disabilities and Handicaps. Geneva, Switzerland: World Health Organization; 1990. World Health Organization. International Classification of Impairments, Disabilities and Handicaps. Geneva, Switzerland: World Health Organization; 1990.
43.
go back to reference Wilson AS, Kitas GD, Carruthers DM, Reay C, Skan J, Harris S, Treharne GJ, Young SP, Bacon PA. Computerized information-gathering in specialist rheumatology clinics: an initial evaluation of an electronic version of the Short Form 36. Rheumatology (Oxford). 2002;41:268–273.CrossRef Wilson AS, Kitas GD, Carruthers DM, Reay C, Skan J, Harris S, Treharne GJ, Young SP, Bacon PA. Computerized information-gathering in specialist rheumatology clinics: an initial evaluation of an electronic version of the Short Form 36. Rheumatology (Oxford). 2002;41:268–273.CrossRef
Metadata
Title
Can the Toronto Extremity Salvage Score Produce Reliable Results When Used Online?
Authors
Mark Clayer, MD
Aileen Davis, PhD
Publication date
01-06-2011
Publisher
Springer-Verlag
Published in
Clinical Orthopaedics and Related Research® / Issue 6/2011
Print ISSN: 0009-921X
Electronic ISSN: 1528-1132
DOI
https://doi.org/10.1007/s11999-010-1715-8

Other articles of this Issue 6/2011

Clinical Orthopaedics and Related Research® 6/2011 Go to the issue

Symposium: Update on Hard-on-Hard Bearings in Hip Arthroplasty

Is Resurfacing Arthroplasty Appropriate for Posttraumatic Osteoarthritis?

Symposium: Update on Hard-on-Hard Bearings in Hip Arthroplasty

Does a Plastic Drape Reduce Incidence of Heterotopic Ossification After Hip Resurfacing?

Symposium: Update on Hard-on-Hard Bearings in Hip Arthroplasty

Femoral Head Size Does Not Affect Ion Values in Metal-on-Metal Total Hips