Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on medication adherence

LIVE: Thursday 27th June 2024, 18:00-19:30 (CEST)
Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.
ADVANCED SEARCH
Log in
Top
International Ophthalmology
Published in: International Ophthalmology 1/2024

Open Access 01-12-2024 | Original Paper

Interrater reliability of photographic assessment of thyroid eye disease using the VISA classification

Authors: Antony C. Boynes, Nicholas J. Enright, Thomas G. Hardy, Jwu Jin Khong

Published in: International Ophthalmology | Issue 1/2024

Login to get access

Abstract

Purpose

To determine the interrater reliability (IRR) of thyroid eye disease (TED) photographic assessment using the VISA classification. To assess whether a VISA grading atlas improves ophthalmology trainees’ performance in photographic assessment of TED.

Methods

A prospective, partially randomized, international study conducted from September 2021 to May 2022. Online study invitation was emailed to a volunteer sample group of 68 ophthalmology college accredited consultants and trainees, and 6 were excluded from the study. Participants were asked to score 10 patient photographs of TED using only the inflammation and motility restriction components of the VISA classification. IRR was compared between groups of practitioners by their level of experience. A clinical activity grading atlas was randomized to 50% of the ophthalmology trainees.

Results

Overall rater ICC was 0.96 for inflammation and 0.99 for motility restriction. No statistically significant difference in IRR between rater groups was identified. Trainees with a grading atlas had the highest IRR for inflammation (ICC = 0.95). Each subcomponent of the inflammation and motility restriction components of VISA classification had an ICC considered good to excellent. The mean overall rater score was 4.6/9 for inflammation and 3.5/12 for motility restriction. For motility restriction there was a reduced mean score variance among all raters when scoring photographs with more severe motility restriction.

Conclusion

IRR using the inflammation and motility restriction components of the VISA classification was excellent. A VISA grading atlas improved trainee performance in grading inflammation.
Appendix
Available only for authorised users
Literature
1.
Patel A, Yang H, Douglas RS (2019) A new era in the treatment of thyroid eye disease. Am J Ophthalmol 208:281–288PubMedCrossRef
2.
Wang Y, Padnick-Silver L, Francis-Sedlak M, Holt RJ, Foley C, Douglas RS (2022) Inflammatory and noninflammatory thyroid eye disease: comparison of disease signs, symptoms, and quality of life in patients in the United States. Endocr Pract 28(9):842–846PubMedCrossRef
3.
Otto EA, Ochs K, Hansen C, Wall JR, Kahaly GJ (1996) Orbital tissue-derived T lymphocytes from patients with Graves’ ophthalmopathy recognize autologous orbital antigens. J Clin Endocrinol Metab 81(8):3045–3050PubMed
4.
Mourits MP, Koornneef L, Wiersinga WM, Prummel MF, Berghout A, van der Gaag R (1989) Clinical criteria for the assessment of disease activity in Graves’ ophthalmopathy: a novel approach. Br J Ophthalmol 73(8):639–644PubMedPubMedCentralCrossRef
5.
Gontarz-Nowak K, Szychlińska M, Matuszewski W, Stefanowicz-Rutkowska M, Bandurska-Stankiewicz E (2020) Current knowledge on graves' orbitopathy. J Clin Med. 10(1).
6.
7.
Tsui S, Naik V, Hoa N, Hwang CJ, Afifiyan NF, Sinha Hikim A et al (2008) Evidence for an association between thyroid-stimulating hormone and insulin-like growth factor 1 receptors: a tale of two antigens implicated in Graves’ disease. J Immunol 181(6):4397–4405PubMedCrossRef
8.
Douglas RS, Kahaly GJ, Patel A, Sile S, Thompson EHZ, Perdok R et al (2020) Teprotumumab for the treatment of active thyroid eye disease. N Engl J Med 382(4):341–352PubMedCrossRef
9.
Ugradar S, Kang J, Kossler AL, Zimmerman E, Braun J, Harrison AR et al (2022) Teprotumumab for the treatment of chronic thyroid eye disease. Eye (Lond) 36(8):1553–1559PubMedCrossRef
10.
Denisova K, Barmettler A (2021) Evaluating the thyroid eye disease patient. Int Ophthalmol Clin 61(2):33–52PubMedCrossRef
11.
12.
Smith TJ, Kahaly GJ, Ezra DG, Fleming JC, Dailey RA, Tang RA et al (2017) Teprotumumab for thyroid-associated ophthalmopathy. N Engl J Med 376(18):1748–1761PubMedPubMedCentralCrossRef
13.
Ma L, Hui S, Li Y, Hou Z, Liu Z, Chang Q et al (2022) Different characteristics of orbital soft tissue expansion in graves orbitopathy: extraocular muscle expansion is correlated to disease activity while fat tissue volume with duration. J Craniofac Surg 33(8):2354–2359PubMedCrossRef
14.
Li Q, Ye H, Ding Y, Chen G, Liu Z, Xu J et al (2017) Clinical characteristics of moderate-to-severe thyroid associated ophthalmopathy in 354 Chinese cases. PLoS ONE 12(5):e0176064PubMedPubMedCentralCrossRef
15.
Van Dyk HJ (1981) Orbital Graves' disease. A modification of the "NO SPECS" classification. Ophthalmology 88(6):479–483.PubMedCrossRef
16.
Bartalena L, Kahaly GJ, Baldeschi L, Dayan CM, Eckstein A, Marcocci C et al (2021) The 2021 European Group on Graves’ orbitopathy (EUGOGO) clinical practice guidelines for the medical management of Graves’ orbitopathy. Eur J Endocrinol 185(4):G43–G67PubMedCrossRef
17.
Dolman PJ (2018) Grading severity and activity in thyroid eye disease. Ophthalmic Plast Reconstr Surg 34(4S Suppl 1):S34–S40.PubMedCrossRef
18.
Mourits MP, Prummel MF, Wiersinga WM, Koornneef L (1997) Clinical activity score as a guide in the management of patients with Graves’ ophthalmopathy. Clin Endocrinol (Oxf) 47(1):9–14PubMedCrossRef
19.
Rundle FF, Wilson CW (1945) Development and course of exophthalmos and ophthalmoplegia in Graves’ disease with special reference to the effect of thyroidectomy. Clin Sci 5(3–4):177–194PubMed
20.
Perros P, Baldeschi L, Boboridis K, Dickinson AJ, Hullo A, Kahaly GJ et al (2006) A questionnaire survey on the management of Graves’ orbitopathy in Europe. Eur J Endocrinol 155(2):207–211PubMedCrossRef
21.
Bartalena L, Baldeschi L, Dickinson A, Eckstein A, Kendall-Taylor P, Marcocci C et al (2008) Consensus statement of the European Group on Graves’ orbitopathy (EUGOGO) on management of GO. Eur J Endocrinol 158(3):273–285PubMedCrossRef
22.
Dolman PJ, Cahill K, Czyz CN, Douglas RS, Elner VM, Feldon S et al (2012) Reliability of estimating ductions in thyroid eye disease: an International Thyroid Eye Disease Society multicenter study. Ophthalmology 119(2):382–389PubMedCrossRef
23.
Dickinson AJ, Perros P (2001) Controversies in the clinical evaluation of active thyroid-associated orbitopathy: use of a detailed protocol with comparative photographs for objective assessment. Clin Endocrinol (Oxf) 55(3):283–303PubMedCrossRef
24.
Wiersinga WM, Perros P, Kahaly GJ, Mourits MP, Baldeschi L, Boboridis K et al (2006) Clinical assessment of patients with Graves’ orbitopathy: the European Group on Graves’ Orbitopathy recommendations to generalists, specialists and clinical researchers. Eur J Endocrinol 155(3):387–389PubMedCrossRef
25.
Dagi LR, Zoumalan CI, Konrad H, Trokel SL, Kazim M (2011) Correlation between extraocular muscle size and motility restriction in thyroid eye disease. Ophthalmic Plast Reconstr Surg 27(2):102–110PubMedCrossRef
26.
Bontzos G, Papadaki E, Mazonakis M, Maris TG, Tsakalis NG, Drakonaki EE et al (2022) Extraocular muscle volumetry for assessment of thyroid eye disease. J Neuroophthalmol 42(1):e274–e280PubMedCrossRef
27.
Dolman PJ, Rootman J (2006) VISA classification for graves orbitopathy. Ophthalmic Plast Reconstr Surg 22(5):319–324PubMedCrossRef
28.
Dolman PJ (2012) Evaluating Graves’ orbitopathy. Best practice and research. Clin Endocrinol Metabolism 26:229–248.
29.
Hutchings KR, Fritzhand SJ, Esmaeli B, Koka K, Zhao J, Ahmed S, et al (2023) Graves' Eye disease: clinical and radiological diagnosis. Biomedicines 11(2).
30.
Mawn LA, Dolman PJ, Kazim M, Strianese D, Genol I, Chong KKL et al (2018) Soft tissue metrics in thyroid eye disease: an international thyroid eye disease society reliability study. Ophthalmic Plast Reconstr Surg 34(6):544–546PubMedCrossRef
31.
Koo TK, Li MY (2016) A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 15(2):155–163PubMedPubMedCentralCrossRef
32.
Bartko JJ (1966) The intraclass correlation coefficient as a measure of reliability. Psychol Rep 19(1):3–11PubMedCrossRef
33.
Schuck P (2004) Assessing reproducibility for interval data in health-related quality of life questionnaires: which coefficient should be used? Qual Life Res 13(3):571–586ADSPubMedCrossRef
34.
35.
Li L, Zeng L, Lin ZJ, Cazzell M, Liu H (2015) Tutorial on use of intraclass correlation coefficients for assessing intertest reliability and its application in functional near-infrared spectroscopy-based brain imaging. J Biomed Opt 20(5):50801PubMedCrossRef
36.
Edwards DT, Bartley GB, Hodge DO, Gorman CA, Bradley EA (2004) Eyelid position measurement in Graves’ ophthalmopathy: reliability of a photographic technique and comparison with a clinical technique. Ophthalmology 111(5):1029–1034PubMedCrossRef
37.
38.
39.
40.
Moon JH, Shin K, Lee GM, Park J, Lee MJ, Choung H et al (2022) Machine learning-assisted system using digital facial images to predict the clinical activity score in thyroid-associated orbitopathy. Sci Rep 12(1):22085ADSPubMedPubMedCentralCrossRef
41.
Schulz CB, Clarke H, Makuloluwe S, Thomas PB, Kang S (2023) Automated extraction of clinical measures from videos of oculofacial disorders using machine learning: feasibility, validity and reliability. Eye (Lond), pp 1–7.
42.
Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A et al (2016) Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316(22):2402–2410PubMedCrossRef
43.
Hood DC, De Moraes CG (2018) Efficacy of a deep learning system for detecting glaucomatous optic neuropathy based on color fundus photographs. Ophthalmology 125(8):1207–1208PubMedCrossRef
44.
Wiersinga WM, Bartalena L (2002) Epidemiology and prevention of Graves’ ophthalmopathy. Thyroid 12(10):855–860PubMedCrossRef
45.
Wilson WB, Prochoda M (1995) Radiotherapy for thyroid orbitopathy. Effects on extraocular muscle balance. Arch Ophthalmol 113(11):1420–1425.CrossRef
46.
Mimura M, Yang PT, Ko AC, Korn BS, Kikkawa DO (2020) Analysis of periorbital soft tissue in thyroid eye disease. Ophthalmic Plast Reconstr Surg 36(1):30–33PubMedCrossRef
47.
Wiersinga WM, Prummel MF, Mourits MP, Koornneef L, Buller HR (1991) Classification of the eye changes of Graves’ disease. Thyroid 1(4):357–360PubMedCrossRef
48.
Bobak CA, Barr PJ, O’Malley AJ (2018) Estimation of an inter-rater intra-class correlation coefficient that overcomes common assumption violations in the assessment of health measurement scales. BMC Med Res Methodol 18(1):93PubMedPubMedCentralCrossRef
49.
Shrout PE, Fleiss JL (1979) Intraclass correlations: uses in assessing rater reliability. Psychol Bull 86(2):420–428PubMedCrossRef
50.
Qin S, Nelson L, McLeod L, Eremenco S, Coons SJ (2019) Assessing test-retest reliability of patient-reported outcome measures using intraclass correlation coefficients: recommendations for selecting and documenting the analytical formula. Qual Life Res 28(4):1029–1033PubMedCrossRef
51.
Mehta S, Bastero-Caballero RF, Sun Y, Zhu R, Murphy DK, Hardas B et al (2018) Performance of intraclass correlation coefficient (ICC) as a reliability index under various distributions in scale reliability studies. Stat Med 37(18):2734–2752MathSciNetPubMedPubMedCentralCrossRef
Metadata
Title
Interrater reliability of photographic assessment of thyroid eye disease using the VISA classification
Authors
Antony C. Boynes
Nicholas J. Enright
Thomas G. Hardy
Jwu Jin Khong
Publication date
01-12-2024
Publisher
Springer Netherlands
Published in
International Ophthalmology / Issue 1/2024
Print ISSN: 0165-5701
Electronic ISSN: 1573-2630
DOI
https://doi.org/10.1007/s10792-024-02934-z

Other articles of this Issue 1/2024

International Ophthalmology 1/2024 Go to the issue

Original Paper

Vision-related quality of life in patients with retinal vein occlusion

Original Paper

The association in diabetic retinopathy and stroke finding from NHANES evidence

Original Paper

Evaluation of the (Baha) technique of scleral indentation using a self-retained scleral indenter during vitrectomy surgery: a randomized trial

Review

Correlation between gout and dry eye disease

Original Paper

CircRNA HLCS regulates lens epithelial cell apoptosis via miR-338-3p/BPNT1 axis

Case Report

Corneal pseudoectasia: a case series