Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
BMC Neurology
Published in: BMC Neurology 1/2014

Open Access 01-12-2014 | Research article

Optimizing the use of expert panel reference diagnoses in diagnostic studies of multidimensional syndromes

Authors: Ron L H Handels, Claire A G Wolfs, Pauline Aalten, Patrick M M Bossuyt, Manuela A Joore, Albert F G Leentjens, Johan L Severens, Frans R J Verhey

Published in: BMC Neurology | Issue 1/2014

Login to get access

Abstract

Background

In the absence of a gold standard, a panel of experts can be invited to assign a reference diagnosis for use in research. Available literature offers limited guidance on assembling and working with an expert panel for this purpose. We aimed to develop a protocol for an expert panel consensus diagnosis and evaluated its applicability in a pilot project.

Methods

An adjusted Delphi method was used, which started with the assessment of clinical vignettes by 3 experts individually, followed by a consensus discussion meeting to solve diagnostic discrepancies. A panel facilitator ensured that all experts were able to express their views, and encouraged the use of argumentation to arrive at a specific diagnosis, until consensus was reached by all experts. Eleven vignettes of patients suspected of having a primary neurodegenerative disease were presented to the experts. Clinical information was provided stepwise and included medical history, neurological, physical and cognitive function, brain MRI scan, and follow-up assessments over 2 years. After the consensus discussion meeting, the procedure was evaluated by the experts.

Results

The average degree of consensus for the reference diagnosis increased from 52% after individual assessment of the vignettes to 94% after the consensus discussion meeting. Average confidence in the diagnosis after individual assessment was 85%. This did not increase after the consensus discussion meeting. The process evaluation led to several recommendations for improvement of the protocol.

Conclusion

A protocol for attaining a reference diagnosis based on expert panel consensus was shown feasible in research practice.
Appendix
Available only for authorised users
Literature
1.
Knottnerus JA, Buntinx F: The Evidence Base of Clinical Diagnosis: Theory and Methods of Diagnostic Research. 2009, Wiley-Blackwell Pub./BMJ Books, Oxford; Hoboken, NJ
2.
Reitsma JB, Rutjes AWS, Khan KS, Coomarasamy A, Bossuyt PM: A review of solutions for diagnostic accuracy studies with an imperfect or missing reference standard. J Clin Epidemiol. 2009, 62: 797-806. 10.1016/j.jclinepi.2009.02.005.CrossRefPubMed
3.
Rutjes AWS, Reitsma JB, Coomarasamy A, Khan KS, Bossuyt PMM: Evaluation of diagnostic tests when there is no gold standard: a review of methods. Health Technol Assess. 2007, 11 (iii): ix-51.
4.
McKhann GM, Knopman DS, Chertkow H, Hyman BT, Jack CR, Kawas CH, Klunk WE, Koroshetz WJ, Manly JJ, Mayeux R, Mohs RC, Morris JC, Rossor MN, Scheltens P, Carrillo MC, Thies B, Weintraub S, Phelps CH: The diagnosis of dementia due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's disease. Alzheimers Dement. 2011, 7: 335-337. 10.1016/j.jalz.2011.03.005.CrossRef
5.
Albert MS, DeKosky ST, Dickson D, Dubois B, Feldman HH, Fox NC, Gamst A, Holtzman DM, Jagust WJ, Petersen RC, Snyder PJ, Carrillo MC, Thies B, Phelps CH: The diagnosis of mild cognitive impairment due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's disease. Alzheimers Dement. 2011, 7: 270-279. 10.1016/j.jalz.2011.03.008.CrossRefPubMedPubMedCentral
6.
Sperling RA, Aisen PS, Beckett LA, Bennett DA, Craft S, Fagan AM, Iwatsubo T, Jack CR, Kaye J, Montine TJ, Park DC, Reiman EM, Rowe CC, Siemers E, Stern Y, Yaffe K, Carrillo MC, Thies B, Morrison-Bogorad M, Wagster MV, Phelps CH: Toward defining the preclinical stages of Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's disease. Alzheimers Dement. 2011, 7: 280-292. 10.1016/j.jalz.2011.03.003.CrossRefPubMedPubMedCentral
7.
Handels RL, Aalten P, Wolfs CA, Olde Rikkert M, Scheltens P, Visser PJ, Joore MA, Severens JL, Verhey FR: Diagnostic and economic evaluation of new biomarkers for Alzheimer’s disease: the research protocol of a prospective cohort study. BMC Neurol. 2012, 12: 72-10.1186/1471-2377-12-72.CrossRefPubMedPubMedCentral
8.
Scheltens P, Rockwood K: How golden is the gold standard of neuropathology in dementia?. Alzheimers Dement. 2011, 7: 486-489. 10.1016/j.jalz.2011.04.011.CrossRefPubMed
9.
Savva GM, Wharton SB, Ince PG, Forster G, Matthews FE, Brayne C: Age, neuropathology, and dementia. N Engl J Med. 2009, 360 (22): 2302-2309. 10.1056/NEJMoa0806142.CrossRefPubMed
10.
Mason SE, McShane R, Ritchie CW: Diagnostic tests for Alzheimer’s disease: rationale, methodology, and challenges. Int J Alzheimers Dis. 2010, 2010: 1-7. 10.4061/2010/972685.CrossRef
11.
Noel-Storr AH, Flicker L, Ritchie CW, Nguyen GH, Gupta T, Wood P, Walton J, Desai M, Solomon DF, Molena E, Worrall R, Hayen A, Choudhary P, Ladds E, Lanctôt KL, Verhey FR, McCleery JM, Mead GE, Clare L, Fioravanti M, Hyde C, Marcus S, McShane R: Systematic review of the body of evidence for the use of biomarkers in the diagnosis of dementia. Alzheimers Dement. 2013, 9: e96-e105. 10.1016/j.jalz.2012.01.014.CrossRefPubMed
12.
Laméris W, Van Randen A, Dijkgraaf MG, Bossuyt PM, Stoker J, Boermeester MA: Optimization of diagnostic imaging use in patients with acute abdominal pain (OPTIMA): design and rationale. BMC Emerg Med. 2007, 7: 9-10.1186/1471-227X-7-9.CrossRefPubMedPubMedCentral
13.
Cowie MR, Struthers AD, Wood DA, Coats AJ, Thompson SG, Poole-Wilson PA, Sutton GC: Value of natriuretic peptides in assessment of patients with possible new heart failure in primary care. Lancet. 1997, 350: 1349-1353. 10.1016/S0140-6736(97)06031-5.CrossRefPubMed
14.
Maisel AS, Krishnaswamy P, Nowak RM, McCord J, Hollander JE, Duc P, Omland T, Storrow AB, Abraham WT, Wu AHB, Clopton P, Steg PG, Westheim A, Knudsen CW, Perez A, Kazanegra R, Herrmann HC, McCullough PA: Rapid measurement of B-type natriuretic peptide in the emergency diagnosis of heart failure. N Engl J Med. 2002, 347: 161-167. 10.1056/NEJMoa020233.CrossRefPubMed
15.
McKeith I, O’Brien J, Walker Z, Tatsch K, Booij J, Darcourt J, Padovani A, Giubbini R, Bonuccelli U, Volterrani D, Holmes C, Kemp P, Tabet N, Meyer I, Reininger C: Sensitivity and specificity of dopamine transporter imaging with 123I-FP-CIT SPECT in dementia with Lewy bodies: a phase III, multicentre study. Lancet Neurol. 2007, 6: 305-313. 10.1016/S1474-4422(07)70057-1.CrossRefPubMed
16.
Reeves MJ, Wehner S, Organek N, Birbeck GL, Mullard AJ, Jacobs BS, Kothari R, Hickenbottom S: Accuracy of identifying acute stroke admissions in a Michigan stroke registry. Prev Chronic Dis. 2011, 8: A62-PubMedPubMedCentral
17.
Rutten FH, Moons KGM, Cramer M-JM, Grobbee DE, Zuithoff NPA, Lammers J-WJ, Hoes AW: Recognising heart failure in elderly patients with stable chronic obstructive pulmonary disease in primary care: cross sectional diagnostic study. BMJ. 2005, 331: 1379-10.1136/bmj.38664.661181.55.CrossRefPubMedPubMedCentral
18.
Whiteley WN, Wardlaw JM, Dennis MS, Sandercock PAG: Clinical scores for the identification of stroke and transient ischaemic attack in the emergency department: a cross-sectional study. J Neurol Neurosurg Psychiatry. 2011, 82: 1006-1010. 10.1136/jnnp.2010.235010.CrossRefPubMed
19.
Van Dijk N, Boer KR, Colman N, Bakker A, Stam J, Van Grieken JJM, Wilde AAM, Linzer M, Reitsma JB, Wieling W: High diagnostic yield and accuracy of history, physical examination, and ECG in patients with transient loss of consciousness in FAST: the fainting assessment study. J Cardiovasc Electrophysiol. 2008, 19: 48-55. 10.1111/j.1540-8167.2008.01215.x.CrossRefPubMed
20.
Carnero-Pardo C, Espejo-Martinez B, Lopez-Alcalde S, Espinosa-Garcia M, Saez-Zea C, Vilchez-Carrillo R, Hernandez-Torres E, Navarro-Espigares JL: Effectiveness and costs of phototest in dementia and cognitive impairment screening. BMC Neurol. 2011, 11: 92-10.1186/1471-2377-11-92.CrossRefPubMedPubMedCentral
21.
Gabel MJ, Foster NL, Heidebrink JL, Higdon R, Aizenstein HJ, Arnold SE, Barbas NR, Boeve BF, Burke JR, Clark CM, Dekosky ST, Farlow MR, Jagust WJ, Kawas CH, Koeppe RA, Leverenz JB, Lipton AM, Peskind ER, Turner RS, Womack KB, Zamrini EY: Validation of consensus panel diagnosis in dementia. Arch Neurol. 2010, 67: 1506-1512. 10.1001/archneurol.2010.301.CrossRefPubMedPubMedCentral
22.
Egglin TK, Feinstein AR: Context bias: a problem in diagnostic radiology. JAMA. 1996, 276: 1752-1755. 10.1001/jama.1996.03540210060035.CrossRefPubMed
23.
Cushman L, Scherer MJ: Psychological Assessment in Medical Rehabilitation. 1995, American Psychological Association, Washington, DCCrossRef
24.
Hughes CP, Berg L, Danziger WL, Coben LA, Martin RL: A new clinical scale for the staging of dementia. Br J Psychol. 1982, 140: 566-572. 10.1192/bjp.140.6.566.CrossRef
25.
Morris JC: The Clinical Dementia Rating (CDR): current version and scoring rules. Neurology. 1993, 43: 2412-2414. 10.1212/WNL.43.11.2412-a.CrossRefPubMed
26.
Knopman DS, DeKosky ST, Cummings JL, Chui H, Corey-Bloom J, Relkin N, Small GW, Miller B, Stevens JC: Practice parameter: diagnosis of dementia (an evidence-based review): report of the quality standards subcommittee of the American academy of neurology. Neurology. 2001, 56: 1143-1153. 10.1212/WNL.56.9.1143.CrossRefPubMed
27.
Hort J, O’Brien JT, Gainotti G, Pirttila T, Popescu BO, Rektorova I, Sorbi S, Scheltens P: EFNS guidelines for the diagnosis and management of Alzheimer’s disease. Eur J Neurol. 2010, 17: 1236-1248. 10.1111/j.1468-1331.2010.03040.x.CrossRefPubMed
28.
Almeida OP, Almeida SA: Short versions of the geriatric depression scale: a study of their validity for the diagnosis of a major depressive episode according to ICD-10 and DSM-IV. Int J Geriatr Psychiatry. 1999, 14: 858-865. 10.1002/(SICI)1099-1166(199910)14:10<858::AID-GPS35>3.0.CO;2-8.CrossRefPubMed
29.
Cummings JL: The neuropsychiatric inventory: assessing psychopathology in dementia patients. Neurology. 1997, 48 (5 Suppl 6): S10-S16. 10.1212/WNL.48.5_Suppl_6.10S.CrossRefPubMed
30.
Gélinas I, Gauthier L, McIntyre M, Gauthier S: Development of a functional measure for persons with Alzheimer’s disease: the disability assessment for dementia. Am J Occup Ther. 1999, 53: 471-481. 10.5014/ajot.53.5.471.CrossRefPubMed
31.
Rey A: L’ Examen Clinique En Psychologie. 1958, Presses universitaires de France, Oxford, England
32.
Brand N, Jolles J: Learning and retrieval rate of words presented auditorily and visually. J Gen Psychol. 1985, 112: 201-210. 10.1080/00221309.1985.9711004.CrossRefPubMed
33.
Lindeboom J, Schmand B, Tulner L, Walstra G, Jonker C: Visual association test to detect early dementia of the Alzheimer type. J Neurol Neurosurg Psychiatry. 2002, 73: 126-133. 10.1136/jnnp.73.2.126.CrossRefPubMedPubMedCentral
34.
Wechsler D: WMS-III: Wechsler Memory Scale Administration and Scoring Manual. 1997, Psychological Corporation, San Antonio
35.
Natu MV, Agarwal AK: Digit letter substitution test (DLST) as an alternative to digit symbol substitution test (DSST). Hum Psychopharmacol. 1995, 10: 339-343. 10.1002/hup.470100414.CrossRef
36.
Stroop JR: Studies of interference in serial verbal reactions. Can J Exp Psychol. 1935, 18: 643-662. 10.1037/h0054651.CrossRef
37.
Reitan R: Trail Making Test: Manual for Administration, Scoring and Interpretation. 1956, Section of Neuropsychology, Department of Neurology, Indiana University Medical Centre, Bloomington
38.
Reitan RM: Validity of the trail making test as an indicator of organic brain damage. Percept Mot Skills. 1958, 8: 271-276. 10.2466/pms.1958.8.3.271.CrossRef
39.
Bossuyt PM: The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Clin Chem. 2003, 138: 7-18. 10.1373/49.1.7.CrossRef
40.
Moons KGM, Grobbee DE: When should we remain blind and when should our eyes remain open in diagnostic studies?. J Clin Epidemiol. 2002, 55: 633-636. 10.1016/S0895-4356(02)00408-0.CrossRefPubMed
41.
Powell C: The Delphi technique: myths and realities. J Adv Nurs. 2003, 41: 376-382. 10.1046/j.1365-2648.2003.02537.x.CrossRefPubMed
42.
43.
Keeney S, Hasson F, McKenna H: Consulting the oracle: ten lessons from using the Delphi technique in nursing research. J Adv Nurs. 2006, 53: 205-212. 10.1111/j.1365-2648.2006.03716.x.CrossRefPubMed
44.
Thornbury JR, Fryback DG: Technology assessment–an American view. Eur J Radiol. 1992, 14: 147-156. 10.1016/0720-048X(92)90228-2.CrossRefPubMed
45.
Van Den Bruel A, Cleemput I, Aertgeerts B, Ramaekers D, Buntinx F: The evaluation of diagnostic tests: evidence on technical and diagnostic accuracy, impact on patient outcome and cost-effectiveness is needed. J Clin Epidemiol. 2007, 60: 1116-1122. 10.1016/j.jclinepi.2007.03.015.CrossRefPubMed
Metadata
Title
Optimizing the use of expert panel reference diagnoses in diagnostic studies of multidimensional syndromes
Authors
Ron L H Handels
Claire A G Wolfs
Pauline Aalten
Patrick M M Bossuyt
Manuela A Joore
Albert F G Leentjens
Johan L Severens
Frans R J Verhey
Publication date
01-12-2014
Publisher
BioMed Central
Published in
BMC Neurology / Issue 1/2014
Electronic ISSN: 1471-2377
DOI
https://doi.org/10.1186/s12883-014-0190-3

Other articles of this Issue 1/2014

BMC Neurology 1/2014 Go to the issue

Research article

Brainstem infarcts predict REM sleep behavior disorder in acute ischemic stroke

Research article

Ipsilateral foetal-type posterior cerebral artery is associated with cognitive decline after carotid revascularisation

Research article

Can testing of six individual muscles represent a screening approach to upper limb neuropathic conditions?

Study protocol

Long-term sequelae of severe sepsis: cognitive impairment and structural brain alterations – an MRI study (LossCog MRI)

Research article

Changes of liver enzymes and bilirubin during ischemic stroke: mechanisms and possible significance

Research article

Co-morbidities increase the risk of disability pension among MS patients: a population-based nationwide cohort study