Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2003

Open Access 01-12-2003 | Research article

Development and evaluation of a quality score for abstracts

Authors: Antje Timmer, Lloyd R Sutherland, Robert J Hilsden

Published in: BMC Medical Research Methodology | Issue 1/2003

Login to get access

Abstract

Background

The evaluation of abstracts for scientific meetings has been shown to suffer from poor inter observer reliability. A measure was developed to assess the formal quality of abstract submissions in a standardized way.

Methods

Item selection was based on scoring systems for full reports, taking into account published guidelines for structured abstracts. Interrater agreement was examined using a random sample of submissions to the American Gastroenterological Association, stratified for research type (n = 100, 1992–1995). For construct validity, the association of formal quality with acceptance for presentation was examined. A questionnaire to expert reviewers evaluated sensibility items, such as ease of use and comprehensiveness.

Results

The index comprised 19 items. The summary quality scores showed good interrater agreement (intra class coefficient 0.60 – 0.81). Good abstract quality was associated with abstract acceptance for presentation at the meeting. The instrument was found to be acceptable by expert reviewers.

Conclusion

A quality index was developed for the evaluation of scientific meeting abstracts which was shown to be reliable, valid and useful.
Appendix
Available only for authorised users
Literature
1.
Scherer RW, Langenberg P: Full publication of results initally presented in abstracts (Cochrane Methodology Review). The Cochrane Library. 2001, Oxford: Update Softw (4)
2.
Scherer RW, Dickersin K, Langenberg P: Full publication of results initially presented in abstracts. A meta-analysis. JAMA. 1994, 272: 158-162. 10.1001/jama.272.2.158.CrossRefPubMed
3.
Sacks JJ, Peterson DE: Improving conference abstract selection [letter]. Epidemiology. 1994, 5: 636-637.CrossRefPubMed
4.
Conn HO: An experiment in blind program selection. Clin Res. 1974, 22: 128-134.
5.
6.
Kemper KJ, McCarthy PL, Cicchetti DV: Improving participation and interrater agreement in scoring Ambulatory Pediatric Association abstracts. How well have we succeeded?. Arch Pediatr Adolesc Med. 1996, 150: 380-383.CrossRefPubMed
7.
Vilstrup H, Sorensen HT: A comparative study of scientific evaluation of abstracts submitted to the 1995 European Association for the Study of the Liver Copenhagen meeting. Dan Med Bull. 1998, 45: 317-319.PubMed
8.
Rubin HR, Redelmeier DA, Wu AW, Steinberg EP: How reliable is peer review of scientific abstracts? Looking back at the 1991 Annual Meeting of the Society of General Internal Medicine. J Gen Intern Med. 1993, 8: 255-258.CrossRefPubMed
9.
Koren G, Graham K, Shear H, Einarson T: Bias against the null hypothesis: the reproductive hazards of cocaine. Lancet. 1989, 1140-1142.
10.
McIntosh N: Abstract information and structure at scientific meetings [letter]. Lancet. 1996, 347: 544-545. 10.1016/S0140-6736(96)91177-0.CrossRefPubMed
11.
Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S: Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials. 1995, 16: 62-73. 10.1016/0197-2456(94)00031-W.CrossRefPubMed
12.
Juni P, Witschi A, Bloch R, Egger M: The hazards of scoring the quality of clinical trials for meta-analysis. JAMA. 1999, 282: 1054-1060. 10.1001/jama.282.11.1054.CrossRefPubMed
13.
Chalmers TC, Smith H, Blackburn B, Silverman B, Schroeder B, Reitman D, Ambroz A: A method for assessing the quality of a randomized controlled trial. Control Clin Trials. 1981, 2: 31-49. 10.1016/0197-2456(81)90056-8.CrossRefPubMed
14.
Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, McQuay HJ: Assessing the quality of reports of randomized clinical trials: is blinding necessary?. Control Clin Trials. 1996, 17: 1-12. 10.1016/0197-2456(95)00134-4.CrossRefPubMed
15.
Friedenreich CM: Methods for pooled analyses of epidemiologic studies. Epidemiology. 1993, 4: 295-302.CrossRefPubMed
16.
Cho MK, Bero LA: Instruments for assessing the quality of drug studies published in the medical literature. JAMA. 1994, 272: 101-104. 10.1001/jama.272.2.101.CrossRefPubMed
17.
Haynes RB: More informative abstracts: current status and evaluation. J Clin Epidemiol. 1993, 46: 595-597.CrossRefPubMed
18.
Froom P, Froom J: Deficiencies in structured medical abstracts. J Clin Epidemiol. 1993, 46: 591-594.CrossRefPubMed
19.
Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics. 1977, 33: 159-174.CrossRefPubMed
20.
Seigel DG, Podgor MJ, Remaley NA: Acceptable values of kappa for comparison of two groups. Am J Epidemiol. 1992, 135: 571-578.PubMed
21.
Streiner DL, Norman GR: Health measurement scales. A practical guide to their development and use. 1989, Oxford, New York, Tokyo: Oxford University Press
22.
Feinstein AR: The theory and evaluation of sensibility. In Clinimetrics. 1987, New Haven, London: Yale University Press, 141-166.
23.
Oxman AD, Guyatt GH, Cook DJ, Jaeschke R, Heddle N, Keller J: An index of scientific quality for health reports in the lay press. J Clin Epidemiol. 1993, 46: 987-1001. 10.1016/0895-4356(93)90166-X.CrossRefPubMed
24.
Oxman AD, Guyatt G, Guyatt GH: Validation of an index of the quality of review articles. J Clin Epidemiol. 1991, 11: 1271-1278. 10.1016/0895-4356(91)90160-B.CrossRef
25.
Panush RS, Delafuente JC, Connelly CS, Edwards NL, Greer JM, Longley S, Bennett F: Profile of a meeting: how abstracts are written and reviewed. J Rheumatol. 1989, 16: 145-147.PubMed
26.
Chalmers I, Adams M, Dickersin K, Hetherington J, Tarnow-Mordi W, Meinert C, Tonascia s, Chalmers TC: A cohort study of summary reports of controlled trials. JAMA. 1990, 263: 1401-1405. 10.1001/jama.263.10.1401.CrossRefPubMed
27.
Narine L, Yee DS, Einarson TR, Ilersich AL: Quality of abstracts of original research articles in CMAJ in 1989. CMAJ. 1991, 144: 449-453.PubMedPubMedCentral
28.
Scherer RW, Crawley B: Reporting of randomized clinical trial descriptors and use of structured abstracts. JAMA. 1998, 280: 269-272. 10.1001/jama.280.3.269.CrossRefPubMed
29.
Deeks JJ, Altman DG: Inadequate reporting of controlled trials as short reports. Lancet. 1998, 352: 1908-1909.CrossRefPubMed
30.
Weintraub WH: Are published manuscripts representative of the surgical meeting abstracts? An objective appraisal. J Ped Surg. 1987, 22: 11-13.CrossRef
31.
Weber EJ, Callaham ML, Wears RL, Barton C, Young G: Unpublished research from a medical specialty meeting. Why investigators fail to publish. JAMA. 1998, 280: 257-259. 10.1001/jama.280.3.257.CrossRefPubMed
32.
Moher D, Jadad AR, Tugwell P: Assessing the quality of randomized controlled trials. Current issues and future directions. International Journal of Technology Assessment in Health Care. 1996, 12: 195-208.CrossRefPubMed
33.
Schulz KF, Chalmers I, Hayes RJ: Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment in controlled trials. JAMA. 1995, 273: 408-412. 10.1001/jama.273.5.408.CrossRefPubMed
34.
Verhagen AP, de Vet HC, de Bie RA, Kessels AG, Boers M, Bouter LM, Knipschild PG: The Delphi list: a criteria list for quality assessment of randomized clinical trials for conducting systematic reviews developed by Delphi consensus. J Clin Epidemiol. 1998, 51: 1235-1241. 10.1016/S0895-4356(98)00131-0.CrossRefPubMed
35.
Timmer A, Hilsden RJ, Sutherland LR: Determinants of abstract acceptance for the Digestive Diseases Week – a cross sectional study. BMC Med Res Methodol. 2001, 1: 13-10.1186/1471-2288-1-13.CrossRefPubMedPubMedCentral
36.
Timmer A, Hilsden RJ, Cole J, Hailey D, Sutherland LR: Publication bias in gastroenterological research. BMC Med Res Methodol. 2002, 2: 7-10.1186/1471-2288-2-7.CrossRefPubMedPubMedCentral
Metadata
Title
Development and evaluation of a quality score for abstracts
Authors
Antje Timmer
Lloyd R Sutherland
Robert J Hilsden
Publication date
01-12-2003
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2003
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/1471-2288-3-2

Other articles of this Issue 1/2003

BMC Medical Research Methodology 1/2003 Go to the issue

Debate

Pragmatic controlled clinical trials in primary care: the struggle between external and internal validity

Erratum

Erratum to: The transitive fallacy for randomized trials: If A bests B and B bests C in separate trials, is A better than C?

Debate

Choosing a control intervention for a randomised clinical trial

Research article

Performance of health-status scales when used selectively or within multi-scale questionnaire

Research article

Estimation of the correlation coefficient using the Bayesian Approach and its applications for epidemiologic research

Debate

Does anybody read "evidence-based" articles?