Beth M. Sundheim
ABSTRACT
The latest in a series of natural language processing system evaluations was concluded in October 1995 and was the topic of the Sixth Message Understanding Conference (MUC-6) in November. Participants were invited to enter their systems in as many as four different task-oriented evaluations. The Named Entity and Coreference tasks entailed Standard Generalized Markup Language (SGML) annotation of texts and were being conducted for the first time. The other two tasks, Template Element and Scenario Template, were information extraction tasks that followed on from the MUC evaluations conducted in previous years. The evolution and design of the MUC-6 evaluation are discussed in the paper by Grishman and Sundheim in this volume. All except the Scenario Template task are defined independently of any particular domain.
- Proceedings of the Fourth Message Understanding Conference (MUC-4), June 1992, San Mateo: Morgan Kaufmann.Google Scholar
Recommendations
Statistical significance of MUC-6 results
MUC6 '95: Proceedings of the 6th conference on Message understandingThe results of the MUC-6 evaluation must be analyzed to determine whether close scores significantly distinguish systems or whether the differences in those scores are a matter of chance. In order to do such an analysis, a method of computer intensive ...
Design of the MUC-6 evaluation
MUC6 '95: Proceedings of the 6th conference on Message understandingThe sixth in a series of "Message Understanding Conferences", which are designed to promote and evaluate research in information extraction, was held last fall. MUC-6 introduced several innovations over prior MUCs, most notably in the range of different ...
Overview of results of the MUC-6 evaluation
TIPSTER '96: Proceedings of a workshop on held at Vienna, Virginia: May 6-8, 1996The latest in a series of natural language processing system evaluations was concluded in October 1995 and was the topic of the Sixth Message Understanding Conference (MUC-6) in November. Participants were invited to enter their systems in as many as ...
Comments