| 1 |
Overview of Results of the MUC-6 Evaluation
|
|
|
|
In: DTIC (1995)
|
|
Abstract:
The latest in a series of natural language processing system evaluations was concluded in October 1995 and was the topic of the Sixth Message Understanding Conference (MUC-6) in November. Participants were invited to enter their systems in as many as four different task-oriented evaluations. The Named Entity and Conference tasks entailed Standard Generalized Markup Language (SGML) annotation of texts and were being conducted for the first time. The other two tasks, Template Element and Scenario Template, were information extraction tasks that followed on from the MUC evaluations conducted in previous years. The evolution and design of the MUC- 6 evaluation are discussed in the paper by Grishman and Sundheim in this volume. All except the Scenario Template task are defined independently of any particular domain. This paper surveys the results of the evaluation on each task and, to a more limited extent, across tasks. Discussion of the results for each task is organized generally under the following topics: * Results on task as whole; * Results on some aspects of task; * Performance on "walkthrough article." The walkthrough article is an article selected from the test set. Participants were asked to analyze their system's performance on that article and comment on it in their presentations and papers. Permission has been granted by Dow Jones for the full text of the article to be reprinted in this proceedings. It appears in full in the first part of appendix A, and various site reports may contain excerpts from it or annotated versions of it. Also in appendix A are representations of the information contained in the answer key for the walkthrough article for each of the four tasks.
|
|
Keyword:
*MARKUP LANGUAGES; *MESSAGE PROCESSING; *TEST SETS; *TEXTBOOKS; *WORDS(LANGUAGE); EXTRACTION; INFORMATION RETRIEVAL; Information Science; Linguistics; MUC-6; MUC-6(SIXTH MESSAGE UNDERSTANDING CONFERENCE); PAPER; REPORTS; SCENARIOS; SGML(STANDARD GENERALIZED MARKUP LANGUAGE); SITES; SURVEYS; TEMPLATES
|
|
URL: http://oai.dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA458881
http://www.dtic.mil/docs/citations/ADA458881
|
|
BASE
|
|
Hide details
|
|
|
| 2 |
TMSS Parsing Test, MIL-M-9977J, Appendix G, Loading Procedures Checklists.
|
|
In: DTIC AND NTIS (1994)
|
|
BASE
|
|
Show details
|
|
|
| 3 |
MIL-M-9977J, Appendix E, Integrated Combat Turnaround Procedures, Document Type Definition, MAXIMA Corporation.
|
|
|
|
In: DTIC AND NTIS (1994)
|
|
BASE
|
|
Show details
|
|
|
| 4 |
TMSS Parsing Test, MIL-M-9977J, Appendix J, NATO Stage B Cross-Servicing Checklist (CL2), Document Type Definition.
|
|
In: DTIC AND NTIS (1994)
|
|
BASE
|
|
Show details
|
|
|
| 5 |
TMSS Parsing Test, MIL-M-9977J, Appendix F. Nonnuclear Munitions Loading Standard Data Packages, Document Type Definition.
|
|
In: DTIC AND NTIS (1994)
|
|
BASE
|
|
Show details
|
|
|
| 6 |
TMSS Parsing Test, MIL-M-9977J. Appendix C. Nonnuclear Munitions Basic Information. Document Type Definition.
|
|
In: DTIC AND NTIS (1994)
|
|
BASE
|
|
Show details
|
|
|
| 7 |
TMSS Parsing Test MIL-M-38784C, Appendix B, Template for Technical Manuals, Amendment Three. Document Type Definition MAXIMA Corporation DISA Contract No. DCA100-93-D-0065 Quick Short Test Report.
|
|
|
|
In: DTIC AND NTIS (1994)
|
|
BASE
|
|
Show details
|
|
|
| 8 |
TMSS Parsing Test, MIL-M-77OOE, Flight Manual Document Type Definition.
|
|
In: DTIC AND NTIS (1994)
|
|
BASE
|
|
Show details
|
|
|
| 9 |
TMSS Parsing Test, MIL-M-9977J, Appendix H, Integrated Combat Turnaround Procedures Checklist.
|
|
In: DTIC AND NTIS (1994)
|
|
BASE
|
|
Show details
|
|
|
| 10 |
MIL-M-83495A, Appendix B. General System, Amendment One. Document Type Definition.
|
|
In: DTIC AND NTIS (1994)
|
|
BASE
|
|
Show details
|
|
|
| 11 |
TMSS Parsing Test MIL-M-9977J, Appendix D, Nonnuclear Munitions Loading Procedures.
|
|
In: DTIC AND NTIS (1994)
|
|
BASE
|
|
Show details
|
|
|
| 12 |
TMSS Parsing Test MIL-T-388O4B, Appendix A; Time Compliance Technical Orders; Document Type Definition; MAXIMA Corporation; DISA Contract Number DCAA100-93-D-0065 Quick Short Test Report.
|
|
In: DTIC AND NTIS (1994)
|
|
BASE
|
|
Show details
|
|
|
| 13 |
TMSS Parsing Test MIL-M-5O96E Flow/Sequence Chart Document Type Definition MAXIMA Corporation Quick Short Test Report.
|
|
In: DTIC AND NTIS (1994)
|
|
BASE
|
|
Show details
|
|
|
| 14 |
TMSS Parsing Test, MIL-M-83495A, Wiring Diagram, Appendix F, Document Type Definition.
|
|
In: DTIC AND NTIS (1994)
|
|
BASE
|
|
Show details
|
|
|
| 15 |
MITRE-Bedford: Description of the ALEMBIC System as Used for MUC-4
|
|
|
|
In: DTIC (1992)
|
|
BASE
|
|
Show details
|
|
|
|
|
