| 2 |
Performance Assessments of Two-Way, Free-Form, Speech-to-Speech Translation Systems for Tactical Use
|
|
|
|
In: DTIC (2011)
|
|
BASE
|
|
Show details
|
|
|
| 3 |
An Analysis of Specware and Its Usefulness in the Verification of High Assurance Systems
|
|
|
|
In: DTIC (2006)
|
|
BASE
|
|
Show details
|
|
|
| 4 |
Inter-Rater Agreement Measures and the Refinement of Metrics in the PLATO MT Evaluation Paradigm
|
|
|
|
In: DTIC (2005)
|
|
Abstract:
The PLATO machine translation (MT) evaluation (MTE) research program has as a goal the systematic development of a predictive relationship between discrete, well-defined MTE metrics and the specific information processing tasks that can be reliably performed with MT output. Traditional measures of quality, informed by International Standards for Language Engineering (ISLE), namely, clarity, coherence, morphology, syntax, general and domain-specific lexical robustness, and named-entity translation, as well as a DARPA-inspired measure of adequacy are at the core of the program. For robust validation, indispensable for refinement of test and guidelines, we conduct tests of inter-rater reliability on the assessments. Here we discuss development and report on results of our inter-rater reliability tests, focusing on results for Clarity and the Coherence, the first two assessments in the PLATO suite, and we discuss our method for iteratively refining our linguistic metrics and the guidelines for applying them within the PLATO evaluation paradigm. Finally, we discuss reasons why kappa might not be the best measure of inter-rater agreement for our purposes, and suggest directions for future investigation. ; Prepared in collaboration with the U.S. Army Research Laboratory, Adelphi, MD.
|
|
Keyword:
*MACHINE TRANSLATION; AUTOMATED MACHINE TRANSLATION EVAULATION; AUTOMATION; COMPUTATIONAL LINGUISTICS; Computer Programming and Software; Cybernetics; INTER-RATER RELIABILITY TESTS; Linguistics; TEST AND EVALUATION; TEXT PROCESSING; VALIDATION
|
|
URL: http://www.dtic.mil/docs/citations/ADA456393
http://oai.dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA456393
|
|
BASE
|
|
Hide details
|
|
|
| 5 |
Advanced Capabilities for Evidence Extraction (ACEE)
|
|
|
|
In: DTIC AND NTIS (2004)
|
|
BASE
|
|
Show details
|
|
|
| 8 |
Numerical Control Lathe Language Study.
|
|
|
|
In: DTIC AND NTIS (1979)
|
|
BASE
|
|
Show details
|
|
|
| 9 |
An Evaluation of Process and Experiment Automation Realtime Language (PEARL)
|
|
|
|
In: DTIC AND NTIS (1977)
|
|
BASE
|
|
Show details
|
|
|
| 10 |
Methodology for Comprehensive Software Testing.
|
|
|
|
In: DTIC AND NTIS (1975)
|
|
BASE
|
|
Show details
|
|
|
| 11 |
Compass Test Language (CTL) Syntax and Parser.
|
|
|
|
In: DTIC AND NTIS (1973)
|
|
BASE
|
|
Show details
|
|
|
|
|
