| 1 |
Comparing Evaluation Metrics for Sentence Boundary Detection
|
|
|
|
In: DTIC (2007)
|
|
Abstract:
In recent NIST evaluations on sentence boundary detection, a single error metric was used to describe performance. Additional metrics, however, are available for such tasks, in which a word stream is partitioned into subunits. This paper compares alternative evaluation metrics including the NIST error rate, classification error rate per word boundary, precision and recall, ROC curves, DET curves, precision-recall curves, and area under the curves and discusses advantages and disadvantages of each. Unlike many studies in machine learning, we use real data for a real task. We find benefit from using curves in addition to a single metric. Furthermore, we find that data skew has an impact on metrics, and that differences among different system outputs are more visible in precision-recall curves. Results are expected to help us better understand evaluation metrics that should be generalizable to similar language processing tasks. ; Presented at the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) (32nd) held in Honolulu, Hawaii on 15-20 April 2007. Published in the Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) (32nd), 2007. The original document contains color images.
|
|
Keyword:
*COMPUTATIONAL LINGUISTICS; *EVALUATION METRICS; *LANGUAGE PROCESSING; *SENTENCE BOUNDARY DETECTION; *SPEECH RECOGNITION; *TEST AND EVALUATION; Equipment and Methods; ERRORS; LEARNING MACHINES; Linguistics; MACHINE LEARNING; NATURAL LANGUAGE; PRECISION; RECALL; ROC CURVES; ROC(RECEIVER OPERATING CHARACTERISTIC); SYMPOSIA; Test Facilities; Voice Communications
|
|
URL: http://oai.dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA463058
http://www.dtic.mil/docs/citations/ADA463058
|
|
BASE
|
|
Hide details
|
|
|
| 2 |
Scaling Up of Action Repertoire in Linguisitic Cognitive Agents
|
|
|
|
In: DTIC (2007)
|
|
BASE
|
|
Show details
|
|
|
|
|
