| 1 |
How to Evaluate ASR Output for Named Entity Recognition?
|
|
|
|
In: 16th Annual Conference of the International Speech Communication Association (Interspeech'15) ; https://hal.archives-ouvertes.fr/hal-01251370 ; 16th Annual Conference of the International Speech Communication Association (Interspeech'15), Sep 2015, Dresden, Germany (2015)
|
|
Abstract:
International audience ; The standard metric to evaluate automatic speech recognition (ASR) systems is the word error rate (WER). WER has proven very useful in stand-alone ASR systems. Nowadays, these systems are often embedded in complex natural language processing systems to perform tasks like speech translation, manmachine dialogue, or information retrieval from speech. This exacerbates the need for the speech processing community to design a new evaluation metric to estimate the quality of automatic transcriptions within their larger applicative context. We introduce a new measure to evaluate ASR in the context of named entity recognition, which makes use of a probabilistic model to estimate the risk of ASR errors inducing downstream errors in named entity detection. Our evaluation, on the ETAPE data, shows that ATENE achieves a higher correlation than WER between the performances in named entities recognition and in automatic speech transcription.
|
|
Keyword:
[SHS.LANGUE]Humanities and Social Sciences/Linguistics
|
|
URL: https://hal.archives-ouvertes.fr/hal-01251370
|
|
BASE
|
|
Hide details
|
|
|
| 2 |
How to assess the quality of automatic transcriptions for the extraction of named entities? ; Comment évaluer la qualité des transcriptions automatiques pour la détection d’entités nommées ?
|
|
|
|
In: Actes des XXXe Journées d'Études sur la Parole (JEP'14) ; XXXe Journées d'Études sur la Parole (JEP'14) ; https://hal.archives-ouvertes.fr/hal-01134868 ; XXXe Journées d'Études sur la Parole (JEP'14), Jun 2014, Le Mans, France. pp.430-437 ; http://www-lium.univ-lemans.fr/jep2014/ (2014)
|
|
BASE
|
|
Show details
|
|
|
| 3 |
ETER: a New Metric for the Evaluation of Hierarchical Named Entity Recognition
|
|
|
|
In: Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14) ; Ninth International Conference on Language Resources and Evaluation (LREC'14) ; https://hal.archives-ouvertes.fr/hal-01134713 ; Ninth International Conference on Language Resources and Evaluation (LREC'14), European Language Resources Association (ELRA), May 2014, Reykjavik, Iceland. pp.3987-3994 ; http://lrec2014.lrec-conf.org/en/ (2014)
|
|
BASE
|
|
Show details
|
|
|
| 4 |
Automatic named entity pre-annotation for out-of-domain human annotation
|
|
|
|
In: Linguistic Annotation Workshop ; https://hal.archives-ouvertes.fr/hal-01831229 ; Linguistic Annotation Workshop, ACL, Jan 2013, Sofia, Bulgaria (2013)
|
|
BASE
|
|
Show details
|
|
|
| 5 |
Extended named entities annotation on OCRed documents: from corpus constitution to evaluation campaign
|
|
|
|
In: International Conference on Language Resources and Evaluation ; https://hal.archives-ouvertes.fr/hal-01831254 ; International Conference on Language Resources and Evaluation, Jan 2012, Istanbul, Turkey (2012)
|
|
BASE
|
|
Show details
|
|
|
| 6 |
Manual Corpus Annotation: Giving Meaning to the Evaluation Metrics
|
|
|
|
In: Proceedings of the International Conference on Computational Linguistics (COLING 2012) ; International Conference on Computational Linguistics ; https://hal.archives-ouvertes.fr/hal-00769639 ; International Conference on Computational Linguistics, Dec 2012, Mumbaï, India. pp.809--818 (2012)
|
|
BASE
|
|
Show details
|
|
|
| 7 |
Structured Named Entities in two distinct press corpora: Contemporary Broadcast News and Old Newspapers
|
|
|
|
In: Proceedings of the Sixth ACL Linguistic Annotation Workshop ; 6th Linguistics Annotation Workshop (The LAW VI) ; https://hal.archives-ouvertes.fr/hal-00709193 ; 6th Linguistics Annotation Workshop (The LAW VI), Jul 2012, Jeju, South Korea. pp.40-48 (2012)
|
|
BASE
|
|
Show details
|
|
|
| 8 |
Proposal for an Extension of Traditional Named Entitites: from Guidelines to Evaluation, an Overview
|
|
|
|
In: Proceedings of the Fifth ACL Linguistic Annotation Workshop ; 5th Linguistics Annotation Workshop (The LAW V) ; https://hal.archives-ouvertes.fr/hal-00604369 ; 5th Linguistics Annotation Workshop (The LAW V), Jun 2011, Portland, United States. pp.92--100 (2011)
|
|
BASE
|
|
Show details
|
|
|
| 9 |
Named and specific entity detection in varied data: the Quaero named entity baseline evaluation
|
|
|
|
In: 7. Conference on international language resources and evaluation ; https://hal.inrae.fr/hal-02754184 ; 7. Conference on international language resources and evaluation, May 2010, Valletta, Malta ; http://www.lrec-conf.org/proceedings/lrec2010/index.html (2010)
|
|
BASE
|
|
Show details
|
|
|
| 10 |
Named and specific entity detection in varied data: The Quaero Named Entity baseline evaluation
|
|
|
|
In: Seventh International Language Resources and Evaluation (LREC'10) ; https://hal.archives-ouvertes.fr/hal-02496836 ; Seventh International Language Resources and Evaluation (LREC'10), 2010, La Valette, Malta (2010)
|
|
BASE
|
|
Show details
|
|
|
| 11 |
Question answering on web data : the QA evaluation in Quaero
|
|
|
|
In: Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC’10) ; International Conference on Language Resources and Evaluation ; https://hal.archives-ouvertes.fr/hal-02282126 ; International Conference on Language Resources and Evaluation, Jan 2010, Valetta, Malta (2010)
|
|
BASE
|
|
Show details
|
|
|
| 13 |
Semantic Relations for an Oral and Interactive Question-Answering System.
|
|
|
|
In: Semantic Representation of Spoken Language 2007 ; SRSL7 (Semantic Representation of Spoken Language 2007) ; https://hal.archives-ouvertes.fr/hal-00515902 ; SRSL7 (Semantic Representation of Spoken Language 2007), Nov 2007, Salamanca, Spain. pp.12-19 (2007)
|
|
BASE
|
|
Show details
|
|
|
|
|
