1 |
Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP
|
|
|
|
In: https://hal.inria.fr/hal-03540069 ; 2022 (2022)
|
|
BASE
|
|
Show details
|
|
2 |
SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection ...
|
|
|
|
BASE
|
|
Show details
|
|
3 |
SIGTYP 2020 Shared Task: Prediction of Typological Features ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
Linguistic calibration through metacognition: aligning dialogue agent responses with expected correctness ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
Processing South Asian Languages Written in the Latin Script: the Dakshina Dataset ...
|
|
|
|
Abstract:
This paper describes the Dakshina dataset, a new resource consisting of text in both the Latin and native scripts for 12 South Asian languages. The dataset includes, for each language: 1) native script Wikipedia text; 2) a romanization lexicon; and 3) full sentence parallel data in both a native script of the language and the basic Latin alphabet. We document the methods used for preparation and selection of the Wikipedia text in each language; collection of attested romanizations for sampled lexicons; and manual romanization of held-out sentences from the native script collections. We additionally provide baseline results on several tasks made possible by the dataset, including single word transliteration, full sentence transliteration, and language modeling of native script and romanized text. Keywords: romanization, transliteration, South Asian languages ... : Published at LREC 2020 ...
|
|
Keyword:
Computation and Language cs.CL; FOS Computer and information sciences
|
|
URL: https://arxiv.org/abs/2007.01176 https://dx.doi.org/10.48550/arxiv.2007.01176
|
|
BASE
|
|
Hide details
|
|
6 |
Spell Once, Summon Anywhere: A Two-Level Open-Vocabulary Language Model ...
|
|
|
|
BASE
|
|
Show details
|
|
8 |
Unsupervised Disambiguation of Syncretism in Inflected Lexicons ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|