1 |
One Size Does Not Fit All: Finding the Optimal Subword Sizes for FastText Models across Languages ...
|
|
|
|
Abstract:
Unsupervised representation learning of words from large multilingual corpora is useful for downstream tasks such as word sense disambiguation, semantic text similarity, and information retrieval. The representation precision of log-bilinear fastText models is mostly due to their use of subword information. In previous work, the optimization of fastText's subword sizes has not been fully explored, and non-English fastText models were trained using subword sizes optimized for English and German word analogy tasks. In our work, we find the optimal subword sizes on the English, German, Czech, Italian, Spanish, French, Hindi, Turkish, and Russian word analogy tasks. We then propose a simple n-gram coverage model and we show that it predicts better-than-default subword sizes on the Spanish, French, Hindi, Turkish, and Russian word analogy tasks. We show that the optimization of fastText's subword sizes matters and results in a 14% improvement on the Czech word analogy task. We also show that expensive parameter ...
|
|
Keyword:
68T50; Computation and Language cs.CL; FOS Computer and information sciences; I.2.7
|
|
URL: https://arxiv.org/abs/2102.02585 https://dx.doi.org/10.48550/arxiv.2102.02585
|
|
BASE
|
|
Hide details
|
|
2 |
EDS-MEMBED: Multi-sense embeddings based on enhanced distributional semantic structures via a graph walk over word senses ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
Exploration of metaphor and affect sensing using semantic interpretation in an intelligent agent
|
|
|
|
BASE
|
|
Show details
|
|
8 |
Did You Say What I Think You Said? - Towards a Language-Based Measurement of a Speech Recognizer's Confidence
|
|
|
|
BASE
|
|
Show details
|
|
10 |
Project EuDML--A First Year Demonstration
|
|
|
|
In: Intelligent Computer Mathematics ; Intelligent Computer Mathematics 18th Symposium, Calculemus 2011, and 10th International Conference, MKM 2011 ; https://hal.archives-ouvertes.fr/hal-00658076 ; Intelligent Computer Mathematics 18th Symposium, Calculemus 2011, and 10th International Conference, MKM 2011, Jul 2011, Bertinoro, Italy. pp.281-284, ⟨10.1007/978-3-642-22673-1_21⟩ (2011)
|
|
BASE
|
|
Show details
|
|
|
|