3 |
Contextual Embeddings: When Are They Worth It? ...
|
|
|
|
Abstract:
We study the settings for which deep contextual embeddings (e.g., BERT) give large improvements in performance relative to classic pretrained embeddings (e.g., GloVe), and an even simpler baseline---random word embeddings---focusing on the impact of the training set size and the linguistic properties of the task. Surprisingly, we find that both of these simpler baselines can match contextual embeddings on industry-scale data, and often perform within 5 to 10% accuracy (absolute) on benchmark tasks. Furthermore, we identify properties of data for which contextual embeddings give particularly large gains: language containing complex structure, ambiguous word usage, and words unseen in training. ... : ACL 2020 ...
|
|
Keyword:
Computation and Language cs.CL; FOS Computer and information sciences; Machine Learning cs.LG
|
|
URL: https://arxiv.org/abs/2005.09117 https://dx.doi.org/10.48550/arxiv.2005.09117
|
|
BASE
|
|
Hide details
|
|
4 |
Translational regulation of Chk1 expression by eIF3a via interaction with the RNA-binding protein HuR
|
|
|
|
In: Biochem J (2020)
|
|
BASE
|
|
Show details
|
|
5 |
Lexical richness of Chinese candidates in the graded oral English examinations
|
|
|
|
BASE
|
|
Show details
|
|
|
|