DE eng

Search in the Catalogues and Directories

Hits 1 – 8 of 8

1
Focus on the Target's Vocabulary: Masked Label Smoothing for Machine Translation ...
Chen, Liang; Xu, Runxin; Chang, Baobao. - : arXiv, 2022
BASE
Show details
2
StableMoE: Stable Routing Strategy for Mixture of Experts ...
Abstract: The Mixture-of-Experts (MoE) technique can scale up the model size of Transformers with an affordable computational overhead. We point out that existing learning-to-route MoE methods suffer from the routing fluctuation issue, i.e., the target expert of the same input may change along with training, but only one expert will be activated for the input during inference. The routing fluctuation tends to harm sample efficiency because the same input updates different experts but only one is finally used. In this paper, we propose StableMoE with two training stages to address the routing fluctuation problem. In the first training stage, we learn a balanced and cohesive routing strategy and distill it into a lightweight router decoupled from the backbone model. In the second training stage, we utilize the distilled router to determine the token-to-expert assignment and freeze it for a stable routing strategy. We validate our method on language modeling and multilingual machine translation. The results show that ... : ACL-2022 ...
Keyword: Computation and Language cs.CL; FOS Computer and information sciences; Machine Learning cs.LG
URL: https://arxiv.org/abs/2204.08396
https://dx.doi.org/10.48550/arxiv.2204.08396
BASE
Hide details
3
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning ...
BASE
Show details
4
Document-level Event Extraction via Heterogeneous Graph-based Interaction Model with a Tracker ...
BASE
Show details
5
Inductively Representing Out-of-Knowledge-Graph Entities by Optimal Estimation Under Translational Assumptions ...
BASE
Show details
6
Incorporating Glosses into Neural Word Sense Disambiguation ...
Luo, Fuli; Liu, Tianyu; Xia, Qiaolin. - : arXiv, 2018
BASE
Show details
7
Chinese-English Parallel Corpus Construction and its Application
In: http://dspace.wul.waseda.ac.jp/dspace/bitstream/2065/583/1/interactive-9.pdf (2004)
BASE
Show details
8
Translation and information technology
Zhang, Huarui (Mitarb.); Yu, Shiwen (Mitarb.); Jernudd, Björn H. (Mitarb.). - Hong Kong : Chinese Univ. Press, 2002
BLLDB
UB Frankfurt Linguistik
Show details

Catalogues
1
0
0
0
0
0
0
Bibliographies
1
0
0
0
0
0
0
0
0
Linked Open Data catalogues
0
Online resources
0
0
0
0
Open access documents
7
0
0
0
0
© 2013 - 2024 Lin|gu|is|tik | Imprint | Privacy Policy | Datenschutzeinstellungen ändern