1 |
Examining Scaling and Transfer of Language Model Architectures for Machine Translation ...
|
|
|
|
BASE
|
|
Show details
|
|
2 |
Multilingual Document-Level Translation Enables Zero-Shot Transfer From Sentences to Documents ...
|
|
|
|
BASE
|
|
Show details
|
|
3 |
Share or Not? Learning to Schedule Language-Specific Capacity for Multilingual Translation ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
Edinburgh’s End-to-End Multilingual Speech Translation System for IWSLT 2021 ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
On Sparsifying Encoder Outputs in Sequence-to-Sequence Models ...
|
|
|
|
BASE
|
|
Show details
|
|
6 |
Sparse Attention with Linear Units ...
|
|
|
|
Abstract:
Anthology paper link: https://aclanthology.org/2021.emnlp-main.523/ Abstract: Recently, it has been argued that encoder-decoder models can be made more interpretable by replacing the softmax function in the attention with its sparse variants. In this work, we introduce a novel, simple method for achieving sparsity in attention: we replace the softmax activation with a ReLU, and show that sparsity naturally emerges from such a formulation. Training stability is achieved with layer normalization with either a specialized initialization or an additional gating function. Our model, which we call Rectified Linear Attention (ReLA), is easy to implement and more efficient than previously proposed sparse attention mechanisms. We apply ReLA to the Transformer and conduct experiments on five machine translation tasks. ReLA achieves translation performance comparable to several strong baselines, with training and decoding speed similar to that of the vanilla attention. Our analysis shows that ReLA delivers high ...
|
|
Keyword:
Computational Linguistics; Machine Learning; Machine Learning and Data Mining; Natural Language Processing
|
|
URL: https://underline.io/lecture/37313-sparse-attention-with-linear-units https://dx.doi.org/10.48448/ghvm-kp96
|
|
BASE
|
|
Hide details
|
|
7 |
Beyond Sentence-Level End-to-End Speech Translation: Context Helps ...
|
|
|
|
BASE
|
|
Show details
|
|
8 |
Exploring Dynamic Selection of Branch Expansion Orders for Code Generation ...
|
|
|
|
BASE
|
|
Show details
|
|
9 |
Share or Not? Learning to Schedule Language-Specific Capacity for Multilingual Translation
|
|
|
|
In: Zhang, Biao; Bapna, Ankur; Sennrich, Rico; Firat, Orhan (2021). Share or Not? Learning to Schedule Language-Specific Capacity for Multilingual Translation. In: International Conference on Learning Representations, Virtual, 3 May 2021 - 7 May 2021, ICLR. (2021)
|
|
BASE
|
|
Show details
|
|
10 |
ParBLEU: Augmenting Metrics with Automatic Paraphrases for the WMT'20 Metrics Shared Task
|
|
|
|
In: Proceedings of the 5th Conference on Machine Translation ; 5th Conference on Machine Translation ; https://hal.archives-ouvertes.fr/hal-02981143 ; 5th Conference on Machine Translation, Nov 2020, Online, Unknown Region (2020)
|
|
BASE
|
|
Show details
|
|
11 |
Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation ...
|
|
|
|
BASE
|
|
Show details
|
|
12 |
Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation ...
|
|
|
|
BASE
|
|
Show details
|
|
13 |
Adaptive Feature Selection for End-to-End Speech Translation ...
|
|
|
|
BASE
|
|
Show details
|
|
15 |
Brain Structural and Functional Alterations Specific to Low Sleep Efficiency in Major Depressive Disorder
|
|
|
|
BASE
|
|
Show details
|
|
16 |
Improving Deep Transformer with Depth-Scaled Initialization and Merged Attention ...
|
|
|
|
BASE
|
|
Show details
|
|
17 |
Revisiting Low-Resource Neural Machine Translation: A Case Study ...
|
|
|
|
BASE
|
|
Show details
|
|
19 |
BattRAE: Bidimensional Attention-Based Recursive Autoencoders for Learning Bilingual Phrase Embeddings ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|