1 |
SemEval-2022 Task 2: Multilingual Idiomaticity Detection and Sentence Embedding ...
|
|
|
|
BASE
|
|
Show details
|
|
2 |
Improving Tokenisation by Alternative Treatment of Spaces ...
|
|
|
|
Abstract:
Tokenisation is the first step in almost all NLP tasks, and state-of-the-art transformer-based language models all use subword tokenisation algorithms to process input text. Existing algorithms have problems, often producing tokenisations of limited linguistic validity, and representing equivalent strings differently depending on their position within a word. We hypothesise that these problems hinder the ability of transformer-based models to handle complex words, and suggest that these problems are a result of allowing tokens to include spaces. We thus experiment with an alternative tokenisation approach where spaces are always treated as individual tokens. Specifically, we apply this modification to the BPE and Unigram algorithms. We find that our modified algorithms lead to improved performance on downstream NLP tasks that involve handling complex words, whilst having no detrimental effect on performance in general natural language understanding tasks. Intrinsically, we find our modified algorithms give ...
|
|
Keyword:
Computation and Language cs.CL; FOS Computer and information sciences
|
|
URL: https://dx.doi.org/10.48550/arxiv.2204.04058 https://arxiv.org/abs/2204.04058
|
|
BASE
|
|
Hide details
|
|
3 |
NILC-Metrix: assessing the complexity of written and spoken language in Brazilian Portuguese ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
Multistage BiCross encoder for multilingual access to COVID-19 health information ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
Multistage BiCross encoder for multilingual access to COVID-19 health information
|
|
|
|
In: PLoS One (2021)
|
|
BASE
|
|
Show details
|
|
6 |
The (un)suitability of automatic evaluation metrics for text simplification
|
|
|
|
BASE
|
|
Show details
|
|
7 |
ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations
|
|
|
|
In: ACL 2020 - 58th Annual Meeting of the Association for Computational Linguistics ; https://hal.inria.fr/hal-02889823 ; ACL 2020 - 58th Annual Meeting of the Association for Computational Linguistics, Jul 2020, Seattle / Virtual, United States (2020)
|
|
BASE
|
|
Show details
|
|
8 |
Toxic Language Detection in Social Media for Brazilian Portuguese: New Dataset and Multilingual Analysis ...
|
|
|
|
BASE
|
|
Show details
|
|
9 |
ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations ...
|
|
|
|
BASE
|
|
Show details
|
|
10 |
Linguistic analysis model for monitoring user reaction on satirical news for brazilian portuguese
|
|
|
|
BASE
|
|
Show details
|
|
11 |
ASSET: A dataset for tuning and evaluation of sentence simplification models with multiple rewriting transformations
|
|
|
|
BASE
|
|
Show details
|
|
12 |
Data-Driven Sentence Simplification: Survey and Benchmark
|
|
|
|
In: Computational Linguistics, Vol 46, Iss 1, Pp 135-187 (2020) (2020)
|
|
BASE
|
|
Show details
|
|
13 |
Text Simplification From Professionally Produced Corpora ...
|
|
|
|
BASE
|
|
Show details
|
|
14 |
SimPA: A Sentence-Level Simplification Corpus for the Public Administration Domain ...
|
|
|
|
BASE
|
|
Show details
|
|
15 |
SimPA: A Sentence-Level Simplification Corpus for the Public Administration Domain ...
|
|
|
|
BASE
|
|
Show details
|
|
16 |
Text Simplification From Professionally Produced Corpora ...
|
|
|
|
BASE
|
|
Show details
|
|
17 |
SimPA: A Sentence-Level Simplification Corpus for the Public Administration Domain ...
|
|
|
|
BASE
|
|
Show details
|
|
18 |
SimPA: A Sentence-Level Simplification Corpus for the Public Administration Domain ...
|
|
|
|
BASE
|
|
Show details
|
|
19 |
Sheffield submissions for the WMT18 quality estimation shared task
|
|
|
|
In: 807 ; 813 (2018)
|
|
BASE
|
|
Show details
|
|
|
|