1 |
Can Character-based Language Models Improve Downstream Task Performance in Low-Resource and Noisy Language Scenarios?
|
|
|
|
In: Seventh Workshop on Noisy User-generated Text (W-NUT 2021, colocated with EMNLP 2021) ; https://hal.inria.fr/hal-03527328 ; Seventh Workshop on Noisy User-generated Text (W-NUT 2021, colocated with EMNLP 2021), Jan 2022, punta cana, Dominican Republic ; https://aclanthology.org/2021.wnut-1.47/ (2022)
|
|
Abstract:
International audience ; Recent impressive improvements in NLP, largely based on the success of contextual neural language models, have been mostly demonstrated on at most a couple dozen high-resource languages. Building language models and, more generally, NLP systems for non-standardized and low-resource languages remains a challenging task. In this work, we focus on North-African colloquial dialectal Arabic written using an extension of the Latin script, called NArabizi, found mostly on social media and messaging communication. In this low-resource scenario with data displaying a high level of variability, we compare the downstream performance of a character-based language model on part-of-speech tagging and dependency parsing to that of monolingual and multilingual models. We show that a character-based model trained on only 99k sentences of NArabizi and fined-tuned on a small treebank of this language leads to performance close to those obtained with the same architecture pre-trained on large multilingual and monolingual models. Confirming these results a on much larger data set of noisy French user-generated content, we argue that such character-based language models can be an asset for NLP in low-resource and high language variability set-tings.
|
|
Keyword:
[INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI]; [INFO.INFO-CL]Computer Science [cs]/Computation and Language [cs.CL]; [INFO.INFO-IR]Computer Science [cs]/Information Retrieval [cs.IR]; [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG]; [INFO.INFO-SI]Computer Science [cs]/Social and Information Networks [cs.SI]; [INFO.INFO-TT]Computer Science [cs]/Document and Text Processing
|
|
URL: https://hal.inria.fr/hal-03527328
|
|
BASE
|
|
Hide details
|
|
2 |
Cross-lingual few-shot hate speech and offensive language detection using meta learning
|
|
|
|
In: ISSN: 2169-3536 ; EISSN: 2169-3536 ; IEEE Access ; https://hal.archives-ouvertes.fr/hal-03559484 ; IEEE Access, IEEE, 2022, 10, pp.14880-14896. ⟨10.1109/ACCESS.2022.3147588⟩ (2022)
|
|
BASE
|
|
Show details
|
|
3 |
Ensemble of Opinion Dynamics Models to Understand the Role of the Undecided in the Vaccination Debate ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
The Online Behaviour of the Algerian Abusers in Social Media Networks ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
Discussion Networks and Resilience of College Students: Explicating Tie Strength in Communicative Interaction
|
|
|
|
In: International Journal of Communication; Vol 16 (2022); 25 ; 1932-8036 (2022)
|
|
BASE
|
|
Show details
|
|
6 |
“Thou Shalt Not Take the Lord’s Name in Vain”: A Methodological Proposal to Identify Religious Hate Content on Digital Social Networks
|
|
|
|
In: International Journal of Communication; Vol 16 (2022); 22 ; 1932-8036 (2022)
|
|
BASE
|
|
Show details
|
|
7 |
Conceptual structure and the growth of scientific knowledge ...
|
|
|
|
BASE
|
|
Show details
|
|
8 |
INNOVATIVE APPROACHES AND METHODS IN TEACHING FOREIGN LANGUAGES ...
|
|
|
|
BASE
|
|
Show details
|
|
9 |
INNOVATIVE APPROACHES AND METHODS IN TEACHING FOREIGN LANGUAGES ...
|
|
|
|
BASE
|
|
Show details
|
|
10 |
Multilingual Abusiveness Identification on Code-Mixed Social Media Text ...
|
|
|
|
BASE
|
|
Show details
|
|
11 |
MuMiN: A Large-Scale Multilingual Multimodal Fact-Checked Misinformation Social Network Dataset ...
|
|
|
|
BASE
|
|
Show details
|
|
13 |
Discovering Affinity Relationships between Personality Types ...
|
|
|
|
BASE
|
|
Show details
|
|
14 |
Networks and Identity Drive Geographic Properties of the Diffusion of Linguistic Innovation ...
|
|
|
|
BASE
|
|
Show details
|
|
16 |
Cyberbullying Classifiers are Sensitive to Model-Agnostic Perturbations ...
|
|
|
|
BASE
|
|
Show details
|
|
17 |
Feature-rich multiplex lexical networks reveal mental strategies of early language learning ...
|
|
|
|
BASE
|
|
Show details
|
|
18 |
It Takes a Village: Using Network Science to Identify the Effect of Individual Differences in Bilingual Experience for Theory of Mind
|
|
|
|
In: Brain Sciences; Volume 12; Issue 4; Pages: 487 (2022)
|
|
BASE
|
|
Show details
|
|
19 |
Analysis of the Full-Size Russian Corpus of Internet Drug Reviews with Complex NER Labeling Using Deep Learning Neural Networks and Language Models
|
|
|
|
In: Applied Sciences; Volume 12; Issue 1; Pages: 491 (2022)
|
|
BASE
|
|
Show details
|
|
20 |
Transformer-Based Abstractive Summarization for Reddit and Twitter: Single Posts vs. Comment Pools in Three Languages
|
|
|
|
In: Future Internet; Volume 14; Issue 3; Pages: 69 (2022)
|
|
BASE
|
|
Show details
|
|
|
|