21 |
Cross-Lingual Transfer Learning for Arabic Task-Oriented Dialogue Systems Using Multilingual Transformer Model mT5
|
|
|
|
In: Mathematics; Volume 10; Issue 5; Pages: 746 (2022)
|
|
Abstract:
Due to the promising performance of pre-trained language models for task-oriented dialogue systems (DS) in English, some efforts to provide multilingual models for task-oriented DS in low-resource languages have emerged. These efforts still face a long-standing challenge due to the lack of high-quality data for these languages, especially Arabic. To circumvent the cost and time-intensive data collection and annotation, cross-lingual transfer learning can be used when few training data are available in the low-resource target language. Therefore, this study aims to explore the effectiveness of cross-lingual transfer learning in building an end-to-end Arabic task-oriented DS using the mT5 transformer model. We use the Arabic task-oriented dialogue dataset (Arabic-TOD) in the training and testing of the model. We present the cross-lingual transfer learning deployed with three different approaches: mSeq2Seq, Cross-lingual Pre-training (CPT), and Mixed-Language Pre-training (MLT). We obtain good results for our model compared to the literature for Chinese language using the same settings. Furthermore, cross-lingual transfer learning deployed with the MLT approach outperform the other two approaches. Finally, we show that our results can be improved by increasing the training dataset size.
|
|
Keyword:
Arabic language; cross-lingual transfer learning; mixed-language pre-training; mT5; multilingual transformer model; natural language processing; task-oriented dialogue systems
|
|
URL: https://doi.org/10.3390/math10050746
|
|
BASE
|
|
Hide details
|
|
22 |
Measuring Terminology Consistency in Translated Corpora: Implementation of the Herfindahl-Hirshman Index
|
|
|
|
In: Information; Volume 13; Issue 2; Pages: 43 (2022)
|
|
BASE
|
|
Show details
|
|
23 |
Comparative Study of Multiclass Text Classification in Research Proposals Using Pretrained Language Models
|
|
|
|
In: Applied Sciences; Volume 12; Issue 9; Pages: 4522 (2022)
|
|
BASE
|
|
Show details
|
|
24 |
The Role of Task Complexity and Dominant Articulatory Routines in the Acquisition of L3 Spanish
|
|
|
|
In: Languages; Volume 7; Issue 2; Pages: 90 (2022)
|
|
BASE
|
|
Show details
|
|
25 |
Leveraging Frozen Pretrained Written Language Models for Neural Sign Language Translation
|
|
|
|
In: Information; Volume 13; Issue 5; Pages: 220 (2022)
|
|
BASE
|
|
Show details
|
|
26 |
Analyzing COVID-19 Medical Papers Using Artificial Intelligence: Insights for Researchers and Medical Professionals
|
|
|
|
In: Big Data and Cognitive Computing; Volume 6; Issue 1; Pages: 4 (2022)
|
|
BASE
|
|
Show details
|
|
27 |
The Effects of Event Depictions in Second Language Phrasal Vocabulary Learning
|
|
|
|
BASE
|
|
Show details
|
|
30 |
ETHNOCULTURAL AND SOCIOLINGUISTIC FACTORS IN TEACHING RUSSIAN AS A FOREIGN LANGUAGE ...
|
|
|
|
BASE
|
|
Show details
|
|
31 |
The Effects of Event Depictions in Second Language Phrasal Vocabulary Learning ...
|
|
|
|
BASE
|
|
Show details
|
|
34 |
Toward an Epistemic Web
|
|
|
|
In: 197 ; RatSWD Working Paper Series ; 22 (2022)
|
|
BASE
|
|
Show details
|
|
35 |
StaResGRU-CNN with CMedLMs: a stacked residual GRU-CNN with pre-trained biomedical language models for predictive intelligence
|
|
|
|
BASE
|
|
Show details
|
|
37 |
An Empirical Study of Factors Affecting Language-Independent Models
|
|
|
|
BASE
|
|
Show details
|
|
38 |
„A Hund is er scho’“. Die Migration eines Ausdrucks und seine bayerisch-ungarische Transfergeschichte
|
|
|
|
BASE
|
|
Show details
|
|
39 |
Neural-based Knowledge Transfer in Natural Language Processing
|
|
|
|
BASE
|
|
Show details
|
|
40 |
Chinese Idioms: Stepping Into L2 Student’s Shoes
|
|
|
|
In: Acta Linguistica Asiatica, Vol 12, Iss 1 (2022) (2022)
|
|
BASE
|
|
Show details
|
|
|
|