1 |
Multi-SimLex: A Large-Scale Evaluation of Multilingual and Cross-Lingual Lexical Semantic Similarity
|
|
Vulic, Ivan; Baker, Simon; Ponti, Edoardo Maria; Petti, Ulla; Leviant, Ira; Wing, Kelly; Majewska, Olga; Bar, Eden; Malone, Matt; Poibeau, Thierry; Reichart, Roi; Korhonen, Anna
|
|
In: ISSN: 0891-2017 ; EISSN: 1530-9312 ; Computational Linguistics ; https://hal.archives-ouvertes.fr/hal-02975786 ; Computational Linguistics, Massachusetts Institute of Technology Press (MIT Press), 2020, 46 (4), pp.847-897 ; https://direct.mit.edu/coli/article/46/4/847/97326/Multi-SimLex-A-Large-Scale-Evaluation-of (2020)
|
|
Abstract:
Données et informations liées à la publication : https://multisimlex.com/ ; International audience ; We introduce Multi-SimLex, a large-scale lexical resource and evaluation benchmark covering datasets for 12 typologically diverse languages, including major languages (e.g., Mandarin Chinese, Spanish, Russian) as well as less-resourced ones (e.g., Welsh, Kiswahili). Each language dataset is annotated for the lexical relation of semantic similarity and contains 1,888 semantically aligned concept pairs, providing a representative coverage of word classes (nouns, verbs, adjectives, adverbs), frequency ranks, similarity intervals, lexical fields, and concreteness levels. Additionally, owing to the alignment of concepts across languages, we provide a suite of 66 cross-lingual semantic similarity datasets. Due to its extensive size and language coverage, Multi-SimLex provides entirely novel opportunities for experimental evaluation and analysis. On its monolingual and cross-lingual benchmarks, we evaluate and analyze a wide array of recent state-of-the-art monolingual and cross-lingual representation models, including static and contextualized word embeddings (such as fastText, M-BERT and XLM), externally informed lexical representations, as well as fully unsupervised and (weakly) supervised cross-lingual word embeddings. We also present a step-by-step dataset creation protocol for creating consistent, Multi-Simlex-style resources for additional languages. We make these contributions -- the public release of Multi-SimLex datasets, their creation protocol, strong baseline results, and in-depth analyses which can be be helpful in guiding future developments in multilingual lexical semantics and representation learning -- available via a website which will encourage community effort in further expansion of Multi-Simlex to many more languages. Such a large-scale semantic resource could inspire significant further advances in NLP across languages.
|
|
Keyword:
[INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI]; [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG]; [INFO.INFO-TT]Computer Science [cs]/Document and Text Processing; [SCCO.COMP]Cognitive science/Computer science; [SCCO.LING]Cognitive science/Linguistics; [SHS.INFO]Humanities and Social Sciences/Library and information sciences; [SHS.LANGUE]Humanities and Social Sciences/Linguistics; [SHS.STAT]Humanities and Social Sciences/Methods and statistics; Lexicon; Linguistic Resource; Multilinguality; Semantics; Typology
|
|
URL: https://hal.archives-ouvertes.fr/hal-02975786/file/coli_a_00391.pdf https://hal.archives-ouvertes.fr/hal-02975786 https://hal.archives-ouvertes.fr/hal-02975786/document
|
|
BASE
|
|
Hide details
|
|
2 |
Semi-Supervised Learning on Meta Structure: Multi-Task Tagging and Parsing in Low-Resource Scenarios
|
|
|
|
In: Conference of the Association for the Advancement of Artificial Intelligence ; https://hal.archives-ouvertes.fr/hal-02895835 ; Conference of the Association for the Advancement of Artificial Intelligence, Association for the Advancement of Artificial Intelligence, Feb 2020, New York, United States ; https://aaai.org/Conferences/AAAI-20/ (2020)
|
|
BASE
|
|
Show details
|
|
3 |
Automatic Dialect Adaptation in Finnish and its Effect on Perceived Creativity
|
|
|
|
In: 11th International Conference on Computational Creativity (ICCC’20) ; https://hal.archives-ouvertes.fr/hal-02977153 ; 11th International Conference on Computational Creativity (ICCC’20), Sep 2020, Coimbra, Portugal ; http://computationalcreativity.net/iccc20/ (2020)
|
|
BASE
|
|
Show details
|
|
4 |
Babel 2.0. Où va la traduction automatique ?
|
|
|
|
In: https://hal.archives-ouvertes.fr/hal-02265228 ; Odile Jacob, 2019, 9782738148490 ; https://www.odilejacob.fr/catalogue/sciences-humaines/linguistique-psycholinguistique/babel-20_9782738148490.php (2019)
|
|
BASE
|
|
Show details
|
|
5 |
"A Novel of Character": Towards the Automatic Annotation of Characters in a Large Corpus of French Novels
|
|
|
|
In: PROCEEDINGS OF THE INTERNATIONAL CONFERENCE «CORPUS LINGUISTICS–2019 ; International Conference on Corpus Linguistics 2019 ; https://hal.archives-ouvertes.fr/hal-02265134 ; International Conference on Corpus Linguistics 2019, Jun 2019, Saint Petersbourg, Russia ; https://events.spbu.ru/eventsContent/events/2019/corpora/corp_sborn.pdf (2019)
|
|
BASE
|
|
Show details
|
|
6 |
Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing
|
|
|
|
In: https://hal.archives-ouvertes.fr/hal-01856176 ; 2018 (2018)
|
|
BASE
|
|
Show details
|
|
7 |
Multilingual Dependency Parsing for Low-Resource Languages: Case Studies on North Saami and Komi-Zyrian
|
|
|
|
In: LREC 2018 Proceedings ; Language Resource and Evaluation Conference ; https://hal.archives-ouvertes.fr/hal-01856178 ; Language Resource and Evaluation Conference, ELRA, May 2018, Miyazaki, Japan ; http://www.lrec-conf.org/proceedings/lrec2018/pdf/600.pdf (2018)
|
|
BASE
|
|
Show details
|
|
8 |
Language, Cognition, and Computational Models
|
|
|
|
In: https://hal.archives-ouvertes.fr/hal-01722351 ; Cambridge University Press, 2018 ; https://www.cambridge.org/core/books/language-cognition-and-computational-models/90CC7DBA6CADB1FE361266D311CB4413 (2018)
|
|
BASE
|
|
Show details
|
|
9 |
Introduction: Cognitive Issues in Natural Language Processing
|
|
|
|
In: Language, Cognition, and Computational Models ; https://hal.archives-ouvertes.fr/hal-01722353 ; Language, Cognition, and Computational Models, Cambridge University Press, 2018, 9781316676974 ; https://doi.org/10.1017/9781316676974 (2018)
|
|
BASE
|
|
Show details
|
|
10 |
UDLex: Towards Cross-language Subcategorization Lexicons
|
|
|
|
In: Proceedings of the Fourth International Conference on Dependency Linguistics ; Fourth International Conference on Dependency Linguistics (Depling 2017) ; https://hal.archives-ouvertes.fr/hal-01856180 ; Fourth International Conference on Dependency Linguistics (Depling 2017), University of Pisa, Sep 2017, Pise, Italy (2017)
|
|
BASE
|
|
Show details
|
|
11 |
Machine Translation
|
|
|
|
In: https://hal.archives-ouvertes.fr/hal-01674140 ; MIT Press, 2017, 9780262534215 ; https://mitpress.mit.edu/books/machine-translation-0 (2017)
|
|
BASE
|
|
Show details
|
|
12 |
Lexical Knowledge Acquisition: Towards a Continuous and Flexible Representation of the Lexicon
|
|
|
|
In: Workshop on Cognitive Knowledge Acquisition and Applications ; https://hal.archives-ouvertes.fr/hal-01420714 ; Workshop on Cognitive Knowledge Acquisition and Applications, Jul 2016, New York, United States ; http://cognitum.ws/ (2016)
|
|
BASE
|
|
Show details
|
|
13 |
Exploring Natural Language Processing Methods for Finno-Ugric Langages
|
|
|
|
In: Second International Workshop on Computational Linguistics for Uralic Languages ; https://hal.archives-ouvertes.fr/hal-01273769 ; Second International Workshop on Computational Linguistics for Uralic Languages, Jan 2016, Szeged, Hungary (2016)
|
|
BASE
|
|
Show details
|
|
14 |
Traduire sans comprendre ? La place de la sémantique en traduction automatique
|
|
|
|
In: ISSN: 0458-726X ; EISSN: 1958-9549 ; Langages ; https://hal.archives-ouvertes.fr/hal-01273768 ; Langages, Armand Colin (Larousse jusqu'en 2003), 2016, 201 (2016)
|
|
BASE
|
|
Show details
|
|
15 |
The Role of Intrinsic Motivation in Artificial Language Emergence: a Case Study on Colour
|
|
|
|
In: COLING 2016, the 26th International Conference on Computational Linguistics ; 26th International Conference on Computational Linguistics (COLING 2016) ; https://hal.archives-ouvertes.fr/hal-01422478 ; 26th International Conference on Computational Linguistics (COLING 2016), Dec 2016, Osaka, Japan. pp.1646 - 1656 ; http://www.aclweb.org/anthology/C/C16/C16-1155.pdf (2016)
|
|
BASE
|
|
Show details
|
|
16 |
Lexical Knowledge Acquisition: Towards a Continuous and Flexible Representation of the Lexicon
|
|
|
|
In: Workshop on Cognitive Knowledge Acquisition and Applications ; https://hal.archives-ouvertes.fr/hal-01420714 ; Workshop on Cognitive Knowledge Acquisition and Applications, Jul 2016, New York, United States ; http://cognitum.ws/ (2016)
|
|
BASE
|
|
Show details
|
|
17 |
Combining Open Source Annotators for Entity Linking through Weighted Voting
|
|
|
|
In: Joint Conference on Lexical and Computational Semantics (*SEM 2015) ; https://hal.archives-ouvertes.fr/hal-01173967 ; Joint Conference on Lexical and Computational Semantics (*SEM 2015), Association for Computational Linguistics, Jun 2015, Denver, United States. pp.211-215 (2015)
|
|
BASE
|
|
Show details
|
|
18 |
EL92: Entity Linking Combining Open Source Annotators via Weighted Voting
|
|
|
|
In: 9th International Workshop on Semantic Evaluation (SemEval 2015) ; https://hal.archives-ouvertes.fr/hal-01173968 ; 9th International Workshop on Semantic Evaluation (SemEval 2015), Association for Computational Linguistics, Jun 2015, Denver, United States. pp.355-359 (2015)
|
|
BASE
|
|
Show details
|
|
19 |
Archaeology in the Digital Age: From Paper to Databases
|
|
|
|
In: Digital Humanities 2015 ; https://hal.archives-ouvertes.fr/hal-01173964 ; Digital Humanities 2015, Jun 2015, Sydney, Australia (2015)
|
|
BASE
|
|
Show details
|
|
20 |
Visualizing Data for Digital humanities ; Visualizing Data for Digital humanities: Producing Semantic Maps with Information extracted from Corpora and other Media
|
|
|
|
In: https://hal.archives-ouvertes.fr/hal-01173965 ; Jun 2015, Sydney, Australia. 2015, Proceedings of the conference "Digital Humanities 2015" (2015)
|
|
BASE
|
|
Show details
|
|
|
|