1 |
Towards a Cleaner Document-Oriented Multilingual Crawled Corpus
|
|
|
|
In: https://hal.inria.fr/hal-03536361 ; 2022 (2022)
|
|
Abstract:
12 pages, 6 figures, 2 tables ; The need for raw large raw corpora has dramatically increased in recent years with the introduction of transfer learning and semi-supervised learning methods to Natural Language Processing. And while there have been some recent attempts to manually curate the amount of data necessary to train large language models, the main way to obtain this data is still through automatic web crawling. In this paper we take the existing multilingual web corpus OSCAR and its pipeline Ungoliant that extracts and classifies data from Common Crawl at the line level, and propose a set of improvements and automatic annotations in order to produce a new document-oriented version of OSCAR that could prove more suitable to pre-train large generative language models as well as hopefully other applications in Natural Language Processing and Digital Humanities.
|
|
Keyword:
[INFO.INFO-CL]Computer Science [cs]/Computation and Language [cs.CL]; Common Crawl; Language Modeling; Web corpus
|
|
URL: https://hal.inria.fr/hal-03536361
|
|
BASE
|
|
Hide details
|
|
2 |
Towards a Cleaner Document-Oriented Multilingual Crawled Corpus ...
|
|
|
|
BASE
|
|
Show details
|
|
3 |
MOR Digital: The Advent of a New Lexicographical Portuguese Project
|
|
|
|
In: eLex 2021 - Seventh biennial conference on electronic lexicography ; https://hal.inria.fr/hal-03195362 ; eLex 2021 - Seventh biennial conference on electronic lexicography, Jul 2021, Brno, Czech Republic ; https://elex.link/elex2021/ (2021)
|
|
BASE
|
|
Show details
|
|
4 |
MORDigital: The Advent of a New Lexicographical Portuguese Project
|
|
|
|
In: eLex 2021 - Seventh biennial conference on electronic lexicography ; https://hal.inria.fr/hal-03195362 ; eLex 2021 - Seventh biennial conference on electronic lexicography, Jul 2021, Brno, Czech Republic ; https://elex.link/elex2021/ (2021)
|
|
BASE
|
|
Show details
|
|
5 |
Ungoliant: An Optimized Pipeline for the Generation of a Very Large-Scale Multilingual Web Corpus
|
|
|
|
In: CMLC 2021 - 9th Workshop on Challenges in the Management of Large Corpora ; https://hal.inria.fr/hal-03301590 ; CMLC 2021 - 9th Workshop on Challenges in the Management of Large Corpora, Jul 2021, Limerick / Virtual, Ireland. ⟨10.14618/ids-pub-10468⟩ ; https://www.cl2021.org/ (2021)
|
|
BASE
|
|
Show details
|
|
6 |
Lexicographic Data Seal of Compliance
|
|
|
|
In: https://hal.archives-ouvertes.fr/hal-03344267 ; [Research Report] ELEXIS; DARIAH. 2021 (2021)
|
|
BASE
|
|
Show details
|
|
7 |
Building, Encoding, and Annotating a Corpus of Parliamentary Debates in XML-TEI: A Cross-Linguistic Account
|
|
|
|
In: ISSN: 2162-5603 ; EISSN: 2162-5603 ; Journal of the Text Encoding Initiative ; https://halshs.archives-ouvertes.fr/halshs-03097333 ; Journal of the Text Encoding Initiative, TEI Consortium, 2021 (2021)
|
|
BASE
|
|
Show details
|
|
8 |
Expanding the content model of annotationBlock
|
|
|
|
In: Next Gen TEI, 2021 - TEI Conference and Members’ Meeting ; https://hal.archives-ouvertes.fr/hal-03380805 ; Next Gen TEI, 2021 - TEI Conference and Members’ Meeting, Oct 2021, Virtual, United States (2021)
|
|
BASE
|
|
Show details
|
|
9 |
From disparate disciplines to unity in diversity How the PARTHENOS project has brought European humanities Research Infrastructures together
|
|
|
|
In: ISSN: 1753-8548 ; EISSN: 1755-1706 ; International Journal of Humanities and Arts Computing ; https://hal.inria.fr/hal-03402145 ; International Journal of Humanities and Arts Computing, Edinburgh University Press, 2021, 15 (1-2), pp.101-116. ⟨10.3366/ijhac.2021.0264⟩ (2021)
|
|
BASE
|
|
Show details
|
|
10 |
MORDigital ; The advent of a new lexicographical portuguese project
|
|
|
|
BASE
|
|
Show details
|
|
11 |
Ungoliant: An optimized pipeline for the generation of a very large-scale multilingual web corpus ...
|
|
|
|
BASE
|
|
Show details
|
|
12 |
Modelling Etymology in LMF/TEI: The 'Grande Dicionário Houaiss da Língua Portuguesa' Dictionary as a Use Case. ...
|
|
|
|
BASE
|
|
Show details
|
|
13 |
Modelling Etymology in LMF/TEI: The 'Grande Dicionário Houaiss da Língua Portuguesa' Dictionary as a Use Case. ...
|
|
|
|
BASE
|
|
Show details
|
|
14 |
Establishing a New State-of-the-Art for French Named Entity Recognition
|
|
|
|
In: LREC 2020 - 12th Language Resources and Evaluation Conference ; https://hal.inria.fr/hal-02617950 ; LREC 2020 - 12th Language Resources and Evaluation Conference, May 2020, Marseille, France ; http://www.lrec-conf.org (2020)
|
|
BASE
|
|
Show details
|
|
15 |
Modelling Etymology in LMF/TEI: The Grande Dicionário Houaiss da Língua Portuguesa Dictionary as a Use Case
|
|
|
|
In: LREC 2020 - 12th Language Resources and Evaluation Conference ; https://hal.inria.fr/hal-02618067 ; LREC 2020 - 12th Language Resources and Evaluation Conference, May 2020, Marseille, France ; http://www.lrec-conf.org (2020)
|
|
BASE
|
|
Show details
|
|
16 |
CamemBERT: a Tasty French Language Model
|
|
|
|
In: ACL 2020 - 58th Annual Meeting of the Association for Computational Linguistics ; https://hal.inria.fr/hal-02889805 ; ACL 2020 - 58th Annual Meeting of the Association for Computational Linguistics, Jul 2020, Seattle / Virtual, United States. ⟨10.18653/v1/2020.acl-main.645⟩ (2020)
|
|
BASE
|
|
Show details
|
|
17 |
A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages
|
|
|
|
In: ACL 2020 - 58th Annual Meeting of the Association for Computational Linguistics ; https://hal.inria.fr/hal-02863875 ; ACL 2020 - 58th Annual Meeting of the Association for Computational Linguistics, Jul 2020, Seattle / Virtual, United States. ⟨10.18653/v1/2020.acl-main.156⟩ ; https://acl2020.org (2020)
|
|
BASE
|
|
Show details
|
|
18 |
Building, Encoding, and Annotating a Corpus of Parliamentary Debates in XML-TEI: A Cross-Linguistic Account
|
|
|
|
In: https://halshs.archives-ouvertes.fr/halshs-03097333 ; 2020 (2020)
|
|
BASE
|
|
Show details
|
|
19 |
Modelling Etymology in LMF/TEI ; The Grande Dicionário Houaiss da Língua Portuguesa Dictionary as a Use Case
|
|
|
|
BASE
|
|
Show details
|
|
20 |
A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|