1 |
TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing ...
|
|
Gui, Tao; Wang, Xiao; Zhang, Qi; Liu, Qin; Zou, Yicheng; Zhou, Xin; Zheng, Rui; Zhang, Chong; Wu, Qinzhuo; Ye, Jiacheng; Pang, Zexiong; Zhang, Yongxin; Li, Zhengyan; Ma, Ruotian; Fei, Zichu; Cai, Ruijian; Zhao, Jun; Hu, Xingwu; Yan, Zhiheng; Tan, Yiding; Hu, Yuan; Bian, Qiyuan; Liu, Zhihua; Zhu, Bolin; Qin, Shan; Xing, Xiaoyu; Fu, Jinlan; Zhang, Yue; Peng, Minlong; Zheng, Xiaoqing; Zhou, Yaqian; Wei, Zhongyu; Qiu, Xipeng; Huang, Xuanjing. - : arXiv, 2021
|
|
Abstract:
Various robustness evaluation methodologies from different perspectives have been proposed for different natural language processing (NLP) tasks. These methods have often focused on either universal or task-specific generalization capabilities. In this work, we propose a multilingual robustness evaluation platform for NLP tasks (TextFlint) that incorporates universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analysis. TextFlint enables practitioners to automatically evaluate their models from all aspects or to customize their evaluations as desired with just a few lines of code. To guarantee user acceptability, all the text transformations are linguistically based, and we provide a human evaluation for each one. TextFlint generates complete analytical reports as well as targeted augmented data to address the shortcomings of the model's robustness. To validate TextFlint's utility, we performed large-scale ...
|
|
Keyword:
Artificial Intelligence cs.AI; Computation and Language cs.CL; FOS Computer and information sciences
|
|
URL: https://arxiv.org/abs/2103.11441 https://dx.doi.org/10.48550/arxiv.2103.11441
|
|
BASE
|
|
Hide details
|
|
2 |
SpanNER: Named Entity Re-/Recognition as Span Prediction ...
|
|
|
|
BASE
|
|
Show details
|
|
3 |
Align Voting Behavior with Public Statements for Legislator Representation Learning ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
fastHan: A BERT-based Multi-Task Toolkit for Chinese NLP ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
{K-Adapter}: {I}nfusing {K}nowledge into {P}re-{T}rained {M}odels with {A}dapters ...
|
|
|
|
BASE
|
|
Show details
|
|
6 |
Defense against Synonym Substitution-based Adversarial Attacks via Dirichlet Neighborhood Ensemble ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
Causal Direction of Data Collection Matters: Implications of Causal and Anticausal Learning for NLP
|
|
|
|
In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (2021)
|
|
BASE
|
|
Show details
|
|
8 |
Classifying Dyads for Militarized Conflict Analysis
|
|
|
|
In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (2021)
|
|
BASE
|
|
Show details
|
|
9 |
Efficient Sampling of Dependency Structure
|
|
|
|
In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (2021)
|
|
BASE
|
|
Show details
|
|
10 |
Searching for More Efficient Dynamic Programs
|
|
|
|
In: Findings of the Association for Computational Linguistics: EMNLP 2021 (2021)
|
|
BASE
|
|
Show details
|
|
11 |
“Let Your Characters Tell Their Story”: A Dataset for Character-Centric Narrative Understanding
|
|
|
|
In: Findings of the Association for Computational Linguistics: EMNLP 2021 (2021)
|
|
BASE
|
|
Show details
|
|
12 |
A Bayesian Framework for Information-Theoretic Probing
|
|
|
|
In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (2021)
|
|
BASE
|
|
Show details
|
|
13 |
Improving Dialogue State Tracking with Turn-based Loss Function and Sequential Data Augmentation
|
|
|
|
BASE
|
|
Show details
|
|
14 |
Come hither or go away? Recognising pre-electoral coalition signals in the news
|
|
|
|
BASE
|
|
Show details
|
|
15 |
K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters ...
|
|
|
|
BASE
|
|
Show details
|
|
17 |
A Graph-based Model for Joint Chinese Word Segmentation and Dependency Parsing
|
|
|
|
In: Transactions of the Association for Computational Linguistics, Vol 8, Pp 78-92 (2020) (2020)
|
|
BASE
|
|
Show details
|
|
19 |
GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge ...
|
|
|
|
BASE
|
|
Show details
|
|
20 |
Distantly Supervised Named Entity Recognition using Positive-Unlabeled Learning ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|