| 21 |
Sentence Level Embedding Detoxification via Toxic Component Removal ...
|
|
: University of Virginia, 2022
|
|
Abstract:
When state of the art pre-trained language models are trained on internet discourse, they may unintentionally encode and perpetuate some of the hateful and toxic behavior known to exist within the training data. We hypothesize that the encoding of toxicity can be explained by a "toxic subspace" within sentence-level language model embeddings, the existence of which suggests that toxic features follow some underlying pattern and are thus removable. To identify this toxic subspace, we propose a method to find patterns in toxic lexical markers. Through our experiments, we demonstrate that the subspace we find can separate toxic and non-toxic examples in the latent space. We use this subspace to extract a toxic component within sentence embeddings, and demonstrate that the removal of this component causes the number of toxic representations to decrease. These results are promising and suggest that encoders can be reasonably detoxified. ...
|
|
Keyword:
Language Models; NLP for Social Good; Representation Learning
|
|
URL: https://dx.doi.org/10.18130/5pch-v272
https://libraetd.lib.virginia.edu/public_view/k06988420
|
|
BASE
|
|
Hide details
|
|
|
| 22 |
МОДЕЛИ ПРОСТОГО ПРЕДЛОЖЕНИЯ И ИХ СИНОНИМИЯ В ТУВИНСКОМ ЯЗЫКЕ ... : MODELS OF A SIMPLE SENTENCE AND THEIR SYNONYMY IN THE TUVAN LANGUAGE ...
|
|
|
|
BASE
|
|
Show details
|
|
|
| 23 |
LEXICAL AND SEMANTIC CHARACTERISTICS OF HYPONOMIC RELATIONS AND DEEPLY ANALYZING ITS FEATURES IN ENGLISH LINGUISTUCS ...
|
|
|
|
BASE
|
|
Show details
|
|
|
| 24 |
MIss RoBERTa WiLDe: Metaphor Identification Using Masked Language Model with Wiktionary Lexical Definitions
|
|
|
|
In: Applied Sciences; Volume 12; Issue 4; Pages: 2081 (2022)
|
|
BASE
|
|
Show details
|
|
|
| 25 |
Considering Commonsense in Solving QA: Reading Comprehension with Semantic Search and Continual Learning
|
|
|
|
In: Applied Sciences; Volume 12; Issue 9; Pages: 4099 (2022)
|
|
BASE
|
|
Show details
|
|
|
| 26 |
Analysis of the Full-Size Russian Corpus of Internet Drug Reviews with Complex NER Labeling Using Deep Learning Neural Networks and Language Models
|
|
|
|
In: Applied Sciences; Volume 12; Issue 1; Pages: 491 (2022)
|
|
BASE
|
|
Show details
|
|
|
| 27 |
Commonsense Knowledge-Aware Prompt Tuning for Few-Shot NOTA Relation Classification
|
|
|
|
In: Applied Sciences; Volume 12; Issue 4; Pages: 2185 (2022)
|
|
BASE
|
|
Show details
|
|
|
| 28 |
Transformer-Based Abstractive Summarization for Reddit and Twitter: Single Posts vs. Comment Pools in Three Languages
|
|
|
|
In: Future Internet; Volume 14; Issue 3; Pages: 69 (2022)
|
|
BASE
|
|
Show details
|
|
|
| 29 |
Correcting Diacritics and Typos with a ByT5 Transformer Model
|
|
|
|
In: Applied Sciences; Volume 12; Issue 5; Pages: 2636 (2022)
|
|
BASE
|
|
Show details
|
|
|
| 30 |
Predicting the Success of Internet Social Welfare Crowdfunding Based on Text Information
|
|
|
|
In: Applied Sciences; Volume 12; Issue 3; Pages: 1572 (2022)
|
|
BASE
|
|
Show details
|
|
|
| 31 |
Language Competition and Language Shift in Friuli-Venezia Giulia: Projection and Trajectory for the Number of Friulian Speakers to 2050
|
|
|
|
In: Sustainability; Volume 14; Issue 6; Pages: 3319 (2022)
|
|
BASE
|
|
Show details
|
|
|
| 32 |
Fuzzy Approach to Computational Classification of Burnout—Preliminary Findings
|
|
|
|
In: Applied Sciences; Volume 12; Issue 8; Pages: 3767 (2022)
|
|
BASE
|
|
Show details
|
|
|
| 33 |
An Information Theoretic Approach to Symbolic Learning in Synthetic Languages
|
|
|
|
In: Entropy; Volume 24; Issue 2; Pages: 259 (2022)
|
|
BASE
|
|
Show details
|
|
|
| 34 |
Emotion Classification from Speech and Text in Videos Using a Multimodal Approach
|
|
|
|
In: Multimodal Technologies and Interaction; Volume 6; Issue 4; Pages: 28 (2022)
|
|
BASE
|
|
Show details
|
|
|
| 35 |
Methods, Models and Tools for Improving the Quality of Textual Annotations
|
|
|
|
In: Modelling; Volume 3; Issue 2; Pages: 224-242 (2022)
|
|
BASE
|
|
Show details
|
|
|
| 36 |
How Cognitive Abilities May Support Children’s Bilingual Literacy Development in a Multilingual Society
|
|
|
|
In: Languages; Volume 7; Issue 1; Pages: 33 (2022)
|
|
BASE
|
|
Show details
|
|
|
| 37 |
Comparison of Text Mining Models for Food and Dietary Constituent Named-Entity Recognition
|
|
|
|
In: Machine Learning and Knowledge Extraction; Volume 4; Issue 1; Pages: 254-275 (2022)
|
|
BASE
|
|
Show details
|
|
|
| 38 |
Performance Study on Extractive Text Summarization Using BERT Models
|
|
|
|
In: Information; Volume 13; Issue 2; Pages: 67 (2022)
|
|
BASE
|
|
Show details
|
|
|
| 39 |
Cohort-Based Education and Other Factors Related to Student Peer Relationships: A Mixed Methods Social Network Analysis
|
|
|
|
In: Education Sciences; Volume 12; Issue 3; Pages: 205 (2022)
|
|
BASE
|
|
Show details
|
|
|
|
|
