2 |
Navigating the Kaleidoscope of COVID-19 Misinformation Using Deep Learning ...
|
|
|
|
BASE
|
|
Show details
|
|
3 |
HittER: Hierarchical Transformers for Knowledge Graph Embeddings ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
HETFORMER: Heterogeneous Transformer with Sparse Attention for Long-Text Extractive Summarization ...
|
|
|
|
BASE
|
|
Show details
|
|
6 |
Not All Negatives are Equal: Label-Aware Contrastive Loss for Fine-grained Text Classification ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
Contrastive Code Representation Learning ...
|
|
|
|
Abstract:
Anthology paper link: https://aclanthology.org/2021.emnlp-main.482/ Abstract: Recent work learns contextual representations of source code by reconstructing tokens from their context. For downstream semantic understanding tasks like code clone detection, these representations should ideally capture program functionality. However, we show that the popular reconstruction-based RoBERTa model is sensitive to source code edits, even when the edits preserve semantics. We propose ContraCode: a contrastive pre-training task that learns code functionality, not form. ContraCode pre-trains a neural network to identify functionally similar variants of a program among many non-equivalent distractors. We scalably generate these variants using an automated source-to-source compiler as a form of data augmentation. Contrastive pre-training outperforms RoBERTa on an adversarial code clone detection benchmark by 39% AUROC. Surprisingly, improved adversarial robustness translates to better accuracy over natural code; ContraCode ...
|
|
Keyword:
Computational Linguistics; Language Models; Machine Learning; Machine Learning and Data Mining; Natural Language Processing
|
|
URL: https://dx.doi.org/10.48448/ypyw-wq29 https://underline.io/lecture/37746-contrastive-code-representation-learning
|
|
BASE
|
|
Hide details
|
|
8 |
Unsupervised Multi-View Post-OCR Error Correction With Language Models ...
|
|
|
|
BASE
|
|
Show details
|
|
9 |
AttentionRank: Unsupervised Keyphrase Extraction using Self and Cross Attentions ...
|
|
|
|
BASE
|
|
Show details
|
|
10 |
Automatic Fact-Checking with Document-level Annotations using BERT and Multiple Instance Learning ...
|
|
|
|
BASE
|
|
Show details
|
|
11 |
Towards the Early Detection of Child Predators in Chat Rooms: A BERT-based Approach ...
|
|
|
|
BASE
|
|
Show details
|
|
12 |
Semantic Categorization of Social Knowledge for Commonsense Question Answering ...
|
|
|
|
BASE
|
|
Show details
|
|
13 |
Pre-train or Annotate? Domain Adaptation with a Constrained Budget ...
|
|
|
|
BASE
|
|
Show details
|
|
14 |
Stepmothers are mean and academics are pretentious: What do pretrained language models learn about you? ...
|
|
|
|
BASE
|
|
Show details
|
|
15 |
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization ...
|
|
|
|
BASE
|
|
Show details
|
|
16 |
Automatic Text Evaluation through the Lens of Wasserstein Barycenters ...
|
|
|
|
BASE
|
|
Show details
|
|
17 |
Combining sentence and table evidence to predict veracity of factual claims using TaPaS and RoBERTa ...
|
|
|
|
BASE
|
|
Show details
|
|
18 |
Meta Distant Transfer Learning for Pre-trained Language Models ...
|
|
|
|
BASE
|
|
Show details
|
|
20 |
Temporal Adaptation of BERT and Performance on Downstream Document Classification: Insights from Social Media ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|