3 |
Textual Time Travel: A Temporally Informed Approach to Theory of Mind ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
Modeling Event Plausibility with Consistent Conceptual Abstraction ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
On the Systematicity of Probing Contextualized Word Representations: The Case of Hypernymy in BERT ...
|
|
|
|
Abstract:
Contextualized word representations have become a driving force in NLP, motivating widespread interest in understanding their capabilities and the mechanisms by which they operate. Particularly intriguing is their ability to identify and encode conceptual abstractions. Past work has probed BERT representations (Devlin et al., 2019) for this competence, finding that BERT can correctly retrieve noun hypernyms in cloze tasks. In this work, we ask the question: do probing studies shed light on systematic knowledge in BERT representations? As a case study, we examine hypernymy knowledge encoded in BERT representations. In particular, we demonstrate through a simple consistency probe that the ability to correctly retrieve hypernyms in cloze tasks, as used in prior work, does not correspond to systematic knowledge in BERT. Our main conclusion is cautionary: even if BERT demonstrates high probing accuracy for a particular competence, it does not necessarily follow that BERT‘understands’ a concept, and it cannot be ...
|
|
Keyword:
Computer and Information Science; Information and Knowledge Engineering; Intelligent System; Natural Language Processing; Neural Network
|
|
URL: https://dx.doi.org/10.48448/sa1q-9g43 https://underline.io/lecture/6663-on-the-systematicity-of-probing-contextualized-word-representations-the-case-of-hypernymy-in-bert
|
|
BASE
|
|
Hide details
|
|
8 |
Learning Efficient Task-Specific Meta-Embeddings with Word Prisms ...
|
|
|
|
BASE
|
|
Show details
|
|
9 |
Resolving Event Coreference with Supervised Representation Learning and Clustering-Oriented Regularization ...
|
|
|
|
BASE
|
|
Show details
|
|
10 |
Leveraging Lexical Resources for Learning Entity Embeddings in Multi-Relational Data ...
|
|
|
|
BASE
|
|
Show details
|
|
11 |
Distributional Semantics for Robust Automatic Summarization
|
|
|
|
BASE
|
|
Show details
|
|
|
|