DE eng

Search in the Catalogues and Directories

Hits 1 – 5 of 5

1
Vision-and-Language or Vision-for-Language? On Cross-Modal Influence in Multimodal Transformers ...
Abstract: Anthology paper link: https://aclanthology.org/2021.emnlp-main.775/ Abstract: Pretrained vision-and-language BERTs aim to learn representations that combine information from both modalities. We propose a diagnostic method based on cross-modal input ablation to assess the extent to which these models actually integrate cross-modal information. This method involves ablating inputs from one modality, either entirely or selectively based on cross-modal grounding alignments, and evaluating the model prediction performance on the other modality. Model performance is measured by modality-specific tasks that mirror the model pretraining objectives (e.g. masked language modelling for text). Models that have learned to construct cross-modal representations using both modalities are expected to perform worse when inputs are missing from a modality. We find that recently proposed models have much greater relative difficulty predicting text when visual information is ablated, compared to predicting visual object ...
Keyword: Computational Linguistics; Language Models; Machine Learning; Machine Learning and Data Mining; Natural Language Processing
URL: https://dx.doi.org/10.48448/j08d-fv18
https://underline.io/lecture/37781-vision-and-language-or-vision-for-languagequestion-on-cross-modal-influence-in-multimodal-transformers
BASE
Hide details
2
Multimodal Pretraining Unmasked: A Meta-Analysis and a Unified Framework of Vision-and-Language BERTs ...
BASE
Show details
3
Visually Grounded Reasoning across Languages and Cultures ...
BASE
Show details
4
On Language Models for Creoles ...
BASE
Show details
5
Visually Grounded Reasoning across Languages and Cultures ...
BASE
Show details

Catalogues
0
0
0
0
0
0
0
Bibliographies
0
0
0
0
0
0
0
0
0
Linked Open Data catalogues
0
Online resources
0
0
0
0
Open access documents
5
0
0
0
0
© 2013 - 2024 Lin|gu|is|tik | Imprint | Privacy Policy | Datenschutzeinstellungen ändern