2 |
Understanding language through visual imagination
|
|
|
|
Abstract:
This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections. ; Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2019 ; Cataloged from student-submitted PDF version of thesis. ; Includes bibliographical references (pages 57-60). ; This thesis introduces a multimodal approach to natural language understanding by presenting a generative language-vision model that can generate videos for sentences and a comprehensive approach for using this capability to solve natural language inference, video captioning and video completion without task-specific training. The only training required is for acquiring a lexicon from captioned videos similar to the way children learn language through exposure to perceptual cues. The model generates videos by sampling the visual features of objects described in the target sentences over time. The evaluation results show that the model can reliably generate videos for sentences describing multiple concurrent and sequential actions, and that the ability to reason about language using visual scenes enables language tasks to be reduced to vision tasks and be solved more robustly using information obtained via vision. ; by Cheahuychou Mao. ; M. Eng. ; M.Eng. Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science
|
|
Keyword:
Electrical Engineering and Computer Science
|
|
URL: https://hdl.handle.net/1721.1/124257
|
|
BASE
|
|
Hide details
|
|
4 |
Minimal characterization of linguistic phenomena for robust ternary expression construction
|
|
|
|
BASE
|
|
Show details
|
|
6 |
Using a symbolic language parser to Improve Markov language models
|
|
|
|
BASE
|
|
Show details
|
|
9 |
Natural Language and Spatial Reasoning
|
|
|
|
In: http://people.csail.mit.edu/stefie10/publications/stefie10-thesis-proposal.pdf (2010)
|
|
BASE
|
|
Show details
|
|
11 |
Answering English Questions using ForeignLanguage, Semi-Structured Sources
|
|
|
|
In: http://groups.csail.mit.edu/infolab/publications/Katz-ForeignLanguage.pdf (2007)
|
|
BASE
|
|
Show details
|
|
12 |
Nuggeteer: Automatic Nugget-Based Evaluation Using Descriptions and Judgements
|
|
|
|
BASE
|
|
Show details
|
|
13 |
Lexical chains and sliding locality windows in contentbased text similarity detection
|
|
|
|
In: http://publications.csail.mit.edu/tmp/MIT-CSAIL-TR-2005-034.pdf (2005)
|
|
BASE
|
|
Show details
|
|
14 |
Using Syntactic Information to Identify Plagiarism
|
|
|
|
In: http://acl.ldc.upenn.edu/W/W05/W05-0207.pdf (2005)
|
|
BASE
|
|
Show details
|
|
15 |
A Comparative Study of Language Models for Book and Author Recognition
|
|
|
|
In: http://people.csail.mit.edu/ozlem/ijcnlp-05-UzunerO.pdf (2005)
|
|
BASE
|
|
Show details
|
|
16 |
Syntactic and semantic decomposition strategies for question answering from multiple resources
|
|
|
|
In: http://www.hlt.utdallas.edu/workshop2005/papers/Katz-Multiple-Resources.pdf (2005)
|
|
BASE
|
|
Show details
|
|
17 |
Syntactic and semantic decomposition strategies for question answering from multiple resources
|
|
|
|
In: http://www.aaai.org/Papers/Workshops/2005/WS-05-05/WS05-05-006.pdf (2005)
|
|
BASE
|
|
Show details
|
|
18 |
Lexical chains and sliding locality windows in contentbased text similarity detection
|
|
|
|
In: http://acl.ldc.upenn.edu/I/I05/I05-2026.pdf (2005)
|
|
BASE
|
|
Show details
|
|
19 |
Capturing Expression Using Linguistic Information
|
|
|
|
In: https://www.aaai.org/Papers/AAAI/2005/AAAI05-178.pdf (2005)
|
|
BASE
|
|
Show details
|
|
20 |
Identifying expression fingerprints using linguistic information
|
|
|
|
BASE
|
|
Show details
|
|
|
|