1 |
IR-GAN: Image Manipulation with Linguistic Instruction by Increment Reasoning ...
|
|
|
|
Abstract:
Conditional image generation is an active research topic including text2image and image translation. Recently image manipulation with linguistic instruction brings new challenges of multimodal conditional generation. However, traditional conditional image generation models mainly focus on generating high-quality and visually realistic images, and lack resolving the partial consistency between image and instruction. To address this issue, we propose an Increment Reasoning Generative Adversarial Network (IR-GAN), which aims to reason the consistency between visual increment in images and semantic increment in instructions. First, we introduce the word-level and instruction-level instruction encoders to learn user's intention from history-correlated instructions as semantic increment. Second, we embed the representation of semantic increment into that of source image for generating target image, where source image plays the role of referring auxiliary. Finally, we propose a reasoning discriminator to measure ...
|
|
Keyword:
Computer Vision and Pattern Recognition cs.CV; FOS Computer and information sciences
|
|
URL: https://dx.doi.org/10.48550/arxiv.2204.00792 https://arxiv.org/abs/2204.00792
|
|
BASE
|
|
Hide details
|
|
2 |
DVCFlow: Modeling Information Flow Towards Human-like Video Captioning ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
Knowledge-guided Pairwise Reconstruction Network for Weakly Supervised Referring Expression Grounding ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
Adaptive Reconstruction Network for Weakly Supervised Referring Expression Grounding ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|