1 |
COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics ...
|
|
|
|
Abstract:
Many applications of text generation require incorporating different constraints to control the semantics or style of generated text. These constraints can be hard (e.g., ensuring certain keywords are included in the output) and soft (e.g., contextualizing the output with the left- or right-hand context). In this paper, we present Energy-based Constrained Decoding with Langevin Dynamics (COLD), a decoding framework which unifies constrained generation as specifying constraints through an energy function, then performing efficient differentiable reasoning over the constraints through gradient-based sampling. COLD decoding is a flexible framework that can be applied directly to off-the-shelf left-to-right language models without the need for any task-specific fine-tuning, as demonstrated through three challenging text generation applications: lexically-constrained generation, abductive reasoning, and counterfactual reasoning. Our experiments on these constrained generation tasks point to the effectiveness of ... : Updated. code: https://github.com/qkaren/COLD_decoding ...
|
|
Keyword:
Artificial Intelligence cs.AI; Computation and Language cs.CL; FOS Computer and information sciences; Machine Learning cs.LG
|
|
URL: https://arxiv.org/abs/2202.11705 https://dx.doi.org/10.48550/arxiv.2202.11705
|
|
BASE
|
|
Hide details
|
|
2 |
Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions? ...
|
|
|
|
BASE
|
|
Show details
|
|
3 |
Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions? ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristics ...
|
|
|
|
BASE
|
|
Show details
|
|
6 |
ParsiNLU: A Suite of Language Understanding Challenges for Persian ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
Evaluating Models' Local Decision Boundaries via Contrast Sets ...
|
|
|
|
BASE
|
|
Show details
|
|
8 |
TransOMCS: From Linguistic Graphs to Commonsense Knowledge ...
|
|
|
|
BASE
|
|
Show details
|
|
10 |
On the Possibilities and Limitations of Multi-hop Reasoning Under Linguistic Imperfections ...
|
|
|
|
BASE
|
|
Show details
|
|
11 |
Question Answering as Global Reasoning over Semantic Abstractions ...
|
|
|
|
BASE
|
|
Show details
|
|
12 |
Seeing Things from a Different Angle: Discovering Diverse Perspectives about Claims ...
|
|
|
|
BASE
|
|
Show details
|
|
13 |
Distributed knowledge based clinical auto-coding system
|
|
Kaur, Rajvir (S33301). - : U.S., Association for Computational Linguistics, 2019
|
|
BASE
|
|
Show details
|
|
|
|