We develop an algorithm that improves language models’ performance on code generation tasks using minimal human-written feedback during training, making it user-friendly and sample-efficient.

We propose methods for pretraining language models with human preferences, resulting in much better preference satisfaction than standard pretraining-then-finetune paradigm.

We find that language models can self-correct their own biases against different demographic groups.

We’ve developed an automated way to generate evaluations with LMs. We test LMs using >150 LM-written evaluations, uncovering novel LM behaviors and risks.

Human participants who chat with an unreliable language model assistant substantially outperform both the model alone and their own unaided performance.

We describe our early efforts to red team language models in order to simultaneously discover, measure, and attempt to reduce their potentially harmful outputs.

We’re announcing the ISP: a $100k grand prize + $150k in additional prizes for finding an important task where larger language models do *worse*.

Training on odd data (e.g. tables from support.google.com) improves few-shot learning with language models in the same way as diverse NLP data.

Dataset of QA explanations with the goal of helping humans more reliably determine the correct answer when the ground truth can’t be directly determined.

KL penalties in RL with language models aren’t a hack; KL penalties have a principled, Bayesian justification.

We found a way to learn from language feedback (not ratings), enabling us to finetune GPT3 to human-level summarization with just 100 feedback samples.

Language models often generate undesirable text. We introduce methods for finding undesirable behaviors and training them away.

Language models (LMs) generate harmful text. We generate test cases (“red teaming”) using another LM, to catch harmful behaviors before impacting users.

Language models do much worse at few-shot learning when choosing prompts in a few-shot way instead of using large held-out sets (prior work).

Retrieval-augmented generation achieves SOTA on knowledge base question-answering.

We propose a theoretically-justified way to “probe datasets” for what capabilities they require of a model.

We present a single, retrieval-based architecture that can learn a variety of knowledge-intensive tasks: extractive and generative alike.

We decompose a hard question into several, easier questions with unsupervised learning, improving multi-hop question answering without extra supervision.

An honest reflection on FiLM conditioning layers based on the work that followed, including when (not) to use FiLM layers and how to train…

We find text evidence for an answer to a question by finding text that convinces Q&A models to pick that answer.

We introduce a simple yet effective baseline for multimodal BERT-like architectures that jointly finetunes unimodally pretrained text and image encoders.

We introduce a dataset for abstractive question-answering where answers are 100+ words long (many “how” and “why” questions).

Decoding FiLM conditioning parameters in multiple hops helps for more advanced vision-and-language tasks such as visual dialogue.

A review of a simple and surprisingly effective class of neural conditioning mechanisms.

We introduce a simulated environment for agents to learn from vision, audio, semantics, physics, and object-interaction within a realistic, household context.

We introduce a general-purpose neural network layer to integrate multimodal input to answer reasoning questions about images.

We achieve state-of-the-art semi-supervised image classification using a probabilistic graphical model underlying CNNs.