Combining retrieval and generation improves knowledge-intensive NLP tasks

RAG models integrate a pre-trained seq2seq generator with a neural retriever over Wikipedia, achieving state-of-the-art results on open-domain QA and generating more factual, diverse text than parametric-only baselines.

Based on the paper Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks by Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni and 8 others

Challenges in knowledge-intensive NLP and limitations of parametric models

Large pre-trained language models store factual knowledge in their parameters and perform well on many NLP tasks, but they struggle to precisely access and manipulate knowledge for knowledge-intensive tasks.

Such models cannot easily update their knowledge or provide provenance for their outputs, and they sometimes hallucinate incorrect facts.

Hybrid models that combine parametric memory with retrieval-based non-parametric memory can address these issues by allowing direct access to external knowledge sources, but prior work has focused mainly on extractive QA tasks rather than general generation.

You have read the opening. The rest is free with a Bezen account, and no card is needed.

Create a free account Log in

In the rest of this paper

How the method works, step by step, with a flow chart
Three key formulas from the paper, explained symbol by symbol
Three figures from the paper, and how to read them
How it was tested, what was found, and where it falls short

Challenges in knowledge-intensive NLP and limitations of parametric models

Sign in to read the whole paper