Viet-Anh on Software Logo

What is: RAG?

SourceRetrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

Retriever-Augmented Generation, or RAG, is a type of language generation model that combines pre-trained parametric and non-parametric memory for language generation. Specifically, the parametric memory is a pre-trained seq2seq model and the non-parametric memory is a dense vector index of Wikipedia, accessed with a pre-trained neural retriever. For query xx, Maximum Inner Product Search (MIPS) is used to find the top-K documents z_iz\_{i}. For final prediction yy, we treat zz as a latent variable and marginalize over seq2seq predictions given different documents.