Language models are few-shot learners

OpenAI's 2020 paper, 'Language models are few-shot learners,' introduced GPT-3, a 175-billion parameter language model that can perform a wide variety of natural language tasks with just a handful of examples, without any gradient updates or fine-tuning. According to the OpenAI Blog, this demonstrated that scaling model size leads to emergent few-shot abilities, allowing the model to infer task intent from a few input-output pairs. This was a departure from prior approaches that required task-specific datasets and fine-tuning. The practical implication for developers building AI workflows is that they can leverage large language models like GPT-3 as general-purpose 'task engines'—simply provide a few examples in a prompt to get high-quality outputs for translation, summarization, question answering, and more. This reduces the barrier to integrating AI into custom workflows, as there is no need to collect large labeled datasets or train separate models. The paper has since influenced many subsequent tools and APIs that adopt in-context learning as a core feature.