Large language model

A large language model (LLM) is a computational model notable for its ability to achieve general-purpose language generation and other natural language processing tasks such as classification. Based on language models, LLMs acquire these abilities by learning statistical relationships from vast amounts of text during a computationally intensive self-supervised and semi-supervised training process.^[1] LLMs can be used for text generation, a form of generative AI, by taking an input text and repeatedly predicting the next token or word.^[2]

LLMs are artificial neural networks that utilize the transformer architecture, invented in 2017. The largest and most capable LLMs, as of June 2024^[update], are built with a decoder-only transformer-based architecture, which enables efficient processing and generation of large-scale text data.

Historically, up to 2020, fine-tuning was the primary method used to adapt a model for specific tasks. However, larger models such as GPT-3 have demonstrated the ability to achieve similar results through prompt engineering, which involves crafting specific input prompts to guide the model's responses.^[3] These models acquire knowledge about syntax, semantics, and ontologies^[4] inherent in human language corpora, but they also inherit inaccuracies and biases present in the data they are trained on.^[5]

Some notable LLMs are OpenAI's GPT series of models (e.g., GPT-3.5 and GPT-4, used in ChatGPT and Microsoft Copilot), Google's Gemini (the latter of which is currently used in the chatbot of the same name), Meta's LLaMA family of models, Anthropic's Claude models, and Mistral AI's models.