Concept

Large language model (LLM)

A type of AI trained on huge amounts of text to predict the next word, which lets it write, summarize, translate and answer questions.

Last reviewed · 2 sources · Suggest a correction

Contents
  1. In one line
  2. How it works, simply
  3. What it means for you
  4. Timeline
  5. In the news
  6. Sources

In one line

A large language model is a program that has learned, from enormous amounts of text, which word is most likely to come next. Repeated very fast and at huge scale, that one skill lets it hold a conversation, write an email or summarize a report.

How it works, simply

  1. Training. The model reads a vast amount of text and adjusts billions of internal settings so its next-word guesses get better.
  2. Tuning. Developers then train it further on examples of helpful, safe answers, often with human feedback.
  3. Using it. When you type a prompt, the model generates a reply one piece at a time, each piece based on everything before it.

Almost all modern LLMs use the transformer design, introduced by Google researchers in 2017.

What it means for you

An LLM predicts plausible text. It does not look facts up unless it is connected to search, which is why it can hallucinate. And because it follows instructions in whatever text it reads, it can be tricked by prompt injection.

Timeline

  1. Google researchers publish “Attention Is All You Need”, introducing the transformer architecture used by today’s LLMs.

In the news

Sources

  1. Attention Is All You Need, arXiv
  2. Introduction to large language models, Google for Developers

Every fact on this page comes from the sources above. Entries are re-checked when things change. Spotted something out of date? Tell us. Read our editorial policy.