How large language models work, in plain English

You type a question into ChatGPT, Claude or Gemini and a fluent answer appears in seconds. Many people picture a search engine or a giant database of facts sitting behind the box. Neither picture is right, and both lead people to trust these tools in the wrong places. A large language model, or LLM, writes its answer one small piece at a time by predicting what is likely to come next.

This guide covers the four ideas you need to use these tools well: tokens, training, next-token prediction and the limits that follow from them. No maths is needed.

What a large language model is

Wikipedia describes an LLM as a model, usually a neural network, trained on huge amounts of text and used mainly for language tasks such as writing, summarising, translating and analysing text. The chatbots you know are applications built on top of these models. The application adds the chat window, safety filters and sometimes tools such as web search. The model underneath does the writing.

“Large” refers to two things: the amount of text the model learned from, and the number of parameters, which are the internal numbers adjusted during training. Google’s Machine Learning Crash Course notes that LLMs contain far more parameters than the recurrent models that came before them.

Tokens: how the model sees text

A model does not read words the way you do. Text is first split into tokens, which OpenAI’s help centre describes as “pieces of words”. A token might be a whole common word, part of a longer word, a punctuation mark or a word with its leading space. Google’s course gives the example of “cats” being split into “cat” and “s”.

OpenAI gives these rough rules of thumb for English:

• 1 token is about 4 characters.
• 1 token is about three quarters of a word.
• 100 tokens is about 75 words.

Tokens matter for two practical reasons. Paid AI services usually charge by the token, for what you send and for what comes back. And every model has a ceiling on how many tokens it can handle in one go, called the context window.

Training: where the knowledge comes from

Training happens in stages, long before you type a prompt.

1. Pretraining: the model is shown enormous amounts of text and asked again and again to predict hidden or next tokens. Each wrong guess nudges its parameters slightly. Google’s course points out that most of this training is unsupervised: nobody labels the answers, because the text itself supplies them. Grammar, facts, writing styles and patterns of reasoning all help with prediction, so the model picks them up along the way.
2. Instruction tuning: a pretrained model is good at continuing text but poor at following requests. Further training on examples of instructions paired with good responses teaches it to behave like an assistant.
3. Feedback training: many providers then use reinforcement learning from human feedback (RLHF). People rank responses, a reward model learns those preferences, and the LLM is tuned to score well against it. Another approach, Constitutional AI, guides this stage with a written set of principles.

Two consequences follow. A model’s built-in knowledge stops at its training cut-off, so it will not know about last week unless a tool fetches the information. And the model does not keep documents like a filing cabinet. What it holds is a compressed set of patterns, which is why it can explain a concept clearly and still misremember a specific figure.

Next-token prediction: how an answer gets written

When you send a prompt, the model reads every token in its context window: hidden instructions from the provider, the conversation so far, any attached files and your message. It then calculates a probability for each possible next token. Google’s course shows this with a fill-in-the-blank sentence about rain on a roof, where “cook soup” gets a 9.4% chance as the next phrase. The application picks a token, either the most likely one or a weighted random choice, adds it to the text and repeats. A 500-word answer takes several hundred of these steps.

The design behind this is the transformer. Its key mechanism, self-attention, asks for each token how much every other token affects its meaning. That is how the model can work out that “it” in a long sentence refers to the invoice and not the customer. Google’s course sums up the result as a very advanced form of autocomplete.

Because the choice of token usually involves some randomness, the same question asked twice can come back with different wording, and occasionally a different answer. That is normal, and a good reason to run an important prompt more than once.

The context window: the model’s working memory

Anthropic’s documentation defines the context window as all the text a model can reference when generating a response, including the response itself, and calls it a “working memory”. System instructions, earlier messages, uploaded files and the output all count towards it.

Two points catch people out. First, a model remembers nothing between separate requests unless the application sends the earlier conversation again, or a memory feature saves notes for it. Second, more context is not automatically better. Anthropic notes that as the token count grows, accuracy and recall degrade, an effect it calls context rot. A focused prompt with the relevant material usually beats pasting in everything you have.

The limits that follow from how they work

• Fluent is not the same as correct. The model is trained to produce likely text, not checked text. OpenAI’s 2025 research on hallucinations explains that arbitrary facts, such as a particular person’s birthday, cannot be predicted from patterns, so the model may produce a plausible guess instead.
• Bias from the training data: models can inherit and amplify biases in the text they learned from, including gender stereotypes and English-centric views.
• Agreeing too readily: Wikipedia lists sycophancy as a known weakness: a model may tell you what you want to hear rather than what is accurate.
• Hidden reasoning: LLMs are still widely regarded as black boxes. You cannot open one up to see why a particular answer came out.
• Prompt injection: instructions hidden inside a document or web page the model reads can redirect it. Treat outside content as untrusted.

What this means when you use one

An example (illustration only): the owner of a small bakery asks a chatbot, “What allergens are in our new pistachio cake?” The model has never seen her recipe, so it predicts a typical answer for pistachio cakes and lists nuts, gluten, eggs and milk. It sounds right. But her recipe also uses a sesame praline, which the model had no way of knowing. When she pastes the full recipe into the prompt and asks for the allergens found in that text only, quoting the ingredient behind each one, the answer is based on her data and she can check every line against the recipe in two minutes.

The same logic gives you a short set of working rules:

1. Give the model the material. Paste the policy, data or document instead of relying on what it remembers.
2. Ask for output you can check, such as quotes, references or a table.
3. Use it for drafts, structure, summaries and options, and verify facts, figures and anything legal, financial or medical yourself.
4. Keep prompts focused and start a fresh conversation when the topic changes.
5. Check your provider’s data settings and your own policies before pasting in personal or confidential information.

Common mistakes come from the wrong mental model: treating the chatbot as a search engine and accepting a confident answer about a niche fact, assuming it remembers last week’s chat, or judging a prompt on a single run.

When to use it and when not to

LLMs are a good fit when the work is mostly language and a person will review the result. That covers first drafts, rewriting for a different audience, summarising text you supply, brainstorming, explaining an unfamiliar topic and turning rough notes into structure.

They are a poor fit on their own when you need guaranteed accuracy with no review, up-to-date facts without a search tool, exact calculations without a calculator or code tool, or decisions about people without a human in the loop. In those cases the model can still help, but as one step in a process with checks around it.