What is an LLM? Large language models for business readers
Easy Insight Team ·
An LLM, or large language model, is an AI system trained on very large amounts of text that generates language by repeatedly predicting the most likely next piece of text. It is the engine behind tools such as ChatGPT, Claude and Microsoft Copilot. It drafts, summarises and classifies well, but it predicts rather than knows, so check its answers.
How does a large language model work?
The UK Government's AI Playbook (February 2025) describes LLMs as large neural networks trained on text and natural language data. Training tunes the network on a huge body of text until it is very good at one job: continuing a passage plausibly.
Everything else follows from that. Microsoft Learn (updated June 2026) puts it plainly: when you ask a question, the model is not following a separate question-and-answer routine. It produces an answer because an answer is the most likely kind of text to follow a question.
What are tokens and the context window?
Two terms come up in every pricing page and product spec:
| Term | What it means | Why a business should care |
|---|---|---|
| Token | The unit of text the model reads and writes; common words are often one token, rarer words several | API prices and limits are counted in tokens, not words |
| Context window | Everything the model can see while answering: instructions, documents, conversation so far and its own reply | It is the model's working memory for one request, separate from what it learned in training |
Anthropic's context window documentation lists windows of up to one million tokens for its current models, as of October 2026, but also notes that accuracy and recall degrade as the context grows. Bigger is not automatically better: what you put in matters more than how much fits.
Why do LLMs get things wrong?
Because plausible is not the same as true. The AI Playbook warns that LLMs can create content that appears plausible but may be factually incorrect, can repeat bias from their training data, and often have no live access to the internet. These are often called hallucinations.
The practical fix is grounding: giving the model the source material to answer from. Microsoft calls this one of the most effective ways to get reliable answers, and recommends still validating outputs even with careful prompting.
What does a worked example look like?
Take a wholesaler that wants help answering trade customers' questions about delivery charges.
- Ungrounded: a member of staff asks a general chatbot "what is our delivery charge to Northern Ireland?". The model has never seen the price list, so it either says it does not know or, worse, produces a confident, believable figure.
- Grounded: the same question goes to an assistant that first retrieves the current delivery policy and pastes the relevant paragraph into the context window, with an instruction to answer only from it and say "not found" otherwise. Now the answer comes from the business's own document.
- Checked: for the first few weeks a person reviews each answer before it is sent, and logs what needed changing.
Step two is the retrieval layer we describe in our chatbot development cost guide, and step three is how a sensible AI proof of concept measures whether the approach works.
What does this mean for a small business?
In our view, treat an LLM as a fast, fluent first-draft writer that needs a source and a reviewer, not as a database. It suits drafting, summarising and sorting text where a person checks the result. If staff paste customer or employee details into a public tool, that is a data protection decision too: our GDPR and AI guide covers the checks.
We build grounded assistants on top of LLMs in our AI development work, as part of the wider AI practice.
Frequently asked questions
What is an LLM?
An LLM, or large language model, is an AI system trained on very large amounts of text that generates language by repeatedly predicting the most likely next piece of text. It is the engine behind tools such as ChatGPT, Claude and Microsoft Copilot.
Why do LLMs make things up?
Because they produce the most likely continuation of the text, not a checked fact. The UK Government's AI Playbook warns that LLMs can create content that looks plausible but is factually wrong. Giving the model your own documents to answer from, and checking its outputs, reduces the risk but does not remove it.
What is a token?
A token is the unit of text an LLM reads and writes. Common words are often one token, while rarer words are split into several pieces. Model limits and API prices are both counted in tokens rather than words.
Does an LLM know about my business?
No. It only knows what was in its training data and what you put in front of it in the current request. To answer from your policies, prices or records, those documents have to be supplied to the model each time, usually by a retrieval step that finds the relevant passages.
Easy Insight is a UK consultancy for AI, web, apps and data — senior specialists only, no juniors.
Next step
Wondering where AI would actually pay in your business?
An AI strategy and readiness review (from £2,500) tells you where AI pays, where it doesn't, and what to leave alone. The price is fixed in writing before we start, and the advice comes from a team that runs AI in its own products.