How generative AI works

The first time Claude writes you a clean email in four seconds, it feels like it understands you. It doesn't, not the way a person does. And once you see what it is really doing, you stop being fooled by it and start being good with it. That is the whole point of this lesson: not to make you an engineer, but to give you the mental model that separates people who trust AI blindly from people who use it well.

No maths ahead. Just the idea underneath.

The one thing it does: guess the next word

Here is the engine, stripped to the metal. A language model predicts the next chunk of text, then does it again.

That's it. You give it some words, it works out the most likely next word, adds it, then looks at everything so far and works out the next next word, and on and on until the answer is done. It is autocomplete on your phone, grown up and gone to university.

Type "the capital of France is" and it will almost certainly say "Paris", because in everything it ever read, that is overwhelmingly what came next. Type "thanks for your email, I'll get back to you" and it can carry on in a hundred sensible ways, so it picks one that fits. The model is not fetching a stored answer. It is building the sentence forward, one likely step at a time.

Sounds almost too simple to be useful. The surprise of the last few years is that when you do this with a big enough model that has read enough text, "predict the next word" stops looking like a party trick and starts looking like reasoning. To predict the next word in a legal clause, a recipe, or a Python function well, the model has to have soaked up a great deal about how the world is put together. Depth comes out of that simple loop, not from some separate "thinking" part.

Where the smarts come from: training

So how does it know that "Paris" is the likely word? It learned patterns from an enormous amount of text: books, articles, websites, code, conversations. Not a tidy list of facts, but the raw stuff of how humans write.

Picture reading more than any person could in a thousand lifetimes, with one job: get better and better at guessing the hidden next word. Cover a word, guess it, check the real answer, nudge yourself when wrong. Do that billions of times and you slowly absorb grammar, tone, facts, arguments, the shape of a good email and a working formula. That long, one-off process is training, and it is where nearly all the cost and computing power go.

Two things follow that matter every day at work:

Its knowledge has an end date. Training finishes at some point, so the model's built-in knowledge stops there. Ask about something after that cut-off and, unless it can search the web, it is guessing from before. (Some tools, including Claude, can look things up live. That is a bolt-on, not the model's memory.)

It learned patterns, not a filing cabinet. The model did not save the pages it read. It distilled them into a sense of what usually follows what. That is why it can write about your situation, which appears in no book, and also why it cannot always pull back an exact fact on demand. Hold on to that second point. It explains the next section.

Tokens: how it reads and writes

One quick piece of plumbing, because the word shows up everywhere. Models don't read letter by letter, or quite word by word. They work in tokens: chunks of text, often about three-quarters of a word. "Cat" is one token. "Unbelievable" might be three ("un", "believ", "able"). A rough rule: 100 tokens is about 75 words.

Why should you care? Tokens are the unit for three things you will bump into:

• Limits. A model can only hold so many tokens at once (its "context window", the star of a later lesson). That cap is in tokens, not pages.
• Cost. If you use AI through an API or a paid tier, you often pay per token, in and out. Longer prompts and longer answers cost more.
• Speed. The model writes one token at a time, which is why you watch answers stream in rather than land all at once.

You will almost never count tokens by hand. But when someone says "it ran out of context" or "that was a lot of tokens", now you know they mean chunks of text, and roughly how many.

Why it sometimes makes things up

This is the most important part of the lesson, so here it is in one line: a model is a prediction machine, not a fact database, and it will confidently produce text that looks right but isn't. The polite word for this is a hallucination.

It helps to see why it happens, because then you can predict it. The model's whole drive is to produce plausible next words. Usually plausible and true line up, because true things are what it mostly read. But when it has no strong pattern to lean on, an obscure fact, a specific figure, a court case, a citation, it does not stop and say "I don't know". It does what it always does: it generates the most plausible-looking text. And a made-up statistic or a fake but believable book title can look extremely plausible. Fluent and wrong at the same time.

The tell is confidence. A hallucination doesn't arrive hedged and nervous. It arrives in the same calm, well-written voice as everything true the model says, which is exactly what makes it dangerous. Never read fluency as proof.

None of this makes the tool broken. It makes it a tool with a known failure mode, like a very fast, very well-read assistant who would rather give you an answer than admit a blank. Once you expect that, you manage it: ask for sources, paste in the real document instead of trusting memory, and check anything specific. The next section sorts the safe jobs from the risky ones.

What it's brilliant at, and where to take care

Generative AI is not equally good at everything. Match the task to the tool and it feels like a superpower. Hand it the wrong kind of job unchecked and it burns you. The split follows straight from how it works.

Notice the pattern. It is strongest when the raw material is in front of it: rewrite this, summarise that, turn these notes into an email, explain this paragraph. There is little to invent, so little to get wrong. It is weakest when you ask it to recall something specific from memory, because that is exactly when it might smoothly fill a gap.

So the working rule for the rest of your AI life: let it do the language, and you own the facts. Give it the source instead of trusting its memory. Ask it to show where an answer came from. Double-check every number, name, date, and quote before it leaves your hands. Do that and hallucinations shrink from a scary flaw to a manageable quirk.

And a reassuring note: the tools are getting better at this. Newer models make things up less, hedge more honestly when unsure, and can cite sources or search the web. The skill of checking never goes out of date, but you are not fighting the tool. You are steering it.