How AI actually works. No analogies that fall apart on the second question.

What "AI" actually means in 2026.

When people say "AI" today, they almost always mean one specific kind of system: a large language model, or LLM. ChatGPT is an LLM. Claude is an LLM. Gemini, Llama, Mistral, DeepSeek — all LLMs. The same family of math underneath. A different brand on top.

An LLM is, mechanically, a function. You give it text. It gives you back text. That is it. Every dazzling output you have ever seen — the poem, the working Python code, the medical opinion you should not have asked for — is the same function being called with a different input.

What makes the function feel like magic is not what it does. It is the scale at which it does it. A modern LLM has somewhere between 50 billion and 2 trillion parameters: numbers that the model has learned, one at a time, from reading roughly the entire indexed internet. Each call to the function involves multiplying input through those parameters in a precise sequence. The output is, mathematically, the most likely next chunk of text given the input. Run that loop once and you get a word. Run it five hundred times and you get an essay.

A useful way to hold this in your head: the AI is not thinking. It is predicting. But it has been trained on so much human thinking that prediction at this scale starts to look indistinguishable from thinking. Whether the difference matters depends on the question you are asking it. — The honest one-paragraph summary of the field

The chapters that follow open up that function. Token by token, layer by layer. By the end of this page, "How does ChatGPT work?" will not be a hand-wavy question for you any more.

The token — the smallest thing the model sees.

An LLM does not read letters. It does not even read words, not really. It reads tokens. A token is a small chunk of text — usually three to six characters — produced by a separate program called a tokenizer that runs before the language model itself. The tokenizer's job is to split text into a small fixed vocabulary, often around 50,000 to 200,000 unique tokens.

Most common English words are a single token. Longer or rarer words split. Punctuation, spaces, even emojis become tokens.

Once text is tokens, and tokens are integer IDs, the model can begin its actual work. Everything downstream — every embedding, every attention head, every output probability — operates on these IDs, not on the original characters.

This has consequences. It is why the model can be terrible at counting letters ("how many r's in strawberry?" tripped LLMs for years — they were counting tokens, not letters). It is why some languages cost more to talk to than others — Tamil and Bengali tokenise into more tokens per word than English, so the same conversation costs more. And it is why prompt engineering sometimes feels arbitrary: you are not talking to a reader, you are giving instructions to a sequence-of-integers processor.

The embedding — meaning as coordinates.

An integer ID like 42816 has no meaning. The number itself is arbitrary. The model needs a way to turn each token into something that encodes meaning. The trick the field landed on is to map each token to a long vector of real numbers — typically 768, 1536, 4096 or more numbers — called an embedding.

Each number in the vector is a coordinate in a "meaning space" the model has learned. Tokens with similar meanings sit close together in that space. Tokens with different meanings sit far apart. This is one of the most important and least understood facts about how LLMs work — meaning lives in geometry.

The famous king − man + woman ≈ queen result, discovered by Mikolov in 2013, is the first hint of this. It said: if you take the embedding of "king", subtract the embedding of "man", add the embedding of "woman", you arrive remarkably close to the embedding of "queen". Meaning had become arithmetic. The field has been built on that observation ever since.

In a modern LLM, every input token is converted to its embedding before anything else happens. The embedding is what enters the network. The token ID is left at the door.

The attention mechanism, demystified.

If embeddings are the input, attention is the engine. Attention is what lets an LLM read a long passage and understand which earlier words matter most for predicting the next word. It is the single most important architectural idea in modern AI — and it is much simpler than it sounds.

The idea, in one sentence

For each word in the sequence, look at every other word in the sequence. Decide how relevant each one is. Take a weighted average of their embeddings using those relevance scores. That weighted average is what flows forward to the next layer.

That is it. The cleverness is in how the model decides relevance — which it does by giving each token three little projections of itself: a query, a key, and a value.

Each attention layer typically has between 12 and 96 heads running in parallel. Each head learns to attend to a different kind of relationship. One head might track grammatical subjects. Another tracks subject–verb agreement. Another tracks paragraph-level themes. Nobody programmed these. The training process discovered them.

This is the moment in the explanation when most learners pause and say: "wait, the model figures out that 'it' refers to 'mat' on its own?" Yes. It does. From the gradient signal alone, given billions of training examples. It is the single most impressive emergent behaviour in the field.

The transformer block, in one diagram.

A modern LLM is a stack of identical building blocks, called transformer blocks, stacked between 24 and 120 deep. Each block does two things in sequence. First, an attention layer mixes information across tokens. Second, a simple feedforward layer transforms each token individually. Both layers are wrapped in a small mathematical trick called a residual connection that lets information flow around the layer if needed.

Six lines. Stack 96 of them on top of each other. Wrap the input in a token embedding lookup. Wrap the output in a layer that maps the final hidden states back into a probability distribution over the vocabulary. That is the architecture of GPT-4. That is also, more or less, the architecture of Claude and Gemini and Llama. The differences are in scale, training data, and dozens of small tricks. The skeleton is the same.

This is the bewildering thing for someone meeting the architecture for the first time. The model is conceptually so simple. You can fit the entire algorithm on a postcard. And yet — out the other end — it can write a sonnet, debug your TypeScript, and explain quantum tunnelling to a teenager. The intelligence is not in the architecture. The intelligence is in the billions of learned numbers the architecture moves around.

Training — how an empty network learns anything.

At the start, every parameter in the network is a random number. The model is functionally nonsense. Feed it text and it will produce gibberish. Training is the process by which the random numbers become, slowly, the numbers that turn an empty network into GPT.

The training task is mechanically simple. You take a piece of real text — say, the first sentence of a Wikipedia article. You hide the last word. You ask the network to predict it. If the network's predicted word is right, you do nothing. If it is wrong, you compute exactly how wrong (this is the loss) and you nudge every parameter in the direction that would have produced a slightly less wrong answer.

You repeat this with another piece of text. And another. And another. Trillions of times. In modern training runs, the model sees somewhere between two and twenty trillion tokens. By the end, the nudges have accumulated into something extraordinary: the network has internalised the statistical structure of human writing.

What the model is actually learning

It is not memorising the training text. The text is far too big for the model's parameters to memorise. What the model is learning is the function behind the text. The patterns. The grammar. The logical structures. The factual associations. Slowly, statistically, painfully. By the millionth example, a small grammar starts to crystallise. By the billionth, a sense of subject-matter coherence. By the trillionth, what looks an awful lot like reasoning.

This is what people mean by "training" when they say GPT-4 cost a hundred million dollars to train. They mean: a few thousand specialised GPUs running this prediction-and-nudge loop continuously for several months, on the largest curated text dataset humanity has ever assembled. Every nudge is a tiny shift in some parameter. Add a trillion such nudges together and you get the modern LLM.

RLHF — why ChatGPT is polite.

The model that comes out of the training loop above can predict text. It cannot yet help you. It will happily complete your question with three more questions. It will produce unsafe answers. It will be inconsistent. It is, after the first training stage, a stunningly good autocomplete and not a useful assistant.

The second stage — the one that turns GPT into ChatGPT — is called reinforcement learning from human feedback, or RLHF. Human labellers are paid to rank the model's outputs. Better answer first. Worse answer second. Over hundreds of thousands of rankings, a small "reward model" learns to mimic human preference. Then the main LLM is fine-tuned to maximise that reward.

RLHF is what gives ChatGPT its voice — the politeness, the refusal to do dangerous things, the long structured answers with bullet points. Different labs make different RLHF choices and the result is that Claude sounds like Claude, ChatGPT sounds like ChatGPT, and Gemini sounds like Gemini, even though their base models are architecturally similar.

This is also the stage where most of the model's safety behaviour lives. The base model knows how to write malware. The RLHF-trained model has been heavily reinforced toward refusing. Whether this works is a research field of its own — and the fact that "jailbreaks" are still a category of attack tells you the RLHF layer is more like a polite suggestion than a hard wall.

What is still genuinely magical.

You now know roughly how an LLM works. Tokens, embeddings, attention, transformer stacks, the prediction loss, RLHF. That is, with surprising honesty, the whole picture. There is no secret module we left out.

And yet — most people who work on these models full time will tell you the same thing: we know how to build them, but we still do not fully understand why they work as well as they do.

Why does an LLM with enough parameters suddenly become capable of multi-step arithmetic when a slightly smaller one cannot? Why do certain training data mixtures produce dramatically better reasoning? Why does attention learn to look up factual knowledge in some heads and grammar in others, without being told to? Why is the loss curve smooth even when the capabilities emerging from it are jagged?

This is the active research field of mechanistic interpretability — opening up trained networks neuron by neuron, trying to figure out what each one has learned. We are years away from a satisfying answer. The architecture is on a postcard. The reason it works is not.

If you have read this far, you now have a more honest mental model of how modern AI works than ninety-nine per cent of the people writing op-eds about it. The next question is: what do you do with that understanding?

The honest answer is the obvious one. You learn to build with it. The chapters above are the conceptual skeleton. The skill itself comes from writing the code, training the tiny models, breaking them, and watching them work. That is what every Modern Age Coders AI course is built around — at every age, from eight to fifty.