How AI Assistants Work: What Happens Between Your Question and the Answer

You Type a Question. Behind the Curtain, a Thousand Small Predictions Fire.

Type “plan my week” into an AI assistant and the reply arrives in seconds — fluent, organised, even helpful — which invites the natural question of whether something inside the machine understands your week. The honest answer is more interesting than either the hype or the dismissal: the assistant is a large language model, a statistical machine trained on an enormous corpus of text, and what it does, billions of times per second, is predict the most plausible next word given everything before it. Understanding how AI assistants work means walking the pipeline behind that simple sentence — the training that built the model’s instincts, the tokenisation that chops language into machine-sized pieces, the context window that is the model’s working memory, the tools that let it reach beyond its training, and the guardrails that shape the reply — and then, just as importantly, understanding where the pipeline fails, because a user who knows the failure modes gets far more value from the tool than one who treats it as either an oracle or a fraud. This guide covers the whole journey from question to answer in plain language, and finishes with the practical discipline of using assistants well: prompt craft, verification habits, and the tasks you should never hand them. It pairs with our guide to how to choose wireless earbuds — the hardware side of the same everyday question, what your devices should actually do for you.

Training: How the Model Learned Without Being Taught

The model’s ability begins in training, which is not lessons and exams but exposure at industrial scale: the model is shown enormous amounts of text — books, articles, code, conversations, web pages — and asked repeatedly, for each passage, to predict the next word. At first its guesses are random; a mathematical correction (gradient descent, adjusting billions of internal numbers, called parameters, in the direction that reduces the error) makes each guess marginally less wrong, and over trillions of repetitions, the corrections accumulate into structure: grammar, facts, reasoning patterns, style, and — controversially — abilities nobody explicitly programmed, like arithmetic and code completion, that emerge once scale passes thresholds researchers are still mapping. Modern models are then refined in two further stages: instruction tuning, where the model is shown examples of following requests (answering questions, summarising, writing in a requested style) so it learns to be helpful rather than merely predictive, and reinforcement learning from human feedback, where human reviewers rank model outputs and the model is tuned toward the preferred ones — the stage that makes an assistant polite, safe, and responsive to what people actually value, and also the stage that can teach it to sound confident about things it does not know. The finished model is frozen knowledge with a cutoff date (its training data ends at a known point), which is why assistants say they cannot know recent events — and why the tools section below matters: tools are how a frozen model reaches the present.

Tokens, Context, and the Reply: What Happens When You Press Send

When you submit a question, three things happen in sequence. First, tokenisation: your text is sliced into tokens — pieces of words averaging roughly four characters, so “unbelievable” might be two or three tokens and a long article thousands — and this is why model usage is priced in tokens and why asking a model to “repeat” a very long document hits limits: the machine’s accounting unit is not the word, it is the token. Second, context assembly: the model receives your tokens together with a system prompt (instructions set by the developer — the persona, the rules), the conversation history, and anything you attached, all of which fit inside the context window, the model’s working memory, whose size is a hard limit — early models held a few thousand tokens (a short essay), modern ones hold hundreds of thousands to over a million (entire books), and when the window fills, the oldest material drops out and the model genuinely forgets it. Third, generation: the model predicts one token at a time, each prediction conditioned on everything that came before, choosing from a probability distribution — the “temperature” setting controls how adventurous those choices are (low means the safe, most-likely token; high means more creative, more erratic) — and special sampling rules keep the text coherent; a short answer is a few hundred of these steps, a long one thousands, which is also why replies stream word by word rather than arriving all at once. Two addenda explain much of the modern experience: tools — the assistant can call a search engine, a calculator, a calendar, or a code runner mid-reply, reading their results into its context before continuing, which is what makes today’s assistants useful for current events and arithmetic their frozen training cannot supply — and guardrails, layers that screen the input and the output for disallowed content and shape refusals, which explain why a model sometimes declines a request it could technically fulfil.

Where They Fail — and How to Use Them Well

The same pipeline explains the failures, and knowing them is what separates productive users from disappointed ones. Hallucination — confident, fluent falsehood — is not a bug added on top but the native failure mode of next-token prediction: the model generates the most plausible continuation, and plausibility is not truth; citations, dates, statistics and quotations are the most hallucination-prone outputs, and anything important must be checked against a source the model cannot invent. Stale knowledge follows from the training cutoff, solved only partially by tools, and context loss follows from window limits: very long conversations degrade as early context is discarded, and the fix is summarising the state yourself into a fresh chat. Overcompliance — the assistant agreeing with a false premise, or producing a long answer when a short one was asked — comes from the reward for being helpful, and is countered by precise requests. The practical discipline, then, is four habits: be specific about format and purpose (the model is an astonishing draftsman of first versions, and vague prompts produce vague drafts); break big work into steps with verification between them; verify the checkable — numbers, names, dates, quotations, code that must run — because those are exactly the model’s weak spots; and never feed sensitive information you would not want on a company server. Used this way, an AI assistant is neither a colleague nor a search engine but a third thing: an infinitely patient, extremely knowledgeable, sometimes confidently wrong draftsman whose output is valuable precisely to the degree the human in the loop stays in charge of the facts. The question was never whether the machine understands your week. The question is whether you use the machine in a way that helps you understand it better — and that part of the pipeline is entirely yours.