Imagine you’re texting with a friend who has read a huge pile of books and messages. Every time you send a word, they don’t “look up the truth” in a database first — they guess what word should come next based on patterns they’ve seen before.
That’s the core idea behind an LLM: it is a very advanced autocomplete. It works with small pieces of text, remembers only a limited amount of the recent conversation at once, and keeps choosing the next piece over and over until it finishes an answer. Sometimes it sounds brilliant, and sometimes it confidently guesses wrong, because it is predicting text, not checking facts like a search engine.
What an LLM actually is
An LLM (large language model) is a model trained to predict the next token in a sequence. A token is a chunk of text: sometimes a word, sometimes part of a word, sometimes punctuation. Tokenization matters because token count affects both cost and the maximum amount of text the model can consider at once.
LLMs are usually part of the transformer family, which is a neural architecture good at using context efficiently. The model does not “store” every fact explicitly in a human-readable way; instead, it learns statistical patterns from massive training data.
This distinction explains a lot:
Training = learning pattern weights from data.
Inference = using those learned weights to generate new text.
Hallucinations happen when generation sounds plausible but is not grounded in a reliable source.
Tokens and tokenization
Before the model sees text, it is broken into tokens by a tokenizer. For example, common words may be a single token, while rare words might split into several pieces.
Why this matters:
More tokens usually means higher cost.
The model has a fixed context window, measured in tokens, so long prompts can push out earlier messages.
Tokenization is why two sentences with the same number of characters can cost very differently.
A useful mental model is that the model reads text as a sequence of numbered symbols, not as neat human words. That is also why spelling, formatting, and verbosity can affect behavior in subtle ways.
Next-token prediction and generation
The model’s job is to estimate a probability distribution over the next token. Given the current tokens, it assigns scores to many possible continuations, then one token is selected.
Generation is iterative:
Read the current context.
Predict the next token.
Append that token to the context.
Repeat until the answer is done.
This is why LLM output can drift: each new token depends on earlier generated tokens, so small mistakes can compound. The model is not writing a whole answer all at once; it is continuously choosing the next step.
Context windows and attention
The context window is the maximum number of tokens the model can use at once. If the conversation is too long, older tokens may be truncated or become less influential.
Transformers use attention to decide which earlier tokens matter most for the next prediction. That does not mean perfect memory. It means the model can weigh parts of the input differently, but only within the available window.
In practice, context limits matter when you:
paste large documents,
rely on long chat history,
ask the model to compare many details.
If important information falls outside the window, the model may answer as if it never existed.
Training vs inference, plus sampling
During training, the model sees huge amounts of text and adjusts its internal weights to reduce next-token prediction errors. During inference, those weights are frozen; the model is no longer learning from your prompt unless the system is doing separate fine-tuning or memory storage.
The final text also depends on sampling parameters:
Temperature controls randomness: lower is more conservative, higher is more varied.
Top-p limits choices to the smallest set of tokens whose total probability reaches a threshold.
“Deterministic” output is a myth in the general sense: even with low randomness, tiny changes in prompt, system settings, or model version can change the result.
These knobs shape style and repeatability, but they do not make the model magically more factual.
Why a confident answer can still be wrong
Suppose you ask: “Who won the 2025 Best Paper Award at X conference?” If the model has seen similar conference names, award patterns, and paper titles, it may generate a very plausible-looking answer even if it does not truly know the exact winner.
That is a hallucination: the output is fluent and contextually believable, but unsupported or incorrect. The cause is not malice; it is the model doing what it was trained to do — predict likely text.
A better prompt often reduces risk:
ask for the answer with sources,
provide the relevant document in the context window,
ask it to say “I don’t know” when evidence is missing.
This is why retrieval, citations, and verification are common mitigation strategies.
Model families at a glance
Different model families solve the same basic language problem in different ways or with different tradeoffs. The most common family today is the transformer, because it scales well and works strongly on text generation tasks.
You’ll also hear these terms:
Encoder-only models: good at understanding or classifying text.
Decoder-only models: good at generating text, which is what chat LLMs usually are.
Encoder-decoder models: good at transforming one sequence into another, like translation.
For interviews, the key point is not memorizing every architecture. It is knowing that “LLM” usually means a transformer-based model trained for next-token prediction, with behavior shaped by tokenization, context limits, and sampling.
Common interview traps
A few easy traps come up often:
Thinking the model stores facts like a database. It does not; it predicts text from learned patterns.
Confusing training with inference. Training changes weights; inference uses them.
Assuming bigger context means perfect recall. Important tokens can still be ignored or truncated.
Believing low temperature makes the model truthful. It only makes outputs more stable, not more correct.
Forgetting that token count drives both price and context usage.
If you want to sound sharp in an interview, always separate plausible text generation from grounded knowledge. Hallucination is not a bug in the narrow sense; it is a natural failure mode of next-token prediction when the model lacks reliable evidence.