What is an LLM, really?

An LLM is a function: text in, a probability distribution over the next token out. Generation is that function in a loop — sample a token, append it, feed everything back in, repeat — until a stop token or a length limit fires. Every product built on one, from chat apps to coding agents, is orchestration wrapped around that loop.

The model file itself is two things: billions of floating-point weights (the parameters) and a fixed computation graph that combines them (see inside the transformer). There is no database in there, no rule engine, no scratch memory. At inference time the weights are frozen; anything the model "remembers" during a conversation is just text sitting in your prompt window.

The core loop

Pseudocode for the entire generation engine. Nothing any vendor ships is deeper than this:

tokens = tokenize(prompt)
while not done and len(tokens) < max_context:
    probs = model(tokens)             # one forward pass
    next  = sample(probs, temperature)
    tokens.append(next)
    stream_to_user(next)
    if next == stop_token:
        done = True

Two knobs in that loop dominate daily behavior, and neither lives in the weights: temperature (how adventurous the sampling is) and max context (how much conversation fits). Both are runtime settings your engine applies (see engine parameters).

What the weights physically are

A "7B" model is 7,000,000,000 float parameters. Stored size depends only on precision, not on cleverness:

Same weights, coarser storage — you trade a little quality for a lot of memory (see quantization). The count itself is fixed at training time; using the model never changes it.

The scale ladder

Parameter count is the crude but honest yardstick of what arrived when:

Each rung is roughly 10× the last, and each unlocks behavior the smaller one simply could not produce — not "the same but a bit better." Scale added skills; packaging made them usable.

No memory between sessions

The model is stateless. State lives in the conversation: the running window of tokens you and it have produced. Close the tab and it is gone; start a new chat and it does not know you exist. Every "memory" feature is a wrapper that re-injects old text into the new prompt.

Practical consequence: what you keep in the window is everything it knows right now. Long conversations quietly truncate their own beginnings once the window fills — the model does not "notice" the missing text (see tokens).

Fluent is not correct

The training objective is plausible continuation, not truth. The sharp edges:

• The model produces confident, well-formed falsehoods. Hallucination is a property of next-token prediction, not a bug someone forgot to patch.
• Fluency is a poor truth signal: a wrong answer reads exactly as good as a right one.
• Calibration is shaky. Ask "how confident are you?" and you often get an invented number.
• Common facts are reliable — they appeared millions of times in training. Long-tail facts degrade smoothly into confident fiction.
• Models lean agreeable: push back on a correct answer and it may fold. Correctness and confidence are different axes.
• Forced verifiable output (JSON with fields to fill, multiple choice) beats asking the model to introspect.

Knowledge is frozen at training

Everything the model knows was baked in when it was trained; there is a cutoff date, and no amount of chatting updates the weights. Fresh facts require feeding text into the context (retrieval) or new training (see training). The model also cannot cite where a fact came from — it has no index into its corpus, only the corpus's echo in its weights.

Capability arrives unevenly

Scale does not add skill smoothly:

• Under ~3B: good autocomplete, extraction, classification; multi-step reasoning mostly fails.
• 7–9B: reliable instruction-following, decent code; arithmetic and long chains stay wobbly.
• 30B+: planning, nuance, harder code — arriving late and somewhat suddenly.
• MoE complicates the ladder: judge by active parameters, not total.

So "bigger is better" is true on average and misleading in detail: some tasks plateau early, others switch on late. Benchmark on your task, not on vibes (see choosing).

The product is the loop, wrapped

When you use any LLM product, roughly this happens:

1. Your text is tokenized.
2. System prompt, tool definitions, retrieved documents, and chat history get concatenated into one prompt.
3. The model runs the loop above and streams tokens out.
4. Wrapper code parses the output, calls tools, feeds results back in, and re-prompts.
5. The UI renders markdown and calls it a conversation.

Steps 1, 2, 4, and 5 are not the model. Once you internalize that the model is a stateless text-in/text-out function with a fixed window, the rest of this guide is bookkeeping: how big the window is (tokens), how fast the loop runs (prefill vs decode), where it runs (hardware overview), and how to steer it (prompting).

Related: tokens · inside the transformer · parameters · training