Inference (LLM)

At inference time the model takes everything in its context window, instructions, conversation, retrieved passages, and generates a response token by token, each step a full forward pass through billions of parameters. Settings like temperature control how deterministic the output is; serving infrastructure controls speed and cost.

Training happens rarely; inference happens billions of times daily. The economics of those runs, GPU time per token, drive nearly every product decision in AI search.

Because every token costs compute, engines optimize ruthlessly: they retrieve a handful of sources instead of hundreds, prefer concise passages they can use efficiently, truncate long inputs, and route easy queries to smaller, cheaper models. Reasoning modes that spend more inference compute are reserved for hard questions.

For content owners, the implication is consistency with everything else in GEO: dense, structured passages are not just easier to retrieve, they are cheaper for an engine to consume, and product pipelines are tuned to favor exactly that.

Inference is probabilistic: the same prompt can yield different answers across runs, with different brands mentioned, because of sampling randomness, model routing, and shifting retrieval. A single manual check of "what does ChatGPT say about us" is therefore an anecdote, not a measurement, the basis of answer volatility.

Sound measurement samples repeatedly: running tracked prompts daily and computing mention rates over time. Geonimo does precisely this across engines, replacing one-off screenshots with statistically meaningful visibility trends.