What Is a Large Language Model? Tokens, Transformers and Parameters Without the Hype

From text to output: the short version

When someone enters a prompt, the application first converts it into tokens. The model processes those tokens using numerical values learned during training, then produces a probability distribution over possible next tokens. A decoding method selects one, appends it to the sequence, and repeats the process until the response is complete or a stopping condition is reached.

That description sounds like autocomplete because, at the mechanical level, it is related to autocomplete. The striking capability comes from scale, training, architecture, post-training and the breadth of patterns represented in the learned parameters. Predicting tokens over many steps can produce an explanation, a software function, a structured record or a conversation. It still remains prediction, not a guarantee that the result is correct.

Four terms that make LLMs easier to understand

Training is not the same as using the model

During training, software adjusts parameters based on examples and an objective. This is computationally demanding and produces model checkpoints or weights. Post-training may further shape instruction following, safety behaviour or a particular task. Fine-tuning adapts an existing model by updating some or all of its parameters.

During ordinary inference, those trained values are loaded and applied. Your prompt may influence the current answer, but it does not normally rewrite the base model. A chat product can preserve conversation history, retrieve documents or store separate memories; those are application features around the model. This distinction matters when a provider says it does not train on customer data. That statement does not, by itself, answer where prompts are processed, how long they are retained, who can administer the system or which subprocessors receive data.

The model is not the whole AI system

A useful business application normally includes an interface or API, identity and access controls, a serving engine, storage, logging, monitoring and sometimes document retrieval or tools. The same weights can be served from a laptop, an internal GPU server, a Canadian facility or a foreign API. Those deployments can have very different control and risk profiles even though the underlying model is identical.

What “large” does—and does not—tell you

There is no single parameter threshold that turns a language model into a large language model. Dense models use all of their relevant parameters for a request, while mixture-of-experts architectures can contain many total parameters but activate only a subset for a token. Parameter count therefore cannot be treated as a direct purchasing score.

A larger model may handle some difficult tasks better, but it also requires more memory, power or distributed hardware. A smaller model can be a better production choice when the task is narrow, latency matters, data must remain local or the organization can ground the model with well-managed documents. The defensible approach is to test candidate models on representative work and measure useful outcomes, failure modes, latency and resource use.

Why fluent answers can still be wrong

An LLM produces a plausible continuation under its training and current context. It does not consult an authoritative source unless the surrounding application gives it one, and it does not acquire human judgment merely because the prose sounds confident. The Canadian Centre for Cyber Security advises users to expect that generative-AI output can be incorrect, biased, incomplete or nonsensical and to validate information before acting on it.

For a business, that means matching controls to consequence. Drafting internal brainstorming notes is different from approving a payment, interpreting a contract or issuing safety instructions. Useful safeguards include source retrieval, structured outputs, automated checks, human review, limited tool permissions and evaluations based on real tasks.

Open weights change where the model can run

With an API-only system, customers send requests to a provider-operated service. With an open-weight release, the trained parameters can be downloaded under stated terms and loaded into compatible infrastructure. That can support Canadian-hosted or on-premises inference and reduce dependence on one vendor endpoint.

Open weight is not automatically the same as open source. Licences differ, and access to final weights may not include the training data information and code required by the Open Source Initiative’s Open Source AI Definition. Our open weights versus open source guide explains the terminology; the dated Canadian open-weight model comparison covers deployment candidates without treating one benchmark as a procurement decision.

Frequently asked questions

Is an LLM a database?

No. Its parameters encode patterns learned during training; they are not a dependable record system. Retrieval can supply selected documents at inference time, while the source system should remain authoritative.

Does a local LLM learn every prompt?

Not in ordinary inference. An application may store chats or use them in later context, but updating model parameters requires a separate training or fine-tuning process.

Does running an LLM locally make every answer private?

Local execution can keep model processing on controlled equipment, but privacy also depends on interfaces, logs, plugins, tools, updates, remote administration and access controls. Verify the whole data path.

Primary sources reviewed August 24, 2026: Google Machine Learning Glossary; Google, What’s a large language model?; Vaswani et al., Attention Is All You Need; Canadian Centre for Cyber Security, Generative AI.