Hallucination (LLM)

A hallucination is an output from a language model that sounds plausible and confident but is factually incorrect, fabricated, or unsupported by the information the model was given. The danger is precisely that hallucinations are fluent: the grammar is perfect, the tone is authoritative, and a wrong answer reads exactly like a right one. There is no stammer, no lowered voice, no tell. For anyone using an LLM on technical work — mining diagnostics, firmware questions, electrical sizing — this is the single most important failure mode to internalize before trusting a single answer.

Why it happens

A language model is trained to predict the most likely next token given everything before it — not to retrieve verified facts. When the training data is sparse, contradictory, or silent on a query, the model still produces its best statistical continuation rather than saying "I don't know," because fluent continuation is literally the only thing the machinery does. The result can be invented citations, fake part numbers, plausible-but-wrong voltages, or confidently described features that no firmware has ever shipped. Hallucination is intrinsic to how these systems work, not a bug awaiting a patch; newer models hallucinate less often, but the failure mode cannot be trained away entirely, and the residual errors get more convincing as fluency improves.

Reducing it

The most effective mitigation is grounding: give the model authoritative material and instruct it to answer from that material rather than from memory. That is the premise of retrieval-augmented generation, where relevant documents are fetched into the context window at question time — models are far more reliable at reading than at recalling. Beyond grounding: lower the temperature for factual tasks, ask for sources and check that they exist, prefer questions the model can answer by transforming supplied text over questions that demand recall, and validate anything that will touch hardware or money against trusted references. An AI agent with tools to look up real data will hallucinate far less than one answering from weights alone — but every mitigation reduces the rate, and none takes it to zero.

The workshop rule

Where hallucination bites hardest is exactly where LLMs are most useful: obscure, long-tail technical questions with thin public documentation — which describes most of ASIC repair. Ask about a specific error code on a specific board revision and the model faces the sparsest region of its training data with the same confident voice it uses for everything else. A fabricated "expected voltage" acted upon at the bench costs real hardware. This is why D-Central treats a model as a drafting tool, never a source of truth: every factual claim about mining hardware gets cross-checked against verified documentation before it ships or gets soldered. Adopt the same discipline with a local LLM: sovereignty means owning the stack and owning the verification. The model proposes; your documentation, your meter, and your judgment dispose.

Spotting one before it costs you

Fluency defeats casual review, so use structural tells instead. Be most suspicious of hyper-specific details you cannot immediately verify — exact register addresses, precise voltages, part numbers with plausible formatting — because specificity is where fabrication hides best. Check citations by existence, not by vibe: hallucinated references have real journals, real-sounding authors, and no existence. Ask the same question twice in different words; a model recalling knowledge answers consistently, while one confabulating tends to produce confident variations. Demand the source's actual wording when the answer matters, and require page-level pointers into documents you supplied. Above all, calibrate by stakes: a hallucinated movie recommendation costs nothing, a hallucinated "safe" voltage costs a hashboard, and a hallucinated command run as root costs a weekend. The model's confidence is constant across all three — your verification effort is the variable that has to change.