Self-Hosted LLM & AI-Agent Security Hardening (OWASP LLM Top 10, 2025)
Quick answer
Map your setup against the OWASP Top 10 for LLM Applications (2025) and apply the matching defense. The five that bite self-hosters hardest are prompt injection (treat all model input and output as untrusted, and gate agent tools behind approval), improper output handling (never eval or shell-exec model output), excessive agency (least-privilege tools + human-in-the-loop for anything irreversible), supply chain (verify model checksums before loading) and sensitive information disclosure (never put secrets in a prompt). The matrix below gives the attack, the concrete defense, a severity and whether it applies to chat, RAG or agent deployments, cross-referenced to MITRE ATLAS.
Running your own model is a sovereignty win — your data never leaves the box. It also makes you the security team. This page is the checklist we would hand a self-hoster: the OWASP Top 10 for LLM Applications (2025), each mapped to what the attack actually looks like when you run Ollama, vLLM or llama.cpp with your own RAG and agents, and what to do about it. As-of 2026-07-21. Severity and Applies to are D-Central's own field ratings; the OWASP 2025 list is itself an ordered ranking (LLM01 highest), not severity-scored. MITRE ATLAS technique names reflect the current AI-renamed taxonomy.
The matrix
| OWASP | Risk | Severity | Applies to | MITRE ATLAS |
|---|---|---|---|---|
LLM01:2025 |
Prompt Injection | Critical | Chat, RAG, Agent | LLM Prompt Injection (AML.T0051), incl. indirect (AML.T0051.001) |
LLM02:2025 |
Sensitive Information Disclosure | High | Chat, RAG, Agent | LLM Data Leakage |
LLM03:2025 |
Supply Chain | High | All | AI Supply Chain Compromise (AML.T0010) |
LLM04:2025 |
Data and Model Poisoning | High | RAG, Fine-tune | Poison Training Data (AML.T0020); Manipulate AI Model (AML.T0018) |
LLM05:2025 |
Improper Output Handling | High | Agent, App | n/a (a downstream injection sink — classic AppSec, not a distinct ATLAS technique) |
LLM06:2025 |
Excessive Agency | High | Agent | AI Agent Tool Invocation (AML.T0053) |
LLM07:2025 |
System Prompt Leakage | Medium | Chat, RAG, Agent | Extract LLM System Prompt (AML.T0056) |
LLM08:2025 |
Vector and Embedding Weaknesses | High | RAG | Exfiltration via AI Inference API / Extract AI Model (AML.T0024); LLM Prompt Injection: Indirect (AML.T0051.001) |
LLM09:2025 |
Misinformation | Medium | All | n/a (a reliability failure, not a single ATLAS technique) |
LLM10:2025 |
Unbounded Consumption | Medium | All | Denial of AI Service (AML.T0029); Extract AI Model (AML.T0024.002) |
Each risk, with the fix
LLM01:2025 Prompt Injection Critical
Crafted input (direct) or hostile text hidden in a retrieved document, web page or tool output (indirect) overrides your instructions — making a chatbot ignore its rules, a RAG assistant follow commands buried in a PDF, or an agent call a tool it should not.
Keep system, user and tool content in separate roles; never paste retrieved or tool text into the privileged instruction slot; provenance-tag retrieved content. Treat sanitization, regex filters and guard models as defense-in-depth, not a fix — they are bypassable, and model "roles" are not a hard security boundary. The load-bearing control is constraining what the model can DO: least privilege on tools, and human approval on every side-effectful action.
LLM02:2025 Sensitive Information Disclosure High
The model surfaces secrets it should not: PII from its RAG corpus, another user's data, credentials pasted into a prompt, or fragments memorized during fine-tuning.
Never place secrets, keys or credentials in a prompt or system prompt. Scrub PII from RAG sources before indexing; enforce per-user / per-tenant isolation in the vector store. Add a DLP-style output filter. Do not fine-tune on data you would not publish. Turn off verbose stack-trace error messages.
LLM03:2025 Supply Chain High
A model file, adapter, dataset or dependency is tampered with: a backdoored model on a hub, a typosquatted Python package, a poisoned base image — or worst of all, a checkpoint that runs code the moment you load it.
Prefer the safetensors format over pickle-based ones (.bin / .pt / .ckpt): many model and adapter formats execute arbitrary code the moment you load them, so a malicious checkpoint is remote code execution. Never load an untrusted pickle; scan model files (ModelScan / picklescan). Pull weights only from trusted publishers. A SHA256 proves the file is unaltered, not that it is safe or authentic — pair it with a publisher signature for authenticity (this is why D-Central publishes a hash for every flashable image). Pin dependency versions, scan containers, keep an SBOM, and rebuild from source where you can.
LLM04:2025 Data and Model Poisoning High
Poisoned pre-training, fine-tuning, RAG or embedding data implants a backdoor, bias or hidden trigger that only fires on specific inputs.
Vet the provenance of every training / RAG source; clean and validate datasets; isolate untrusted data from trusted. Version your datasets so you can diff and roll back. Red-team the model after any fine-tune — but note that testing with known triggers only finds backdoors you already know to look for; unknown-trigger backdoors can survive it. Use embeddings from a source you trust.
LLM05:2025 Improper Output Handling High
Model output is passed unsanitized into something powerful — a shell, a SQL query, an HTML page (stored XSS), or eval() — turning a hallucination or an injection into remote code execution.
Treat model output as hostile user input at the downstream boundary. Context-aware encode before rendering to HTML; use parameterized queries, never string-built SQL; never eval() or shell-exec model output; run any generated code in a throwaway sandbox. Apply a strict CSP on any web UI.
LLM06:2025 Excessive Agency High
An agent is handed too many tools, too much permission or too much autonomy, so one bad decision (or one successful injection) becomes a destructive real-world action — deleted files, spent money, sent messages.
Least privilege: give an agent the smallest set of tools and the narrowest scopes that do the job. Require human-in-the-loop for anything irreversible or costly. Use scoped, short-lived API tokens; set rate and spend caps; cap the agent's iteration count; log every action for audit.
LLM07:2025 System Prompt Leakage Medium
An attacker extracts your system prompt, exposing hidden logic, guardrail wording, or — worst case — credentials and connection strings someone stuffed into it.
Assume the system prompt is public. Never rely on it for a security control and never put a secret in it. Enforce authorization and business rules server-side, in code, not in prose the model can be tricked into repeating.
LLM08:2025 Vector and Embedding Weaknesses High
RAG-specific: embeddings can be inverted to reconstruct source text; a shared vector store leaks across tenants; poisoned or unauthorized documents get retrieved into the context (a delivery path for indirect injection).
Enforce access control on the vector store per user / tenant — retrieval must respect the same permissions as the source documents. Do not embed secrets. Sanitize and validate retrieved chunks before they hit the prompt. Encrypt the index at rest and monitor for anomalous retrieval patterns.
LLM09:2025 Misinformation Medium
The model states something false with total confidence — a wrong fact, a fabricated citation, or subtly broken code — and a human or a downstream system trusts it.
Ground answers in retrieval with visible citations; require human review for high-stakes output; never auto-execute generated code without a test/sandbox; make the UI communicate uncertainty. For factual work, cross-check against a source of truth rather than the model's memory.
LLM10:2025 Unbounded Consumption Medium
Expensive or looping queries exhaust your GPU/CPU, run up a cloud bill, or (on an agent) spin forever — and mass querying can extract enough behavior to clone the model.
Rate-limit per user and per IP; set hard token, timeout and input-size limits; cap concurrency and queue depth. Cap agent iterations. On self-hosted GPUs, bound context length and batch size so one request cannot starve the box. Alert on unusual usage.
Where this sits in your stack
Security is one layer of running local AI well. See also: local AI runtime comparison · self-hosted vector databases · local-AI telemetry audit · self-hosted MCP server directory · prompt injection · indirect prompt injection · LLM jailbreak.
Frameworks: the risk taxonomy is the OWASP Top 10 for LLM Applications 2025; the attacker-technique cross-references are from MITRE ATLAS. The defenses are D-Central's practical mapping for self-hosted deployments, not a substitute for a full threat model of your own system.
Related products, repair, and setup paths
- self-hosted AI for Bitcoiners hub
- plebs guide to self-hosted AI
- install Ollama in 10 minutes
- LM Studio vs Ollama vs llama.cpp
- connect local AI to Home Assistant and Obsidian
- self-hosted AI troubleshooting
- repurpose mining hardware into an AI hashcenter
- local AI model leaderboards
Last reviewed July 21, 2026.