Securing AI Infrastructure: The Network Behind the Models

By Taylor Thomas · 2026-08-30

Most AI security coverage looks at the model — what a prompt can trick it into, what its outputs leak. Look one layer down instead. A production AI deployment is built from infrastructure: typically storage holding training or fine-tuning data, a registry or artifact store holding model weights, often a vector database feeding retrieval context, and an API serving the model to callers. Not every deployment has every piece, but every piece a deployment has is an asset. That stack is attacked with ordinary techniques — stolen credentials, exposed services, lateral movement — and defended with ordinary controls. The work is knowing where the new assets are and applying the old discipline to them.

The Stack, Asset by Asset

Training data stores come first, because they shape everything downstream. Whoever can read them holds a copy of what may be the organization’s most concentrated data asset; whoever can write to them can poison what the model learns — quietly, since a model trained on tampered data misbehaves without leaving anything conventional malware scanning is built to catch; dataset validation, provenance tracking, and post-training behavioral evaluation are the controls that stand a chance of seeing the problem. Read access is exfiltration risk, write access is integrity risk, and both deserve controls matched to the corpus’s actual sensitivity — for a confidential corpus, that means crown-jewel treatment, not a shared file store.

Model artifacts are the second asset class, and they behave like a software supply chain. Teams routinely pull third-party base models, datasets, and fine-tuning adapters from public repositories — dependencies, in every sense that matters. OWASP’s current LLM Top 10 lists supply chain risks among its categories for exactly this reason: an unvetted artifact can carry compromise into the environment the way an unvetted package does. The registry holding your weights deserves signature verification against a trusted identity — or expected digests obtained through an independently protected channel, since a hash fetched from the same repository proves only consistency — plus provenance records and tight write control. Conventional malware scanning alone cannot reliably detect model substitution or establish artifact integrity; serialization-safety and model-scanning tools can prevent or detect some executable-payload cases, integrity verification catches unauthorized artifact modification, and behavioral testing can surface some backdoors — no single control covers every case.

Vector databases are the asset teams most consistently misjudge. A RAG store is built by embedding source documents, and it typically keeps the source passages to return at retrieval time. The store carries the documents’ content forward — queries hand back the retained passages, and the embeddings themselves can leak content through inversion and membership attacks. OWASP’s same list calls out vector and embedding weaknesses; the practical translation is that a vector store built from confidential material should be protected at least according to the sensitivity of the chunks and metadata it retains, with embeddings treated as potentially sensitive in their own right — and its network placement and access control should say so.

Inference endpoints round out the stack, and the rule there is the least exotic: a network-served inference endpoint is an API. Authentication and authorization proportionate to exposure, rate and resource limiting, input size limits, careful logging — the API security baseline applies before a single AI-specific concern enters the picture. An endpoint with no access control at any layer — no application auth, no mTLS or gateway in front — risks becoming somebody else’s free compute, and risks leaking whatever private data or state it fronts; prompt-level threats then stack on top of whatever the API layer failed to check. And because prompts and retrieved passages are often sensitive, security logging should be access-controlled and minimized, not an indiscriminate payload archive.

The Network Around the Stack

Two network disciplines do most of the containment work. Segmentation first: a training cluster needs its data pipeline and its administrators, not the campus network; a vector store needs its retrieval service, not every workstation. AI infrastructure is often built quickly, on insufficiently segmented networks, by teams optimizing for iteration speed — which makes deliberate east-west boundaries around it among the highest-leverage moves a network engineer can make, the same segmentation casework the SCOR blueprint’s network security domain drills (objective 2.3).

Egress control second. Constrained, monitored outbound paths raise the cost of two important failure modes: exfiltration of stolen artifacts and training data, and compromised components — a poisoned dependency, a tampered model runtime — calling out. Not a guarantee: attackers can stage data internally or abuse an allowed destination, which is why the monitoring half matters as much as the constraint. But a training cluster with unrestricted internet access is still a design smell; its legitimate destinations can be pinned down through authenticated proxies, fixed service endpoints, or private connectivity, and traffic outside them is signal. Identity is part of the network story too: service credentials and workload identity govern which components may talk at all, ground that zero trust for AI agents covers in depth. None of it works against infrastructure nobody inventoried — shadow AI applies to internal buildouts as much as to unsanctioned SaaS.

Where the Blueprint Meets This

The certification track now names this ground directly. The SCOR 350-701 v2.0 blueprint’s Security Concepts domain asks candidates to describe vulnerabilities in AI and LLM models — prompt injection, system prompt leakage, vector and embedding weaknesses, and supply chain (objective 1.3) — two of which are infrastructure problems as much as model problems. Study them the way this guide frames them: as assets on your network, with attack surfaces your existing craft already knows how to close.

Frequently Asked Questions

How is securing AI infrastructure different from securing the model itself?

Model-level security covers threats to the model's behavior — adversarial inputs like prompt injection, but also poisoning, backdoors, extraction, and privacy leakage. Infrastructure security protects the systems around the model: the data stores it was trained from, the registry its weights live in, the vector database behind retrieval, and the API serving it. The layers can fail separately — a weakly protected model-serving host on a flat network raises the blast radius of everything around it — and they interact: an injected prompt can drive tools toward exfiltration, and a compromised registry changes what the model is. Both need defending.

Why is a vector database a sensitive asset?

Because it carries your documents' content forward. Embeddings are numerical abstractions of source content — but they can still leak it through inversion, linkage, and membership attacks — and RAG stores typically keep the source passages themselves, returning them to queries — which is why retrieval should be permission-aware at the document level, and why a store exposed without proper authorization leaks to any caller. A store built from confidential material deserves access control and network placement at least matching the chunks and metadata it holds — treating it as harmless mathematical output is the mistake.

What can a network engineer contribute to AI security without ML expertise?

The controls that contain the blast radius. Map where training data, model registries, vector stores, and inference endpoints actually live; segment them so a GPU cluster is not broadly reachable from the campus network; constrain and monitor their egress; give inference APIs authentication proportionate to what they expose, and resource limits regardless; and make sure the telemetry covering those segments actually exists. Every one of those is network engineering — the assets are new, the craft is not.