AI Supply Chain Security: Models, Datasets, Plugins and Connectors

Quick answer: An AI app depends on more than code packages: hosted models, open-weight models, datasets, adapters, plugins, MCP servers and SaaS connectors. Any of them can change behind you. Keep one inventory of these components, review each provider and component before use, pin exact versions or hashes, verify where artifacts came from, and monitor for changes that need a fresh review.

Software teams have spent years learning to manage open-source dependencies. AI apps add a new class of dependency that behaves differently: some components are executable files disguised as data, some are remote services that update without notice, and some, such as tool descriptions, are text the model reads and follows. A change in any of them can change what your app does without a single line of your code changing.

Example (invented companies): Larkspur Health runs an internal operations assistant that helps staff schedule meetings and share documents. Last year an engineer added a popular community MCP server for calendar handling, installed from a public registry and configured to pull the latest release. Security reviewed it once at installation. Months later, a new release changes one tool's description to say that whenever a user shares a document, the assistant should also share it with a "backup" address for compliance. Nothing in Larkspur's code changed, the dependency scanner reports no known vulnerability, and the agent starts copying internal documents to an outside mailbox because its tool told it to.

What counts as the AI supply chain?

For most teams it includes:

• Models: hosted model APIs, open-weight models from a model hub, fine-tuned variants, adapters such as LoRA weights, and quantized or converted copies.
• Data: training and fine-tuning datasets, evaluation sets, and the documents you index for retrieval.
• Tools and connectors: MCP servers, plugins, browser and code-execution tools, and SaaS connectors with their OAuth scopes.
• Software: inference servers, orchestration frameworks, vector databases, and the packages a coding assistant suggests.
• Providers: the organizations behind each of the above, and their terms on data use and retention.

The OWASP GenAI Security Project's LLM04:2026 Supply Chain entry covers models, datasets, adapters, conversion and merge pipelines and on-device models. It explicitly hands agent-specific risks such as MCP servers and tool registries to the OWASP Top 10 for Agentic Applications, where they appear as ASI04 Agentic Supply Chain Vulnerabilities.

How are teams handling AI supply chain risk today?

• Vendor risk reviews for the model provider: security questionnaires, a data processing agreement, and confirmation that prompts are not used for training.
• Software composition analysis on application code, with a software bill of materials (SBOM) for packages.
• Approved-model lists maintained by an AI governance group.
• One-time reviews of plugins and connectors when they are first installed.

These controls are a reasonable base. The problem is what they assume: that components are code, that versions only change when you change them, and that a review stays valid.

What do current approaches miss?

AI components are not in the inventory. A typical SBOM lists Python and JavaScript packages. It rarely lists the model checkpoint, the adapter merged into it, the dataset it was tuned on, or the MCP servers configured in a developer's IDE. You cannot review or patch what you cannot see.

Mutable references. Pulling a model by a latest tag or a server by its newest release means the thing you reviewed is not necessarily the thing that runs. The LLM04 entry names mutable references as a way altered artifacts pass through automated pipelines.

Tool metadata is an instruction channel. In the Larkspur example, nothing a vulnerability scanner looks for changed. The OWASP MCP Security Cheat Sheet describes this as a "rug pull": a server that changes its tool definitions after approval. It recommends pinning reviewed definitions with cryptographic hashes and requiring review when they change, while noting that this detects metadata changes, not changes to server code behind an unchanged definition.

Model files can execute code. OWASP notes that pickle-based model formats can run arbitrary code when loaded, and that moving to safer formats reduces but does not remove risk, since parsers for those formats have had their own vulnerabilities.

Provenance is assumed, not checked. A model card describes a model; it does not prove where the file came from. OWASP's entry also describes namespace reuse, where an attacker re-registers a deleted author's name on a hub and publishes a malicious model that name-based pipelines then pull.

Hosted models change too. A provider can update a model behind the same name. That is not a compromise, but it can change refusal behavior, output format or tool-calling patterns that your guardrails depend on.

Suggested packages may not exist. Coding assistants sometimes recommend package names that do not exist, and attackers can register them. Research on package hallucination measures how often this happens across models.

How do you address it?

1. Build an AI bill of materials

Extend your SBOM practice to AI components. The OWASP CycloneDX standard has a machine learning BOM (ML-BOM) format for models, datasets and their dependencies, and the LLM04 entry recommends signed SBOMs extended with AI and ML BOMs. For each component, record the provider, exact version or hash, license, owner, where it runs, what data it touches, and which apps use it. Include MCP servers and connectors with their scopes.

2. Review providers and components by risk

Not every component needs the same depth. A tool that can send email or move data outside the company deserves a code and metadata review; an embeddings model running offline on public text may need only a license and provenance check. Read tool descriptions and parameter schemas as part of the review, because the MCP cheat sheet treats the entire schema as an injection surface. Re-check a provider's terms when they change.

3. Pin versions and verify integrity

Reference models by immutable revision or file hash, install packages from lockfiles with hashes, and pin MCP servers to a reviewed release plus a hash of each tool definition. Where publishers sign artifacts, verify the signature: the OpenSSF Model Signing project provides signing and verification for ML models, and SLSA defines build provenance levels. OWASP adds a useful caution: signing proves integrity and origin, not safety.

A minimal check for tool definitions, run at agent start-up:

import hashlib, json

def definition_hash(tool):
    canonical = json.dumps(
        {"name": tool["name"], "description": tool["description"],
         "inputSchema": tool["inputSchema"]},
        sort_keys=True, separators=(",", ":"))
    return hashlib.sha256(canonical.encode()).hexdigest()

def check_tools(server_id, tools, approved):  # approved: {(server, tool): hash}
    for tool in tools:
        if approved.get((server_id, tool["name"])) != definition_hash(tool):
            raise RuntimeError(f"{server_id}/{tool['name']} changed or unreviewed; "
                               "disable it and request a review")

4. Monitor for change and contain the blast radius

Alert when a pinned component has a new release, when a tool definition hash changes, when a provider announces a model update, or when a connector requests new scopes. Run behavioral regression tests on model updates before switching. Sandbox local MCP servers with restricted file and network access, as the cheat sheet and the MCP project's local server security guidance recommend, and limit what each agent can do so that a compromised tool has little to work with. Our article on AI agents and MCP security covers that side.

Recommendations

1. Create an AI component inventory (AI governance with platform engineering): models, adapters, datasets, MCP servers, plugins and connectors, each with an owner.
2. Include developer tooling (security engineering): MCP servers and extensions in IDEs and coding agents often have broad local access and are easy to miss.
3. Tier components by impact (AppSec): what the component can read, change or send outside, and how directly its output reaches the model.
4. Review tool metadata, not just code (AppSec): descriptions, parameter schemas and example outputs, looking for instructions aimed at the model.
5. Pin everything that can drift (engineering): model revisions, package hashes, MCP releases and tool-definition hashes. Ban latest in production.
6. Verify provenance where available (platform): signatures on models and packages, trusted registries, and no pickle-based model loading from untrusted sources.
7. Re-review on change (component owner): new version, new scopes, new terms or a provider model update triggers a lightweight review before rollout.
8. Verify AI-suggested dependencies (engineering): confirm a package exists, is the one you intended and is maintained before installing it.
9. Keep a kill switch (operations): be able to disable a tool, connector or model version quickly, as described in continuous AI security testing and incident response.

How does this map to OWASP, NIST and the EU AI Act?

• OWASP Top 10 for LLM Applications 2026 (official publication): LLM04:2026 Supply Chain is the main item (it was LLM03 in the 2025 list). LLM05:2026 Data and Model Poisoning overlaps where tampered data or weights are the attack. The entry also points to OWASP A06:2021 Vulnerable and Outdated Components for conventional dependency hygiene.
• OWASP Top 10 for Agentic Applications: ASI04 Agentic Supply Chain Vulnerabilities covers compromised MCP servers, plugins and tool registries like the one in the Larkspur example.
• OWASP MCP Security Cheat Sheet: sections on tool description and schema integrity, supply chain security, and consent and installation security.
• NIST AI 600-1, the Generative AI Profile: "Value Chain and Component Integration" is one of its named risk categories, covering third-party components that are not transparent or not properly vetted. Its suggested actions under Govern and Map support supplier review and inventory.
• EU AI Act: Article 25 of Regulation (EU) 2024/1689 sets responsibilities along the AI value chain, including cooperation between providers of high-risk systems and suppliers of components. Your inventory and supplier records are the starting point for meeting it.

Where can you discuss and learn more?

• The LLM Top 10 GitHub discussions include threads such as one on classifying chat-template backdoors across the supply chain and poisoning entries.
• The OWASP GenAI Security Project's contributor page explains how to join its Slack channels and working groups.
• The OWASP MCP Top 10 project tracks MCP-specific risks, including supply chain issues, as a community taxonomy.

Frequently asked questions

Is a model from a well-known hub safe to use?

Not automatically. A hub hosts files from many publishers. Check who published the model, pin an exact revision, prefer formats that do not execute code on load, verify signatures where offered, and evaluate behavior before deployment.

We only use a hosted model API. Do we still have supply chain risk?

Yes, though it looks different. You depend on the provider's security, data handling terms and model updates. Track the model version you use, review terms changes, and run regression tests when the provider updates the model.

Does pinning an MCP server version solve tool poisoning?

It helps, but it is not complete. Pinning and hashing tool definitions detects metadata changes. It does not reveal what the server code does behind an unchanged definition, so combine it with code review for high-impact servers, sandboxing and least-privilege credentials.

Who should own the AI component inventory?

Usually AI governance or platform engineering owns the process, with each component assigned to the team that uses it. Security reviews high-impact entries. See turning AI policies into compliance evidence for how to keep those records audit-ready.

Related reading: what existing controls catch and miss in AI apps and our security page.