The AI Supply Chain

Why the Model You Didn’t Build Is Still Your Responsibility

7 MIN Read

You didn’t train the model. You didn’t write the weights. But when it gets exploited, it’s your breach to own.

INTRODUCTION

Introduction

Supply chain security entered the mainstream security consciousness through a series of high-profile incidents that demonstrated how a compromise at a single point in a software dependency chain could cascade into breaches affecting thousands of organizations downstream. The lesson was hard-learned but clear: you are responsible for the security of everything your systems depend on, not just the code your own team writes.

AI systems have introduced a new and more complex version of that same challenge. When an organization deploys an AI-powered application, the dependency chain beneath it is deep, often opaque, and frequently sourced from parties whose security posture has never been evaluated. Base models trained by third parties. Open-source model weights pulled from public repositories. Training datasets assembled from the internet, licensed data providers, or user contributions. Inference APIs operated by external vendors. Fine-tuning pipelines built on open-source frameworks maintained by communities with no security guarantee.

Every layer of that chain is a potential attack surface. And the organization deploying the AI application owns the risk of all of it.

02 / AI SECURITY & SUPPLY CHAIN

The AI Dependency Chain

To reason about AI supply chain risk, it helps to map the layers that sit beneath a typical AI application deployment:

Base models. The foundation of most AI applications today is a large pre-trained model — a general-purpose language model, vision model, or multimodal model that has been trained at scale on large datasets. These base models are typically produced by a small number of well-resourced organizations and accessed either through APIs or as downloadable model weights.

Fine-tuning layers. Organizations frequently fine-tune base models on their own data to adapt model behavior to specific use cases. Fine-tuning introduces the organization’s proprietary data into the training pipeline and creates a model whose behavior is a product of both the base model’s training and the fine-tuning data. Vulnerabilities or backdoors in the base model can survive fine-tuning and persist in the deployed application.

Third-party datasets. Training and fine-tuning data is frequently sourced from third parties — licensed data providers, public datasets, web scrapes, and user-contributed content. The integrity of that data is rarely validated with the rigor applied to software dependencies. Poisoned training data introduced at this layer can produce models that behave maliciously under specific trigger conditions.

Open-source frameworks and libraries. AI development relies heavily on open-source frameworks for training, inference, and deployment. These frameworks carry the same supply chain risks as any other open-source software dependency — malicious packages, compromised maintainer accounts, and vulnerabilities in widely used components.

Inference APIs. Organizations that access AI capabilities through APIs rather than hosting models directly are trusting the API provider with the availability of their AI functionality, the integrity of the model’s outputs, and the confidentiality of the data submitted through the API.

03 / AI SECURITY & SUPPLY CHAIN

How Backdoored Models Work

One of the most technically sophisticated AI supply chain attacks is the introduction of a backdoor into a model during training. A backdoored model behaves normally under ordinary conditions, producing outputs indistinguishable from a clean model. But when a specific trigger is present in the input — a particular word, phrase, image pattern, or data configuration — the model behaves in a predetermined malicious way.

The attack is dangerous for several reasons. It is difficult to detect because the model passes standard functional testing. It can survive fine-tuning if the backdoor was introduced in a base model that is subsequently fine-tuned. And the trigger can be designed to be obscure enough that it is never encountered in normal use — meaning the backdoor may remain dormant for extended periods before being activated.

For organizations that deploy AI models sourced from third parties — including open-source repositories — backdoor risk is a genuine and underappreciated concern. The barrier to downloading and deploying a model from a public repository is low. The validation rigor applied to those downloads is typically minimal.

04 / AI SECURITY & SUPPLY CHAIN

The Open-Source Model Risk Landscape

Public model repositories have democratized access to AI capabilities in ways that have been enormously beneficial to the security community and the broader technology ecosystem. They have also created a distribution channel for AI models with no consistent security vetting process.

The risk profile of open-source AI models is similar to the risk profile of open-source software packages, with some important differences:

• Model weights cannot be inspected the way source code can. Understanding what a model has learned requires behavioral testing, not code review.
• The provenance of a model — how it was trained, on what data, by whom — is often poorly documented and difficult to verify independently.
• Models are large artifacts, making comprehensive behavioral testing expensive and time-consuming compared to running a software test suite.
• The community governance structures that provide some security assurance for major open-source software projects are less mature in the AI model ecosystem.

This does not mean open-source models should not be used. It means they should be treated with the same scrutiny applied to any third-party dependency — and that scrutiny should be AI-specific.

05 / AI SECURITY & SUPPLY CHAIN

The AI Model Bill of Materials

Software Bill of Materials (SBOM) has become an established practice for documenting the components that make up a software artifact and their provenance. The concept translates directly to AI systems as an AI Model Bill of Materials (MBOM) — a structured record of the components, data sources, and dependencies that constitute a deployed AI model.

An MBOM for a production AI system should capture:

• The base model used, including its version, source, and the organization that produced it
• The training and fine-tuning datasets, including their sources, licensing terms, and any known integrity issues
• The frameworks and libraries used in training and inference pipelines, with version information
• The infrastructure and serving environment, including cloud provider and relevant configuration details
• Any third-party APIs or model services the application depends on

The MBOM serves multiple functions: it enables security teams to assess the risk profile of an AI deployment, respond more effectively when a vulnerability is disclosed in a dependency, and demonstrate governance maturity to regulators and auditors who are increasingly asking these questions.

06 / AI SECURITY & SUPPLY CHAIN

Practical Steps for AI Supply Chain Due Diligence

Establish model provenance requirements. Before deploying any AI model — open-source or commercial — document where it came from, who produced it, and what it was trained on. Make provenance documentation a deployment prerequisite, not an afterthought.

Apply behavioral testing before production deployment. Functional testing validates that the model performs its intended task. Security-oriented behavioral testing probes the model for unexpected behaviors, sensitivity to adversarial inputs, and responses to potential trigger conditions. Both are necessary.

Treat AI vendors as third-party risk subjects. AI API providers, model vendors, and fine-tuning platform operators should be assessed through the same third-party risk management process applied to any critical vendor. AI-specific assessment criteria — data handling practices, incident response capabilities, model integrity controls — should supplement the standard vendor questionnaire.

Monitor for supply chain disclosures. Vulnerabilities and security disclosures affecting AI frameworks, base models, and model repositories should be tracked as part of the organization’s vulnerability management program. This requires extending existing monitoring capabilities to cover AI-specific sources.

Maintain an MBOM for production AI systems. Build and maintain an MBOM for every AI system in production. Review it when dependencies change, when security disclosures are issued, and on a regular cadence aligned with the organization’s risk review processes.

KEY TAKEAWAYS

Key Takeaways

Recap: AI supply chain risk is the next frontier of third-party and dependency risk management. The organization deploying the AI application owns the risk of every layer beneath it.

• The AI dependency chain includes base models, fine-tuning layers, training datasets, open-source frameworks, and inference APIs — each with distinct risk characteristics
• Backdoored models can survive fine-tuning, pass functional testing, and remain dormant until triggered
• Open-source AI models require security-specific due diligence, not just capability evaluation
• An AI Model Bill of Materials provides the visibility needed to assess, manage, and respond to AI supply chain risk
• AI vendors should be assessed through the organization’s existing third-party risk management process with AI-specific criteria appended

Security / Strategy / Trust

Security isn't just technology.
It's trust.

Exploring identity, security strategy, AI security, and the human side of cybersecurity.