Building an AI Security and Risk Framework
8 MIN Read
Most organizations have an AI strategy. Almost none of them have an AI risk strategy. That gap is where the exposure lives.
INTRODUCTION
Introduction
AI governance conversations in most organizations are dominated by ethics, fairness, and regulatory compliance. These are legitimate and important concerns. But they frequently crowd out a parallel conversation that security and risk professionals need to be having: how do we identify, assess, and manage the security risks that AI systems introduce?
An organization can have a thoughtful AI ethics policy and be entirely unprepared for a model theft attack. It can be fully compliant with emerging AI regulations and have no visibility into the AI capabilities embedded in its vendor products. It can have an AI strategy that the board has approved and no process for assessing the security posture of the AI systems that strategy depends on.
This article provides a framework for building AI governance that takes security seriously — drawing on established risk management disciplines and adapting them to the specific characteristics of AI systems.
02 / AI SECURITY & GOVERNANCE
The AI Risk Taxonomy
Effective AI risk governance begins with a clear taxonomy of the risk categories that AI systems introduce. Five categories deserve structured treatment:
Model risk. The risk that an AI model produces incorrect, biased, or manipulated outputs that lead to harmful decisions or outcomes. Model risk includes training data quality, model drift, adversarial manipulation, and the consequences of over-reliance on model outputs.
Data risk. The risk associated with the data used to train, fine-tune, or augment AI models. Data risk includes the exposure of sensitive training data, the integrity of the training pipeline, and the privacy implications of data used in model development.
Operational risk. The risk that AI systems fail, perform unexpectedly, or are exploited in ways that disrupt business operations. Operational risk includes availability of AI infrastructure, resilience against adversarial inputs, and the consequences of AI system failures on dependent processes.
Third-party risk. The risk introduced by AI capabilities sourced from external vendors, model providers, and API services. Third-party AI risk includes the security posture of AI providers, the data handling practices of AI platforms, and the supply chain risk of AI components embedded in vendor products.
Regulatory risk. The risk of non-compliance with emerging AI regulations and the legal consequences of AI systems that cause harm. Regulatory risk is evolving rapidly as jurisdictions develop AI-specific legislation and sector regulators issue AI guidance.
03 / AI SECURITY & GOVERNANCE
Existing Frameworks: Building on What Exists
Security practitioners do not need to build AI risk governance from scratch. Several frameworks provide structured approaches that can be adapted to organizational context:
NIST AI Risk Management Framework (AI RMF). Published by the National Institute of Standards and Technology, the AI RMF provides a voluntary framework for managing risks to the trustworthiness of AI systems. It is organized around four core functions — Govern, Map, Measure, and Manage — and is designed to integrate with existing risk management processes, including NIST’s Cybersecurity Framework.
MITRE ATLAS. The Adversarial Threat Landscape for Artificial-Intelligence Systems provides a knowledge base of adversarial tactics and techniques targeting AI systems, structured in a format parallel to MITRE ATT&CK. ATLAS is the most operationally useful resource available for AI threat modeling and provides the vocabulary security practitioners need to reason about AI-specific attack patterns.
ISO/IEC 42001. The international standard for AI management systems provides a framework for organizations that develop, deploy, or use AI systems, covering governance, risk management, and continuous improvement. ISO 42001 is particularly relevant for organizations seeking external certification of their AI governance practices.
OWASP Top 10 for LLM Applications. For organizations specifically deploying large language model applications, the OWASP LLM Top 10 provides a focused list of the most critical security risks, including prompt injection, insecure output handling, training data poisoning, and supply chain vulnerabilities.
04 / AI SECURITY & GOVERNANCE
Building an AI Asset Inventory
You cannot govern what you cannot see. The first operational step in AI risk governance is building a comprehensive inventory of AI assets across the organization. This is more complex than it sounds.
AI is present in the enterprise in three distinct forms that the inventory must capture:
- AI systems the organization owns and operates: models developed internally, fine-tuned on organizational data, or deployed on organizational infrastructure. These represent the highest level of governance responsibility because the organization controls the full stack.
- AI capabilities accessed through APIs and cloud services: third-party AI models accessed through APIs, including large language model services, computer vision APIs, and specialized AI platforms. The organization controls how these capabilities are used but not the underlying model or infrastructure.
- AI embedded in vendor products: AI capabilities that are part of SaaS applications, security tools, productivity platforms, and operational systems that the organization uses. This category is the most frequently overlooked because the AI is not visible as a distinct system — it is simply part of a product the organization already uses.
Each AI asset in the inventory should be classified by risk level, with classification driven by the sensitivity of the data the system processes, the access it holds, the decisions it informs, and the consequences of its failure or compromise.
05 / AI SECURITY & GOVERNANCE
AI-Specific Threat Modeling
Threat modeling is a well-established security practice. Applying it to AI systems requires extending the traditional approach to account for AI-specific attack vectors.
A structured AI threat modeling exercise should address:
- What data does this AI system process, and what are the consequences of that data being extracted or manipulated?
- What access does this system hold to downstream resources, and what could an attacker do with that access through prompt injection or other manipulation techniques?
- How could the model’s outputs be manipulated to cause harm, and what decisions or actions depend on those outputs?
- What are the trust boundaries in this system — where does the model receive input from potentially untrusted sources, and how are those inputs validated?
- What is the supply chain of this AI system — what training data, base models, and third-party components does it depend on, and what are the security implications of each dependency?
MITRE ATLAS provides the vocabulary for mapping identified threats to known adversarial techniques, enabling organizations to connect AI threat modeling outputs to detection and mitigation strategies.
06 / AI SECURITY & GOVERNANCE
Third-Party AI Risk
Third-party AI risk deserves specific attention because it is frequently underestimated. When an organization integrates an AI API into a product, it is trusting the AI provider with the data submitted through that API, the integrity of the model’s outputs, and the security of the provider’s infrastructure. These are significant trust commitments that should be evaluated explicitly.
Key questions for third-party AI vendor assessment include:
- What data retention and usage policies govern data submitted through the API? Are these commitments contractually enforceable?
- What security certifications does the AI provider hold, and are they relevant to the organization’s risk profile?
- How does the provider handle security incidents that affect the integrity of the model or the confidentiality of customer data?
- What is the provider’s approach to model updates — and what notification and testing processes exist before updated models are deployed to production?
- What supply chain dependencies does the AI provider have, and what is their approach to supply chain security?
07 / AI SECURITY & GOVERNANCE
Connecting AI Governance to Existing GRC Structures
One of the most common mistakes in AI governance is building a parallel governance structure that operates independently of existing GRC processes. The result is duplicated effort, inconsistent standards, and an AI governance program that lacks the organizational standing and integration needed to be effective.
AI governance is most effective when it is integrated into existing structures:
- AI assets should be included in the enterprise risk register alongside other technology risks, using the same risk rating methodology and reporting cadence
- AI vendor assessments should be incorporated into the existing third-party risk management process, using AI-specific assessment criteria as an extension of the standard vendor questionnaire
- AI security controls should be included in the existing control framework — mapped to existing control categories where applicable and documented as AI-specific controls where not
- AI incidents should be handled through the existing incident response process, with AI-specific playbooks as supplements rather than a separate IR program
- AI compliance requirements should be tracked in the existing compliance calendar alongside other regulatory obligations
KEY TAKEAWAYS
Key Takeaways
Recap: AI governance is most effective when it is built on established risk management disciplines — extended with AI-specific frameworks, not replaced by them.
- The AI risk taxonomy covers model risk, data risk, operational risk, third-party risk, and regulatory risk — each requires distinct governance treatment
- NIST AI RMF, MITRE ATLAS, ISO 42001, and OWASP LLM Top 10 provide complementary frameworks that together cover the governance, threat, standards, and application security dimensions of AI risk
- An AI asset inventory must capture owned AI systems, third-party AI APIs, and AI embedded in vendor products
- Threat modeling for AI systems extends traditional approaches with AI-specific attack vectors from MITRE ATLAS
- Third-party AI risk requires explicit vendor assessment with AI-specific criteria
- Integrating AI governance into existing GRC structures is more effective and sustainable than building parallel programs
The organizations that govern AI effectively will not be those that build the most elaborate AI-specific frameworks. They will be those that take AI risk seriously, apply established risk management discipline to a new problem domain, and build governance practices that scale with their AI adoption — before the adoption outpaces the governance.
Security / Strategy / Trust
Security isn't just technology.
It's trust.
Exploring identity, security strategy, AI security, and the human side of cybersecurity.
