One API key, straight to the vendor.
Fits: Seed or Series A teams moving fast.
Top risks
- Keys copied across projects and laptops
- No inspection of what data leaves
- Shared keys hide who spent what
AI Security Reference Architectures
How AI reaches a model, takes action and ships to your customers, in eleven diagrammed models from a single API key to an enterprise control plane.
Most companies already run more AI than they govern. In Verizon's 2026 Data Breach Investigations Report, 45% of employees regularly use AI on their corporate devices, up from 15%, and 67% of users reach it through non-corporate accounts.
FlowsPrompt / responseTool callIdentity / credentialTelemetryUnsafe pathBlocked
BoundariesYour boundaryThird party
Start here
Most companies need two or three of these models, not all eleven. Find the rows that describe you and open those models first. Setup times assume a team of two to four with executive cover.
- Every company A2 Workforce AI AppsB1 Employee AI Agents Your people already use AI chat apps and hand agents their own access.
- Engineers call model APIs A1 Direct to ProviderA3 Gateway and Router Start direct with expiring keys and spend caps. Add a gateway once several teams share AI access.
- Engineers use coding agents B4 Developer AI In the IDE, in CI, or working tickets on their own.
- You build agents or workflows that take actions B3 Agents You Build From n8n flows to custom agents with their own tools.
- You buy an AI SOC or SRE agent B2 Vendor-Run Agents The vendor holds standing access to your systems.
- You ship AI to customers C1 Customer Agents on Your APIC2 AI Features in Your Product Their agents on your API, and AI features inside your product.
- Regulated, residency-bound or IP-sensitive data A4 In-Boundary Inference Inference inside your cloud or on models you run.
- About 200 people or more, with security and platform staff T Enterprise Reference The target state: every track under one control plane.
Never skip, even for speed
- No long-lived prod credentials where an agent can read them
- No long-lived personal access tokens to source code or sensitive SaaS
- Human review before code ships to production or any irreversible action
- Scoped, short-lived tokens for tool and repo access
- Spend limits on every model project or workspace
- Some form of logging, even if it is just a bucket
Track A: Model access
How prompts reach a model. Ordered from least to most control.
Employees using AI chat apps and the AI inside their SaaS.
Fits: Every company.
Top risks
- Sensitive data pasted into personal accounts
- Shadow AI tools with unknown data terms
- AI features switched on by default in existing SaaS
2 variants: Chat apps · AI inside your SaaS
Diagram, data flow and rolloutOne choke point for keys, cost, routing and inspection.
Fits: Series B to mid-market with several teams using AI.
Top risks
- Gateway outage stops all AI traffic
- Gateway logs hold sensitive prompts
- Bypass if teams keep direct keys
2 variants: Self-hosted · Managed or aggregator
Diagram, data flow and rolloutInference stays inside your cloud or cluster.
Fits: Regulated, residency-bound or IP-sensitive orgs.
Top risks
- Model catalogs differ by cloud
- Invocation logging off by default, with gaps on newer endpoints
- Malicious or backdoored open weights
2 variants: Managed cloud AI · Self-hosted open weights
Diagram, data flow and rolloutTrack B: Agent runtimes
How AI takes action. Ordered from least to most control.
Agents that act with one employee's access, inside your AI tenant or outside it.
Fits: Every company.
Top risks
- Instructions hidden in email, invites or web pages steer the agent
- The agent sends, shares or deletes as the user
- Broad OAuth scopes granted to third-party apps
2 variants: In your AI tenant · Outside your tenant
Diagram, data flow and rolloutAgents a vendor runs with its own access to your data and systems.
Fits: Companies that buy an AI SOC analyst or an AI SRE or on-call agent.
Top risks
- Stolen vendor tokens reach every connected system
- Write scopes granted for read-only work
- Attacker-written log data steers the analyst
Agents and AI workflows you run, with governed tool access.
Fits: Companies building agents or AI workflows that take actions, from an ops team's n8n flows to product engineering's custom agents.
Top risks
- One credential store holding tokens for everything
- Internet-exposed, under-patched workflow editors
- Excessive agency from broad tool scopes
2 variants: Low-code workflow · Custom agents
Diagram, data flow and rolloutAI that writes, reviews and ships code.
Fits: Engineering teams using AI across the SDLC.
Top risks
- Injection via issues, READMEs and dependencies
- Malicious MCP servers, extensions, skills and rules files
- Secrets exposed to fork or untrusted CI runs
2 variants: Attended and CI · Autonomous fleet
Diagram, data flow and rolloutTrack C: AI you ship
AI your customers use: their agents calling your product, and AI features inside it. Ordered from least to most control. Both call a Track A model and reuse the same gateways and toolshed.
Your customers' AI agents calling your product.
Fits: B2B SaaS whose customers connect AI agents to your API or an MCP server you publish.
Top risks
- Long-lived API keys pasted into agent configs
- Prompt-injected agents calling your tools
- Over-broad scopes across tenants
AI your customers use inside your product.
Fits: SaaS companies shipping chat, search, summaries or agents to customers.
Top risks
- One customer's data in another customer's answer
- Prompt injection from customer-uploaded content
- Customer data used for training without consent
Target state
Every track under one control plane.
Every track under one control plane, from vendor tenants to the AI you ship.
Fits: Companies of about 200 or more with dedicated security or platform staff.
Top risks
- Complexity slows adoption and drives shadow AI
- Misclassified data routed to the wrong tier
- Developer tooling outside the control plane
Reference
Comparison, the fast path guide, other frameworks and vendors by category.
Compare the models
Sorted by control, across all three tracks. The dots are the nine building blocks that recur across the models: build or buy each one once and every runtime reuses it.
| ID | Model | Control | Gateway | Toolshed | Identity | Secrets | Sandbox | Telemetry | Approval | Inventory | Endpoint |
|---|---|---|---|---|---|---|---|---|---|---|---|
| A1 | Direct to Provider | Low | AI gateway: not used | Toolshed / MCP gateway: not used | Identity (SSO, workload, agent ID, OBO): recommended | Secrets manager: recommended | Sandboxed execution: not used | Telemetry to SIEM: recommended | Human approval gate: not used | AI inventory and posture: not used | Endpoints (MDM + EDR): recommended |
| A2 | Workforce AI AppsChat apps · AI inside your SaaS | Low to Med | AI gateway: not used | Toolshed / MCP gateway: not used | Identity (SSO, workload, agent ID, OBO): core | Secrets manager: not used | Sandboxed execution: not used | Telemetry to SIEM: recommended | Human approval gate: not used | AI inventory and posture: core | Endpoints (MDM + EDR): core |
| B1 | Employee AI AgentsIn your AI tenant · Outside your tenant | Low to Med | AI gateway: not used | Toolshed / MCP gateway: recommended | Identity (SSO, workload, agent ID, OBO): core | Secrets manager: recommended | Sandboxed execution: recommended | Telemetry to SIEM: core | Human approval gate: provided by the vendor | AI inventory and posture: core | Endpoints (MDM + EDR): core |
| A3 | Gateway and RouterSelf-hosted · Managed or aggregator | Med to Med-High | AI gateway: core | Toolshed / MCP gateway: not used | Identity (SSO, workload, agent ID, OBO): core | Secrets manager: core | Sandboxed execution: not used | Telemetry to SIEM: core | Human approval gate: not used | AI inventory and posture: recommended | Endpoints (MDM + EDR): not used |
| B2 | Vendor-Run Agents | Med | AI gateway: not used | Toolshed / MCP gateway: not used | Identity (SSO, workload, agent ID, OBO): core | Secrets manager: core | Sandboxed execution: not used | Telemetry to SIEM: core | Human approval gate: core | AI inventory and posture: core | Endpoints (MDM + EDR): not used |
| C1 | Customer Agents on Your API | Med to High | AI gateway: not used | Toolshed / MCP gateway: recommended | Identity (SSO, workload, agent ID, OBO): core | Secrets manager: core | Sandboxed execution: not used | Telemetry to SIEM: core | Human approval gate: recommended | AI inventory and posture: recommended | Endpoints (MDM + EDR): not used |
| B3 | Agents You BuildLow-code workflow · Custom agents | Med to High | AI gateway: core | Toolshed / MCP gateway: core | Identity (SSO, workload, agent ID, OBO): core | Secrets manager: core | Sandboxed execution: core | Telemetry to SIEM: core | Human approval gate: core | AI inventory and posture: core | Endpoints (MDM + EDR): not used |
| B4 | Developer AIAttended and CI · Autonomous fleet | Med to High | AI gateway: core | Toolshed / MCP gateway: core | Identity (SSO, workload, agent ID, OBO): core | Secrets manager: core | Sandboxed execution: core | Telemetry to SIEM: core | Human approval gate: core | AI inventory and posture: core | Endpoints (MDM + EDR): core |
| C2 | AI Features in Your Product | High | AI gateway: core | Toolshed / MCP gateway: recommended | Identity (SSO, workload, agent ID, OBO): core | Secrets manager: core | Sandboxed execution: recommended | Telemetry to SIEM: core | Human approval gate: recommended | AI inventory and posture: recommended | Endpoints (MDM + EDR): not used |
| A4 | In-Boundary InferenceManaged cloud AI · Self-hosted open weights | High to Highest | AI gateway: recommended | Toolshed / MCP gateway: not used | Identity (SSO, workload, agent ID, OBO): core | Secrets manager: recommended | Sandboxed execution: not used | Telemetry to SIEM: core | Human approval gate: not used | AI inventory and posture: recommended | Endpoints (MDM + EDR): not used |
| T | Enterprise Reference | Highest | AI gateway: core | Toolshed / MCP gateway: core | Identity (SSO, workload, agent ID, OBO): core | Secrets manager: core | Sandboxed execution: core | Telemetry to SIEM: core | Human approval gate: core | AI inventory and posture: core | Endpoints (MDM + EDR): core |
Core to the modelRecommendedProvided by the vendor
Fast path for risk takers
Five places a team can cut corners and still defend the decision. On each model's page, the fast path overlay marks which components in the diagram are safe to defer, which can be simplified, and which are load bearing. The never skip list above still applies.
- Use a managed gateway instead of self-hosting.Cloudflare AI Gateway, Prisma AIRS AI Gateway, or provider project keys with spend caps get most of A3 in a day.You lose: one more vendor in the data path.
- Defer prompt DLP and guardrails.Rely on zero-data-retention or enterprise terms, plus a written "what data can go in" policy.You lose: catching accidental PII or secrets in prompts.
- Use a static tool allowlist before a policy engine.Hardcode which tools each agent can call and start read-only. Add OPA or Cedar past a handful of agents.You lose: fine-grained, context-aware decisions.
- Send logs to a bucket, not the SIEM.Immutable storage with retention now, detections later.You lose: real-time alerting. You keep forensics.
- Use persistent dev containers instead of ephemeral sandboxes.Only with no prod credentials inside and an egress allowlist.You lose: a clean slate after a compromised session.
KEEPload bearing SIMPLIFYlighter option is fine DEFERadd later
How these models map to other frameworks
Where each model sits on the AWS scoping matrices and the OWASP LLM and Agentic Top 10.
Show the AWS and OWASP mapping
The AWS Generative AI Security Scoping Matrix runs from scope 1 (consumer apps) to scope 5 (models you train yourself), which none of these models cover. The AWS Agentic AI Security Scoping Matrix runs from no agency to full agency, which none of these models recommend. The OWASP column lists the top risks from the LLM Top 10 (2026) and the Agentic Top 10 that each model's controls address.
| ID | Model | AWS GenAI scope | AWS agentic scope | Main OWASP risks |
|---|---|---|---|---|
| A1 | Direct to Provider | 3 | No agency | LLM02 LLM06 |
| A2 | Workforce AI Apps | 1 and 2 | No agency to supervised (AI browsers, suite agents) | LLM01 LLM02 ASI02 |
| A3 | Gateway and Router | 3 | No agency | LLM01 LLM02 LLM04 LLM06 |
| A4 | In-Boundary Inference | 3 and 4 | No agency | LLM02 LLM04 LLM05 |
| B1 | Employee AI Agents | 1 and 2 | Prescribed (user approves) to supervised; on unmanaged devices only consent and token policy apply | ASI01 ASI03 ASI04 ASI06 ASI09 |
| B2 | Vendor-Run Agents | 2 | Prescribed (analyst approves actions) to supervised | ASI01 ASI03 ASI04 |
| B3 | Agents You Build | 3 | Prescribed to supervised | ASI01 ASI02 ASI03 |
| B4 | Developer AI | 2 and 3 | Prescribed (attended) to supervised (fleet) | ASI01 ASI04 ASI05 |
| C1 | Customer Agents on Your API | 3 | Prescribed (out-of-band confirm) | ASI02 ASI03 ASI09 |
| C2 | AI Features in Your Product | 3 and 4 | No agency to prescribed | LLM01 LLM02 LLM09 |
| T | Enterprise Reference | 1 to 4 | Prescribed to supervised | LLM01 LLM02 ASI01 ASI02 ASI03 |
ASI01 Agent Goal HijackASI02 Tool Misuse and ExploitationASI03 Identity and Privilege AbuseASI04 Agentic Supply Chain VulnerabilitiesASI05 Unexpected Code ExecutionASI06 Memory and Context PoisoningASI09 Human-Agent Trust ExploitationLLM01 Prompt InjectionLLM02 Sensitive Information DisclosureLLM04 Supply ChainLLM05 Data and Model PoisoningLLM06 Unbounded ConsumptionLLM09 Vector and Embedding Weaknesses
Above all eleven sits the governance layer: the NIST AI RMF for structure, ISO 42001 for certification, and MITRE ATLAS for red teaming. Our AI governance cheatsheet covers those, plus the AI laws and their dates.
Find your target AI architecture in one call
Bring the AI tools your teams use today and the data they touch. In a 30-minute call a CISO maps you to these models and names the controls to add first. No sales pitch.
Talk to a CISO