AI Security for CTOs: 50 Questions About PHI, PII and LLMs

Start with inventory and classification, because you cannot write an AI policy for data you have not classified. Four tiers cover most companies. Public and Internal are self-explanatory, Confidential holds customer data, PII and source code, and Regulated holds PHI, PCI, government data and anything with a contract restriction. Map each tier to the AI tools approved for it, and that one table answers most everyday employee questions. For the full sequence after that, our 21-step AI security hardening checklist lists the five things to do in your first 30 days.

The CISO, security lead or fractional CISO owns the risk and the controls. Engineering owns the gateway and the integrations, legal and privacy own contracts and notification decisions, and one named person owns the approved tools list. An "AI committee" with no single owner is how shadow AI wins.

Think of it as a spectrum.

• Expense anything. The policy says no PHI, and you trust the team. That is cheap and fast, but you get zero visibility and no contractual protection.
• Central control plane. Everyone uses company accounts and every request goes through an AI control plane (a gateway for API and agent traffic, and MDM plus your EDR's AI module or a web proxy for chat). You get the same security posture everywhere and observability over all of it.

Bans don't belong on the spectrum at all, since every ban we have audited had active shadow usage on personal accounts.

For any company that holds PHI, PII or customer data, we recommend leaning hard toward the control plane end and building it as a paved road rather than a wall. In practice that means company accounts with SSO on a short approved list, the control plane in front of them, consumer AI blocked, and an intake process that approves new tools in days so nobody needs to expense around you. Have finance reject AI subscriptions that are not on the list.

A 20 person company with no regulated data can start with enterprise accounts plus SSO and add the gateway later, but once you hold PHI, observable by default is the standard.

Not for anything beyond public information. Personal plans run on consumer terms, which means training settings the employee controls, consumer retention, no DPA, no BAA and no audit trail. The company also has no way to offboard the account, so the conversation history, with your code, your architecture and your customer's stack traces, walks out the door with the employee. A rule of "fine as long as there is no PHI" sounds reasonable but can't be enforced, because the leaks are rarely deliberate. They come from a log file pasted to debug an error, a database row inside a stack trace, or a .env file an agent read. Expensing personal Pro subscriptions gets you the worst of both worlds, since you pay for the tool and still get consumer terms. You can enforce part of this. MDM can lock Claude Desktop to your company account and block unapproved AI apps, but blocking a personal account in the browser takes a web proxy that adds the vendor's tenant header.

It fits on one page, because nobody reads a longer one. It lists approved tools by data tier and a short set of things never to paste (credentials, PHI outside approved tools, customer data in personal accounts, unreleased financials). It says humans own the output, explains how to request a new tool, and tells people how to report a mistake without fear.

Keep it to ten minutes, use real examples from your own business, and give people a clear path for what to do when they make a mistake. Most leaks happen because AI is the fastest way to finish a task, so an approved tool that is just as fast solves most of the problem.

On commercial and API terms from the major providers, not by default. On consumer plans it depends on settings, and those change. That is the biggest reason to get people off personal accounts, because paying for Enterprise means little if half the team is still logged into a personal account in another browser tab.

Yes, on the right plan with the right configuration. Anthropic signs a Business Associate Agreement for its HIPAA-ready services, which are the first-party API (a HIPAA-ready API organization) and Claude Enterprise plans with the HIPAA configuration enabled. On Enterprise, the organization's Primary Owner can now turn that configuration on. Anthropic's BAA page also lists Claude in Microsoft Foundry, but its API documentation says HIPAA readiness is not available there, so confirm with Anthropic before relying on it. Consumer plans (Free, Pro, Max) and Team plans are not eligible. A BAA and a correct configuration are the minimum, and together they still don't make your workflow compliant on their own.

Probably not. Anthropic states that BAAs signed for API usage before December 2, 2025 cover only the API and do not extend to HIPAA-ready Enterprise plans. The BAA also covers only the single organization that accepted it, so a company with three Claude orgs may have one covered and two that are not.

All three offer HIPAA paths, and all three exclude consumer tiers.

• OpenAI. The API BAA covers the endpoints on its HIPAA eligible list and requires a modified retention setup. Organizations with a history of API usage can accept it themselves in their API Platform organization settings, and custom terms go through OpenAI's sales team. ChatGPT Enterprise and Edu are eligible only on sales-managed accounts, and OpenAI also offers ChatGPT for Healthcare. ChatGPT Business and the consumer plans are not eligible.
• Microsoft. Microsoft Copilot and Microsoft Copilot Chat (formerly Microsoft 365 Copilot and Microsoft 365 Copilot Chat), used with a work Entra ID account, are on Microsoft's HIPAA in-scope list. Web search queries that Copilot generates fall outside the BAA, and personal Copilot on a consumer Microsoft account is not covered.
• Google. Google covers the Gemini app and Gemini in Workspace under the Workspace BAA on eligible editions, but not Gemini in Chrome or the Workspace version of Gemini Notebook (formerly NotebookLM). Gemini Notebook Enterprise is on the Google Cloud BAA list instead. Gemini Enterprise and Gemini Enterprise Agent Platform (which now includes Vertex AI) fall under the separate Google Cloud BAA, and consumer Gemini and Google AI Studio are not on Google's covered list.

With all three vendors, coverage comes down to an enterprise tier, an executed BAA, a named service and the correct configuration.

Often, yes. Amazon Bedrock and Bedrock AgentCore are on AWS's HIPAA Eligible Services list under the standard AWS BAA, so if you already run PHI workloads in AWS, Claude on Bedrock sits inside a boundary your auditors already understand, with your VPC, IAM, CloudTrail and KMS keys. AWS does not list models individually, so confirm the specific model with your AWS account team. On Google Cloud, Google says Claude supports HIPAA workloads, but Claude is not named on the Google Cloud BAA product list, so confirm partner model coverage in your agreement. The tradeoff is that feature availability differs by platform. The Files API, batch processing, web fetch and code execution, for example, are on Anthropic's own API but not on Bedrock or Google Cloud.

Zero data retention (ZDR) means the provider does not store your prompts and outputs after the response, apart from narrow exceptions such as data flagged by trust and safety systems or required by law. It reduces breach exposure and discovery risk. The catch is that some newer, more capable models require 30-day retention and are not available to ZDR organizations, and some features need state and will not work under ZDR. Anthropic positions HIPAA readiness as an alternative to ZDR rather than an add-on, and recommends separate organizations for HIPAA and non-HIPAA workloads. Make the call workload by workload instead of setting one rule for the whole company.

Look for data use and no-training language, retention periods, the subprocessor list with notice of changes, data residency, breach notification timelines shorter than HIPAA's outer limit, audit rights or current SOC 2 Type II and ISO 27001 reports, indemnity for IP claims on outputs, and an exit clause that covers deletion. For personal data from the EU, you need either the vendor's EU US Data Privacy Framework certification or a DPA with Standard Contractual Clauses. Many buyers want SCCs as a fallback because the Framework is under appeal at the EU Court of Justice.

The law treats them differently, but in practice the risk looks similar. PHI has HIPAA, while PII is governed by a patchwork of state privacy laws, GDPR for EU residents, and your customer contracts, which are often stricter than the law. Many B2B contracts quietly prohibit sending customer data to new subprocessors without notice, so read your top ten customer agreements before you route their data to any LLM.

Grounding (retrieval at query time) is far easier to defend than fine tuning, because data stays in your store and access can be enforced per request. Fine tuning bakes data into weights, where it cannot be deleted or access controlled and can be extracted. For regulated data our default answer is to use retrieval and skip training. If you must fine tune, de-identify first and get explicit contractual rights.

For structured data with known fields, yes. For free text such as clinical notes, support tickets and emails, automated redaction catches most identifiers but misses some. Names in odd positions, a rare condition plus a ZIP code, and dates combine into re-identification. Treat redaction as a way to reduce risk rather than a guarantee, and if the downstream tool is not BAA-covered, redaction alone is a thin defense.

Enforce access at retrieval time using the requesting user's identity instead of the service account's. The common failure is a vector store indexed with an admin account that happily returns documents the user could never open directly. Store permissions as metadata on every chunk, filter before ranking, and test it with a low-privilege account.

Yes. Embeddings can be inverted to recover much of the source text (one 2023 study recovered 92% of 32-token inputs exactly, including full names from clinical notes), and they encode content well enough to identify individuals. Treat a vector store with PHI embeddings as a PHI data store, with encryption, access control, BAA coverage for any managed vector database, and deletion when the source is deleted.

Both count as retention. Prompt caches hold your content at the provider for a short window (Anthropic's default is 5 minutes, with an optional 1 hour), and conversation history in chat products may persist until deleted. Know the retention behavior of each feature and confirm it is within BAA scope, because features in beta are frequently excluded.

The pattern is sound, but building it yourself is usually the wrong call. A gateway that inspects prompts, classifies data and routes regulated traffic to BAA-covered endpoints, while everything else goes to the cheapest or best model, is the architecture we recommend for most mid-size companies. Buy or adopt the gateway layer (commercial AI gateways, cloud-native options, or a well-maintained open source proxy) and build only the policy, meaning your classification rules, routing table and logging. At 50 to 500 people, start with a managed option or your cloud provider's native controls unless you have an engineer who will own a self-hosted proxy as production infrastructure. Building the plumbing yourself puts a security-critical system on the hot path of every AI request, with prompt parsing, streaming, retries and tool calls to keep current.

It makes sense when you have unusual constraints, such as air-gapped or on-prem models, a classification engine you already trust, strict latency budgets, or a gateway you sell as part of your product. Even then, start from an open source proxy and add your logic. Treat it as critical infrastructure, because it holds every provider key. In March 2026, attackers used stolen publishing credentials to push credential-stealing versions of the popular open source LiteLLM proxy to PyPI. Pin versions, verify package hashes, build from your own image, and keep its secrets in a vault.

The classifier becomes the weak point. If routing depends on detecting PHI, a missed detection sends PHI to an uncovered endpoint. Design the system to fail safe, so the default route goes to the BAA-covered endpoint and only traffic positively classified as non-sensitive goes elsewhere. Better still, route by source (which app, which user group, which data store) instead of relying on content inspection alone, since source is deterministic and content detection is probabilistic.

Log who sent the request, which app and model it used, the data classification decision, token counts, cost, and a hash or redacted copy of the prompt. Be careful with full prompt logging, because your log store then contains PHI and inherits all the controls that come with it. Many teams log metadata everywhere and keep full content only for the regulated route, in a store with HIPAA-grade controls and short retention.

We recommend one. A dedicated account or subscription with its own BAA scope, KMS keys, network boundary and access group is easier to audit and harder to mix up with dev experiments by accident. It also gives "where does PHI go?" a short answer.

Make PHI access the exception, granted per use case and only while it is needed.

• Separate environments by data class. Use a HIPAA-ready organization or workspace for PHI work and a standard one for everything else. Anthropic recommends separate organizations for HIPAA and non-HIPAA workloads, and its HIPAA configuration is a one-way switch that turns off features outside the BAA, so splitting also lets everyone else keep the full feature set.
• Drive membership from your identity provider. An AI PHI group maps to the covered environment through SCIM, and enterprise role controls in Claude and ChatGPT decide which capabilities, connectors and models each group gets.
• Make the group just in time. People request access for a defined use case, a named approver signs off, and membership expires automatically after hours or days instead of lingering for years.
• Agents follow the same rule. Each use case gets a separate identity, with credentials scoped to the PHI data stores and covered endpoints it needs, short-lived tokens issued per task from a secrets manager or cloud workload identity, and PHI-only MCP servers available only in the PHI configuration.
• The gateway enforces it by identity. Traffic from the PHI group or a PHI agent goes only to BAA-covered endpoints, and nothing outside that group can reach PHI data sources at all.

Nobody has PHI AI access by default, every grant is logged, and access is reviewed at least quarterly.

Examples: at 50 to 200 people, an approval ticket plus a group membership with a scheduled removal is enough. Automatic expiry needs an identity governance add-on, such as Microsoft Entra Privileged Identity Management (Entra ID P2, beyond Business Premium). For agents, your cloud's secrets manager or AWS STS issues short-lived credentials.

In some cases it makes sense. Self-hosting removes the third party, but it makes you responsible for model serving security, patching, GPU capacity, evaluation and the quality gap with frontier models. For most companies, a BAA-covered frontier model inside their own cloud account is safer than a self-hosted model run by a team that has never operated one. Self-host when data cannot leave a boundary at all, or when volume makes the economics obvious.

Without observation you don't know. Policy and training lower the odds, but they produce no evidence, and evidence is what an auditor, a customer or a regulator will ask for. Observability comes in layers:

• Identity. Enterprise tenants with SSO put work accounts where you can see them, and MDM can lock Claude Desktop to your company account. Blocking personal accounts takes more: a web proxy or SSE that inspects traffic and adds the tenant restriction headers OpenAI and Anthropic offer on their Enterprise plans, or Microsoft Entra tenant restrictions for Copilot. Most smaller companies don't run one, and OpenAI's header covers ChatGPT on the web, not its desktop apps.
• Endpoints. MDM blocks unapproved AI apps, AI browsers and extensions (Chrome Enterprise Core is free). Your EDR vendor's AI module, such as SentinelOne AI Usage Control or CrowdStrike Falcon Guardian, finds AI use and redacts or blocks sensitive data in browser prompts, in the console your MDR already watches. Desktop app coverage is thinner, especially on Macs.
• Network. If you already run Cloudflare or an SSE, it can block consumer AI domains and apply DLP to text and file uploads. Most companies of this size don't need to buy one for AI alone. An SSE (security service edge) bundles a secure web gateway that inspects encrypted traffic, a CASB (cloud access security broker) that finds and controls SaaS and AI apps, zero trust network access and shared DLP. Buy an SSE and you get a CASB with it.
• AI gateway. It sees all API and agent traffic and handles classification and logging.
• Vendor audit logs. ChatGPT Enterprise, Claude Enterprise and other enterprise plans now offer audit log and compliance APIs. Export them to your SIEM, because some vendors keep these logs for only 30 days.
• Developer machines. They need the EDR AI module and approved tool configuration pushed by MDM, since coding agents read files directly.
• Sweeps. Periodic sweeps of SSO and OAuth grants, expense reports, browser extensions and DNS logs find what slipped through.

Close the remaining gap upstream. If PHI is only reachable from managed devices through conditional access, a personal phone never sees it in the first place.

What you can credibly tell an auditor is that you can show what share of AI usage runs through observed paths, that you detect and block sensitive data on those paths, and that you sweep for the rest. One caution: inspecting prompts means you are now collecting PHI and employee data in your logs, so protect that store like any PHI system and tell employees what is monitored.

On enterprise plans with no-training terms, yes for most codebases, and the productivity gain is real. Watch for secrets in the repo and for production data in fixtures or local databases, since the tool will read both, and control which commands it can run. Set permission policies centrally rather than letting each developer choose.

Take the complaint seriously, because friction is how most AI security programs fail at this size. When the approved path is slower than a personal account, engineers route around it, and every control you built stops seeing anything. Treat developer experience as a security requirement and make the sanctioned setup the fastest one. Seats should appear on day one through SSO and SCIM, with tools preconfigured through MDM instead of a setup wiki, and shared skills, MCP servers and plugins should make the company tool more useful than anything a developer could expense. Put hard limits where the risk lives (secrets, production access, regulated data) and let everything else move. In Claude Code, sandboxing lets commands run inside defined filesystem and network boundaries with far fewer approval prompts, and auto mode has a classifier review actions instead of the developer. Give engineers a named person to ask, approve new tools in days, and ask the team every quarter where the controls get in their way.

Manage it the way you manage code. Keep one repository for your AI platform configuration, holding managed settings, permission rules, the approved MCP server list, and shared skills and plugins. Changes go through pull request review like any other code, because a skill can carry instructions and scripts that run on every developer's machine. Distribution then happens in three layers. Enforced policy goes out through MDM as managed settings that developers cannot override. Convenience defaults can come from the claude.ai admin console, which Anthropic is clear is not a security boundary. Project context lives in each repository, in the instruction files most AI coding tools read (CLAUDE.md for Claude Code), so conventions travel with the code. For Claude Code, host a private plugin marketplace in a Git repository and use strictKnownMarketplaces so developers install only from it. For Claude on the web and desktop, Enterprise admins can provision skills to the whole organization and turn on security scanning for them. Give the repository an owner and a changelog, and announce changes where engineers already talk.

Say yes to the editor and keep the control plane.

• MDM pushes configuration so every approved tool starts with your settings (for Claude Code, managed settings that lock login to your organization, turn on the sandbox and set permission rules).
• Every developer gets a company seat on the Team or Enterprise plan for each tool, never a personal subscription, so SSO, organization privacy settings and offboarding apply.
• Point custom endpoints at your LLM gateway wherever a tool allows it, so model traffic is logged and routed by policy no matter which editor sends it.
• Publish integrations from one internal source, such as a managed plugin marketplace or an MCP allowlist, so approved skills, MCP servers and plugins and their controls follow the developer from editor to editor.

Two cautions apply. Some editors send traffic through their own servers even when you supply your own endpoint or key, which makes the editor vendor a subprocessor that needs its own review. A gateway can also take a tool outside its vendor's BAA. Anthropic's HIPAA configuration for Claude Code, for example, applies to direct connections to the Claude API and not to traffic through LLM gateways. Pointing Claude Code at a gateway also makes it skip the settings pushed from its admin console, so deliver settings through MDM.

A short intake review for each new editor keeps this manageable.

Yes, and a container or VM is the only place it belongs. Yolo mode (Claude Code's --dangerously-skip-permissions, and similar modes in other agents) removes the human review of each command. The risk that matters is a prompt injection hidden in a dependency, an issue or a web page, or an honest mistake, executing with everything the session can reach.

A container or VM shrinks what it can reach, and Anthropic's own documentation is direct about the limit: run this way, a dev container does not prevent a malicious project from exfiltrating anything accessible inside the container, including the agent's own credentials. So build the container for the job:

• Use a centrally maintained image rather than one each repo defines, because anyone with write access can edit a repo's Dockerfile.
• Run as a non-root user with default-deny network egress, allowing only the model API, your git host and package registries (Anthropic publishes a reference dev container with a firewall script for exactly this).
• Mount no host secrets, which means no ~/.ssh, no cloud credential files and no Docker socket, and pass only repository-scoped, short-lived tokens.
• Keep production access out, make environments disposable, send telemetry to your SIEM, and stick to trusted repositories.
• Let work leave the box only as a pull request a human reviews.

On laptops, keep bypass mode disabled through managed settings and allow it only in the sanctioned image. If the goal is simply fewer prompts, Claude Code's auto mode, where a classifier reviews each action before it runs, is a middle ground.

The code itself is usually not PHI. The trouble is everything near it, like test fixtures copied from production, log files, database dumps on laptops, and agents with access to production consoles. Keep PHI out of dev environments with synthetic data, and if a coding tool will touch PHI, use a BAA-covered configuration. For Claude Code that means local mode on a HIPAA-ready Enterprise plan with the HIPAA configuration applied, or ZDR, and the IDE extensions are not covered.

Prompt injection is instructions hidden in content the model reads, such as a web page, an email, a PDF or a support ticket, that hijack what the model does. It matters most when the model can take actions or see private data. An agent that reads customer emails and can also send email can be told by an attacker's email to forward your data. There is no complete fix today, so the defense is architectural. Limit what the agent can do, separate untrusted input from privileged tools, and require human approval for sensitive actions.

The Model Context Protocol lets AI tools connect to external systems such as databases, ticketing, CRMs and file stores. Each MCP server is a new integration with its own credentials and permissions. The MCP specification's own security guidance covers over-broad token scopes, token passthrough, malicious local servers and server-side request forgery, and it warns that tool descriptions from untrusted servers should not be trusted. Add prompt injection reaching a server that has write access, and the risk compounds. Maintain an allowlist of approved MCP servers, prefer read-only scopes, and review community servers like any other dependency.

Once you have more than a handful of agents, put an MCP gateway in front of them. It is a single control point that every tool call passes through. It holds the registry of approved MCP servers, swaps the user's identity for short-lived on-behalf-of tokens where the downstream API supports them, asks a policy engine such as OPA or Cedar whether the call is allowed, routes risky writes to human approval, and logs the call, its arguments and the result. Agents then never hold broad standing credentials to production systems. Architecture B3 shows the pattern.

Match autonomy to how reversible the action is. Agents can read and draft on their own, and they can write to internal systems on their own as long as the writes are logged. Anything external, financial, destructive or involving regulated data needs human approval per action. Send that approval to the person outside the agent's own channel, so an agent hijacked by injected instructions can't approve itself. This lines up with the AWS Agentic AI Security Scoping Matrix: an agent that needs approval for each action is prescribed agency, one that acts on its own inside a task a human started is supervised agency, and full agency, where it starts its own work, should wait until least-privilege tools, spend limits, a kill switch and live monitoring are proven. Agents should run under their own identities with least privilege, never under a human admin's credentials. A dedicated identity per agent lets you scope permissions, audit actions, rotate credentials and kill one agent without disrupting people, while an agent acting as a human user makes every audit log ambiguous.

Egress controls do most of the work. Coding agents and agent sandboxes should have network allowlists instead of open internet access. Watch for exfiltration through tool calls (fetching a URL with data in the query string), rendered images in chat (markdown image URLs) and generated links. Many agent platforms now ship sandbox and network policy controls, so turn them on.

Yes, and a serious one. They often request access to every page you visit, including your EHR, admin consoles and email. Block them by default and allowlist specific vetted extensions, pushed through MDM as Chrome or Edge policies (ExtensionInstallBlocklist and ExtensionInstallAllowlist). AI browsers carry the same risk at a larger scale, since they read and act on every page. Block unapproved ones with MDM.

They record and transcribe everything, often to a third party, sometimes join meetings automatically, and frequently lack BAAs. A clinical or client call recorded by an unapproved notetaker is a data leak with timestamps. Approve one tool, set auto-join policies, and require disclosure to participants, since about a dozen states, including California, Florida, Illinois, Pennsylvania and Washington, require consent from all parties to record.

Treat it as exposed and rotate it the same day, without debating whether the provider kept a copy. Rotating a credential takes minutes, while investigating a compromise takes weeks. Then check logs for use of that credential since the time of exposure, and add secret scanning to your gateway, browser controls or your EDR's AI module so it is caught next time.

Contain it first by stopping the workflow and identifying exactly what was sent, to which product, under which account. Next, establish the contractual status, because whether it went to an enterprise account with a DPA or BAA and no training, or to a personal consumer account, drives everything else. Then request deletion from the provider in writing and assess notification obligations under your client contracts and applicable privacy laws with counsel. Document every step, since a quick, well-documented response is your best defense with clients and regulators.

It is presumed to be one, unless your risk assessment demonstrates a low probability that the PHI was compromised, using HIPAA's four factors: the nature and extent of the data, who received it, whether it was actually acquired or viewed, and the extent of mitigation. A consumer chatbot with training enabled is a hard case to argue as low probability, while an enterprise account with no training, no retention and a prompt deletion confirmation puts you in a much better position. Involve your privacy officer and counsel, and do not make the call alone in Slack.

On enterprise and API plans, usually yes, through your account team or support, and under ZDR there may be nothing retained to delete. On consumer plans, deleting the conversation helps, but you have less contractual assurance about backups and training pipelines. Get the deletion confirmation in writing for your incident file.

Yes, as a named scenario with its own runbook covering data leakage to an AI tool, prompt injection causing an agent to take an unauthorized action, a compromised AI vendor, and model output causing harm. Run a tabletop on it. Most teams discover during the tabletop that nobody knows who owns the AI vendor relationship.

Not through the model weights, as long as you are not training on customer data. The real risks sit in your application, in shared conversation context, incorrectly keyed caches, retrieval without per-tenant filtering, and logs visible across tenants. Test tenant isolation in your AI features with the same rigor as the rest of your multi-tenant app.

Start with the ones that already applied before AI, which are HIPAA, GLBA, PCI DSS, state privacy laws, GDPR and your customer contracts. AI does not change them, but it creates new ways to violate them. On top of those sit the EU AI Act (prohibitions since February 2025, general purpose model duties since August 2025, transparency duties since August 2026 that require you to tell people when they're talking to AI and to label AI-generated content, an AI literacy duty that the 2026 amendments softened to taking reasonable measures, and high-risk obligations moved by the 2026 Digital Omnibus to December 2027 and August 2028); Colorado's automated decision-making law (SB 26-189, which in May 2026 replaced the 2024 AI Act before it ever took effect, with core obligations starting January 2027 for consequential decisions such as hiring, lending, housing and insurance, though a pending lawsuit could delay it); other state rules on automated decisions; and FTC enforcement on deceptive AI claims, such as its Operation AI Comply sweep. The landscape is moving, so assign someone to track it. Our AI governance cheatsheet keeps a dated timeline of the EU and US obligations.

SOC 2 has no AI-specific criteria. The AICPA's September 2026 guidance (Technical Q&A section 9561) explains how a company's AI use is assessed within existing SOC 1 and SOC 2 examinations, and your auditor will expect AI vendors in your vendor management, AI tools in your access reviews, and AI data flows in your system description. Expect AI questions in every enterprise security questionnaire as well. An AI policy, an approved tools list and a gateway log make those answers easy.

They do different jobs. The NIST AI RMF is a free, voluntary framework (Govern, Map, Measure, Manage), and NIST has announced a revision, that works well as a checklist to structure your program, and it maps cleanly to customer questionnaires and the EU AI Act. ISO 42001 is a certifiable AI management system standard. If you build AI into your product and sell to enterprises or into the EU, it is becoming a differentiator. If you only use AI internally, get your ISO 27001 or SOC 2 controls covering AI first and layer 42001 on later instead of making it your first certification. The AI governance cheatsheet compares these frameworks with CSA AICM, the OWASP lists and AIUC-1.

Have a standard AI security statement ready that covers which AI vendors you use, what data reaches them, the contractual terms (no training, retention, BAA where relevant), how you control access, and how customers can opt out if you offer AI features. Most customers want to know that their data is not training someone else's model and that you know where it goes. For a structured answer, fill in the Cloud Security Alliance's AI-CAIQ, the AI questionnaire built on its AI Controls Matrix, and publish it as a STAR for AI Level 1 self-assessment. Many enterprise questionnaires now map to it, so one completed copy answers most of them.

Measure it. Track the percentage of AI usage that runs through approved tools (gateway, SSO and EDR AI module data compared with discovery scans), the number of blocked sensitive prompts, the time to approve a new tool request, AI incidents reported, and clean answers to customer questionnaires. A rising incident count early on is a good sign, because it means people report. If you cannot measure usage, you are not governing it. Once a year, baseline the program against a maturity model, such as the OWASP AI Maturity Assessment or MITRE's AI Maturity Model, so you can show progress rather than just activity.