Privacy and retention terms can change. Treat this as a practical buying guide, then verify the provider’s current API, trust, and legal documentation before sending regulated or sensitive data into production.
Last verified against official provider documentation: April 24, 2026.
When teams ask whether an AI API is "private," they usually mean several different things at once. They want to know whether prompts are used for model training, how long inputs and outputs are retained, whether humans can review them, where the data is processed, and what changes when they use tools like files, grounding, prompt caching, or hosted agents.
Those are different questions, and providers answer them differently. A direct API, a managed cloud wrapper, and a free developer tier can all expose the same model family while giving you different answers on logging, review, residency, and contractual controls.
If you are choosing between AI APIs in 2026, the practical privacy question is not "Which provider says it is secure?" It is "What exactly happens to my prompts, files, outputs, and metadata in the workflow I plan to run?"
Key takeaways
- Privacy with AI APIs is not one setting. Check training use, retention, human review, feature-specific storage, and processing location separately.
- Direct API terms and cloud-hosted versions of the same underlying models can handle your data differently.
- Files, assistants, prompt caching, grounding, and stateful chat features often store data longer than a simple stateless text call.
- Free and paid routes can have materially different data-use terms, especially for Gemini API and Google AI Studio.
What happens to your prompts when you call an AI API?
At a minimum, your prompt is transmitted to the provider, processed by the model, and paired with the returned output. In a real production workflow, that same request can also create or touch several other data surfaces:
- Request content: the prompt, system message, files, images, retrieved documents, tool results, and output.
- Operational metadata: account, project, model, region, token counts, classifier results, latency, errors, and billing data.
- Safety path: automated abuse detection, policy classifiers, and in some products a manual review path for flagged content.
- Application state: threads, conversations, responses, vector stores, batch files, caches, and agent memory created so the feature can work.
- Downstream services: search grounding, maps, code execution, remote tools, MCP servers, connectors, and logging systems.
This is why privacy reviews often fail when teams only read the headline claim about model training. A provider can decline to train on API traffic and still store content for abuse monitoring, product functionality, legal obligations, or a feature you enabled.
The five privacy questions buyers should ask every provider
- Are prompts and outputs used to train the model? Distinguish paid API traffic, unpaid developer tiers, feedback programs, and fine-tuning uploads.
- How long are prompts and outputs retained? Abuse logs, files, caches, and stateful features may all have different clocks.
- Can humans review the data? Ask what triggers review, who can access it, and whether your contract can change that path.
- Where is data stored and processed? Tenant isolation and region selection can matter as much as the model brand.
- Which features override the default? Files, prompt caching, grounding, web search, batch jobs, and assistants often change the answer.
How major AI API pathways handle data
| Provider path | Training on API data | Default retention pattern | Human review | Data residency / region control | Zero-retention availability | Source / notes |
|---|---|---|---|---|---|---|
| OpenAI API | API data is not used to train OpenAI models by default unless the customer opts in.[1] | Abuse monitoring logs are generally retained up to 30 days; application state depends on endpoint and feature.[1] | OpenAI documents rare manual-review retention for certain image or file inputs flagged by CSAM classifiers, even with retention controls.[1] | Regional storage and processing are available for supported regions; non-US regions require approval for abuse-monitoring controls and a Zero Data Retention amendment.[1] | Modified Abuse Monitoring and Zero Data Retention are available to approved customers, but not every endpoint or capability is eligible.[1] | Review the endpoint table before using Responses, conversations, files, vector stores, batches, web search, or hosted tools.[1] |
| Anthropic API | Anthropic says commercial API inputs and outputs are not used for training by default unless otherwise agreed.[3] | API inputs and outputs are automatically deleted from backend systems within 30 days, except for longer-retention features, ZDR agreements, policy enforcement, or law.[3] | The public retention article emphasizes enforcement and legal exceptions rather than routine reviewer access; treat policy exceptions as review-sensitive unless your agreement says otherwise.[3] | Direct API residency controls should be confirmed contractually; use a cloud-hosted route if hard regional control is a requirement. | Some enterprise API customers may receive ZDR arrangements for eligible Anthropic APIs, subject to approval and product scope.[5] | Files API uploads persist until explicitly deleted and are stored for reuse across requests.[4] |
| Gemini API direct | Unpaid Gemini API and Google AI Studio content can be used to improve Google products; paid Gemini API prompts and responses are not used to improve products.[8] | Paid services log prompts and responses for policy enforcement for a limited period; abuse-monitoring documentation states 55 days.[8][9] | Unpaid services may involve human reviewers; flagged or suspicious paid usage can be reviewed by authorized Google personnel.[8][9] | Paid terms say data may be transiently stored or cached in any country where Google or its agents maintain facilities.[8] | No general public ZDR path is described for Gemini API direct in the cited docs; use Vertex AI when cloud governance and ZDR controls are required. | The first privacy decision is often unpaid versus paid, not only Google versus non-Google.[8] |
| Vertex AI | Google says customer data in Vertex AI is not used to train or fine-tune AI/ML models without prior permission or instruction.[10] | Feature-dependent: grounding with Google Search or Maps stores prompt/context/output data for 30 days; Gemini Live session resumption stores cached data up to 24 hours; in-memory caching has a 24-hour TTL and can be disabled.[10] | Abuse monitoring can involve prompt logging for investigation when suspicious activity is detected.[11] | Vertex AI has stronger project, region, and Google Cloud governance controls; abuse-monitoring logs are stored in the same selected region or multi-region when logged.[11] | Zero retention requires specific actions or exceptions, and some feature storage cannot be disabled if you use that feature.[10] | Grounding and caching settings can change the real retention story.[10] |
| Azure OpenAI / Azure Direct Models | Prompts, completions, embeddings, and training data are not available to OpenAI or other model providers and are not used to train foundation models without permission or instruction.[14] | Models are stateless, but files, vector stores, Responses, Threads, batch, stored completions, and related features can store data in the service.[14] | Abuse monitoring can select prompts and completions for automated review and, when necessary, human review by authorized Microsoft employees.[14] | Standard deployments process prompts in the customer-specified geography; Global and DataZone deployments change processing scope while stored data remains in the designated geography.[14] | Managed customers may apply for modified abuse monitoring; automated review can still apply.[14] | Azure changes the hosting, tenancy, and governance model; it does not remove feature review. |
| Amazon Bedrock | AWS says Bedrock does not use prompts and completions to train AWS models and does not distribute them to third parties.[17] | AWS documentation says Bedrock does not store or log prompts and completions; surrounding AWS services and your own logs still matter.[17] | Bedrock docs emphasize no sharing with model providers; review your own CloudTrail, app logs, guardrails, and connected services.[17] | Customer content processed by Bedrock is encrypted and stored at rest in the AWS Region where you use Bedrock.[18] | Bedrock does not usually market this as ZDR, but its documented inference posture is close to that for prompts and completions; verify feature-specific services separately.[17] | Your privacy posture still depends on IAM, VPC design, logging, encryption, and connected AWS services. |
OpenAI API: not training by default does not mean nothing is stored
OpenAI’s API documentation says API data is not used for training by default, but it also separates abuse monitoring logs from application state. By default, abuse monitoring logs may contain prompts, responses, and derived metadata, and OpenAI says those logs are retained for up to 30 days unless legally required longer. OpenAI also documents organization-level controls such as Modified Abuse Monitoring and Zero Data Retention for eligible customers.[1]
The operational detail is endpoint-specific. OpenAI’s current table shows simple endpoints that can have no application state, but it also shows conversations, assistants, threads, vector stores, files, fine-tuning jobs, evals, and batches with longer application-state retention. The same page calls out examples such as Responses API state, background-mode polling data, extended prompt caching, files, third-party MCP servers, and CSAM review exceptions.[1]
Anthropic API: commercial default is no training, with a cleaner baseline for API retention
Anthropic says API inputs and outputs are deleted from backend systems within 30 days by default, except where a longer-retention service is used, the customer has a different agreement such as zero data retention, Usage Policy enforcement requires more time, or law requires retention.[3]
That makes Anthropic relatively straightforward at the headline level, but features can still override the simple story. Anthropic’s Files API docs say uploaded files persist until explicitly deleted, and the ZDR article says eligible ZDR coverage applies only to approved Anthropic API surfaces and products using the commercial organization API key.[4][5]
Gemini API: free and paid usage are not the same privacy product
Google’s Gemini API terms are one of the clearest examples of why you cannot treat an API provider as a single privacy bucket. For unpaid services, including Google AI Studio and unpaid Gemini API quota, Google says submitted content and generated responses can be used to provide, improve, and develop Google products and machine learning technologies. The same terms say human reviewers may read, annotate, and process API input and output.[8]
For paid Gemini API usage, Google says it does not use prompts or responses to improve products. But that still does not mean zero retention. Gemini API abuse-monitoring documentation says prompts, contextual information, and outputs are retained for 55 days for detecting and preventing policy violations and required legal or regulatory disclosures, and flagged content may be assessed by authorized Google employees.[8][9]
For many businesses, the first privacy decision is not merely "Google or not Google." It is whether you are using unpaid Gemini pathways, paid Gemini API pathways, or a Google Cloud product with a different control model.
Vertex AI: stronger cloud controls, but feature-level storage still matters
On Vertex AI, Google says customer data is not used to train or fine-tune AI or ML models without prior permission or instruction. That is often the more relevant Google option for teams that need clearer enterprise controls, regional choices, and contract-backed cloud governance.[10]
But Vertex AI is not automatically zero retention either. Google’s Vertex AI documentation lists limited retention scenarios tied to abuse monitoring, grounding with Google Search or Google Maps, session resumption for Gemini Live, and in-memory caching. Some settings can be disabled or exceptions can be requested, while certain grounding storage is part of using the feature.[10][11]
This is the pattern buyers should internalize: the safer cloud wrapper can still have feature-specific storage rules that matter just as much as the base model choice.
Azure OpenAI: Microsoft-hosted, tenant-oriented, but still a monitored service
Microsoft’s Azure Direct Models documentation says prompts, completions, embeddings, and training data are not available to OpenAI or other model providers and are not used to train foundation models without permission or instruction. Microsoft also positions Azure Direct Models as hosted in Microsoft’s Azure environment rather than forwarding traffic into a provider’s public API environment.[14]
That is meaningful for enterprise buyers because it changes the control plane, contracting, and regional model. But Azure’s docs also make clear that stateful features such as files, Responses, Threads, Assistants, batch, and stored completions can store data, and that abuse monitoring can involve automated and human review for flagged content unless modified abuse monitoring applies.[14]
Amazon Bedrock: no training on customer prompts, and no sharing with model providers
AWS states that Amazon Bedrock does not store or log prompts and completions, does not use prompts and completions to train AWS models, and does not distribute them to third parties. AWS also says model providers do not have access to customer prompts and completions through Bedrock.[17]
That does not make privacy automatic. AWS’s own guidance starts with the shared responsibility model, meaning your IAM policies, VPC and PrivateLink design, logging choices, encryption setup, and connected AWS services still determine whether the overall system is actually private in practice.[17]
The biggest privacy mistakes teams make with AI APIs
- Reading only the training claim. It answers one question, not the whole data-flow review.
- Ignoring feature-specific storage. Files, vector stores, grounding, caching, and agent memory often change retention.
- Confusing product tiers. A free developer route and a paid production route can have different privacy terms.
- Assuming the model name defines the posture. Hosting layer, contract, endpoint, and feature path matter.
- Forgetting downstream tools. Search, remote tools, MCP servers, and connectors can create separate data flows.
How to choose a privacy-safe AI API path
The right choice depends on your data class and workflow shape. Use this as a decision matrix before you get lost in model benchmarks.
| Scenario | Likely starting point | What to verify before production |
|---|---|---|
| Routine non-regulated business text | Paid commercial API or managed cloud API | No-training default, abuse-log retention, DPA terms, deletion behavior, and whether any feature stores application state. |
| Free prototyping | Synthetic or public test data only | Whether the free tier allows product improvement use or human review. Do not test with customer records unless the tier permits it. |
| Healthcare or PHI | BAA-covered path with approved feature list | BAA scope, non-covered features, logging, region, subcontractors, and whether web search, files, batch, code execution, or connectors are excluded.[2][7] |
| EU or strict residency requirement | Managed cloud wrapper or regional API configuration | Regional storage versus regional processing, support personnel location, global routing, subprocessors, and whether caches or abuse logs remain in-region.[1][10][14] |
| Internal documents, RAG, or vector search | Cloud account with explicit storage and deletion controls | Where files and embeddings live, who can delete them, whether indexes are reused, and whether retrieved text is added to prompts. |
| Agents, tools, or workflow automation | Start with a data-flow diagram, not a model shortlist | Every tool call, remote server, browser, code sandbox, connector, credential, audit log, and downstream processor. |
| Minimal retention requirement | ZDR/MAM-approved endpoint or Bedrock-style inference path | Endpoint eligibility, feature exclusions, safety exceptions, and whether your own application logs recreate the data you are trying not to store. |
Before legal or security signs off, write down the exact product, endpoint, model, feature flags, region, retention controls, deletion process, human-review path, BAA/DPA coverage, and downstream tools. If any one of those is unknown, the privacy review is not complete.
This is also where a comparison surface can help, as long as it stays secondary to the legal review. In AI Models, teams can shortlist models by provider, compatibility, access path, and use case, then bring that shortlist into the privacy checklist above.
Compliance snapshot (last verified April 24, 2026)
Compliance badges are useful, but they are not blanket permission to send sensitive data through every feature. Treat them as the start of procurement diligence, then verify the exact service, region, and feature scope.
| Provider path | What the public docs support | Buyer caution |
|---|---|---|
| OpenAI API | OpenAI says API customers need a BAA before processing PHI, and its help center says most API services are covered with exceptions.[2] | Check the BAA exceptions and endpoint retention table before using web search, files, images, videos, hosted tools, or other advanced features.[1][2] |
| Anthropic API | Anthropic lists SOC 2 Type I and Type II, ISO 27001, ISO/IEC 42001, and HIPAA-ready configuration with BAA availability.[6] | Anthropic’s BAA page lists covered and non-covered surfaces; several beta and API features are not covered.[7] |
| Google Vertex AI | Google Cloud documents HIPAA BAA requirements and FedRAMP High program coverage for in-scope Google Cloud services.[12][13] | Confirm the specific Vertex AI, Generative AI, grounding, region, and Assured Workloads configuration in scope for your workload. |
| Azure OpenAI / Azure Direct Models | Azure compliance docs describe HIPAA BAA support and FedRAMP High authorization for Azure and Azure Government in-scope services.[15][16] | Deployment type matters: standard, Global, DataZone, preview features, agents, and data sources can shift processing and storage details.[14] |
| Amazon Bedrock | AWS says Bedrock is in scope for common standards including SOC, ISO, HIPAA eligibility, and FedRAMP Moderate, and that content is not shared with model providers.[18] | Bedrock’s service posture is strong, but your workload still inherits responsibility for IAM, network isolation, CloudTrail, application logs, S3, and connected services.[17] |
The practical retention point is simpler than the old shorthand. A stateless text call, a ZDR-approved endpoint, a Files API upload, a vector store, and an unpaid developer-tier prompt can all have different lifetimes. Do not describe a provider as private or non-private until you know which path you are actually using.
| Workflow example | Why the data lifetime changes |
|---|---|
| Simple text generation call | May have no application state, short abuse-log retention, or no prompt logging depending on provider, endpoint, and approved controls.[1][17] |
| OpenAI files, vector stores, assistants, conversations, or batches | Application state can persist until deleted or for the documented endpoint period, and not all features are ZDR eligible.[1] |
| Anthropic Files API | Files are retained until explicitly deleted through the file deletion endpoint.[4] |
| Gemini API unpaid route | Submitted content and responses can be used to improve Google products, and human reviewers may process inputs and outputs.[8] |
| Vertex AI grounding with Google Search or Maps | Google documents 30-day storage for prompts, contextual information, and generated output when using those grounding features.[10] |
FAQ
Can I send PHI to an AI API?
Only if your contract, BAA, and selected feature path cover that exact use. OpenAI and Anthropic both publish BAA guidance with feature exceptions; Google Cloud, Azure, and AWS also require the right cloud agreement and customer-side configuration.[2][7][12][15][18]
Is Azure OpenAI or Vertex AI automatically more private than a direct API?
Not automatically. Managed cloud routes usually give stronger tenant, region, procurement, and compliance controls, but files, grounding, agents, batch jobs, and stored conversations still need feature-by-feature review.
What should security ask a vendor for?
Ask for the DPA, BAA if relevant, SOC or FedRAMP reports, subprocessors, region controls, retention matrix by endpoint, human-review policy, ZDR or abuse-monitoring options, deletion process, and a data-flow diagram for every tool or connector.
Sources
- OpenAI platform data controls: https://developers.openai.com/api/docs/guides/your-data
- OpenAI API BAA Help Center: https://help.openai.com/en/articles/8660679-how-can-i-get-a-business-associate
- Anthropic commercial data retention: https://privacy.claude.com/en/articles/7996866-how-long-do-you-store-my-organization-s-data
- Anthropic Files API documentation: https://platform.claude.com/docs/en/build-with-claude/files
- Anthropic zero data retention scope: https://privacy.claude.com/en/articles/8956058-i-have-a-zero-data-retention-agreement-with-anthropic-what-products-does-it-apply-to
- Anthropic certifications: https://support.claude.com/en/articles/10015870-what-certifications-has-anthropic-obtained
- Anthropic BAA coverage: https://support.claude.com/en/articles/8114513-business-associate-agreements-baa-for-commercial-customers
- Gemini API Additional Terms of Service: https://ai.google.dev/gemini-api/terms
- Gemini API abuse monitoring policy: https://ai.google.dev/gemini-api/docs/usage-policies
- Vertex AI zero data retention and data governance: https://docs.cloud.google.com/vertex-ai/generative-ai/docs/vertex-ai-zero-data-retention
- Vertex AI abuse monitoring: https://cloud.google.com/vertex-ai/generative-ai/docs/learn/abuse-monitoring
- Google Cloud HIPAA compliance: https://cloud.google.com/security/compliance/hipaa-compliance
- Google Cloud FedRAMP compliance: https://cloud.google.com/security/compliance/fedramp
- Microsoft Azure Direct Models data, privacy, and security: https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/data-privacy
- Microsoft Azure HIPAA compliance: https://learn.microsoft.com/en-us/azure/compliance/offerings/offering-hipaa-us
- Microsoft Azure FedRAMP compliance: https://learn.microsoft.com/en-us/azure/compliance/offerings/offering-fedramp
- Amazon Bedrock data protection: https://docs.aws.amazon.com/bedrock/latest/userguide/data-protection.html
- Amazon Bedrock FAQ security and compliance: https://aws.amazon.com/bedrock/faqs/