Microsoft has introduced a dedicated AI Gateway tier for Azure API Management, now in public preview, giving teams a control plane built around models, MCP servers and tools instead of traditional APIs. The update is aimed at organizations that need one place to govern multiple AI providers, apply policy controls and expose approved capabilities to application teams without rebuilding the gateway from scratch.
A new Azure API Management AI Gateway tier
The dedicated tier is presented as a separate experience rather than an extra layer of policy on the existing gateway. Microsoft said the classic and v2 tiers still retain the AI gateway capabilities they already had, but the preview adds a model-first control plane designed for the way AI applications are now assembled.
According to the announcement, the shift reflects a practical change in operating model: teams are no longer fronting one provider, but many. The gateway is intended to help platform teams publish approved models and tools while giving application teams a self-service path to test and build against them.
Models from multiple providers through one gateway
The preview supports Foundry-hosted models, including OpenAI, Anthropic and Mistral, along with models in AWS Bedrock, Google Vertex AI and OpenAI directly. Microsoft said all OpenAI-compatible providers share a single endpoint path, with routing based on an exact match of the model field, which means every published model needs a unique name.
Anthropic is handled differently through a custom provider with Messages API passthrough. That detail matters for teams planning migrations or consolidations, since the gateway is clearly designed to normalize access without hiding provider-specific behavior where it still matters.
Policy controls are built into the portal
Instead of XML and expressions, policies are configured as cards in the portal. The available controls include token and request limits, quotas, Content Safety and model fallback.
Microsoft also said a gateway can be provisioned in about a minute and does not require scale units to be planned in advance. The resource runs in the customer’s own subscription and Entra tenant, and telemetry is exported as OpenTelemetry token metrics to destinations such as Application Insights, Datadog and Grafana, among others the customer controls.
Tools, MCP and connectors expand the gateway’s reach
The AI Gateway tier also federates backends from three sources: a remote MCP server by URL, an OpenAPI specification, or a built-in connector. Microsoft said that connector catalog covers more than a thousand SaaS applications and removes the need to host a separate server.
Once connected, each backend’s operations are surfaced as tools. For each backend, teams can choose whether the gateway authenticates with none, an API key, OAuth 2.0 or a managed identity, giving platform owners a way to fit the setup to existing security requirements.
How Microsoft is splitting control from self-service
The intended operating model is straightforward: a central platform group connects approved models and tools, publishes them and keeps the guardrails in place. Application teams then use those assets in a test console and build against them without routing every change through the platform team.
That division is likely to be the most important part of the release for larger organizations. It promises a cleaner separation between governance and delivery, while still leaving the platform team with visibility into usage and the ability to enforce policy.
Early reaction: consolidation, but governance questions remain
Initial reactions from architects and platform engineers have been broadly positive, especially around the consolidation of AI controls in one place. At the same time, the launch has prompted questions about where the governance boundary should sit.
Paolo Perrone, who writes the AI Engineer newsletter on production AI systems, said in a comment on the launch that the underrated part is putting cost governance at the gateway. He argued that most teams only add rate limiting and spend tracking after an incident, and that centralizing it gives “one control plane instead of per-app patches.”
Adolph White Jr., an enterprise AI systems architect, focused on a different issue: what happens when an agent run does not finish cleanly. He asked whether useful output is preserved for auditable review or whether the gateway fails over and retries, framing the distinction as “governing AI traffic and governing the full lifecycle.”
Questions about blast radius and coexistence
One design choice stands out: the runtime access key is gateway-scoped, which means it reaches every model and every tool published on that gateway. Microsoft’s guidance is one key per application, but that also means a leaked key exposes the whole gateway rather than a single product.
That is a different boundary from the APIM subscription model many teams already use to scope consumers to a set of APIs. The announcement also leaves coexistence questions open for organizations that have already built AI gateway patterns on Premium or Standard v2.
Preview limits, regions and pricing are still unsettled
The release is explicitly preview software, so Microsoft says availability is best effort and there is no SLA. APIs, telemetry, limits, regions and pricing can change before general availability, and the preview quotas cap models, tools, runtime keys and throughput, with the specific limits not published.
Pricing has not been announced yet, which leaves the cost-governance story incomplete for now. The AI Gateway tier is available in East US 2 and Sweden Central and is free during preview, with labs available for publishing a governed Foundry model and observing token rate limits.
Explore more: Blog Our Services Contact Us
Source: Original report
Was this helpful?
Last Modified: August 10, 2026 at 4:25 pm
0 views
