AgentGateway Explained: How It Compares to AWS, Azure, and Other AI Gateways

If you've been following along with the AgentGateway series, you've seen the four modes, poked at config.yaml, and wired up a few backends. This one's a detour. Someone asked me last week whether "Azure AgentGateway" was better than the open-source project I keep writing about, and I realized I'd never actually laid out where AgentGateway sits next to the rest of the market. So this post zooms out: what a gateway even is, why the AI-native flavor exists, who else is building one, and an honest look at where AgentGateway wins and where it doesn't.

This one's for the newbie who's never touched any of these tools, and for the pro who already runs Kong or LiteLLM and wants to know what the fuss about "agentic" gateways is actually about.

I'm looking at AgentGateway v1.3.0 (July 2026) and whatever the competing products looked like as of this writing. This market moves fast enough that some of this will be stale in a quarter. Check the vendor's own docs before you buy anything based on a blog post, including this one.

What is a gateway, actually

Strip away the AI branding and a gateway is an old idea: a single point of entry that traffic passes through on its way from a caller to a backend. Instead of every client talking directly to every server, everything goes through one place that can enforce rules, log what happened, and swap out backends without the callers noticing. Load balancers, reverse proxies, and ingress controllers are all cousins of the same idea.

The value isn't the routing itself, routing is easy. It's that a gateway becomes a natural chokepoint for the things you don't want scattered across a hundred services: auth, rate limits, retries, and observability.

Without the gateway, that "enforced once" box gets copy-pasted into every one of the N callers instead, and drifts out of sync the moment one team forgets to update it. That's the whole argument for the pattern, before AI ever enters the picture.

What is an API gateway

An API gateway is that same chokepoint idea applied specifically to application APIs, usually REST or gRPC. Kong, Apigee, and AWS API Gateway are the classic names here. They took the generic reverse-proxy pattern and bolted on things API teams actually needed: request transformation, API key management, per-route rate limiting, and a developer portal so other teams could discover what was available. If you've ever put an API behind Kong or Apigee to get consistent auth and throttling without touching the backend code, that's the pattern.

That last dashed box is the whole reason the AI-native gateway category exists. A REST/gRPC API gateway can proxy an LLM call as raw HTTP just fine, rate limit by request count, and log the response, but it has no concept of a token, a model name, or an MCP tool call embedded inside the body. It treats a $40 completion the same as a $0.0001 healthcheck, because to the policy layer above, they're both just bytes on the wire. That gap is what the next section is actually about.

What is an AgentGateway

AgentGateway (lowercase agentgateway as a project) is an open-source proxy that extends the gateway pattern to cover four kinds of traffic in one data plane instead of four separate tools: plain HTTP/gRPC service traffic, LLM provider traffic (OpenAI, Anthropic, Gemini, Bedrock, Azure, and others behind one endpoint), MCP traffic (agents calling tools and data sources), and A2A traffic (agents talking to other agents). It started life at Solo.io, who built it after evaluating Envoy and deciding the AI traffic pattern needed a proxy designed for it from scratch rather than bolted onto an existing one. It's now hosted by the Agentic AI Foundation under the Linux Foundation.

The pro bridge: if you've run Kong or an ingress controller for your services and separately bolted on LiteLLM or a custom proxy for your LLM calls, AgentGateway is what it looks like when someone designs both of those into the same binary instead of stitching them together.

Why we need it

Without something like this, teams end up running separate infrastructure for each traffic type: an API gateway for services, an LLM proxy for model calls, custom code for MCP auth, and nothing at all for agent-to-agent traffic until it becomes a security incident. Each of those has its own auth model, its own logging format, and its own on-call rotation. Multiply that by every team building an agent, and platform engineering ends up governing four things that all do roughly the same job of "sit between a caller and a backend and enforce policy."

The other driver is cost and governance. Token spend is real money now, and without a gateway layer, the only way to see who's burning budget on which model is to instrument every client separately, or find out from the bill.

The market players

This category filled up fast in 2025 and 2026, and it's already consolidating. A rough map of who's building what:

• LiteLLM is the most widely adopted open-source option for LLM-only routing: a self-hosted, OpenAI-compatible proxy in front of 100+ providers, with virtual keys and budgets. It doesn't touch MCP orchestration, A2A, or plain API traffic the way AgentGateway does, and it took a real hit to its reputation when a March 2026 supply-chain compromise shipped malicious code through two PyPI releases.

• Portkey leans into governance, guardrails, and observability on top of multi-provider routing, with a managed platform. It agreed to be acquired by Palo Alto Networks in April 2026, which is worth watching if you're picking a vendor for the long haul rather than a project you can fork.

• Kong AI Gateway extends Kong's established API-management platform with LLM plugins. It's the natural pick if Kong is already your API gateway standard, less so if you're starting from zero on AI-only infrastructure.

• Cloudflare AI Gateway is the managed, edge-native option: caching and basic routing baked into Cloudflare's network, minimal ops, but shallow governance compared to the others.

• Envoy AI Gateway (and Tetrate's commercial layer on top of it) takes the opposite approach from AgentGateway: build the AI routing as an extension of Envoy rather than a new proxy, which suits teams already deep in Envoy/Istio.

• Amazon Bedrock AgentCore Gateway is AWS's fully managed answer, not literally called "AWS AgentGateway" but functionally the closest thing. Per AWS's own developer guide, it has three target categories: MCP targets (it converts OpenAPI specs, Smithy models, and Lambda functions into MCP-compatible tools and aggregates them into one virtual MCP server), HTTP passthrough targets (which explicitly cover fronting other agents, including A2A traffic, without protocol translation), and inference targets (unified, model-based routing across providers). Inbound auth supports OAuth/JWT or IAM (SigV4); observability runs through CloudWatch metrics, logs, and OTEL-compliant spans, invocations, latency, duration, and error rates, though AWS's docs don't describe a dedicated dollar-cost dashboard the way Azure and AgentGateway have.

• Azure's answer isn't a single product either. Per Microsoft Learn, the AI gateway capabilities live inside Azure API Management and explicitly "extend API Management's existing API gateway; it's not a separate offering." It covers OpenAI-compatible and Anthropic-compatible LLM APIs, exposing REST APIs as MCP servers, importing A2A agent APIs, token rate limiting, semantic caching, and load balancing/circuit breakers across backends, plus an optional integration into Microsoft Foundry for governing models, agents, and tools from one place. Separately, and confirmed on Azure's own AKS engineering blog, Azure Kubernetes Service runs the actual open-source agentgateway project itself, as a data plane behind its AKS Application Network add-on. So depending on which "Azure AgentGateway" someone means, they're either talking about API Management's AI gateway (a distinct Microsoft product) or about the open-source agentgateway project running on Azure infrastructure, and those are not the same thing.

AgentGateway vs the field

Treat this table as a starting point, not gospel. Every vendor comparison you'll find online, including some of the sources I used here, is published by a competitor to at least one row in it. Test the fit against your own workload.

Advantages of AgentGateway

The single-proxy scope is the headline advantage: one data plane for services, LLM, MCP, and A2A means one auth model, one observability pipeline, and one team that owns it, instead of four. It's genuinely open source under a vendor-neutral foundation rather than open-core with the good parts held back, which matters if you've been burned by a project changing its license later. Being Rust-based gives it a real performance edge over Python-based alternatives like LiteLLM, which run into GIL and garbage-collection limits under heavy concurrent load. And because it grew out of Envoy and Istio Ambient experience, its policy engine (CEL expressions, OPA-style RBAC) is more expressive than most AI-only gateways bother building.

Disadvantages of AgentGateway

It's young. The project was created in March 2025, hit v1.0 roughly a year later, and ships on a monthly cadence, which is exciting for velocity but means breaking changes between minors are a real risk you need to plan config migrations around. Its ecosystem of integrations, guides, and third-party tooling is smaller than Kong's or LiteLLM's simply because those have had years longer to accumulate community content. If your organization is already fully committed to AWS or Azure and wants a single-vendor support relationship rather than an open-source project you operate yourself, a managed service like AgentCore Gateway or Azure APIM's AI gateway removes operational burden that AgentGateway asks you to own. And because it's self-hosted by default, you're responsible for scaling, patching, and securing it, the same tradeoff LiteLLM users accept and the same one that bit LiteLLM's users when a malicious release shipped through their own package manager.

Latest updates and where it's headed

The most recent minor, v1.3.0, rebuilt the LLM experience around a purpose-built UI with three native views (LLM, MCP, and traffic), added first-class token and dollar cost tracking attributed by model, provider, user, and team, introduced virtual models for weighted/failover/conditional routing, and added 13 new LLM providers along with an Azure provider covering both Azure OpenAI and Azure AI Foundry resources. The project has crossed roughly 2,000 GitHub stars and a million container image pulls in a single recent quarter, and it joined the Agentic AI Foundation as a hosted project, which is meant to keep it vendor-neutral as adoption grows.

Forward-looking work the maintainers have flagged publicly includes finer-grained authorization scoped to individual MCP tools rather than whole servers, progressive disclosure so agents can discover MCP capabilities dynamically instead of relying on static config, and code-execution modes meant to cut down the back-and-forth between agents and tools. None of that is shipped yet, so treat it as direction, not a commitment.

Where this leaves you

You've now got the ladder: gateway, API gateway, AI-native gateway, and a clearer sense of where AgentGateway sits against AWS, Azure, and the open-source alternatives. It's not the only option, and it's not automatically the right one for every team, but it's the only one on this list that treats plain services, LLM traffic, MCP, and A2A as one problem instead of four. Back to the regular series next: we'll pick up the build-and-ship rhythm again with the next numbered post.

Handy links

Primary sources for the AWS/Azure claims in this post:

• Amazon Bedrock AgentCore Gateway developer guide (official AWS docs)

• Core concepts for AgentCore Gateway (target types, incl. A2A via HTTP targets)

• AI gateway capabilities in Azure API Management (official Microsoft Learn docs)

• Control AI spend with Application Network and agentgateway (official AKS Engineering Blog, confirms agentgateway running on AKS)

AgentGateway project sources:

• agentgateway.dev (project home)

• agentgateway on GitHub (source, releases, roadmap discussion)

• AgentGateway v1.3.0 release notes

• agentgateway at the Agentic AI Foundation