9 AI Gateways Compared: What Each One Is Actually For

Posted on by
Contents show
ai gateways 2026 featured 9 AI Gateways Compared: What Each One Is Actually For

An AI gateway sits between your application and the model providers: one API for many providers, retries and fallbacks, spend tracking, and a single place to enforce data policy. In a year when agent workloads multiplied token consumption and model choice became a weekly decision, the AI gateway stopped being optional infrastructure for anyone operating at scale, and it became the layer where costs, reliability and compliance are actually negotiated. In 2026 the category has both consolidated and fragmented at once, three of the best-known names changed owners this year, and the word gateway now describes tools that do genuinely different jobs. Calling a self-hosted proxy, an inference marketplace and an enterprise API plugin by the same name makes comparison lists nearly useless, so this article sorts the nine leading gateways by the job each does, then says what each is actually for.

The facts here come from each product's own documentation and release materials, verified individually: LiteLLM's docs, Bifrost's docs, Portkey's docs, Cloudflare's AI Gateway documentation, and the published terms and pages of the managed platforms. Where a claim comes from a vendor's own marketing rather than its docs, it is marked as such.

What changed in 2026, and why it matters for choosing

Three ownership changes this year reshaped the category, and any gateway decision made from an old comparison list is now working from stale facts. Helicone was acquired by Mintlify in March 2026 and is in maintenance mode, meaning security fixes and new models but no new features, which makes planning a move off its proxy the prudent default. The pattern to note is that all three acquisitions went to companies whose core business is adjacent rather than identical, documentation, security and payments, which means each gateway now serves a parent strategy as well as its users.

Portkey was acquired by Palo Alto Networks in May 2026 and is now sold as Prisma AIRS AI Gateway, which means upgrades route through Palo Alto's sales motion and the roadmap follows enterprise security buyers. OpenRouter was acquired by Stripe in August 2026, and the company states its brand, product and neutrality remain unchanged, but your model marketplace is now owned by a payments company, which is a fact worth weighing for teams that valued its independence.

None of these changes makes any product bad. All of them change what to expect from the roadmap, and roadmap direction matters more in infrastructure than current features, because a gateway is a dependency you build around.

Group one: managed gateways that also sell inference

These AI gateway products give you one API to many models and also sell you the tokens, so you skip a contract with each provider.

Concentrate AI exposes an OpenAI-compatible API, plus an Anthropic-compatible endpoint, for 196 models across more than 20 providers, charging no platform, service or card fee with free bring-your-own-key. Its distinguishing feature is same-model failover: Claude failing over across Anthropic, Bedrock, Azure and Vertex without code changes, which matters because a fallback to a different model changes your output while a fallback to the same model on another cloud does not. The docs claim zero-data-retention enforcement that fails closed on 104 models, per-key PII redaction, budgets counting BYOK spend by default, and SOC 2 Type II. The stated gaps: no OpenTelemetry export yet and US-only data residency.

OpenRouter is the largest model marketplace and the fastest way to try a new model, with no inference markup but a 5.5 percent credit-purchase fee, minimum 0.80 dollars, and 5 percent on BYOK above 25,000 dollars a month. Its job is exploration and model access; teams at scale typically add their own controls on top. Post-acquisition, the question for larger deployments is whether its neutrality promise survives its new owner's incentives, and the honest answer is that only time answers that.

Vercel AI Gateway offers zero markup and zero BYOK fee, budgets at four scopes, and OpenTelemetry trace drains, with team-wide zero-data-retention and provider allowlisting metered at 0.10 dollars per 1,000 requests each. Its notable gap is that BYOK spend is not counted in budgets, which matters for anyone whose spend controls are a compliance requirement rather than a convenience. The natural fit is teams already on Vercel.

Group two: self-hosted proxies

These AI gateway products run on your infrastructure: you keep the keys, and you operate the service.

LiteLLM is the open-source standard: a unified interface to more than 100 LLM providers using the OpenAI format, with a self-hosted gateway proxy that adds virtual keys, cost tracking, budgets, guardrails and an admin UI. The docs position it as a unified AI gateway for LLMs, agents and MCP, one endpoint for models, A2A agents and MCP tools, which reflects where agent deployments have gone this year. The guardrails module covers content filtering and PII masking at the proxy layer, observability integrates with Langfuse,

MLflow and Helicone itself, and the virtual-key system maps budgets and rate limits to teams and users, which is the feature set that turns a proxy from a convenience into a governance point. The free tier covers the core; SSO, SCIM, audit logs and RBAC live in the Enterprise edition. The real cost is operational: you run it, you patch it, you staff the Sunday night, and you still need provider accounts or an inference layer behind it.

Bifrost, from Maxim, is an Apache 2.0 Go gateway built for overhead that is nearly invisible: its sustained benchmarks claim 11 microseconds added per request at 5,000 requests per second. It unifies more than 20 providers, ships virtual keys, budgets, rate limits, semantic caching, MCP gateway functionality including tool filtering and agent mode, and OpenTelemetry out of the box, with clustering, guardrails integrations, RBAC and audit logs in the enterprise tier.

Its docs include a migration guide from LiteLLM, which tells you who it competes with. Choose Bifrost when overhead is the constraint and your team can run Go infrastructure; choose LiteLLM when provider breadth and ecosystem maturity matter more. Both AI gateway projects are actively developed and open source, so the self-hosted path in 2026 is genuinely strong, and the choice between them is about operational fit rather than capability gaps.

Group three: gateways inside a larger platform

Portkey, now Prisma AIRS AI Gateway, is the platform-embedded AI gateway that is strong on guardrails, prompt management, caching and logs, with a documented feature set covering fallbacks, load balancing, canary testing and virtual keys. Production pricing is 49 dollars a month for 100,000 logs, with SSO and granular budgets reserved for enterprise. Under Palo Alto, its trajectory points at security-led enterprise buyers; for teams that want AI reliability as part of a security stack, that is a feature, not a bug.

TrueFoundry prices its Pro tier at 25 dollars per user per month, including pooled team budgets and SSO, and its pitch is bundling the gateway with a deployment platform. Choose it if you want the model gateway and the ML platform from the same vendor; the per-seat pricing model is the trade to model before committing.

Group four: gateways inside your existing stack

Cloudflare AI Gateway is the edge-deployed AI gateway that adds caching, analytics, logging, rate limiting, retries and model fallback to your AI traffic, available on all plans, with the core features free. Its documentation positions this AI gateway as observability and control with one line of code, and the practical trade is scope: it is excellent at the edge layer, and teams needing deep LLM-specific governance, redaction policies or per-team virtual keys find those elsewhere. Unified Billing adds 5 percent to credit purchases. If you already run on Workers, this is the default.

Kong AI Gateway is AI plugins for Kong Gateway, and it makes sense in exactly one situation: your platform team already runs Kong for every other API, and AI traffic should be governed by the same policies, the same tooling and the same people. Adopting Kong solely for AI would be heavy; adopting it as an extension of an existing Kong estate is the obvious move.

The fees nobody reads until the invoice

The unglamorous differentiator across managed AI gateway products is the fee schedule, and it is where sticker-price comparisons mislead. Most gateways do not mark up tokens in 2026; the costs that remain live in the fine print: credit-purchase fees, BYOK fees above thresholds, per-request metered add-ons, and subscriptions for features like zero-data-retention that compliance may require rather than merely want.

The differences are material at scale. A 5.5 percent credit fee on 50,000 dollars of monthly tokens is 2,750 dollars a month for the same models another gateway sells with no fee. A BYOK fee of 5 percent above a threshold changes the calculus for teams that negotiated direct provider contracts and want only the control layer. Metered add-ons, such as per-request charges for zero-data-retention or allowlisting, convert a fixed cost into one that scales with traffic, which is either fine or dangerous depending on growth shape. And the accounting question, whether BYOK spend counts toward the budgets you set, determines whether the gateway's spend controls are a governance tool or a partial view.

The discipline this site recommends for token-level routing decisions applies at the gateway level: model the total cost with your actual traffic, including the fees, before committing. The vendors publish their fee schedules; the work is reading them against your own numbers, and the invoices arrive monthly whether or not you did.

How to choose in five minutes

The decision collapses to two questions: where does your traffic live, and who operates it. If you spend over 10,000 dollars a month on tokens and security is asking questions, the managed gateways with fail-closed data controls are the conversation, and the differentiators are fee structure, BYOK accounting and same-model failover. If you are exploring models on a small budget, OpenRouter remains the fastest path. If you must self-host,

LiteLLM is the ecosystem default and Bifrost is the low-overhead challenger, and both are genuinely good. If your company has already standardized on Palo Alto, Cloudflare or Kong, use that vendor's gateway, because consistency with your existing controls beats marginal feature wins. And if you are all-in on Vercel, its gateway is priced to be a non-decision.

The one AI gateway check that applies to every choice: verify the fee schedule and the BYOK accounting in the current docs, not in a comparison list, because 2026 proved that this category's ownership and pricing can change inside a quarter. And once chosen, the gateway deserves monitoring like any other critical dependency: track its own uptime, watch its changelog for ownership-driven direction changes, and keep the migration path warm, because the cost of switching a gateway grows with everything you build on it.

That is the lesson this site's infrastructure coverage keeps returning to, from model routing economics to the supply chain attack on LiteLLM itself: the gateway is load-bearing infrastructure, it carries every prompt and every key, and choosing it deserves the same scrutiny as choosing the models behind it.

Where the numbers stop

  • The nine-gateway set and each product's gaps are drawn from a comparative snapshot taken in October 2026; pricing and feature locations, especially which tier includes SSO or audit logs, move frequently and should be verified in each vendor's current docs before a purchase decision.
  • The Helicone, Portkey and OpenRouter ownership changes are dated events, but the roadmap consequences described here are inferences from public positioning, not documented product plans.
  • Bifrost's 11-microsecond overhead figure is the vendor's own sustained benchmark at a specific load (5,000 requests per second); other gateways do not publish directly comparable numbers, so no cross-product latency ranking is possible from published data.
  • Concentrate AI's model count (196 models, 104 with fail-closed ZDR), fee claims and compliance certifications are the vendor's own statements from its product materials; the SOC reports are verifiable through its trust center, the rest are marketing until audited.
  • This article does not benchmark any gateway hands-on, and no figure here should be read as an independent performance test.

FAQ

What is an AI gateway and why would I need one?

An AI gateway sits between your application and model providers, giving you one API across many providers, retries and fallbacks, spend tracking and limits, and a single enforcement point for data policy. You need one when you use more than one provider, when you need budgets and visibility across teams, or when compliance requires a control point in front of models. In 2026 the category spans managed gateways that also sell inference (OpenRouter, Vercel AI Gateway), self-hosted proxies (LiteLLM, Bifrost), platform-embedded gateways (Portkey, TrueFoundry) and gateways inside your existing edge stack (Cloudflare, Kong), and the right choice depends on which of those jobs you actually have.

What changed in the AI gateway market in 2026?

Three ownership changes: Mintlify acquired Helicone in March, putting it in maintenance mode; Palo Alto Networks acquired Portkey in May, now selling it as Prisma AIRS AI Gateway; and Stripe acquired OpenRouter in August, with the company stating its product and neutrality stay the same. The practical consequence is that comparison lists written before March are stale, and any gateway decision should start by verifying the current ownership, roadmap and fee schedule in the vendor's own docs.

Which AI gateway should a small team start with?

For exploring many models cheaply, OpenRouter remains the fastest path despite its 5.5 percent credit fee. For a team that has standardized on a platform, the platform's own gateway (Vercel, Cloudflare, Kong) is the path of least resistance. For teams that must keep traffic and keys inside their own infrastructure, LiteLLM is the ecosystem default with the broadest provider support, and Bifrost is the low-overhead alternative built in Go. The universal advice: verify current fees and BYOK accounting in the vendor's docs, because this category's pricing changed multiple times in 2026.

The short version

The 2026 AI gateway market is four different jobs wearing one name: managed gateways that sell inference, self-hosted proxies, platform-embedded gateways and edge-stack plugins. Three of the best-known names changed owners this year, Helicone went to maintenance mode, Portkey became a Palo Alto product, and Stripe bought OpenRouter, which makes every old comparison list stale. The choosing questions are where your traffic lives and who operates the gateway, and the universal checks are the fee schedule and BYOK accounting in current docs.

LiteLLM remains the self-hosted default, Bifrost the low-overhead challenger, and the managed field is now differentiated by data controls and same-model failover rather than model counts. A gateway carries every prompt and every key, so it deserves the scrutiny you would give any load-bearing dependency.

Sources
Subscription Form