Best Open Source LLM Gateways of 2026

LiteLLM wins on community defensibility because Apache 2 / MIT license plus 100-plus provider integrations plus around 15,000 GitHub stars is the strongest free-forever profile in OSS LLM proxies. Portkey wins on production feature breadth specifically. Teams that want a focused routing proxy pick LiteLLM. Teams that want a full production gateway with caching and guardrails pick Portkey. Both are auditable OSS at any revenue level.

On a few. We disclose this on every /best page. Self-host OSS (LiteLLM Apache 2 / MIT, Portkey Apache 2.0, Langfuse MIT) has no affiliate path because there is no transaction. Cloud upgrades on LiteLLM Pro, Portkey Production, and Langfuse Core have paid plans where we earn commission only on conversion. The composite ranking weights price at 40 percent, features at 30, free tier at 15, fit at 15; none tuned by affiliate rate.

LiteLLM plus Langfuse splits the proxy and observability layers; you run two specialized tools that each excel at their layer. Portkey bundles both in one product. The decision pivots on team posture. Teams that want maximum flexibility and accept operational tax for two services pick the LiteLLM plus Langfuse stack. Teams that want one tool covering both layers pick Portkey. Both paths are dollar-zero on licensing.

A small LiteLLM deployment runs on a single five to fifteen dollar VPS with Python. A small Portkey OSS deployment runs on a similar VPS with Node.js. Langfuse self-host needs Postgres plus the Langfuse application; around fifteen to thirty dollars monthly. For staging and prod, double the cost. Above that, you absorb the operational tax of patching, backups, monitoring, and on-call. Cloud upgrades trade labor for managed hosting at fifty to seventy-nine dollars monthly.

Yes for high-concurrency production workloads. Bifrost is Go-based with around 11 microseconds gateway overhead at 5,000 requests per second under sustained traffic. LiteLLM is Python-based; Python overhead can add hundreds of microseconds at high concurrency. For low-volume indie-dev workflows the difference is invisible. For latency-critical production at scale, Bifrost performs measurably better. Bifrost is out of our catalog; from our catalog LiteLLM is the OSS proxy pick.

Yes for routing logic. Application code calling OpenAI SDK works unchanged behind LiteLLM proxy. Cost-tracking and observability data does not migrate; you start fresh on the new tool. Plan one to three weeks for a non-trivial migration of routing logic plus dashboards. The honest framework: do not start a migration without budgeting design time; OSS self-host has different operational characteristics than managed SaaS, and direct ports rarely match feature for feature.

The LLM gateway category emerged in late 2022 and early 2023 as developers building on OpenAI hit the same problems: provider lock-in, no cost visibility, no caching, no fallbacks. Y Combinator funded several teams that winter (W23) at different layers: LiteLLM (OSS proxy), Helicone (observability-first proxy), Langfuse (OSS observability). The timing aligned with ChatGPT launch (Nov 2022) and the GPT-4 API release (March 2023).

All three OSS picks self-host gives full control of where data lives, so EU residency is user-controlled on self-host. Langfuse Cloud is Berlin-based with default EU residency. LiteLLM Cloud has multi-region with EU option. Portkey Cloud has multi-region with EU on Pro and Enterprise. For EU-resident OSS use, all three self-host paths qualify; managed Cloud picks all offer EU regions on paid tiers.

We re-review pricing and feature changes annually at minimum, with mid-year refreshes when major vendor announcements happen. Portkey open-sourced under Apache 2.0 in March 2026 (a major shift). LiteLLM crossed 100 providers in 2024. Langfuse Core launched at twenty-nine dollars monthly in 2024. The lastReviewed date reflects the most recent editorial pass.