LiteLLM is the dominant self-hosted AI gateway, a Python SDK plus proxy server that exposes one OpenAI-compatible endpoint over 100+ provider APIs with routing, budgets, and spend tracking.
Self-hosting the gateway means you own its availability, its config surface, and (since March 2026) its supply-chain risk, and that ownership is the entire trade.
What it is #
BerriAI’s gateway ships in two pieces: a Python SDK, a drop-in OpenAI-format client for 100+ providers, and a self-hosted proxy server with virtual keys, per-key budgets, rate limits, an admin UI, and spend tracking.
The Router adds retries, fallbacks, and load balancing across deployments, so an agent survives a single provider’s outage or rate limit.
The proxy now also fronts MCP servers and A2A agents, making one deployment the gateway for models, tools, and agents at once.
A Rust core is in beta, and the repo description now reads “Rust core with Python SDK”.
The core is MIT licensed; the enterprise/ directory ships under a separate commercial license.
Status #
Very active: 61,356 stars as of 2026-10-16, repo pushed 2026-11-04 (as of 2026-11-04), publicly released July 2023.
The cadence is extreme: a new minor line roughly every week, with v1.98.1, v1.99.4, v1.100.3, v1.101.2, and v1.102.1 all published between September 16 and 25, 2026, then v1.106.1 stable on 2026-10-20 through v1.106.5 on 2026-10-27, v1.106.6 on 2026-10-28 (with v1.105.7 patched the same day), a train carrying v1.109.0-dev.1 (2026-10-28), v1.108.0-rc.1 (2026-10-29, the first tag the then-never-stable v1.108 line had seen), and v1.109.0-dev.2 (2026-10-30), and on 2026-11-04 v1.108.0 went stable (the v1.107 line never reaching stable at all), v1.106.7 patched, and v1.109.0-rc.1 published, with PyPI latest at 1.108.0 and the repo pushed 2026-11-04 (as of 2026-11-04). Since 2026-06-29 only the four most recent stable minor lines receive patches, a policy the company states is a direct cost of that cadence. In March 2026 the project’s PyPI publishing credentials were compromised: poisoned 1.82.7 and 1.82.8 wheels shipped a credential stealer and were quarantined within about a day (PYSEC-2026-2).
Strengths #
- One OpenAI-compatible surface over 100+ providers, with the Router handling retries, fallbacks, and load balancing.
- Virtual keys, per-key and per-team budgets, rate limits, and spend tracking are in the free self-hosted tier, not gated.
- The MCP and A2A gateways converge into the same endpoint, so one deployment fronts models, tools, and agents.
- The observability surface is broad, with callbacks for Langfuse, MLflow, Helicone, and OpenTelemetry.
- Day-0 support for new providers is effectively weekly, which matters when an agent stack adopts a fresh model fast.
Cautions #
- March 2026: compromised 1.82.7/1.82.8 PyPI wheels stole SSH keys, cloud credentials, and Kubernetes tokens, exfiltrating to a lookalike domain; TechCrunch later tied the Mercor breach to the same compromise.
- A gateway aggregates every provider credential you own, so it is a single high-value theft target; pin versions and verify tags before pulling.
- The config surface moves fast: minor lines ship weekly and older lines lose support within four releases, so an unpinned deployment drifts.
- Community pushback exists: a September 2026 “LiteLLM without the bloat” fork hit 177 points on Hacker News, with commenters arguing the SDK has outgrown its size and dependency budget.
- SSO above 5 users, audit logs, and most built-in moderation guardrails require the enterprise license.
- No public enterprise pricing; everything beyond OSS is quote-based, with no published floor.
Pricing #
Self-hosted OSS: free, MIT licensed (excluding the enterprise/ directory).
Enterprise: quote-based license scoped to deployment size, adding SSO/SCIM, audit logs, organization RBAC, IP allowlists, multi-region deployment, and secret-manager integration; SSO is free for up to 5 users.
Standard enterprise support is included with the license, and 24/7 SLAs (Sev 0 in 1 hour) cost extra.
No model-access fees: providers bill tokens directly and LiteLLM takes no cut, as of 2026-10-02.
Compared to #
OpenRouter is the hosted counterpart: the same multi-provider idea, but someone else runs it and takes a 5.5% credit fee; choose LiteLLM when data residency or key custody rules out a third party. OpenCode Zen sells curation of benchmarked endpoints; LiteLLM sells control, with no curation guarantee. Requesty is the category’s other hosted gateway, lighter to adopt; LiteLLM wins on self-hosting and enterprise controls.
Bottom line #
Recommended for platform teams that must keep provider keys and logs inside their own perimeter, or that need per-team budgets across many providers. Not for a single-provider setup or a small project, where the gateway is operational overhead without a payoff. My disagreeable claim: below three providers, self-hosting LiteLLM is negative value, because you are running a stateful service to avoid editing a base URL.
Changes #
- 2026-09-27 - Created when the owner’s GitHub-stars scan surfaced it.
- 2026-10-06 - Converted the Compared-to cross-references from plain-text paths into working links.
- 2026-10-07 - Added the BerriAI/litellm star history chart to the Status section.
- 2026-10-16 - Release train refresh: the v1.105 line went stable with PyPI latest at 1.105.1 (2026-10-14), the first stable minor past 1.104.2, and the GitHub train advanced to v1.106.0-rc.4 (2026-10-15) and v1.107.0-dev.3 (2026-10-16); stars refreshed to 61,356.
- 2026-10-18 - Release train refresh: v1.106.0 left pre-release and went stable on 2026-10-18 (v1.103.5 patched 2026-10-17 behind it) with v1.107.0-dev.3 still on the train; stars stand at their 2026-10-16 verification.
- 2026-10-19 - Release train refresh: v1.107.0-rc.1 published 2026-10-18, the first rc on the v1.107 line, and PyPI latest moved to 1.106.0; stars stand at their 2026-10-16 verification.
- 2026-10-21 - Release train refresh: v1.106.1 went stable 2026-10-20 with v1.105.2, v1.104.4, and v1.103.6 patched the same day and v1.107.0-rc.2 on the train; PyPI latest moved to 1.106.1; stars stand at their 2026-10-16 verification.
- 2026-10-23 - Release train refresh: v1.106.2 went stable 2026-10-21 with v1.105.3, v1.104.5, and v1.103.7 patched the same day and v1.107.0-rc.2 still the train head; PyPI latest moved to 1.106.2 and the repo pushed 2026-10-23; stars stand at their 2026-10-16 verification.
- 2026-10-25 - Release train refresh: v1.106.3 went stable 2026-10-24 with v1.105.4, v1.104.6, and v1.103.8 patched the same day and v1.107.0-rc.2 still the train head; PyPI latest moved to 1.106.3 and the repo pushed 2026-10-25; stars stand at their 2026-10-16 verification.
- 2026-10-26 - Release train refresh: v1.106.4 went stable 2026-10-25 with v1.105.5, v1.104.7, and v1.103.9 patched the same day and v1.107.0-rc.2 still the train head; PyPI latest moved to 1.106.4 and the repo pushed 2026-10-26; stars stand at their 2026-10-16 verification.
- 2026-10-28 - Release train refresh: v1.106.5 went stable 2026-10-27 with v1.105.6, v1.104.8, and v1.103.10 patched the same day; PyPI latest moved to 1.106.5 and the train head jumped two lines to v1.109.0-dev.1 (2026-10-28), the v1.107 line never having passed rc.2 and no v1.108 tag existing at all; the repo pushed 2026-10-28; stars stand at their 2026-10-16 verification.
- 2026-10-29 - Release train refresh: v1.106.6 went stable 2026-10-28 with v1.105.7 patched the same day; PyPI latest moved to 1.106.6 and the v1.108 line, which had never been tagged, got its first rc with v1.108.0-rc.1 published 2026-10-29, so the train now carries both a v1.108 rc and the v1.109 dev tag; the repo pushed 2026-10-29; stars stand at their 2026-10-16 verification.
- 2026-11-01 - Release train refresh: v1.109.0-dev.2 published 2026-10-30 as the new train head, with v1.106.6 stable still the latest release and PyPI latest at 1.106.6; the repo pushed 2026-11-01; stars stand at their 2026-10-16 verification.
- 2026-11-04 - Release train refresh: v1.108.0 went stable on 2026-11-04, the line that had sat at rc.1 since 2026-10-29, with the v1.107 line never reaching stable at all, v1.106.7 patched the same day, and v1.109.0-rc.1 published as the train head; PyPI latest moved to 1.108.0 and the repo pushed 2026-11-04; stars stand at their 2026-10-16 verification.
See also #
- OpenRouter - the hosted gateway most teams compare against this one.
- OpenCode Zen - the curated-endpoint counterpoint to uncurated self-hosting.
- Requesty - the category’s other hosted gateway.
- OpenCode - a coding harness that consumes gateways like this one.
- Model provider feature matrix - where LiteLLM sits among access providers.
References #
https://api.github.com/repos/BerriAI/litellm - 60,260 stars, Python, license metadata, pushed 2026-10-07 (fetched via the GitHub API, 2026-10-07; re-fetched 2026-10-14 at 61,173 stars; re-fetched 2026-10-19, pushed 2026-10-19; re-fetched 2026-10-21, pushed 2026-10-21; re-fetched 2026-10-23, pushed 2026-10-23; re-fetched 2026-10-24, pushed 2026-10-24; re-fetched 2026-10-25, pushed 2026-10-25; re-fetched 2026-10-26, pushed 2026-10-26; re-fetched 2026-10-28, pushed 2026-10-28; re-fetched 2026-10-29, pushed 2026-10-29; re-fetched 2026-11-01, pushed 2026-11-01; re-fetched 2026-11-04, pushed 2026-11-04).
https://docs.litellm.ai/docs/ - SDK and proxy feature claims, 100+ providers, MCP/A2A gateway, Rust beta (200).
https://raw.githubusercontent.com/BerriAI/litellm/main/LICENSE - MIT text with the enterprise/ directory carve-out (200).
https://futuresearch.ai/blog/litellm-attack-transcript/ - minute-by-minute discovery of the March 2026 PyPI compromise, payload behavior, PYSEC-2026-2 (200).
https://hn.algolia.com/api/v1/search?query=LiteLLM&tags=story&hitsPerPage=8 - the 938-point compromise thread, the Mercor/TechCrunch story, the litelm bloat fork (200).
https://pypi.org/pypi/litellm/json - PyPI latest at 1.106.2, published 2026-10-21 (200, re-checked 2026-10-23 and 2026-10-24; re-checked 2026-10-25 reading 1.106.3, published 2026-10-24; re-checked 2026-10-26 reading 1.106.4, published 2026-10-25; re-checked 2026-10-28 reading 1.106.5, published 2026-10-27; re-checked 2026-10-29 reading 1.106.6, published 2026-10-28; re-checked 2026-11-01, 1.106.6 unchanged; re-checked 2026-11-04 reading 1.108.0, published 2026-11-04).