↓ Skip to main content
tomrochette.com
  1. Agents/
  2. Evaluation and review/

Phoenix

Author
glm-5.3-flash
History / Edit / PDF / EPUB / MD
Created: / Updated: / Status: finished / Readability: 3 / 5 minutes read (~830 words)
agent-curated 303 evaluation 13 fully-ai-generated 401 llm=glm-5.3-flash 314 observability 7 opentelemetry 1 research-note 273 tracing 1
Table of Contents

Phoenix is Arize AI’s open-source AI observability and evaluation platform: OpenTelemetry-native tracing, LLM and code evals, versioned datasets and experiments, a prompt playground, and a built-in AI engineering agent, self-hostable or running on the Arize AX cloud.

Phoenix is the observability column of this category, and its defining trade is the ELv2 license: everything is inspectable and self-hostable, but you cannot offer Phoenix itself as a service, and Arize keeps the production-grade monitoring surface in the paid platform.

What it is
#

A Python platform (pip install arize-phoenix, web UI on localhost:6006) built on OpenTelemetry with instrumentation from the companion OpenInference project: dozens of auto-instrumentors for OpenAI, Anthropic, LangChain, LlamaIndex, CrewAI, the Vercel AI SDK, and MCP. Traces flow over OTLP, evals attach to spans (LLM judges, code evaluators, human annotations, with documented interop for Ragas and deepeval), datasets and experiments version prompts, and span replay re-runs a captured step. A remote MCP server lets coding agents query the traces, and the PXI agent works inside the UI. Docker, Helm, and one-click deploys cover laptop through Kubernetes; TypeScript client packages ship alongside. Core platform under Elastic License 2.0 with the client and OTel packages Apache-2.0; made by Arize AI ($70M Series C, February 2025).

Status
#

Mature and busy: 11,689 stars, 1,178 forks, 1,091 open issues and PRs as of 2026-10-03. Created 2022-11-09, pushed 2026-10-03, platform release arize-phoenix 20.19.0 on 2026-10-01, about 650,000 PyPI downloads over the trailing 30 days ending 2026-10-03. Four years in, it is the oldest and most production-proven column in this category, with near-daily commits and weekly releases.

Strengths
#

  • OTel-native end to end: the same OTLP traces feed Phoenix and Arize AX, and OpenLIT and OpenLLMetry data convert in.
  • Covers the full loop, tracing, evals, datasets, experiments, prompt versioning, replay, not just trace storage.
  • Runs anywhere data must stay, from laptop to air-gapped clusters.
  • Modern DX including one-command coding-agent instrumentation and an MCP endpoint.

Cautions
#

  • Not truly open source: ELv2 bars offering Phoenix as a managed service, and a community issue calls the license overly restrictive for OSS compatibility.
  • Production monitoring (dashboards, alerting, issue grouping) lives in Arize AX, not the OSS project.
  • The Azure quick-deploy template serves plain HTTP and the Google Cloud button builds from source, per the README’s own notes.
  • 1,091 open issues and PRs is a large queue even for a project this size.

Pricing
#

Phoenix OSS is free and self-hosted with no paywalled features inside the product. Arize AX: Free (25k spans/month), Pro $50/month, Enterprise custom with SSO, SOC 2, HIPAA, SLAs, and self-hosting.

Price history
#

Date Plan Change Source
2026-09 Arize AX Baseline: Phoenix OSS free and self-hosted; AX Free (25k spans/mo), Pro $50/mo, Enterprise custom. arize.com/pricing

Compared to
#

  • deepeval: eval-first, pytest-style, no trace store; choose deepeval to gate CI, Phoenix when traces and a UI matter, and note they interoperate.
  • Langfuse: the closest feature peer, with a permissive MIT core (under ClickHouse ownership since January 2026) where Phoenix’s is ELv2; choose Langfuse when license terms dominate, Phoenix for OpenInference breadth and the AX upgrade path.
  • Plain OTel backends (Jaeger, Tempo, Datadog): excellent generic span storage with no LLM semantics; choose Phoenix when AI-specific observability is the point.

Bottom line
#

Recommended for teams that need production-grade AI tracing and evaluation under their own infrastructure, accepting ELv2 and the AX upsell gravity. Not for anyone resampling hosted observability on top of it, or strictly open-source shops whose policy excludes Elastic-licensed cores.

Changes
#

  • 2026-08-30 - Created in the Evaluation category, recording Arize’s OTel observability platform with an ELv2 license caution.
  • 2026-09-16 - Recorded the arize-phoenix 20.12.0 release and refreshed repository counts.
  • 2026-09-16 - Langfuse mention linked and expanded into a Compared-to entry.
  • 2026-09-18 - Recorded the arize-phoenix 20.14.0 release and refreshed repository counts.
  • 2026-09-20 - Added the Price history section tracking price changes in a table, per the new owner rule.
  • 2026-09-20 - Corrected the monthly PyPI download figure to about 840,000 per the pypistats source and refreshed repository counts.
  • 2026-09-25 - Recorded the arize-phoenix 20.16.0 release and refreshed repository counts.
  • 2026-09-25 - Re-measured the monthly PyPI figure at about 650,000 for the trailing 30 days ending 2026-09-25, after readings of about 840,000 (2026-09-20) and 770,000 (2026-09-21) on the same source.
  • 2026-10-03 - Refreshed repository counts and corrected the Status open-issues figure, which had drifted from the Cautions line (both now the same 1,091 count as of 2026-10-03).

See also
#

References
#