If you have ever shipped an LLM application and then asked “what did the model actually see, and what did it answer?”, you already understand the problem Arize set out to solve with Phoenix. Phoenix is an open-source, OpenTelemetry-native observability platform for LLM apps: it ingests traces and spans from your agents, RAG systems, and SDK calls, stores them in a real database, and gives you a web UI plus REST and GraphQL APIs to query, annotate, and evaluate them. It also does evals — LLM-graded scoring of outputs like retrieval relevance, hallucination, and question-answering correctness.

What makes Phoenix interesting to read at the source level is that it is not a toy collector. The main package boots a FastAPI application with a Strawberry GraphQL schema, an async gRPC server implementing the standard OpenTelemetry trace service, a background bulk-insert writer, and a SQLAlchemy storage layer that runs on SQLite by default with PostgreSQL as the scale-out option. Everything speaks OTLP, so any OpenTelemetry-instrumented program can ship traces to it without vendor lock-in.

This post is a source tour of the Arize-ai/phoenix repository at its main branch. Phoenix is a monorepo: the core server lives under src/phoenix/, and three lightweight Python sub-packages — arize-phoenix-otel, arize-phoenix-client, and arize-phoenix-evals — live under packages/. I verified every file path, symbol name, and line number below against the checked-out tree, and mapped the whole system into two diagrams. The first shows the shape of the system; the second goes one level deeper into each area.

Overview of Phoenix at `main`: the arize-phoenix-otel package exports spans to either the gRPC or HTTP ingest surface of the server; a bulk inserter writes them into SQLAlchemy-backed storage; REST, GraphQL, and notebook sessions read the same rows; and the client and evals packages talk back over the versioned REST API.

Reading the overview from left to right:

• register(), L65 — the entry point of arize-phoenix-otel. It wraps OpenTelemetry primitives with Phoenix-aware defaults (endpoint, project name, headers) and auto-instruments any installed OpenInference libraries so LangChain, LlamaIndex, OpenAI, and friends start emitting spans with no extra code.
• run(), L217 — what phoenix serve actually executes. It reads every environment variable up front, builds the async database session factory, and hands the app to uvicorn.
• create_app(), L939 — the FastAPI factory that assembles middleware, the v1 REST router, the GraphQL router, the gRPC server, and the bulk inserter in one place. If you only read one file in the repo, read this one.
• Export(), L41 — the gRPC handler implementing the OpenTelemetry TraceService. It walks resource_spans, resolves the project name from resource attributes, and enqueues each decoded span.
• post_traces(), L461 — the HTTP/JSON twin of the gRPC path, mounted at POST /v1/traces and advertised in the boot banner as otlp_http_url.
• decode_otlp_span(), L97 — converts protobuf OTLP spans into Phoenix’s own Span dataclasses, synthesizing OpenInference attributes along the way.
• BulkInserter, L115 — the background writer that batches queued spans and annotations, retries transient failures, and flushes them to the database.
• Span, L867 — the SQLAlchemy ORM model at the bottom of the stack, next to Trace (L808), Project (L740), Dataset (L1438), and Experiment (L1680).

The detail view: instrumentation internals on the left, the server entry chain and OTLP ingest in the middle, the write path into the ORM models, and the three query/read surfaces plus the client and evals packages on the right.

The Write Path: From SDK Call to Database Row

The ingest story is the heart of Phoenix. On the instrumentation side, register() returns a Phoenix-flavored TracerProvider (L200) that wires up either an HTTPSpanExporter (L541) or a GRPCSpanExporter (L619), depending on the endpoint scheme you configured. That means the same register() call works whether your Phoenix is local, containerized, or remote.

On the server side, GrpcServer (L55) owns the grpc.aio server lifecycle — port binding, optional TLS and mutual TLS, and interceptors, including an API-key interceptor when authentication is enabled. Its Servicer.Export runs the protobuf-to-domain translation in a thread pool (so the event loop never blocks on decoding) and then hands each (span, project_name) pair to an async enqueue callback.

Everything lands in the BulkInserter. Rather than writing each span as it arrives, Phoenix queues them and lets _insert_spans() (L283) drain the queue in batches, calling insert_span() (L128) to persist rows inside a session. After each successful insert, DML events flow into DmlEventHandler (L243), which fans them out to GraphQL subscriptions and cache invalidation — that is how a live UI shows new traces without polling.

Two practical details worth knowing: the HTTP ingest endpoint and the gRPC endpoint accept the same OTLP payloads (one in JSON form, one in protobuf form), so you can point the OpenTelemetry collector, arize-phoenix-otel, or any OTLP-capable SDK at either. And spans carry OpenInference semantic-convention attributes — prompt text, completion text, retrieval documents, token counts — which is what makes the UI render actual model conversations instead of raw key-value noise.

From Install to Your First Model

Getting a working Phoenix plus a traced application is a five-minute job. The steps below come straight from the repository README and the verified code paths above.

First, install and boot the server. The README quickstart is two lines, with a no-install alternative via uvx:

pip install arize-phoenix
phoenix serve
# or: uvx arize-phoenix serve

The server listens on port 6006 by default (the client’s documented fallback is http://localhost:6006), prints a boot banner listing its UI, REST, GraphQL, and OTLP endpoints, and creates a SQLite database on first run. run() in the serve command resolves all of this from environment variables, and create_app() applies migrations on boot through the engine factory.

Second, wire up tracing. The fastest path is the coding-agent setup command from the README (npx @arizeai/phoenix-cli setup, or px setup with Phoenix installed), which detects your framework and LLM provider and installs the matching OpenInference instrumentation. If you prefer code, install the small OTel wrapper package:

from phoenix.otel import register

tracer_provider = register()  # auto-instruments installed OpenInference libraries

That single call returns a configured TracerProvider and starts exporting spans to your local server. If you use LangChain or LlamaIndex, the instrumentation hooks emit ready-made traces with prompts, retrievals, and token usage — see our LangChain and LlamaIndex source tours for how those frameworks’ internals look from the outside.

Third, query and export what you collected. Install the client package (pip install arize-phoenix-client) and use the typed REST client:

from phoenix.client import Client

client = Client()  # defaults to http://localhost:6006
df = client.spans.get_spans_dataframe()   # export trace data as pandas
client.datasets.create_dataset(...)       # version prompts and examples

Client (L20 of the client module) exposes resource objects for prompts, projects, spans, traces, sessions, datasets, and experiments, each mapping onto the v1 REST surface. The classic notebook experience still works too: import phoenix as px and call launch_app() (L268), which starts or attaches to a server and returns a Session object for in-notebook browsing.

Datasets, Experiments, and Evals

Phoenix treats evaluation as a first-class workflow rather than an afterthought. A Dataset is versioned: examples you upload (individually via the client’s create_dataset and add_examples_to_dataset, or as CSV/JSONL uploads) are frozen into versions so an experiment’s results always reference the exact inputs used. An Experiment then records the outputs of a task function run over those examples, along with evaluator scores — the client’s Experiments resource (L592 of its module) orchestrates the run and accepts evaluator callables.

The arize-phoenix-evals package supplies the graders. Evaluator (L305) is the abstract base, LLMEvaluator (L515) wraps any LLM with a prompt template to score outputs (relevance, hallucination, QA correctness), and ClassificationEvaluator (L607) maps raw model responses onto labeled choices. Scores land back in Phoenix as span or trace annotations, which is how the UI renders per-row eval badges. If you have used OpenAI’s evals harness, the mental model is similar — see our openai-evals tour for the comparison point — but Phoenix binds the results to specific traces, so a bad score is one click away from the conversation that caused it.

The Query Surfaces

One database, three read paths. The v1 REST router (create_v1_router (L35)) is the stable, versioned surface the Python and TypeScript clients target, with cursor-paginated endpoints like list_project_traces (L148). The GraphQL schema (build_graphql_schema (L67), mounted via create_graphql_router (L801)) is what the React UI actually consumes, including subscriptions fed by the DML event handler. And the notebook Session classes offer an embedded experience where a Phoenix server runs in a background thread or process next to your kernel.

It is worth appreciating the consistency here: all three surfaces read the same SQLAlchemy models, and all writes — OTLP ingest, annotations logged via REST, dataset uploads — flow through the same bulk-inserter write path. That single-writer design is what keeps the SQLite default viable for individual use.

Honest Limits

A few caveats, all checkable in the repository:

• Storage scale. The default engine is SQLite, with PostgreSQL supported for multi-user deployments (both engine factories live in src/phoenix/db/engines.py). The repo ships Docker Compose, Helm charts, and cloud deployment templates for the Postgres path. There is no distributed collector story in-process: it is one server, optionally fronted by an OTel collector.
• The UI is out of scope here. The React frontend lives in js/app (TypeScript) and is a substantial codebase on its own; this tour covered the Python side that serves it.
• Package churn. Phoenix has evolved quickly — MIGRATION.md at the repo root documents breaking changes, and older tutorials using px.Client() predate the newer phoenix.client.Client. Parts of phoenix-evals carry deprecation warnings as templating APIs are consolidated. Pin versions and check the changelog before upgrading.
• Evals are LLM-graded, not ground truth. The eval package’s scores come from whatever model you give it; treat them as measurements that need their own spot-checking, not as verdicts.
• Self-hosting vs cloud. Everything described here runs locally and for free under the project’s license, but Phoenix (the product) also has a hosted cloud offering with additional features — the open-source repo is the self-hosted core.

For context on the wider landscape, Langfuse and Opik are the two other open-source LLM observability servers we have toured; Phoenix’s distinguishing bet is full OpenTelemetry alignment end to end — the same protobuf OTLP service your APM already speaks, applied to LLM traces.

Enjoyed this post? Never miss out on future posts by following us