If you have shipped anything with an LLM in it, you know the feeling: the demo worked, and then production happened. Prompts drift, models get swapped, retrieval silently degrades, and nobody can say which call returned what. Opik is Comet’s answer to that problem — an open-source platform for logging, searching, and evaluating LLM traces, with a self-hostable backend and a Python SDK that is genuinely pleasant to read.

What makes Opik interesting as a codebase is not any single clever algorithm. It is the split between two worlds that have very different performance budgets: the world of your application code, which wants opik.track to return in microseconds, and the world of an HTTP backend that wants well-formed batches of JSON. Almost every interesting design decision in the SDK lives in the gap between those two worlds — an in-process queue, background consumer threads, per-entity batchers, and a flush timer that decides when your trace finally leaves the building.

This post is a source tour of the Python SDK on the main branch (I pinned what I read to the sdks/python tree). We will walk the public surface first, then descend into the message-processing layer where the batching actually happens, and finish with an honest look at what the library does and does not give you. The two diagrams below are clickable — every node links straight to the file on GitHub.

Overview of the opik Python SDK: the public API surface funnels into the Opik client, which feeds a background streamer that batches telemetry to the REST backend.

Reading the overview from left to right:

• Everything starts at the package root, which re-exports the working vocabulary: Opik, L108 (the client class), track, Dataset, Trace, Span, evaluate, and configure are all imported in init.py, L25-L31.
• The track, L60 method on BaseTrackDecorator is the zero-config entry point: wrap any function and its inputs, outputs, timing, and exceptions become a span in a trace, with the default implementation in OpikTrackDecorator, L11.
• The Opik, L108 client is the hub. trace(), L277 and span(), L485 create observation handles, while get_dataset(), L1189 and create_experiment(), L1797 open the evaluation side of the house.
• A Trace, L16 is one end-to-end unit of work; calling end(), L38 finalizes it, and the same pattern holds for Span, L26 with its own end(), L52.
• Dataset, L498 holds versioned test items — insert(), L1024 accepts dicts, pandas DataFrames, or lists — and Experiment, L119 records scored runs against one, with log_experiment_scores(), L746 attaching metric results.
• Nothing here blocks your thread: both trace and span creation end up as messages on the Streamer, L20, whose put(), L52 is the hand-off point, and OpikApi, L55 is the generated REST client that eventually ships the data.

From Install to Your First Model

The README keeps the on-ramp short, and it genuinely is. First, install the SDK:

pip install opik

Then configure it — either interactively with opik configure, which prompts for your Comet API key and workspace (or a self-hosted server address), or programmatically:

import opik

opik.configure(
    api_key="YOUR_API_KEY",
    workspace="YOUR_WORKSPACE",
    project_name="my-first-project",  # optional default project
)

For a local stack, opik.configure(use_local=True) points the SDK at your own instance instead of Comet’s cloud. Under the hood, configuration lands in OpikConfig, L82, a pydantic-settings object persisted to a config file; its api_key field at L127 has a neat trick — Comet keys embed the base URL, and parse_api_key(), L73 extracts it so a single pasted key can configure everything.

Now the tracing part. The decorator is the idiomatic path:

import opik

@opik.track
def my_llm_function(user_question: str) -> str:
    # your LLM call or business logic here
    return "a helpful answer"

my_llm_function("Why is the sky blue?")

That single decorator creates a trace, nests a span for the function, captures the arguments and return value, and logs exceptions if the function blows up. If you need to flush before a process exits (useful in short-lived scripts and Lambda functions), call opik.flush_tracker() — the helper at L90 delegates to the client’s flush(), L2097. There is also a runtime kill-switch: opik.set_tracing_active(False) disables tracing globally without code redeployment, which the README recommends for high-throughput or sampled scenarios.

The Client as a Facade

The Opik class reads like a well-organized table of contents. Tracing methods (trace, span, log_traces_feedback_scores at L862) write data; query methods (search_traces(), L2184) read it back with a filter language; and the evaluation methods hand you dataset and experiment objects. When the SDK needs synchronous request/response calls — fetching a trace by ID, checking workspace names — it goes through the rest_client property at L167, but writes never do; they queue.

One piece worth singling out is the connection layer. ConnectionResourceManager, L278 treats everything expensive — the httpx pool, the REST client, the message-processing chain, the streamer and its threads — as properties of a connection identity (url, workspace, api_key) rather than of a client instance. The SharedConnectionResourcesBundle, L47 holds those objects, and each Opik handle gets a ref-counted Lease, L232. Create ten clients against the same workspace and you get one pool of threads, not ten. That is a small thing that saves real memory in notebook-heavy workflows.

Inside the Message Processing Machinery

The detail view: message flow from decorator and client through the streamer, preprocessing, batching, and out to the REST transport, plus evaluation metrics and provider integrations.

This is the part of the codebase I would make every observability-SDK author read. When you call client.trace(...), the client does not make an HTTP request — it calls self._streamer.put(...) with a CreateTraceMessage, L86, one of several message types derived from BaseMessage, L49. From there the flow is fully asynchronous to your code.

A background QueueConsumer, L17 thread drains the queue into a chain of processors. The first stage is MessagePreprocessor, L7, an abstract step that converts raw messages into REST-shaped payloads; its concrete sibling BatchingPreprocessor, L8 routes those payloads into a BatchManager, L11. The manager keeps one BaseBatcher, L11 per entity type — traces, spans, feedback scores — so a burst of a thousand spans becomes a handful of well-sized API calls instead of a thousand tiny ones. A FlushingThread, L9 wakes up on a fixed interval and drains whatever has accumulated, so low-traffic apps do not hold data hostage waiting for a size threshold.

Two consequences of this design deserve mention. First, failure is asynchronous too: if the backend rejects a batch, the error surfaces in flush reporting rather than in your application call, which is the right trade for telemetry. Second, ordering is only guaranteed per entity, not globally — the queue is a throughput device, not a transaction log. If you have used other LLM observability tools, this is a familiar shape; LangForge-style exporters in /langfuse/ make the same trade, and it is interesting to compare how each SDK surfaces upload failures to the developer.

Evaluation and the Metric Zoo

Tracing tells you what happened; evaluation tells you whether it was any good. The entry point is evaluate(), L138, which takes a dataset, a task callable, and a list of scoring metrics, runs the task across dataset items, and logs everything as an Experiment you can compare in the UI. Sampling is built in — a dataset_sampler argument lets you evaluate on a subset without materializing the whole set.

Metrics implement BaseMetric, L34, whose score(), L85 returns a ScoreResult. The zoo splits into two families. Heuristic metrics are deterministic and free: SentenceBLEU at L110, plus ROUGE, METEOR, Levenshtein ratio, regex match, JSON validity, and sentiment among others under metrics/heuristics/. LLM judges use a model to grade a model: Hallucination, L15 checks groundedness against context, and there are judges for factuality, answer relevance, context precision and recall, moderation, and G-Eval-style custom criteria. The judge family shares template/parser/metric triplets, which keeps adding a new judge to a three-file recipe.

If you are assembling an eval harness for a paper or a regression suite, Opik’s metric implementations pair naturally with standalone harnesses like /lm-eval-harness/: the latter covers academic benchmarks, while Opik covers your actual production prompts. For multi-step agent applications, the same traces that power debugging feed the trajectory-level judges, and callback-based integrations (LangChain, LlamaIndex — see /llama-index/ for context on that ecosystem) mean agent frameworks can log without code changes. Provider coverage is broad: track_openai at L24 monkey-patches the OpenAI client via OpenaiChatCompletionsTrackDecorator, L32, and the integrations/ package ships similar trackers for Anthropic, Gemini, LiteLLM, Bedrock, and more.

Honest Limits

A few things worth knowing before you bet a project on this:

• The SDK is one piece of a three-piece system. sdks/python is only the client; the backend lives under apps/ in the same monorepo, and deployment/ carries the docker-compose stack for self-hosting. Reading the SDK alone tells you nothing about backend scaling characteristics — evaluate the whole stack if you plan to self-host.
• LLM judges are opinionated prompts, not ground truth. The hallucination and factuality judges ship fixed templates; their scores are only as good as the judge model you point them at, and the docs themselves steer you toward reviewing results rather than trusting them blindly. For benchmark-grade measurement, deterministic metrics or an external harness remain the safer instrument.
• The REST client is generated. The rest_api/ package is Fern-generated — thousands of mechanical lines. That is good for correctness and bad for grep-driven reading; the hand-written logic you want to study lives in api_objects/ and message_processing/.
• Async support has caveats. The decorator handles sync and async functions, and an AsyncOpikApi exists at L212, but the message queue is threaded rather than asyncio-native; extremely high-concurrency asyncio services should measure the queue’s behavior under load before committing.
• Benchmarks are yours to run. The repo publishes no latency or overhead numbers for the tracing hot path. The design (fire-and-forget put(), background batching) is sound, but trust your own measurement on your own traffic.

Overall, Opik is one of the more thoughtfully layered SDKs in the LLM-tooling space: a clean facade, a genuinely decoupled write path, and an evaluation layer that grew real depth without losing its toy-like ease of use. If observability is the problem you are shopping for, it belongs on a shortlist of two or three — and the source is readable enough that the shortlisting can start with a quiet afternoon in sdks/python.

Enjoyed this post? Never miss out on future posts by following us