Zipkin Under the Hood: A Source Tour of the Tracing System That Started It All
Every AI stack you operate today — an API gateway routing to a model server, an agent orchestrating tool calls, a RAG chain fanning out to an embedding service and a vector database — creates the same debugging problem Zipkin...
Vector: The Rust Router That Moves Your LLM's Logs, Metrics, and Traces
Ask anyone running a large language model service what their telemetry situation looks like and watch them wince. A single serving cluster can emit millions of structured log lines per day — prompts, completions, tool calls, guardrail verdicts — plus...
Grafana Tempo Under the Hood: Kafka, Block Builders and a TraceQL Engine That...
An AI application in 2026 is not one program. It is a chain: a router picks a model, a retriever fetches context from a vector store, an inference server runs the prompt, tool calls fan out to external APIs, and...
Telegraf Under the Hood: The Plugin-Driven Go Agent That Moves Your Metrics
Training and serving AI models is a telemetry problem as much as a compute problem. A training run is only as trustworthy as the numbers you can see while it happens: GPU utilization and memory pressure from every node, host...
Inside Apache SkyWalking: A Source Tour of the OAP Server That Watches AI Inf...
Modern AI infrastructure is a distributed-systems problem wearing a lab coat. One “ask the model” request can cross a gateway, a queue, a vector database, and a GPU inference service — and when latency spikes, someone must find which hop...
Pyroscope: A Source Tour of Grafana's Continuous Profiling Backend
Anyone who has served a large language model in production knows the feeling: the GPU sits at 97% utilization on your dashboard, yet tokens per second quietly sag during the evening peak. Dashboards built on metrics from Prometheus tell you...
Grafana Mimir Under the Hood: HA Dedup, Tenant TSDBs and Sharded Block Reads ...
Every serious ML platform eventually hits the same wall: metrics. A modern inference fleet — hundreds of replicas across a dozen models, each exporting request latency, GPU utilization, KV-cache pressure, token throughput and queue depth — outgrows a single Prometheus...
Logstash Under the Hood: The JRuby + JVM Engine That Turns Raw Logs into Sear...
Logstash is Elastic’s server-side data processing engine. It collects events from a multitude of sources at once, parses and transforms them with a small DSL that reads like configuration, and then ships each event to a stash of your choice....
Fluent Bit Under the Hood: The C Engine That Ships Logs From Your GPU Fleet
Training and serving large models is now a telemetry problem as much as a compute problem. Every GPU node writes journald and container logs, every inference server emits latency and throughput metrics, every fine-tuning run streams progress lines that someone...
cAdvisor Source Tour: How Google's Container Advisor Turns cgroups into Metrics
Every serious GPU cluster conversation eventually hits the same wall: somebody asks what a specific container was actually doing at 3 a.m. — how much CPU it burned, whether it was network-starved during a checkpoint upload, or why memory pressure...
Trino Under the Hood: A Source Tour of the Distributed SQL Engine Behind Fede...
Modern AI systems are data-starved in a very specific way: the features a model needs rarely live in one place. Click events stream into a log broker, sensor telemetry lands in a time-series store, historical training rows sit on object...
Inside RisingWave: A Source Tour of the Rust Streaming Database Powering Alwa...
If you build AI systems, you eventually hit the same wall: your model was trained on a snapshot, but the world kept moving. Feature vectors go stale, dashboards lag, retrieval indexes drift. RisingWave attacks that problem from the database side....
Inside Redpanda: A C++ Source Tour of the Kafka-Compatible Streaming Engine
Modern ML systems live and die on event streams. Training features arrive as sequences of user actions, model predictions come back as telemetry, and drift detection watches both. The infrastructure that carries those events has traditionally been the JVM: Apache...
Inside QuestDB: A Java Source Tour of the Column Store Built for Real-Time Te...
Every serious ML deployment is a telemetry factory. Training clusters emit GPU utilization, temperature, and job metrics; inference servers log request latencies and prediction distributions; drift monitors compare live feature windows against training snapshots. All of that data is time-stamped,...
Inside Apache Pulsar: The Stateless Broker, BookKeeper Ledgers, and the Manag...
Modern ML systems are, underneath everything else, nervous systems. Feature vectors flow from ingestion jobs to training clusters, prediction requests emit traces, drift monitors and shadow scorers consume the same tensors in parallel, and retraining jobs wake up when enough...
Inside Apache Pinot: A Java Source Tour of Segments, Broker Routing and the R...
Recommendation and personalization models live or die by how fresh their inputs are. A ranking model that cannot see the last five minutes of user behavior is quietly optimizing the past, and an LLM ops dashboard that answers in seconds...
Inside Materialize: A Source Tour of the Rust Streaming Database That Compile...
If you have ever wired an ML feature store, you know the pain: features are computed offline, batched into a table, and served stale by the time your model reads them. Materialize attacks that problem from an unusual direction —...
Inside Apache Druid: A Java Source Tour of Streaming Ingestion, Segments, and...
When you train a model, the easy part is the model. The hard part is everything that happens after it starts serving: millions of prediction events, feature distributions drifting by the hour, error rates that spike in ways your offline...
Bytewax Under the Hood: A Source Tour of Python Stream Processing on a Rust T...
Every serious ML system eventually hits the same wall: the features that feed your model have to be computed continuously, on live data, with state that survives crashes. Batch scoring is not enough once tokens, clicks, sensor readings, or order...
Arroyo Under the Hood: A Source Tour of the Rust SQL Streaming Engine
Modern machine learning is increasingly a streaming problem. Models that score fraud, rank feeds, or trigger alerts do not want a nightly table dump — they want fresh features computed continuously over the same event streams that drive the product....
Inside runc: A Source Tour of the OCI Runtime That Boots Every AI Container
Every GPU inference workload you have ever run bottoms out in the same few thousand lines of Go. Whether a vLLM server is scheduled by Kubernetes, pulled by CRI-O, launched through containerd, or started by the Docker engine, the final...
Prometheus Operator Source Tour: How One Go Binary Turns CRDs Into StatefulSets
Run a fleet of LLM servers — vLLM, SGLang, Triton — and the first operational question after “does it answer?” is “what is it doing right now?”. GPU utilization, KV-cache pressure, request queue depth, tokens per second: all of that...
Inside Moby: A Source Tour of the Docker Engine Daemon
Every modern AI workload ships in a container. You pull a CUDA base image, layer on PyTorch or a quantized LLM runtime, and hand it to a scheduler — and the binary quietly doing the heavy lifting behind docker run...
Kustomize Source Tour: How Template-Free YAML Customization Actually Builds
Serving a large language model on Kubernetes rarely means writing one YAML file. A dev cluster runs one GPU replica with lenient probes; staging and production differ in replicas and limits — the resources are mostly identical while a handful...