Trino Under the Hood: A Source Tour of the Distributed SQL Engine Behind Fede...
Modern AI systems are data-starved in a very specific way: the features a model needs rarely live in one place. Click events stream into a log broker, sensor telemetry lands in a time-series store, historical training rows sit on object...
Inside RisingWave: A Source Tour of the Rust Streaming Database Powering Alwa...
If you build AI systems, you eventually hit the same wall: your model was trained on a snapshot, but the world kept moving. Feature vectors go stale, dashboards lag, retrieval indexes drift. RisingWave attacks that problem from the database side....
Inside Redpanda: A C++ Source Tour of the Kafka-Compatible Streaming Engine
Modern ML systems live and die on event streams. Training features arrive as sequences of user actions, model predictions come back as telemetry, and drift detection watches both. The infrastructure that carries those events has traditionally been the JVM: Apache...
Inside QuestDB: A Java Source Tour of the Column Store Built for Real-Time Te...
Every serious ML deployment is a telemetry factory. Training clusters emit GPU utilization, temperature, and job metrics; inference servers log request latencies and prediction distributions; drift monitors compare live feature windows against training snapshots. All of that data is time-stamped,...
Inside Apache Pulsar: The Stateless Broker, BookKeeper Ledgers, and the Manag...
Modern ML systems are, underneath everything else, nervous systems. Feature vectors flow from ingestion jobs to training clusters, prediction requests emit traces, drift monitors and shadow scorers consume the same tensors in parallel, and retraining jobs wake up when enough...
Inside Apache Pinot: A Java Source Tour of Segments, Broker Routing and the R...
Recommendation and personalization models live or die by how fresh their inputs are. A ranking model that cannot see the last five minutes of user behavior is quietly optimizing the past, and an LLM ops dashboard that answers in seconds...
Inside Materialize: A Source Tour of the Rust Streaming Database That Compile...
If you have ever wired an ML feature store, you know the pain: features are computed offline, batched into a table, and served stale by the time your model reads them. Materialize attacks that problem from an unusual direction —...
Inside Apache Druid: A Java Source Tour of Streaming Ingestion, Segments, and...
When you train a model, the easy part is the model. The hard part is everything that happens after it starts serving: millions of prediction events, feature distributions drifting by the hour, error rates that spike in ways your offline...
Bytewax Under the Hood: A Source Tour of Python Stream Processing on a Rust T...
Every serious ML system eventually hits the same wall: the features that feed your model have to be computed continuously, on live data, with state that survives crashes. Batch scoring is not enough once tokens, clicks, sensor readings, or order...
Arroyo Under the Hood: A Source Tour of the Rust SQL Streaming Engine
Modern machine learning is increasingly a streaming problem. Models that score fraud, rank feeds, or trigger alerts do not want a nightly table dump — they want fresh features computed continuously over the same event streams that drive the product....
Inside runc: A Source Tour of the OCI Runtime That Boots Every AI Container
Every GPU inference workload you have ever run bottoms out in the same few thousand lines of Go. Whether a vLLM server is scheduled by Kubernetes, pulled by CRI-O, launched through containerd, or started by the Docker engine, the final...
Prometheus Operator Source Tour: How One Go Binary Turns CRDs Into StatefulSets
Run a fleet of LLM servers — vLLM, SGLang, Triton — and the first operational question after “does it answer?” is “what is it doing right now?”. GPU utilization, KV-cache pressure, request queue depth, tokens per second: all of that...
Inside Moby: A Source Tour of the Docker Engine Daemon
Every modern AI workload ships in a container. You pull a CUDA base image, layer on PyTorch or a quantized LLM runtime, and hand it to a scheduler — and the binary quietly doing the heavy lifting behind docker run...
Kustomize Source Tour: How Template-Free YAML Customization Actually Builds
Serving a large language model on Kubernetes rarely means writing one YAML file. A dev cluster runs one GPU replica with lenient probes; staging and production differ in replicas and limits — the resources are mostly identical while a handful...
Kubernetes From the Source: How kube-apiserver and kubelet Are Wired in Go
If you serve an LLM in production today, the odds are good that the model runs in a Pod somewhere on a Kubernetes cluster. The scheduler picked a GPU node, the node agent pulled the image, started the container, and...
Helm Source Tour: How the Kubernetes Package Manager Renders Charts and Store...
Serving an LLM today is rarely one container. A realistic inference stack means a model server such as vLLM or Triton, a GPU device plugin, autoscaling rules, service endpoints, and a pile of config that all have to land in...
Flux from the Source: How the GitOps Toolkit Bootstraps, Syncs and Heals a Cl...
Serving a large model in production is an operations problem as much as a modeling problem. A vLLM or Triton inference server is a Deployment, its weights are a PersistentVolume or an object store, its autoscaling is an HPA, and...
CRI-O Source Tour: How Kubernetes' Leanest Runtime Speaks CRI in Go
When you schedule a large language model pod onto a Kubernetes node, someone has to actually fork the container process, mount the model weights, wire the network namespace, and keep the thing supervised after the scheduler moves on. On many...
containerd Source Tour: The Daemon, the Shim Protocol, and the Snapshot Layer
Every AI inference service you deploy — a Triton server, a vLLM worker, or a plain FastAPI wrapper around a fine-tuned model — exists as a container on some Linux host. On most modern clusters, and on plenty of single-box...
Inside Argo CD's Source: How the GitOps Controller Turns Git Commits into Clu...
GitOps is the operating model where Git is the source of truth for what runs in your cluster, and a controller’s only job is to close the loop between the repository and the API server. For AI/ML infrastructure this matters...
Traefik in the Source: How a Go Reverse Proxy Learns Your Fleet and Rewires I...
If you run more than one model server — a vLLM box here, a FastAPI inference container there, a GPU node pool over in Kubernetes — then the most performance-critical piece of software you own is not the model server....
Linkerd: Reading the Rust and Go Source of the Lightest Service Mesh
Modern AI inference is quietly becoming a distributed-systems problem. A production stack today is rarely one container: there is a gateway, a tokenizer, a prefill-versus-decode split across GPU pods, a vector database, maybe a reranker, and a scrape target for...
Inside Kong: The Lua Gateway That Wraps Nginx into an AI Traffic Machine
If you have ever served a machine-learning model to real users, you have run into the same trio of problems: you need to throttle who can call your inference endpoint, authenticate those calls, and decide which model version a request...
Istio: Reading the Go Source of the Service Mesh Control Plane
Serving large language models in production is rarely one process talking to one GPU. A realistic AI platform is a swarm of Kubernetes workloads: a tokenizer service, one or more vLLM or Triton replicas, an embedding server, a vector database,...