One Command to a Serving Cluster - Inside xorbitsai/inference
Most inference servers begin and end with “run this model on this GPU.” Xinference, from the team behind the Xorbits data-processing framework, starts from a different premise: model serving is a cluster problem, and it should feel as routine as...
A Deep Learning Stack That Fits in Your Head - Inside tinygrad/tinygrad
Somewhere between PyTorch’s twenty million lines and micrograd’s two hundred lies the sweet spot for understanding how deep learning frameworks actually work. tinygrad, maintained by tiny corp and released under the MIT license, plants its flag exactly there: an end-to-end...
The Smallest Load-Bearing Code in AI - Inside openai/tiktoken
Few repositories carry more traffic per line of code than openai/tiktoken. It is the fast byte pair encoding tokenizer behind OpenAI’s models — four lines of Python that virtually every cost estimate, context-window check, and prompt-length guard in the ecosystem...
The Server That Powered Hugging Chat - Inside huggingface/text-generation-inf...
Before every lab had its own serving stack, Hugging Face had to solve the problem for the whole ecosystem. Text Generation Inference, Apache-2.0-licensed and written in Rust with a Python GPU core, is the toolkit that powered Hugging Chat, the...
The App That Democratized AI Images - Inside AUTOMATIC1111/stable-diffusion-w...
Few open-source projects have shaped how ordinary people use AI models as much as AUTOMATIC1111’s stable-diffusion-webui. It turned Stable Diffusion from a Python notebook exercise into a one-click desktop application: clone, run a script, open a browser tab, and generate....
The Serving Engine Named After Its Cache - Inside sgl-project/sglang
Most inference engines are named after speed. SGLang is named after a data structure. The project from LMSYS grew out of RadixAttention, the idea that every prompt your users send is a prefix that has probably been seen before —...
The Machine-Written SDK - Inside openai/openai-python
Nearly every Python developer who has touched an AI product has typed the same two lines: import the openai package and instantiate a client. What few of them have considered is what those two lines actually invoke. The openai-python repository...
One Binary Between You and the Weights - Inside ollama/ollama
Ask a million developers what “running a model locally” looks like and most will describe one command: ollama run. The project by the same name is the software behind that moment, and its repository is a masterclass in hiding enormous...
The 300-Line GPT Classroom - Inside karpathy/nanoGPT
Most “minimal GPT” repositories are minimal the way a demo reel is minimal: the training script is a toy, and everything real is hidden in a framework you must first learn. karpathy/nanoGPT refuses that bargain. It is a rewrite of...
One Model, Every Device - Inside mlc-ai/mlc-llm
Most inference frameworks ask you to meet the model on the model’s terms: a datacenter GPU, a CUDA driver, a Python runtime, and a lot of VRAM. mlc-llm from the mlc-ai team flips that arrangement. It treats a large language...
The 13-File Transformer - Inside mistralai/mistral-inference
Production inference stacks have grown into sprawling systems, which makes Mistral AI’s own reference implementation almost contrarian. The mistral-inference repository — now archived, with the company pointing users at its API and vLLM-class servers — runs the full model family...
Two Engines, One Server - Inside InternLM/lmdeploy
LMDeploy from the InternLM team at Shanghai AI Laboratory occupies an interesting spot in the inference landscape. It ships two independent inference engines — a pure-PyTorch engine you can read and hack like any research codebase, and TurboMind, a C++...
A Neural Network With Nowhere to Hide - Inside karpathy/llm.c
Every deep learning framework makes the same bargain: you trade visibility for convenience, and after a few years you can ship models without ever seeing a training loop. llm.c, Andrej Karpathy’s follow-up to nanoGPT, tears up that bargain. It trains...
One Token That Unlocks the Image - Inside haotian-liu/LLaVA
Every multimodal assistant that chats about images owes something to this repository. LLaVA — Large Language and Vision Assistant — is the research codebase behind visual instruction tuning, the NeurIPS 2023 oral that showed a frozen CLIP encoder, a frozen...
The Executable That Is Also a Zip of Weights - Inside Mozilla-Ocho/llamafile
Software distribution is the quiet bottleneck of local AI. Weights live in one place, runtimes in another, and every OS wants a different installer. The llamafile answer is almost absurd in its simplicity: take llama.cpp, compile it with Cosmopolitan Libc...
Node Graphs for Professional Image Work - Inside invoke-ai/InvokeAI
While most Stable Diffusion interfaces grew from scripts into apps, InvokeAI took the opposite path: it was designed from the start as a professional creative tool, and the codebase shows it. The current repository is a full-stack application — a...
The Training Rig Behind Pythia - Inside EleutherAI/gpt-neox
Some of the most influential language models of the open-source era were trained by a volunteer collective with no vendor behind them. GPT-NeoX is EleutherAI’s library for training large-scale autoregressive language models on GPU clusters, built on NVIDIA’s Megatron design...
The Config-Driven Engine Behind SDXL - Inside Stability-AI/generative-models
Every image model has a lab notebook. For Stability AI’s flagship family, that notebook is generative-models, the repository where SDXL, SDXL-Turbo, Stable Video Diffusion, SV3D, and SV4D all live as research code. It is not a product and not a...
The Toolbox Behind Every Diffusion Model - Inside huggingface/diffusers
If you have generated an image with an open model in the last three years, the odds are high that the huggingface/diffusers library was somewhere underneath. It is the reference toolbox for diffusion models — image, video, and audio —...
The Engine That Sharded BLOOM - Inside microsoft/DeepSpeed
When Microsoft’s AI at Scale initiative committed to training language models measured in hundreds of billions of parameters, the binding constraint was not model design but memory arithmetic. A 176-billion-parameter model in mixed precision needs hundreds of gigabytes just to...
The Post-Training Assembly Line - Inside huggingface/trl
Pretraining teaches a model to predict the next token; post-training teaches it to be useful, honest, and steerable. That second stage — supervised fine-tuning, preference optimization, reinforcement learning from feedback — is where open models close the gap to frontier...
How an LLM Writes Wikipedia Articles With Citations - Inside stanford-oval/storm
Asking a language model to write you an article usually gets you a confident essay with invented facts and no receipts. STORM, from Stanford’s Open Virtual Assistant Lab, attacks that problem head on: it is an LLM system that writes...
MCP Reference Servers: The Seven That Defined the Protocol's Manners - Inside...
Every protocol gets one repository that is less a product than a manners guide. For the Model Context Protocol, it is modelcontextprotocol/servers: seven small servers that thousands of client implementations were implicitly trained on. A filesystem server whose security layer...
OpenAI Agents SDK: The Run Loop Beneath the Handoffs - Inside openai/openai-a...
Most agent frameworks sell you an abstraction and hide the loop. This one does the opposite: the abstraction is a plain Python dataclass, and the loop is laid out in a dedicated internals package where every hard decision gets its...