The Post-Training Assembly Line - Inside huggingface/trl
Pretraining teaches a model to predict the next token; post-training teaches it to be useful, honest, and steerable. That second stage — supervised fine-tuning, preference optimization, reinforcement learning from feedback — is where open models close the gap to frontier...
How an LLM Writes Wikipedia Articles With Citations - Inside stanford-oval/storm
Asking a language model to write you an article usually gets you a confident essay with invented facts and no receipts. STORM, from Stanford’s Open Virtual Assistant Lab, attacks that problem head on: it is an LLM system that writes...
MCP Reference Servers: The Seven That Defined the Protocol's Manners - Inside...
Every protocol gets one repository that is less a product than a manners guide. For the Model Context Protocol, it is modelcontextprotocol/servers: seven small servers that thousands of client implementations were implicitly trained on. A filesystem server whose security layer...
OpenAI Agents SDK: The Run Loop Beneath the Handoffs - Inside openai/openai-a...
Most agent frameworks sell you an abstraction and hide the loop. This one does the opposite: the abstraction is a plain Python dataclass, and the loop is laid out in a dedicated internals package where every hard decision gets its...
MarkItDown: Twenty-Five Converters Behind One MCP Tool - Inside microsoft/mar...
Ask an LLM to read a PDF and it cannot; ask it to read markdown and it can. That one-sentence gap is why microsoft/markitdown became infrastructure: a small MIT-licensed library that turns documents into markdown, used everywhere from RAG ingestion...
Your AI Second Brain, Self-Hosted - Inside khoj-ai/khoj
Everyone accumulates a private mountain of notes, PDFs, and half-remembered web pages. Khoj is the AGPL-3.0 personal AI that turns that mountain into something you can actually talk to: a self-hostable second brain that ingests your markdown, PDFs, org-mode files,...
Turn Your Desk Into an AI Cluster - Inside exo-explore/exo
Every Mac on your desk is a supercomputer that spends most of its life idle. exo-explore/exo is the open-source project that bands those machines together into one logical AI cluster, so a model too large for any single device can...
The Node Graph That Runs Any Diffusion Model - Inside comfyanonymous/ComfyUI
If you have generated an AI image in the last two years, odds are ComfyUI touched it. ComfyUI is the GPL-3.0 node-graph engine that has become the de facto way to run diffusion models locally: you wire boxes like Load...
Claude Code: The Open Surface of a Closed Binary - Inside anthropics/claude-code
Claude Code is the most-discussed terminal agent in software, and almost none of its code is public. The binary you install is proprietary, licensed under commercial terms, and built somewhere you cannot read. So why does the repository deserve a...
Google ADK: Workflow Agents That Never Call the Model - Inside google/adk-python
Every agent framework eventually confronts the same problem: when you wire five agents together, the wiring itself becomes code that a language model can get wrong. Google’s Agent Development Kit answers with a pattern no other major framework ships as...
whisper.cpp: Speech Recognition on Anything - Inside ggml-org/whisper.cpp
Every app that listens — the meeting transcriber, the voice assistant, the subtitle generator on your phone — needs speech-to-text that runs without a datacenter, and for most of the industry the answer is this repository. whisper.cpp took OpenAI’s Whisper...
Weaviate: The Vector Database Built Like a Database - Inside weaviate/weaviate
Most vector databases grew out of an ANN index and added a database around it. Weaviate grew out of a database and added the vectors in. The result shows everywhere in the source: writes flow through a write-ahead-logged LSM tree...
Transformers: The Library That Standardized AI - Inside huggingface/transformers
Before 2018, using five model architectures meant reading five papers and porting five codebases; after transformers, it meant changing one string. The library’s contract is deceptively simple — AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-1B-Instruct") returns a ready model, AutoTokenizer handles the text, and model.generate() streams...
TensorRT-LLM: Two Engines, One API - Inside NVIDIA/TensorRT-LLM
Serving a large model on a GPU is a different sport from running one on a laptop: the difference between a demo and a production endpoint is measured in tokens per second per dollar, and every millisecond of kernel launch...
SWE-agent: The Agent That Fixes Real GitHub Issues - Inside SWE-agent/SWE-agent
In 2023 the founders of this project asked a question that sounds naive and turned out to be profound: if you put a language model inside a terminal, can it fix a real GitHub issue? The answer, published with the...
RAGFlow: Deep Document Understanding, Now in Go - Inside infiniflow/ragflow
Most retrieval-augmented generation projects treat documents as text to be split; RAGFlow was built on the conviction that the hard part happens before the first embedding call. Scanned PDFs, multi-column papers, tables spanning pages, contracts with stamps — a naive...
OpenManus: An Open General Agent, Built in Hours - Inside FoundationAgents/Op...
When the closed-source Manus agent went viral behind an invite-code wall, five researchers from the MetaGPT team responded the fastest way they knew how: they built an open alternative and shipped the prototype in three hours. OpenManus — MIT-licensed, Python...
Gradio: The Python Function That Became a Web App - Inside gradio-app/gradio
Every machine learning demo you have clicked in the last five years — on Hugging Face Spaces, in a paper’s supplementary page, inside a model card — was almost certainly built with this library. The promise is absurd and delivered...
Fish Speech: A Language Model That Speaks in Audio Tokens - Inside fishaudio/...
The newest generation of text-to-speech systems stopped pretending to be signal processing and started being language models, and fish-speech is one of the clearest open examples of that shift. Its flagship model, Fish Audio S2, is a 4-billion-parameter transformer trained...
Coqui TTS: The Voice Zoo That Refused to Die - Inside idiap/coqui-ai-TTS
When the original Coqui company shut down in 2024, it left behind something unusual: a full zoo of text-to-speech architectures — fourteen acoustic models, a shelf of vocoders, speaker encoders, voice conversion — with pretrained checkpoints for over a thousand...
Whisper: One Model for Every Voice - Inside openai/whisper
Speech recognition used to mean a stack of specialist components — a voice activity detector, an acoustic model, a pronunciation lexicon, a language model, each with its own failure modes — and Whisper’s central idea is that a single sequence-to-sequence...
Streamlit: The Script That Became an App - Inside streamlit/streamlit
Most web frameworks ask you to learn HTTP. Streamlit asks you to write a Python script. x = st.slider("Select a value") followed by st.write(x, "squared is", x * x) is a complete interactive web app — run streamlit run streamlit_app.py...
Skyvern: LLM Agents That Drive the Browser - Inside skyvern-ai/skyvern
Every Selenium script eventually dies the same death: the website redesigns one button, the XPath goes stale, and the automation that ran for months starts failing on a Tuesday. Skyvern attacks that fragility at the root — instead of hand-written...
Pydantic AI: Typed Agents for Every Model - Inside pydantic/pydantic-ai
Type safety is the oldest promise in the pydantic ecosystem, and Pydantic AI — subtitled simply “How Python does AI” — is that promise carried into the agent era. The README’s pitch is that an agent is a normal Python...