Monkey-Patch Your Way to 2x Training - Inside unslothai/unsloth
Most fine-tuning libraries you meet are wrappers: they take Hugging Face Trainer, add a config file, and hope the stock kernels are fast enough. Unsloth is the opposite bet. Instead of orchestrating around transformers, it reaches inside them, replacing attention,...
The 52K Recipe That Started a Thousand Fine-Tunes - Inside tatsu-lab/stanford...
Some repositories matter because of their engineering, and some because of what they proved. stanford_alpaca, from five Stanford grad students advised by Tatsunori Hashimoto, is the second kind: fourteen files that showed a 7B LLaMA model fine-tuned on 52K machine-generated...
Ray, vLLM, and One Controller - Inside OpenRLHF/OpenRLHF
Reinforcement learning from human feedback is the most distributed-shaped problem in LLM training: you need a policy being trained, a reference model frozen in time, a reward model scoring outputs, a critic estimating values, and a generation engine producing rollouts...
One Framework from Tuning to Deployment - Inside modelscope/ms-swift
ModelScope’s ms-swift has a scope that makes most fine-tuning frameworks look modest: supervised tuning, preference optimization, GRPO-style reinforcement learning, knowledge distillation, multimodal training, Megatron-backed large runs, quantized export, and deployment, all in one package. The current development line, version 4.7.0.dev0...
Five Calls from Data to Fine-Tuned Model - Inside OptimalScale/LMFlow
Every fine-tuning toolbox has a philosophy. axolotl bets on YAML, LLaMA-Factory on a hundred CLI flags, unsloth on monkey-patching, and ms-swift on mixin-composed trainers. LMFlow, from the Statistics and Machine Learning Research Group at HKUST, bets on something subtler: a...
The Ruler Every LLM Paper Trusts - Inside EleutherAI/lm-evaluation-harness
Every benchmark number you have seen attached to an open model - MMLU, HellaSwag, GSM8K, ARC, TruthfulQA - has a decent chance of having been produced by this repository. EleutherAI’s Language Model Evaluation Harness is the framework behind the Open...
100+ Models, One Command - Inside hiyouga/LLaMA-Factory
Where axolotl bets on one exhaustive YAML schema, LLaMA-Factory bets on breadth: one CLI that fine-tunes over a hundred model families, one data loader that speaks every open dataset format, and one stage dispatcher that walks from continuous pretraining through...
Your Language Model Is Secretly a Reward Model - Inside eric-mitchell/direct-...
In May 2023 a Stanford team - Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn - posted “Direct Preference Optimization: Your Language Model Is Secretly a Reward Model” (arXiv 2305.18290), and the title carried...
One YAML, Every Fine-Tune - Inside axolotl-ai-cloud/axolotl
Fine-tuning an open model used to mean writing a training script: dataset glue, tokenizer surgery, trainer arguments, sharding flags, and a prayer that the checkpoint resumes. Axolotl replaces the script with a single YAML file and a codebase whose whole...
The Recipe Book Behind Zephyr and SmolLM3 - Inside huggingface/alignment-hand...
Some repositories impress with volume, and some with restraint. The Alignment Handbook, from Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Shengyi Huang, Kashif Rasul, Alvaro Bartolome, Carlos M. Patino, Alexander M. Rush, and Thomas Wolf at Hugging Face, is...
The Machine Learning Compiler That Speaks Python - Inside apache/tvm
Most deep learning systems ship as fixed inference engines: you bring a model, the engine runs it, and the optimizations are whatever the vendor froze into the release. Apache TVM is the opposite bet. Its README calls it an Open...
A Python Compiler for the GPU - Inside openai/triton
Writing a fast GPU kernel by hand means warp shuffles, shared memory swizzling, and a week in the profiler, which is why most deep learning teams would rather not. Triton, started by Philippe Tillet and described in the 2019 MAPL...
FP8 Without Rewriting Your Stack - Inside NVIDIA/TransformerEngine
Training large models in FP8 promises roughly double the throughput and half the memory traffic of BF16, and the 2022 paper FP8 Formats for Deep Learning is the reason the industry believes it. The catch has always been numerics: eight-bit...
Post-Training, the PyTorch Way - Inside pytorch/torchtune
Fine-tuning an LLM used to mean choosing between a framework that hid PyTorch from you and a notebook that hid correctness from you. Torchtune, released by Meta’s PyTorch team in April 2024 under the BSD 3-Clause license, took a third...
The Clean-Room Trainer That Scales Llama to 512 GPUs - Inside pytorch/torchtitan
Every serious lab eventually writes the same three files: a model, a sharding plan, and a training loop that quietly accretes hundreds of special cases for every parallelism combination. The PyTorch team looked at that pattern and decided to publish...
One Framework, Four Kinds of Retrieval - Inside UKPLab/sentence-transformers
In 2019, two researchers at the UKP Lab in TU Darmstadt published Sentence-BERT and open-sourced the code that made transformer embeddings practical for sentences. Six years later that codebase, UKPLab/sentence-transformers under the Apache-2.0 license and now maintained within Hugging Face,...
Your Science, Their Engineering - Inside Lightning-AI/pytorch-lightning
Every PyTorch project eventually grows the same skeleton: a training function that moves batches to the device, zeroes gradients, calls backward, clips gradients, steps the optimizer, logs metrics, validates at intervals, saves checkpoints, and then gets rewritten when someone says...
Train 0.19 Percent of the Model - Inside huggingface/peft
Full fine-tuning is the luxury the rest of us cannot afford. A 12B-parameter model in fp16 needs around 47GB of GPU memory before the optimizer even gets its share, which is why so many promising fine-tuning projects died the moment...
One API, Twenty-Five Accelerators - Inside microsoft/onnxruntime
Somewhere behind your bank’s fraud model, your phone’s photo search, a Windows Copilot feature, and half the vector databases on the market, the same C library is running the inference: ONNX Runtime, Microsoft’s cross-platform inference and training accelerator, MIT-licensed, at...
The Scan That Outgrew the Paper - Inside state-spaces/mamba
Every sequence model you have served pays attention’s tax: quadratic cost in sequence length, a KV cache that grows with every token, and memory bandwidth that becomes the bottleneck long before the FLOPs do. Mamba, from Albert Gu and Tri...
The Compressor That Feeds vLLM - Inside vllm-project/llm-compressor
Every quantized model you load into vLLM was made by something, and for most of the official checkpoints that something is llmcompressor - the vLLM project’s own optimization library. It takes a Hugging Face checkpoint plus a recipe, runs a...
20 LLMs, One Readable Codebase - Inside Lightning-AI/litgpt
Most LLM libraries hand you a wrapper around someone else’s model class and call it a day. LitGPT took the opposite bet: reimplement every popular open model from scratch - Llama 3, Gemma 2/3, Phi 4, Qwen2.5 and Qwen3, Mistral,...
The Kernel That Rewrote Attention - Inside Dao-AILab/flash-attention
Every transformer trained since 2022 owes part of its existence to a paper proof that attention could be computed without ever writing the full sequence-by-sequence matrix to HBM. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness by Tri Dao, Daniel...
The Library That Indexes Billions - Inside facebookresearch/faiss
Every recommendation feed, duplicate detector, and retrieval-augmented answer starts with the same question: of these billion vectors, which few are closest to mine? Faiss, started by Herve Jegou at Meta’s Fundamental AI Research group and maintained by an author list...