Every Mac on your desk is a supercomputer that spends most of its life idle. exo-explore/exo is the open-source project that bands those machines together into one logical AI cluster, so a model too large for any single device can run comfortably across all of them. It discovers peers with zero configuration, elects a coordinator, weighs each device’s memory and each link’s bandwidth, and then shards the model with tensor parallelism. The README reports up to 1.8x speedup on two devices and 3.2x on four, with day-0 RDMA over Thunderbolt 5 cutting inter-device latency by 99 percent. The project is Apache-2.0 licensed and maintained by exo labs, with a dashboard at localhost:52415 for watching the cluster breathe. The README’s own screenshots show four 512GB M3 Ultra Mac Studios jointly serving DeepSeek v3.1 and Kimi K2 Thinking, which is the kind of demonstration that used to require a rack in a datacenter.

What makes the codebase worth a tour is that it is a complete distributed systems education in miniature: a Rust discovery layer, an election protocol, a placement solver, a supervision tree, and four API dialects, all in a few thousand readable lines. The tree is pleasantly polyglot — Python for every process under src/exo, Rust under rust/networking for peer discovery and transport, a Swift macOS app under app, and a Node dashboard under dashboard — yet each part speaks to the others through one small event vocabulary. As always in this series, what follows is an educational tour of published source code.

The exo architecture at a glance: one API surface and dashboard on top, a discovery-and-election fabric underneath, an elected master computing placements, and workers executing tensor-parallel inference.

Reading the overview from left to right:

• A single command boots the world — the CLI entry starts the API, dashboard, and either master or worker roles on each machine.
• Rust multicast discovery — a UDP beacon shouts hello on the LAN so devices find each other with no manual configuration.
• The live topology graph — nodes and weighted links, updated in realtime as devices join, sleep, or move between networks.
• Master election — a heartbeat protocol with seniority and retry backoff picks one coordinator and survives its crash.
• The master controller — consumes events from the bus, updates cluster state, and issues commands back onto it.
• The placement engine — fits model shards onto devices using memory limits and link quality, choosing the best device cycles.
• The tensor parallel engine — rewrites MLX layers into distributed shards so big models run across small machines.
• The API server — one FastAPI app on port 52415 that accepts OpenAI, Claude, and Ollama requests and streams tokens back.

Why You Need This

The first reason is raw capability. A 671B-parameter model in 8-bit needs roughly 700 GB of unified memory, which no laptop holds, yet four Mac Studios hold it easily. The machinery that makes this real is the placement engine working with the tensor parallel engine: the master computes which shard lands on which device, and the worker rewrites each attention and MLP layer into a distributed version whose partial results are summed across the ring. Because exo accounts for link bandwidth when choosing device cycles, adding a faster path such as Thunderbolt 5 actually makes the cluster faster rather than merely bigger. That is the difference between a science project and a tool: the benchmark images in the repository were produced by an independent blogger who wired four machines over Thunderbolt and published the tokens-per-second numbers, which is exactly the kind of reproducibility open source should enable.

The second reason is that it feels like magic but is honestly engineered. You install nothing on the network, run nothing as administrator, and yet machines find each other. That is the Rust discovery module, which multicasts hello beacons and answers them, plus master election, which resolves the inevitable question of who is in charge. When a master goes silent for thirty seconds, a follower runs a new election with growing backoff, and the cluster reorganizes itself without a restart. Reading these two files teaches more about practical consensus and discovery than most tutorials. There are even practical touches for shared spaces: the EXO_ZENOH_NAMESPACE setting lets two clusters live on the same office network without ever seeing each other, and mismatched versions announce themselves politely instead of fighting over peers.

The third reason is that your existing tools keep working. Every client you already use targets some API dialect, and the API server mounts adapters for OpenAI Chat Completions, Claude Messages, OpenAI Responses, and Ollama, each translating requests into the cluster’s internal task types and streaming chunks back. Point a coding assistant or a chat frontend at localhost:52415 and the whole cluster answers as if it were one model server. The built-in dashboard at the same port shows live topology, memory, and power per device. The adapters are not thin shims either: tool calls, system prompts, and stream interruption are handled per dialect, and the test suite under src/exo/api/tests covers each contract separately, which tells you the compatibility promise is taken seriously rather than asserted.

The detail view: four API adapters feeding a shared kernel of events and tasks, an elected master running the placement solver, worker engines behind a supervision tree, and the download and discovery infrastructure underneath.

The detail diagram rewards a slow read. On the left, the four adapters in src/exo/api/adapters translate their dialects into typed tasks and publish them onto the zenoh-backed event router, which is the single communication spine every process shares. The master folds each event into state with a pure reducer, consults the cycle solver in placement_utils, and issues PlaceInstance and download commands. On the right, each worker reacts through its local planner, which queues shard downloads first and only then asks the supervisor to spawn a runner. The supervisor tracks acknowledgements, restarts failed runners, and drains cleanly when a new master takes over, which is why you can yank a machine from the cluster mid-request and watch the rest of it carry on. The runner loads the model through the tensor parallel engine, whose model-specific adapters handle DeepSeek, GLM, GPT-OSS, and Gemma variants, while the KV prefix cache makes repeated prompts cheap and batched generation keeps short requests from starving long ones.

Two quieter subsystems deserve attention. The download coordinator tracks a per-shard state machine with verification and cancellation, so a flaky network cannot corrupt weights or strand a half-loaded model. And disaggregated serving splits prefill from decode across machines over a typed wire protocol, an architecture pattern usually reserved for datacenter inference farms. Meanwhile the profilers under src/exo/utils/info_gatherer sample memory, power, and Thunderbolt link health — optionally through the macmon tool — and feed those measurements back into the topology graph, closing the loop that keeps placement decisions grounded in reality rather than in guesses.

From Install to a Running Cluster

On a Mac, the fastest path is the app: brew install --cask exo, launch it, and grant the network-profile prompt so RDMA over Thunderbolt can be enabled on macOS 26.2 or later. From source on macOS or Linux, install uv for Python dependencies, node for the dashboard build, and a stable Rust toolchain, then clone the repository, build the dashboard with npm, run uv sync --extra mlx, and start with uv run exo. Every device that runs that command joins the same cluster automatically; open http://localhost:52415 on any of them to see the merged topology, then send a chat request from any OpenAI-compatible client and watch the shards light up. If you use Nix, nix run .#exo skips most setup entirely, and the Cachix binary cache spares you the Xcode Metal toolchain download.

Honest limits: exo is built around Apple Silicon and the MLX backend, so this is the tool for Macs and Apple-silicon Linux boxes rather than a general CUDA cluster manager; RDMA needs Thunderbolt 5 hardware and a very recent macOS; the dashboard and app are young and still rough around permissions; and clusters are meant for one network segment, not the wide area. None of that dims the educational value: few projects let you watch a distributed system elect, schedule, and infer, all in a couple thousand lines you can actually read.

Enjoyed this post? Never miss out on future posts by following us