NVIDIA users get a dizzying menu of inference stacks — we have toured TensorRT-LLM, vLLM, and FlashInfer on that side of the fence. On AMD’s ROCm platform the answer to “how does an ONNX model become fast” is more concentrated: most roads run through MIGraphX, the graph compiler inside the ROCm stack. It is not a server like Triton Inference Server and not a kernel library like FlashInfer; it is a compiler with a familiar shape — frontend, IR, optimization passes, target-specific codegen — and a runtime thin enough that the compiled artifact can be saved to disk and executed without ever seeing the original model again.

The mental model that makes the whole repository click: a program is built once as a graph of ops, then each compile target — gpu, cpu, or ref — receives the graph and answers the same two questions: which passes transform it, and what concrete code runs each instruction. That division is what lets the same program object serve a Radeon GPU through HIP code objects, a server CPU through oneDNN, and a portable reference implementation used for correctness checks. Readers of our TVM and Executorch tours will recognize the compiler-not-runtime philosophy; MIGraphX differs mainly in betting everything on one vendor’s hardware and libraries.

The overview map below follows a model file from left to right: the ONNX frontend fills a module, the program drives passes through the pass manager, each target contributes its own pass sequence, and the GPU path bottoms out in codegen that mixes MIOpen plans, hipBLASLt GEMMs, and Just-In-Time-compiled pointwise kernels.

Overview of MIGraphX: the ONNX parser and its op registry build the module IR; program and pass_manager coordinate optimization; three registered targets (gpu, cpu, ref) each supply passes, with the GPU path ending in HipRTC and library-backed codegen; load_save and the driver complete the toolchain.

Reading the overview from left to right:

• The onnx_parser, L631 walks a GraphProto, looking every node type up in the op_parser registry, L37 that the several dozen parse_*.cpp files self-register into.
• Everything lands in a module, L369 — an ordered instruction list with named parameters, nested for loops and branches — and modules live inside a program, L380 whose compile() is the front door to all optimization.
• The pass_manager, L240 executes whichever passes the chosen target requests, and register_target, L56 is the name-to-target map those requests resolve against.
• The GPU target (target.cpp, L359) assembles a five-stage pass sequence ending in compile_ops, L985, where actual kernel code gets generated or fetched.
• Around the compiler, load_save, L61 persists compiled programs to MXR files and the migraphx-driver, L1143 provides run, perf, and verify commands from the command line.

A Frontend That Speaks Fluent ONNX

The ONNX path starts at parse_from, L314, which loads the protobuf and hands the graph to parse_graph. Each ONNX operator type has a dedicated handler — from parse_attention to parse_convolution — registered at static-initialization time into the map guarded by register_op_parser, L37. That design matters more than it looks: adding support for a new ONNX op means adding one file with one registration call, and the registry never changes. Tensor initializers become literals via parse_tensor, L870, subgraphs become nested modules, and opset versioning is resolved before any instruction exists.

By the time parsing ends, the model is no longer ONNX at all — it is MIGraphX’s own IR: instructions referencing ops like convolution, dot, and softmax with MIGraphX shape objects attached. This is the layer where our ONNX format and KTransformers tours ended; MIGraphX is where the next step usually starts on ROCm. It is also deliberately decoupled: the ONNX support builds as its own library, so a caller who constructs programs programmatically — the way our ONNX Runtime tour showed for its own ORT format — never links the parser at all.

The Pass Factory: One IR, Many Transformations

Optimization in MIGraphX is a flat list of small passes, and the pass_manager’s run_passes, L240 applies each one to every module in the program, with a tracer for debugging and dead-code elimination inserted between stages. A few passes carry most of the weight. propagate_constant, L81 evaluates constant subgraphs and replaces them with literals. dead_code_elimination, L38 prunes anything no longer referenced — you see it invoked dozens of times in the GPU target’s source because every transformation can orphan instructions. fuse_pointwise, L487 chains elementwise ops into single fused instructions, which later become one kernel launch instead of five.

Two passes handle the less glamorous but equally decisive problems. schedule, L533 partitions instructions across concurrent hardware streams so independent branches of the graph execute in parallel. memory_coloring, L300 assigns offsets in one shared scratch buffer so tensors with disjoint lifetimes reuse the same memory — on a 16 GB card this pass often decides whether a large model fits at all. Because all of this operates on the target-neutral IR, the CPU and reference targets get the same benefits for free.

Inside the GPU Target’s Five-Stage Compile

The GPU target is where MIGraphX earns its complexity budget, and target.cpp, L359 is the file to read. Its factory object (defined at L177) assembles five named stages: dynamic-shape handling, a required stage of normalization and quantization cleanups (L198), an optimization/rewrite stage (L231), a fusion stage (L266), and the backend stage (L293) that actually produces code. What is striking is how hardware-aware the wiring is — the source checks the gfx generation to decide whether FP8 uses OCP or FNUZ intrinsics, whether BF16 needs promotion, and which rewrites apply.

The backend stage is a relay race. fuse_ops uses declarative matchers to catch patterns like conv-plus-bias-plus-relu and rewrite them into fused ops (the apply at L452). compile_miopen, L68 hands convolutions and pooling to MIOpen’s plan-finding machinery, while compile_hipblaslt, L46 resolves GEMMs through hipBLASLt with heuristic algorithm selection. Whatever neither library covers falls to compile_ops, L985, the codegen dispatcher whose compile plans (L432) and compile manager (L774) choose between generating source via HipRTC at compile time, loading prebuilt code objects, or consulting the persistent binary_cache, L131 so a second run of the same model skips compilation entirely.

The generated-code path is the most interesting. The JIT compilers in src/targets/gpu/jit/ interpolate C++ templates — pointwise.cpp’s kernel template, L38 is a string containing #include <migraphx/kernels/pointwise.hpp> plus placeholders — and the pointwise_compiler, L60 fills in the transformer expressions per op. The included headers, like pointwise.hpp, L54, are a full header-only device kernel library written to be #included inside generated source. The result is wrapped in a code_object_op, L76 that launches the compiled kernel at eval time. For deployment, hipgraphify, L255 can capture the whole launch sequence into HIP graphs, cutting per-launch CPU overhead the same way CUDA graph capture does on the NVIDIA side.

Ref, CPU, and the Art of Verification

Two smaller targets give the project its testing culture. The reference target (ref/target.cpp) implements every op with portable scalar code — slow, but unambiguous. The CPU target maps generic ops onto oneDNN through cpu/lowering.cpp’s cpu_apply, L306, which rewrites dot, convolution, and elementwise ops into their dnnl:: counterparts (with native fallbacks like cpu_op, L135), and cpu/dnnl.cpp, L80 converts MIGraphX shapes into oneDNN memory descriptors. This is the same runtime-of-record pattern our DeepSpeed and llama.cpp tours met from the training and kernel sides, here serving correctness rather than speed.

The payoff lives in the driver: migraphx-driver verify (main.cpp, L1004) runs the same program on two targets and diffs the outputs, with --reduce and --bisect modes that shrink a failing graph to the minimal instruction pair that diverges. For compiler work that is a superpower — numerical bugs become small reproducible cases instead of full-model mysteries.

From Install to Your First Model

On a ROCm machine the C++ library plus Python wheel install through the platform packages, and from there the driver does the teaching:

migraphx-driver perf resnet50.onnx --iterations 100
migraphx-driver run resnet50.onnx --input @0 1 3 224 224
migraphx-driver perf resnet50.onnx --gpu

perf compiles, then reports per-instruction and end-to-end timings using the loop in perf.cpp’s time_run, L143. The compiled program saves to an MXR file — save, L61 writes the serialized instruction stream, shapes, and weights — and loading MXR later skips both parsing and compilation. Programmatically, the C API mirrors the C++ core: migraphx_program_compile, L2430 and migraphx_program_run, L2516 are the two calls most embedders live in, and the Python bindings in the same directory wrap them so import migraphx gives you the same program/parameter_map interface.

Honest Limits and Where MIGraphX Fits

Limits first. MIGraphX is ROCm-first: every interesting codegen decision assumes HIP, MIOpen, and hipBLASLt exist, so it is the wrong tool for a heterogeneous fleet that includes NVIDIA or Intel accelerators — that is what the ONNX Runtime and MLC-LLM abstraction layers are for. Model coverage tracks its ONNX op registry: a model using an unregistered op fails at parse time, and while the registry is broad, exotic new transformer ops can trail the research curve by weeks. The LLM story is real but narrower than on NVIDIA — inference serving for chat-scale deployments on AMD typically flows through it via the ROCm builds of vLLM and SGLang, which we covered in the SGLang tour, rather than through MIGraphX’s own serving surface, which does not really exist.

As for fit: MIGraphX is the right read (and tool) when you deploy ONNX models on AMD GPUs and want a compiled, cacheable artifact — the MXR path — or when you are debugging why a model is slow on ROCm and need per-instruction timings plus a reference target to diff against. Between the pass factory, the five-stage GPU compile, and the verify workflow, it is also simply one of the best study objects in the open-source compiler space: small enough to read whole, honest enough to test itself, and opinionated enough to teach you what a graph compiler actually decides. For the serving layer that sits on top, our inference overview maps the remaining territory.

Enjoyed this post? Never miss out on future posts by following us