Every retrieval-augmented generation stack, every recommendation feed, and every semantic search box eventually asks the same question: given this vector, which of the hundred million stored vectors are its nearest neighbors? Apache-2.0 licensed cuVS, version 26.12.00 on the RAPIDS release train, is NVIDIA’s answer, and it is the layer that an increasing number of databases call rather than reimplement. The README describes it as state-of-the-art implementations of approximate nearest neighbors and clustering on the GPU, built on top of the RAPIDS RAFT primitives, usable directly or through the libraries that have integrated it. What makes the tree special is the coverage: one C ABI, four idiomatic language APIs, a dozen index types from exact brute force to learned quantization, and a build-on-GPU, deploy-on-CPU story for the edge. As always in this series, what follows is an educational tour of published source code.

The headline algorithm is CAGRA, a GPU-native graph index published in a 2023 paper whose build and search the repository implements end to end. Around it sit the IVF family in four quantization regimes, graph builders like NN-Descent and Vamana, a 3015-line CPU HNSW engine, clustering that includes the cuSLINK single-linkage paper implementation, and a top-k selection stack strong enough that its own research paper is cited in the README. The tree is organized for portability: cpp/include/cuvs holds the public headers, cpp/src the CUDA kernels, c/include the stable C surface, and python, rust, go, and java folders each carry a native-feeling API. Reading the overview from left to right walks one query from a language binding down through graph traversal kernels to a ranked result list.

Overview of cuVS: graph builders and the CAGRA index define the searchable structures, CUDA detail kernels implement build and search, the IVF family trains its lists through the clustering suite, brute force feeds the top-k selection machinery, and one C ABI carries everything to Python, Rust, Go, and Java, with HNSW bridging GPU indexes to CPU deployment and multi-GPU sharding on top.

Reading the overview from left to right:

• cagra.hpp is the flagship at 4979 lines: search_params at 56, ace_params at 173 for partitioned builds that exceed GPU memory, the index class at 486, and merge_params at 3474.
• ivf_flat.hpp at 3579 lines is the inverted-file workhorse, with index_params at 28, list_spec at 87, and the index at 137.
• ivf_rabitq.hpp compresses vectors to 1 through 9 bits per dimension, defaulting to 3 at line 54, with a search_mode enum at 95 offering LUT16, LUT32, QUANT4, and QUANT8 scan modes.
• brute_force.hpp at 952 lines is the exact baseline that papers benchmark against, alongside scann.hpp at 408 lines for the two-level ScaNN design.
• nn_descent.hpp at 623 lines and vamana.hpp at 664 build the k-NN graphs that CAGRA consumes, while tiered_index.hpp at 273 wraps any index for incremental inserts.
• cagra_build.cuh is the 1250-line kernel layer behind the build, with build_knn_graph at 156 and host-side refinement at 110.
• select_k.hpp at 203 lines is the public face of the top-k machinery from the Top-K on GPU paper, shared by every index in the tree.
• cagra.h is the stable C surface: cuvsCagraBuild at 710, cuvsCagraSearch at 804, SerializeToHnswlib at 922, and Merge at 1079, all over DLPack tensors.

Why You Need This

The first reason is the throughput story at database scale. Vector search at a hundred million rows is memory-bound, and the difference between a good index and a great one is how few random reads each query needs. CAGRA’s answer is a graph built specifically for GPU warp-level traversal, and the repository lets you watch the whole construction: build_knn_graph in cagra_build.cuh computes the intermediate neighborhood, graph_core.cuh prunes it to the final degree, and cagra_ace.cuh implements Augmented Core Extraction, the partitioned build that streams clusters through GPU memory when the dataset will not fit. Search runs as multiple coordinated kernels in the 713-line search_multi_kernel.cuh, scoring candidates through compute_distance templates generated for raw and vector-quantized datasets. If you run Milvus, Vespa, or any engine that has integrated cuVS, this is the code your latency is made of.

The second reason is the quantization ladder, which pairs beautifully with the weight quantization we toured in the torchao post. The IVF family spans four regimes: ivf_flat keeps full-precision vectors per inverted list, ivf_pq.hpp at 3451 lines interleaves product-quantized codes for memory-constrained serving, ivf_sq scalar-quantizes per dimension, and ivf_rabitq.hpp implements the GPU-native RaBitQ design with 1 to 9 bits per dimension, kmeans-trained coarse centroids, and lookup-table search modes. RaBitQ’s header even documents streaming construction with streaming_batch_size at line 70 and force_streaming at 84, so an index larger than device memory builds straight from host storage. Understanding when each regime wins is exactly the recall-versus-memory tradeoff that vector database architects are paid to navigate, and here every knob is documented in the same header that implements it.

The third reason is the portability discipline. One C ABI in c/include speaks DLPack, which means tensors arrive from PyTorch, CuPy, or your own buffer format without copies, and the four language bindings all sit on that ABI: Cython modules like cagra.pyx for Python, a cuvs-sys FFI crate for Rust with index.rs exposing build, search, search_filtered, and serialize_to_hnswlib, Go packages per index over dlpack.go, and Java through JDK 22 Panama bindings. The Java folder also ships cuvs-lucene, a pluggable Lucene KnnVectorsFormat that offloads vector indexing to the GPU through a codec, so an existing Lucene or Elasticsearch stack accelerates with minimal code changes. And the SerializeToHnswlib entry at line 922 of cagra.h closes the edge case: an index built on a GPU can be exported to hnswlib format and served by the 3015-line CPU HNSW engine in detail/hnsw.hpp.

Detail view: cagra.hpp drives build and search entries into the CUDA detail kernels, ivf_common.cuh backs the flat, PQ, RaBitQ, and SQ variants with kmeans-trained codebooks, the top-k and bitonic machinery ranks candidates, HNSW and multi-GPU layers extend deployment, and the binding stack from DLPack to Lucene carries it all to users.

The CAGRA internals repay a slow read. The public header declares a variant of graph builders at line 253, so the same index can be built with ivf_pq_params, nn_descent_params, or ace_params depending on dataset size, and search_params at 56 add iterative search for oversized candidate sets. In the detail folder, cagra_build.cuh chains build_knn_graph at 156 through write_to_graph at 86 and refine_host_and_write_graph at 110, while the search side pairs cagra_search.cuh with compute_distance templates instantiated per dtype from matrix.json manifests, the same JIT-style instantiation trick that keeps compile times sane across fp32, fp16, int8, and uint8. The topk_for_cagra folder with its topk.cu and topk_core.cuh implements the radix-style candidate pruning, bitonic.hpp sorts partial result lists, and cagra_merge.cuh combines indexes grown independently, which is also what the C API’s Merge entry at 1079 exposes to databases doing continuous ingestion.

The IVF family shows how much machinery one shared abstraction absorbs. ivf_common.cuh and ivf_list.cuh define the inverted-list layout every variant reuses, and the differences are all in what a list stores: full vectors for flat, interleaved PQ codes with the list_spec_interleaved struct at line 235 of ivf_pq.hpp, per-dimension scalars for SQ, or the low-bit RaBitQ codes with their LUT scan modes. Codebooks come from kmeans.hpp at 1897 lines, with balanced_kmeans_example.cu demonstrating the size-evening variant that keeps GPU lists uniform. Filters integrate at the same layer: roaring_bitmap_filter.cu and sample_filter.cu let a search exclude masked rows, the examples folder shows both a roaring bitmap and a Bloom filter variant, and the Python filters.pyx wraps them for user code. Reachability and sparse k-NN kernels under detail round out the graph-analysis use cases the README lists for UMAP and cuGraph integration.

The binding and deployment stack is the quiet engineering triumph. Python’s cagra.pyx at 1161 lines mirrors the C++ surface with build at 453, search at 823, and extend at 1112, over cydlpack.pyx conversions and resources.pyx stream handling, with mg_resources.pyx and the mg packages exposing single-node multi-GPU built on snmg.cuh. The C API even provides IndexParamsFromHnswParams at line 328, acknowledging the migration path from existing hnswlib deployments. Rust’s index.rs exposes search_filtered at 154 and hnswlib export at 215, Go ships per-index packages with a shared distance.go metric zoo, and the Java side documents its filter bitset device pool down to the RMM 4 MiB initial reservation. Distance itself lives in the 471-line distance.hpp behind pairwise_distance at line 51 of distance.pyx, the same metric set every index consumes.

From Install to a Searched Index

Install the Python package, build a CAGRA index, and search it, exactly as the README shows:

pip install cuvs-cu12
# or: conda install -c rapidsai -c conda-forge -c nvidia cuvs

import cupy as cp
from cuvs.neighbors import cagra

dataset = cp.random.random((100000, 64), dtype=cp.float32)

index_params = cagra.IndexParams(
    intermediate_graph_degree=128, graph_degree=64
)
index = cagra.build(index_params, dataset)

queries = cp.random.random((10, 64), dtype=cp.float32)
search_params = cagra.SearchParams(itopk_size=32)
neighbors, distances = cagra.search(search_params, index, queries, 10)

The same CAGRA flow reads almost identically in C, Rust, and Go, because every binding implements the same lifecycle of params, build, search, and destroy over DLPack tensors.

Honest limits: cuVS is GPU-first, so the Python API expects CUDA arrays and the prebuilt wheels are tied to specific CUDA major versions, with the README noting the CUDA 13 builds are roughly half the binary size of CUDA 12 builds. Exact recall reproduction against research numbers takes parameter work, the Java API is flagged experimental in its own README, and the ACE partitioned build trades build time for memory headroom. None of that changes the core fact: the fastest open vector search kernels on NVIDIA hardware, plus the CPU fallback and the multi-language surface, live in this one Apache-2.0 tree, and reading them will make every downstream vector database make sense.

Enjoyed this post? Never miss out on future posts by following us