Most computer vision code you meet in production is written against OpenCV, which means NumPy arrays, BGR channel order, and operations that quietly destroy gradients. Kornia takes the opposite bet: it re-implements the classic vision toolbox — color conversions, blurs, edge detectors, camera geometry, augmentations, feature detection and matching — as pure PyTorch tensor operations that keep a computation graph. An image stops being a blob of unsigned bytes and becomes a differentiable quantity you can backpropagate through. That single design decision turns calibration, image registration, and even dataset preprocessing into optimizable, trainable components rather than frozen preprocessing steps.

The library is also one of the more pleasant Python codebases to read. Almost every public routine is a thin, batch-first tensor function: it validates input with a family of check helpers, computes something mathematically honest in a handful of lines, and returns a tensor whose shape you can predict from the signature. There is very little hidden state. Modules wrap functions, containers compose modules, and the geometry code reads like a linear algebra textbook that happens to compile.

This post is a source tour of kornia/kornia on the main branch. We mapped the repository into two architecture diagrams — one overview, one detail — and then walk each subsystem with line-linked references so you can jump straight into the code. If you want the sibling library that handles loading and model Zoo plumbing rather than the math, see our torchvision tour; kornia complements it rather than competing with it.

The overview graph: images enter as tensors on the left, flow through color and enhancement into the filter stack, while geometry and augmentation share the warping machinery, and feature matching closes the loop with homography estimation.

Reading the overview from left to right:

• Ingest. image_to_tensor, L28 is the front door: it turns NumPy or PIL images into the CHW float tensors every other routine expects, and its twin tensor_to_image, L188 carries results back for display.
• Color order. rgb_to_bgr, L29 is the small but crucial bridge to OpenCV conventions; the module form RgbToBgr, L446 lets you drop the same conversion into an nn.Sequential chain.
• Normalization. normalize, L133 applies per-channel mean and standard deviation, and the class form Normalize, L43 promotes integer input to float first so uint8 batches do not silently truncate.
• The convolution core. filter2d, L71 is a depthwise 2D convolution that nearly every filter builds on; gaussian_blur2d, L96 constructs its kernel analytically and runs it separably.
• Edges. canny, L117 composes that smoothing with Sobel gradients, non-maximum suppression, and a differentiable hysteresis loop — a good example of translating a classic algorithm into tensor stages.
• Warps. warp_perspective, L254 maps pixels through a 3x3 homography, and find_homography_dlt, L502 estimates that matrix from point correspondences by solving the DLT system with SVD.
• Cameras. project_points, L29 applies a pinhole camera matrix to 3D points, anchoring the geometry side of the graph.
• Augmentation and features. AugmentationSequential, L74 applies one random transform to images, masks, keypoints, and boxes at once, while SIFTDescriptor, L156 and DescriptorMatcher, L509 produce and match descriptors entirely in tensor land.

From Install to Your First Model

Installation is the usual one-liner, and the first useful program is already differentiable end to end:

import torch
import kornia
import kornia.augmentation as K

# images as (B, 3, H, W) float tensors in [0, 1]
img = torch.rand(2, 3, 64, 64)

blur = kornia.filters.GaussianBlur2d((5, 5), (1.5, 1.5))
edges = kornia.filters.Canny(0.1, 0.3)
blurred = blur(img)
magnitudes, edges_map = edges(blurred)

aug = K.AugmentationSequential(
    K.RandomAffine(360, (0.1, 0.1)),
    K.ColorJiggle(0.1, 0.1, 0.1, 0.1),
    data_keys=["input"],
)
augmented = aug(blurred)
augmented.sum().backward()  # gradients flow through blurs and warps

Three details are worth noticing. First, everything accepts a batch dimension, so the same call handles one image or one thousand. Second, the module forms (GaussianBlur2d, Canny, ColorJiggle) are honest nn.Module subclasses, so they compose with standard training loops and torch.compile. Third, the backward pass is real: kornia’s unit tests assert gradient correctness for most operators, which is exactly what you need if you want to optimize, say, a homography against a photometric loss. That property is what separates kornia from wrappers around native code, and it is why libraries like ultralytics and segmentation models such as SAM 2 can slot it into data preparation without leaving tensor land.

Inside the Detail Graph

The detail diagram expands each overview box into its concrete modules. Reading it group by group is the fastest way to build a mental index of the package.

The detail graph with seven groups: ingest and export, color and enhancement, filters, geometry and cameras, augmentation containers, feature matching, and the quality metrics that become training losses.

The color and enhancement group is deliberately boring in the best way: rgb.py alone defines the full family of channel conversions, hsv.py handles hue-space math that the augmentation stack reuses, and adjust.py collects brightness, contrast, and gamma curves. The geometry group is the mathematical heart, so it deserves its own walk.

Geometry: Warps, Cameras, and Epipolar Math

Kornia’s geometry package is where batched linear algebra pays off. The transform utilities start from constructors that produce transformation matrices directly from data: get_perspective_transform, L848 solves the closed-form 3x3 system from four point pairs, and get_rotation_matrix2d, L918 builds affine matrices from a center, angle, and scale. The warping functions then consume those matrices — warp_perspective, L254 for homographies and warp_affine, L385 for the affine case — both implemented by generating a sampling grid and letting PyTorch’s grid sampler do the interpolation. Because the sampling is differentiable, gradients flow back into the transformation matrix itself, which is the trick behind test-time optimization of alignments.

On top of the functions sits a stateful layer: HomographyWarper, L77 precomputes and caches its grid, which matters when you warp many frames with the same transform. The camera package mirrors this split between math and state. PinholeCamera, L36 is a batched container for intrinsics and extrinsics, and project_points, L29 performs the perspective division that maps 3D points to pixels. Depth gets equal treatment: depth_to_3d, L205 lifts a depth map into a point cloud using the camera matrix, and DepthWarper, L487 reprojects one frame into another’s view — the core operation of any planar-RL or pose estimation training loop.

The epipolar subpackage completes the two-view story. find_fundamental, L592 dispatches to the seven- and eight-point algorithms — run_8point, L512 builds the design matrix and enforces the rank-2 constraint — while the generic transform_points, L223 applies rigid transforms to point batches. Everything here is differentiable, so you can, for example, learn to weight correspondences and watch the estimated fundamental matrix improve.

Augmentation That Keeps Labels in Sync

The augmentation package is the most commonly used part of kornia, and its design centers on one problem: when you rotate an image, the boxes, masks, and keypoints must rotate with it. Individual ops like RandomAffine, L30 and ColorJiggle, L122 inherit from a shared base that standardizes parameter generation, sampling probabilities, and the return of transformation parameters. The intensity ops reuse the enhancement functions we saw earlier — the hue and saturation math flows through the same HSV conversions in color/hsv.py.

The container is where it comes together. AugmentationSequential, L74 takes data_keys describing each positional argument — input, mask, keypoints, bbox — and routes each operand through the same sampled transform with the correct resampling rules, nearest-neighbor for masks, exact coordinate math for keypoints. It builds on ImageSequentialBase, L224, which adds the plumbing for invoking children in sequence while collecting each one’s transformation matrix. Because every op reports the matrix it applied, you can invert the whole augmentation chain later — invaluable for mapping detections made on augmented images back to original coordinates. For training loops that need gradients through augmentation itself, the container is also the piece that keeps everything differentiable; the PyTorch autograd engine simply sees a long composition of tensor ops.

Features, Matching, and Losses You Can Train Against

The feature package re-creates the classic SIFT recipe as tensors. SIFTDescriptor, L156 builds gradient-histogram pooling kernels at construction time and then computes descriptors with pure tensor reshapes — no loops, no compiled extensions. Matching lives in matching.py: match_nn, L100 does plain nearest-neighbor correspondence, and DescriptorMatcher, L509 wraps mutual-NN, distance-ratio, and distance-ratio-cross checks behind one module interface. Higher up, LocalFeature, L250 bundles a detector, descriptor, and matcher into one object, LocalFeatureMatcher, L767 runs the full two-view estimation including outlier rejection, and KeyNetHardNet, L697 shows the learnable end of the spectrum — a trained keypoint detector wired to HardNet descriptors.

Finally, the quality group turns evaluation metrics into training signals. ssim, L105 computes the structural similarity index batched over images, and SSIMLoss, L91 wraps it as a loss you can minimize — the standard trick in image restoration and style transfer training. Combined with the differentiable color and filter ops, you can construct photometric objectives without ever leaving the tensor world.

Honest Limits

Kornia is not a drop-in OpenCV replacement, and the source is upfront about it in places:

• I/O is minimal by design. The library deliberately does not read video files or camera streams; it assumes tensors already exist. The newer kornia.image container is explicitly marked experimental, and the legacy image helpers in kornia.utils now emit deprecation warnings pointing at it.
• CPU throughput can lose to native code. For a single small image, a hand-tuned C++ implementation will often beat a chained tensor implementation; kornia’s advantage appears with batches, GPU execution, and the need for gradients, not with one-off scripting.
• API churn is real. Feature module names have moved over the years, and the augmentation container keeps refining subtle semantics around transformation matrices and non-rigid ops. Pin your version in production.
• Numerical parity is close but not exact. Kornia’s Canny or SIFT will not match OpenCV bit-for-bit; thresholds may need retuning when porting.
• 3D and video coverage is thinner than 2D. Some operations exist in 3D form, but the polish and test coverage concentrate on 2D image processing.

None of these are disqualifying — they define the niche precisely. If your work lives in PyTorch and you want vision operations that respect autograd, batch dimensions, and device placement, kornia is the reference implementation to read and reuse. Start from the overview diagram, keep the detail graph beside you on your first dive into kornia/geometry, and the codebase opens up quickly.

Enjoyed this post? Never miss out on future posts by following us