Every reinforcement learning result you have ever read about, from an agent learning to balance a pole to a policy that lands a lunar module, rests on the same two method calls: env.reset() and env.step(action). Gymnasium, maintained by the Farama Foundation, is the library that defines precisely what those calls mean, what they return, and how they must behave. It is the community successor to OpenAI Gym, adopted so thoroughly that essentially every modern reinforcement learning library, from training frameworks to evaluation harnesses, speaks its dialect. When Gym was archived in 2022, Gymnasium became the de facto standard: same ergonomics, stricter semantics, and a long-overdue cleanup of ambiguous return values.

What makes the source worth reading is its economy. The entire environment contract lives in one file, gymnasium/core.py, and it is barely seven hundred lines. Everything else in the library is composition on top of that contract: wrappers that layer behavior onto any environment, vector environments that step dozens of them in parallel, spaces that give observations and actions machine-checkable types, and a registration system that lets a string like "CartPole-v1" resolve into a running simulation. There is very little magic here, and that is exactly the lesson: a package used by thousands of research papers is held together by one small interface used with unusual discipline.

We have walked through the source of training-side systems before, like PyTorch internals, the compact nanoGPT training script, and the RLHF framework in OpenRLHF. But all of those consume experience data; Gymnasium is where that data is produced. Understanding the producer side completes the picture, and it also explains why certain quirks show up everywhere in RL code: why step() returns five values, why "truncated" and "terminated" are separate flags, and why seeding is an argument to reset() rather than a constructor call.

The whole library in one view: a single Env contract at the center, with factories, wrappers, vectorization, typed spaces, and reference environments arranged around it.

Reading the overview from left to right:

• The front door is one function. make() resolves a string id through a plain dictionary registry and returns a ready environment, while make_vec() builds the batched variant. Third-party packages never modify the library; they call register() to add their own ids. The registry is literally a module-level dict, which is refreshing to see in an era of over-engineered plugin systems.
• Env is the entire contract. The base class at core.py, line 30 defines reset(), step(), render(), and two typed attributes, observation_space and action_space. That is all an environment owes the world. The famous five-tuple return of step(), observation, reward, terminated, truncated, info, is documented right here, and the split between "the episode reached a natural end" and "we cut it off" is enforced in the type signature, not by convention.
• Wrapping is the extension mechanism. Wrapper is a transparent proxy: any attribute you do not override is forwarded to the inner environment, so a wrapper can override exactly one behavior and inherit everything else. Three convenience subclasses carve out the common cases, ObservationWrapper, RewardWrapper, and ActionWrapper, each asking you to implement a single transformation function.
• The boring-but-critical guards live in one file. TimeLimit converts infinite-horizon games into finite ones, OrderEnforcing makes it an error to step() before reset(), and RecordEpisodeStatistics accumulates episode returns and lengths into the info dict. Notably, make() applies several of these automatically, which is why a raw class and a registered id are not quite the same object.
• Video capture is just another wrapper. RecordVideo hooks into the render stream and writes mp4s of your episodes, HumanRendering upgrades a headless environment into a windowed one, and RenderCollection simply accumulates frames in memory. Because these are wrappers rather than core features, an environment author can stay completely ignorant of how recording works.
• Vectorization is a class hierarchy, not a flag. The abstract VectorEnv treats N environments as one, SyncVectorEnv steps them sequentially in a Python loop, and AsyncVectorEnv runs each in its own process so stepping happens genuinely in parallel. This is the workhorse behind the "collect a rollout of 4096 transitions" idiom in modern algorithms.
• Spaces make shapes machine-checkable. The base class at space.py, line 18 promises sample(), contains(), and seeding. Concrete subclasses cover the universe: Box for continuous vectors, Discrete for integer actions, plus composites like Dict for structured observations. Network input layers can be derived from spaces instead of hand-configured, which removes a whole class of shape bugs.
• Reference environments show the contract in action. CartPoleEnv is two hundred lines of physics plus rendering, an ideal template for writing your own environment, and LunarLander shows the same contract applied on top of a real physics engine with fuel constraints and shaping rewards.
• A checker keeps the ecosystem honest. check_env() runs a battery of assertions against any environment you hand it: spaces must match what reset() and step() actually emit, determinism with a fixed seed must hold, and dtypes must not drift. Most "my env trains on Monday and crashes on Tuesday" bugs are caught here before training ever starts.

The Detail View: One Step Deeper

The overview compresses the story; the detail graph expands it into the classes you will actually touch when writing or debugging an environment. The left-to-right narrative now becomes a nesting narrative, because almost every arrow in the second diagram is a "wraps" or "contains" relationship rather than a call.

The detail graph: the wrapper family tree on the core, the two vectorization strategies, the space hierarchy with its composites, and the two-tier checker that validates environments both actively and passively.

Three relationships deserve your attention. First, the wrapper family tree: TimeLimit, RecordEpisodeStatistics, and friends all descend from the proxy in core.py, which is why they compose in any order and why the env wrapper chain grows at runtime without any central coordination. Second, the two vectorization strategies share one abstract parent, so an algorithm written against VectorEnv runs on either backend unchanged; the synchronous version is your debugging baseline and the asynchronous version is your speed lever, and swapping them is a one-line change. Third, the checker is two-tiered by design: check_env() performs loud, exhaustive validation when you ask for it, while the passive checker runs quietly inside every make() call, issuing warnings instead of exceptions so that a sloppy third-party environment degrades gracefully rather than breaking the library.

From Install to Your First Model

Install is a single package name, and the canonical first script fits on a Post-it note:

import gymnasium as gym

env = gym.make("CartPole-v1")
obs, info = env.reset(seed=42)
for _ in range(1000):
    action = env.action_space.sample()
    obs, reward, terminated, truncated, info = env.step(action)
    if terminated or truncated:
        obs, info = env.reset()
env.close()

Read it as a conversation with the contract. reset() returns the first observation plus an info dict; the seed argument makes the episode reproducible. step() returns the five-tuple, and the two booleans must be handled separately: terminated means the environment ended for a reason intrinsic to the task, while truncated means an external time limit fired. Conflating the two is a classic bootstrap-error bug, because bootstrapping from a state after a time-limit cut introduces bias that the value function then learns to propagate.

Wrappers attach like layers of clothing:

env = gym.make("CartPole-v1", render_mode="rgb_array")
env = gym.wrappers.RecordEpisodeStatistics(env)
env = gym.wrappers.TimeLimit(env, max_episode_steps=250)
obs, info = env.reset()

Each wrapper returns an object that still looks and smells like an Env, so code written for the naked environment keeps working through any number of layers. When you are ready for speed, gym.make_vec("CartPole-v1", num_envs=8) hands you a batched environment whose step() takes an array of eight actions and returns arrays of eight observations, rewards, and flags. For a taste of where this goes, an environment loop is the part of an RLHF training loop that OpenRLHF and its siblings delegate to exactly this API, while the model updates run in PyTorch; the two worlds meet at the five-tuple.

One more habit worth stealing from the source: define your environment's spaces with the same care the built-ins do. If your observation is a camera image plus a joystick state, declare a Dict space with a Box and a Discrete inside it rather than flattening everything into one vector and remembering the layout by comment. The same philosophy shows up in supervised tooling, where scikit-learn users get the same class of bug prevention from typed feature containers, and it costs nothing at import time.

Honest Limits

Gymnasium is superb infrastructure, and it is worth knowing where the infrastructure stops. The classic control and Box2D environments are pedagogical toys; their dynamics are smooth, low-dimensional, and cheap, which makes them wonderful for validating an algorithm and useless for predicting how it will fare on a real robot or a web-scale recommendation task. The moment you need realistic dynamics you leave the built-in registry and depend on third-party packages, with all the version-drift risk that implies.

The asynchronous vector environment has sharp edges too. It pays for parallelism with process boundaries: every environment object must survive pickling, every observation crosses a pipe, and a crash in one worker surfaces as an opaque error on the next collective call. For small or cheap environments the synchronization overhead can exceed the benefit, and the honest benchmarking advice is to compare SyncVectorEnv and AsyncVectorEnv on your workload before assuming the async one is faster.

Finally, there is the migration tax. The rename from gym to gymnasium happened precisely because the old API's ambiguities, most notoriously the four-tuple step return and hidden seeding state, could not be fixed in place. Ten years of tutorials, blog posts, and GitHub issues still use the old names, and copy-pasting them yields confusing import errors rather than clean failures. The library also deliberately knows nothing about agents, replay buffers, or experiment tracking; if you want those, you compose a training framework on top, or accept that you are reading the wrong source tree. What you get in exchange is a dependency that fits in your head: one contract, wrappers you can reason about locally, and a checker that keeps the ecosystem's promises machine-verified rather than folklore-verified.

Enjoyed this post? Never miss out on future posts by following us