# Environment frameworks: OpenEnv, NeMo Gym and FineEnvs

Upstream documentation checked on 20 October 2026. Both OpenEnv and NeMo Gym
support custom models, existing agent harnesses, evaluation and training. The
practical choice concerns interfaces, deployment and integration effort. Upstream
capabilities and this repository's measured qualification are separate evidence.

| Framework | Environment contract | Deployment and composition |
| --- | --- | --- |
| OpenEnv | Gymnasium-style `reset()`, `step()` and `state()`, with synchronous or asynchronous clients over WebSocket. | Docker environments can run locally or remotely. MCP exposes environment tools to agents; training integrations retain environment actions and rewards. |
| NeMo Gym | Composable FastAPI agent, model and resource servers communicating through asynchronous HTTP. | The model server standardizes inference behind the Responses API. Resource servers manage tools, isolated episode state and verification; an existing harness can manage model calls itself. |

These contracts come from the [OpenEnv repository](https://github.com/huggingface/OpenEnv)
and [NeMo Gym architecture](https://docs.nvidia.com/nemo/gym/about/architecture).
NeMo Gym does not require NVIDIA models or exclusive use of NeMo RL. Its
architecture explicitly supports existing agents and local or provider model
endpoints. An environment framework's vendor does not determine the model family.

OpenEnv offers two relevant routes: a trainer can own the tool loop through an
environment adapter, or an existing harness can retain its loop through Harbor.
The latter captures a typed `TrainingTrace` containing original engine tokens,
log probabilities and explicit loss masks. Current documentation describes TRL's
experimental `AsyncGRPOTrainer` with TRL 1.15 or later; that is a different runtime
from our pinned TRL 0.29 recipe. OpenEnv therefore already addresses exact token
capture for external harnesses. See its [Harbor training integration](https://huggingface.co/docs/openenv/environments/harbor).

NeMo Gym also documents training with an external harness that owns model calls.
Its model-server capture associates calls with rollout identities and reconstructs
their continuation relationships. Missing calls or unproven ancestry cause the
rollout to be masked from training rather than guessed. Native token-bearing
responses can instead be consumed directly. Both frameworks can serve this use
case; their capture and trainer contracts need individual validation. See
[training on external harnesses](https://docs.nvidia.com/nemo/gym/tutorials/training-tutorials/external-agent-harnesses).

Bringing our own model normally means configuring its inference endpoint and
the harness that calls it, for example a self-hosted vLLM deployment inside the
cluster. It does not inherently require uploading checkpoint weights to an
environment website. Evaluation may operate on text, actions and trusted
environment state. Policy-gradient training additionally needs the actual sampled
tokens, conditioning contexts, compatible sampling log probabilities, policy
identity and loss masks excluding external observations. Re-tokenizing a finished
transcript does not establish that contract. Model calls must pass through the
chosen training or capture path.

The [FineEnvs multi-harness guide Space](https://huggingface.co/spaces/FineEnvs/multi-harness-rl)
is an interactive article, not a checkpoint-submission or managed evaluation
service. Its companion [FineEnvs repository](https://github.com/adithya-s-k/FineEnvs)
contains runnable environments and training projects, including examples across
frameworks. We can adapt those projects to our model and harness and run them on
our infrastructure. A public Space cannot automatically reach a cluster-private
endpoint. Our model weights and datasets remain on the cluster; using the guide
does not require publishing them. The article's historical results and current
tutorial code are distinct; version pins matter when reproducing either.

Our qualified path is narrower: a real NeMo Gym CPU resource server supplied
lookup/calculator state and verification while native TRL controlled generation
and optimization. The A100 qualification completed eight steps and 32 on-policy
rollouts, with all 392 trainable adapter tensors changed and independent
observation/token-mask replay. Strict training success was 5/32. This establishes
integration and an update, not held-out improvement or German-math retention.
It does not qualify upstream external-harness capture, arbitrary black-box agents
or transfer across harnesses. See [local evidence and limits](agent-gyms.md).

The [matched OpenEnv arm](use-cases/german-openenv-comparison.md) now implements
the same task, verifier and TRL loop. Native SDK CPU checks establish transport,
state isolation, reward parity, capacity recovery and sticky infrastructure
failure. The arm copies the frozen training data and evaluation protocol and
starts from the original SFT parent. GPU resume qualification and held-out
agent/MGSM assessment remain pending. The sequential controller waits for NeMo
release before submitting one bounded GPU job; the original NeMo runtime stays
frozen.

Hugging Face's [reasoning walkthrough](https://huggingface.co/docs/openenv/tutorials/end-to-end-walkthrough)
is a useful introductory example of Qwen3-1.7B, OpenEnv and GRPO with
`environment_factory`. Its current main-version API and single-step arithmetic
task differ from our pinned multi-turn recipe. The
[compatibility notes](use-cases/german-openenv-comparison.md#hugging-faces-reasoning-walkthrough)
explain the adaptation and why training reward needs separate held-out tests.