# Matched German Qwen experiment with OpenEnv

This is a second, independently selected training arm for the
[long NeMo Gym experiment](german-nemo-long-run.md). It tests the same German
lookup/calculator recipe through a native OpenEnv service and SDK. The driver
is implemented and native CPU-qualified; **no completed OpenEnv longer-run
training or held-out quality result is recorded here**. The native service smoke passed 12 checks,
including poisoned GPU-overlay isolation, concurrency, lifecycle and strict-score
parity. The actual SDK client also passed tool/replay, forged-history rejection,
reset isolation and sticky service-loss checks; native trainer configuration,
tool schemas and runtime bindings were exercised without loading a model.
A new GPU stop/resume qualification remains mandatory before the full arm.
Neither integration success nor parameter updates establish improved reasoning.
The [reviewed CPU evidence](../openenv-validation.json) records exact proof and
source hashes, measured concurrency and qualification limits.

As of 21 October 2026, the 20 October OpenEnv GPU attempt stopped in the initial
continuous branch of resume qualification at the strict sampled-token/context/
tool-mask audit. It did not establish full-four-step versus stop-two/resume-four
equivalence or proceed to the 256-step run and held-out assessment. The NeMo
attempt stopped at the same qualification boundary. These engineering failures
do not measure longer-run policy quality. Preserve their original plans and
evidence; new JAIF attempts need freshly qualified runtimes and path-bound plans.

## Frozen comparison

The OpenEnv arm starts from the original German SFT checkpoint, never a NeMo RL
candidate. It copies the existing NeMo evaluation-plan JSON and TRAIN256 JSONL
**byte-for-byte**. The copied evaluation plan retains its original protocol ID,
data hashes and absolute source-data paths. This is deliberate: the evaluator,
tasks, scorer, templates, seeds and acceptance rule remain identical across arms.
The training config changes only the dataset/output paths. Preparation does not
mutate NeMo's prepared inputs or evidence. Changes needed for a workspace
migration require fresh plans for both arms before either starts.

| Element | Both arms |
| --- | --- |
| Input policy/tokenizer | Original SFT model SHA `b16dc15361e19c155df1f833f6ae9800d81b12091755a6f0d346c8c37527b108`; frozen tokenizer SHA `a08566d4528626b17dc3d9dd0667f1312116c67d09751cf410c45e21dce444be`. |
| TRAIN | The same 256 episodes, seed `20261020`, deterministic schedule and four rollouts per optimizer group. |
| Optimization | Native TRL 0.29.0 GRPO; 256 steps, batch 4, accumulation 1, LR `5e-6`, beta `0.1`; LoRA rank 16/alpha 32/dropout 0/all-linear; BF16. |
| Generation | Thinking enabled; temperature 0.6, top-p 0.95, top-k 20, min-p 0; retained stream 768 tokens, at most 3 tool-calling rounds. |
| DEV | Agent 64 and source-disjoint math 128, seed 42, at optimizer steps 64/128/256 only. |
| Selection | Independently per arm: no math DEV correctness loss and improved strict agent success; rank by strict successes, fewer tool errors, earliest step. |
| Diagnostic fallback | If no DEV candidate qualifies, retain SFT as recommendation and assess the preregistered step 256 candidate once. Final tests cannot promote it. |
| Final agent assessment | The same 256 tasks at seeds 42/1729/20261020, only after that arm's selection. |
| Retention | The same full German MGSM 250 at seeds 42/20261020; no observed loss overall and at either seed. Passing does not prove no harm. |
| Analysis | Shared frozen local evaluators, exact paired wins/losses and 10,000 item-bootstrap resamples, grouping seeds by item; caps, malformed outputs, tokens, timing and planned language inspection stay separate. |
| Resource bound | Independent cumulative eight-hour GPU-worker budget per arm. One GPU at a time, with a release-gated sequential controller. |

No NeMo DEV or final-test result may change OpenEnv hyperparameters, steps,
selection rules or task content. Each arm selects using its own DEV results.
MGSM is never a selection input. Earlier development inspected 64 MGSM items, so
the full 250 set is not wholly unseen. The synthetic agent family has only 80
ordered operand pairs; new IDs/templates do not establish broad generalization.
One training seed cannot establish framework superiority.

## Native execution and evidence

The [new driver](../../src/hessian_recipes/openenv_experiment.py) compiles the
hash-guarded frozen NeMo driver into a private namespace. Exactly six trainer
imports and two trainer CLI literals are adapted; service ownership and plan
validation are injected. The original module's globals and files remain intact.
It reuses evaluation, DEV selection, durable budget accounting, journals,
checkpoint export, saved-BF16 change audits and final ancestry checks.

The [OpenEnv training bridge](../../src/hessian_recipes/openenv_training.py)
likewise reuses the source-guarded native TRL trainer. A pinned OpenEnv 0.9.0 SDK
at commit `17c1edd22168fcb76b086def1ed90a7168fb8749` runs in an isolated Python
process; native persistent WebSocket reset/step/state owns each episode. The model
sees the unchanged two-tool schema. Independent local replay checks actual tool
observations, strict scores and shaped rewards; sampled policy IDs/contexts and
external-observation masks remain audited. RPC failures fail as infrastructure,
not zero-reward examples. Only owned server/SDK children are closed.

Preparation requires a completed native CPU smoke proof and pins its bytes,
server/task/requirements identities, installed SDK source identity and executable
path. It also pins the original NeMo plan, shared evaluation plan, training data,
new driver/client/bridge/server/launcher, dependency lock and sequencing controller.
At launch it rechecks the SDK and service identity. Native checkpoint seals bind
optimizer, scheduler, RNG and evidence prefixes. A separate actual same-GPU
full 4 versus stop 2/resume 4 check must pass before OpenEnv main training; NeMo's
qualification cannot substitute for it.

## Prepare and run

Run from `/pfss/mlde/workspaces/mlde_wsp_JAIF/hessian_recipes` in workspace `JAIF` on
our cluster; other teams can use their own checkout and scheduler. See
[operations](../operations.md) for the mounts and allocation. After a relocation,
prepare a fresh NeMo plan and then this OpenEnv plan from the new paths. The
shared evaluation-plan bytes must match within that new pair; do not edit old
frozen plans, journals or historical result manifests to make them launchable.

Recreate the isolated OpenEnv SDK and GPU environments from the
[gym guide](../agent-gyms.md). Verify the original SFT checkpoint and input bytes
in JAIF, then prepare the corresponding fresh NeMo plan. A native CPU proof from
the old environment does not qualify a relocated SDK. Run a fresh proof before
preparing the OpenEnv arm:

```bash
.venv-openenv/bin/python examples/agent_gyms/openenv_smoke.py \
  --output runs/agent-gyms/my-jaif-openenv-native-check
.venv/bin/python -m hessian_recipes.openenv_experiment prepare \
  --nemo-plan runs/agent-gyms/my-jaif-nemo-experiment/experiment-plan.json \
  --output runs/agent-gyms/my-jaif-openenv-experiment \
  --sdk-python "$PWD/.venv-openenv/bin/python" \
  --native-pin runs/agent-gyms/my-jaif-openenv-native-check/report.json
```

Preparation probes the isolated SDK on CPU; it performs no model inference or GPU
allocation. Preserve the venv executable path instead of resolving its symlink.
The original historical proof was `openenv-native-smoke-04/report.json` (SHA-256
`e57d24dc07ad5813aaa46829653b76bc6feb322fb4531d4f17083275890269ea`);
retain it unchanged, separately from the new JAIF proof. Full raw evidence stays
under ignored `runs/agent-gyms/`.

On an allocated compatible GPU, first run only the new arm's qualification:

```bash
.venv-gpu/bin/python -m hessian_recipes.openenv_experiment run \
  --plan runs/agent-gyms/my-jaif-openenv-experiment/experiment-plan.json \
  --qualification-only
```

Continue with the same command using `--resume` instead of `--qualification-only`
only after qualification succeeds. Use the current JAIF command configuration or
a separately qualified managed-experiment launcher as described in
[operations](../operations.md); managed training is not implemented by this guide.
Do not overlap the two GPU arms.

The [sequential controller](../../scripts/run-openenv-after-nemo.py) is an optional
legacy route. Before using it, satisfy the pinned overlay/archive and GPU-library
prerequisites in [operations](../operations.md). It also requires a predecessor
launched by the legacy NeMo controller, with that fresh plan's matching
`queue-launcher-status.json` release acknowledgement; another submission route
does not automatically supply that journal. Supply the actual JAIF predecessor
GPU shell ID explicitly. Never use the protected zero-GPU continuation shell.

```bash
OPENENV_PLAN=runs/agent-gyms/my-jaif-openenv-experiment/experiment-plan.json
OPENENV_PLAN_SHA=$(sha256sum "$OPENENV_PLAN" | cut -d ' ' -f 1)
.venv/bin/python scripts/run-openenv-after-nemo.py \
  --plan "$OPENENV_PLAN" --expected-plan-sha256 "$OPENENV_PLAN_SHA" --check
.venv/bin/python scripts/run-openenv-after-nemo.py \
  --predecessor-shell-id "$JAIF_PREDECESSOR_GPU_SHELL_ID" \
  --plan "$OPENENV_PLAN" --expected-plan-sha256 "$OPENENV_PLAN_SHA" --launch
```

`--check` contacts no cluster, but still verifies the local plan and pinned archive.
`--launch` waits for acknowledged NeMo release and its native task to be terminal
or absent before requesting one OpenEnv GPU command job. The worker proceeds from
preflight to resume qualification, training and assessment only as each preceding
stage succeeds. It resumes an existing journal only with the same plan/runtime.
Cumulative worker time survives interruptions, including conservatively charged
unclosed attempts; an outer eight-hour deadline also bounds the command job.
Scheduler completion releases its GPU.

## Hugging Face’s reasoning walkthrough

The official [walkthrough](https://huggingface.co/docs/openenv/tutorials/end-to-end-walkthrough)
also pairs Qwen3-1.7B, an OpenEnv environment and `environment_factory`, using a
single-step arithmetic task. Its evolving main-version example is not copied
verbatim into this pinned stack. [TRL 0.29.0](https://github.com/huggingface/trl/blob/v0.29.0/trl/trainer/grpo_trainer.py)
requires explicit `reward_funcs`; this recipe supplies a frozen `train_dataset`.
Public bound methods other than `reset` become model tools, so the tutorial's
`get_reward` would be exposed as a tool here. Our verifier stays private and is
called by the trusted reward callback. The frozen generation, reward and
evaluation protocol remains unchanged.

The tutorial's training-log reward delta is a training signal. Our shaped TRAIN
reward is distinct from strict held-out agent success and MGSM retention; it
cannot replace either assessment. This arm performs no public model/dataset push.

## Whole-suite baseline reuse

By default, the plan freezes this rule: reuse an entirely completed compatible
NeMo SFT-baseline suite, otherwise rerun the entire suite. Eligibility depends on
identity/coverage, never scores. `prepare --rerun-baseline` disables reuse before
execution. DEV and final suites are checked separately; no subset of convenient
items or seeds is reused.

Reuse validates captured reports, the original completed-stage journal, current
package/Python versions and byte-identical newly generated baseline specifications.
It copies immutable report bytes into the OpenEnv journal's own directory. A
sidecar binds source report/spec/manifest hashes, the NeMo selection identity and
the independent OpenEnv selection identity. Baseline timings/tokens then describe
the original capture, not newly spent OpenEnv compute. Candidate reports always
come from that arm's selected checkpoint through the shared evaluator. All final
suite signatures and ancestry are verified again before summary acceptance.

## Limits and publication

Retained budgets include tool observations/framing; native training may sample
and discard longer tails. Matching retained caps does not establish equal physical
compute. Transport overhead is part of operational cost, separate from task
quality. Language inspection is a small qualitative audit, not German fluency or
explanation-correctness certification.

Raw traces, checkpoints, adapters and downloaded datasets remain in ignored local
run directories. Recipe/source and reviewed aggregate evidence may enter the
private Hub; this experiment uploads no model or dataset artifacts. Every report
remains `release_eligible: false`.