# German Qwen agentic RL pilot results

The 15 October 2026 pilot completed real multi-turn GRPO on the German SFT
Qwen3-1.7B checkpoint. It does not establish a quality improvement: strict agent
success changed from 4/32 to 5/32, while MGSM German accuracy fell from 30/64 to
29/64 and failed the zero-drop retention gate. The candidate is not release eligible.

See the [recipe and frozen protocol](german-agent-rl.md) and
[aggregate evidence](german-agent-rl-results.json). Weights and raw trajectories
remain on shared storage; no model was published.

## Experiment

One A100-SXM4-80GB performed 16 optimizer steps with four on-policy rollouts per
group, rank-16 LoRA, BF16, learning rate 0.00001 and KL coefficient 0.02. Training
took 636.9 seconds and consumed 16 unique tasks from the 128-task pool, producing
64 episodes. The 32 development tasks were unused. Each task requires a hidden
order lookup, a calculator call and a final integer answer.

The prompt supplies the exact first lookup JSON; hidden facts become available
only through actual tool execution. Training rewards verified lookup/calculation
milestones, strict completion and tool-error penalties. Its mean reward of
0.140625 is not a success rate: strict training success was 3/64. There was no
supervised tool-format warmup.

Prompt and reward choices used training-only preflights. The 16-step budget was
selected before held-out evaluation; an earlier prepared 32-step plan was never
executed. Main training started from the original SFT checkpoint, without a
preflight adapter. Runtime source commit:
`1fe200a2ab3f3df38bca3e03e357e7e982ec249d`.

## Matched agent evaluation

Both checkpoints used the same 32 frozen test tasks, native tool parser, tokenizer,
thinking mode, decoding settings, three tool-loop iterations and 768-token
completion-stream budget. All episodes completed without infrastructure errors.

| Measure | Before RL | After RL |
| --- | ---: | ---: |
| Strict full-workflow success | 4/32 (12.5%) | 5/32 (15.625%) |
| Valid tool workflow | 5/32 | 5/32 |
| Correct accepted final answer, regardless of workflow | 7/32 | 8/32 |
| Tool calls / tool errors | 70 / 48 | 72 / 51 |
| Malformed final outputs | 23 | 23 |
| Turn-cap / token-cap episodes | 13 / 3 | 15 / 4 |
| Assistant tokens | 8,658 | 9,568 |
| Episode wall time, excluding model loading | 205.8 s | 231.6 s |

The paired result was one win and zero losses: +3.125 percentage points, with a
descriptive paired item-bootstrap 95% interval of [0, +9.375] points. There is only
one discordant item, so this does not establish a reliable improvement. The same
five items had valid workflows under both policies. The extra success supplied
a valid accepted final answer within that set; the baseline final was malformed
or unparseable. This does not demonstrate improved arithmetic. Tool errors, token
use and time increased.

## Mathematical retention and language

The separate retention check reused the exact 64 frozen MGSM German questions,
tokenizer and decoding protocol from the SFT pilot. Strict final-answer protocol
accuracy changed from 30/64 (46.875%) to 29/64 (45.3125%): one paired win, two
losses. The descriptive bootstrap interval for the difference was
[−6.25, +3.125] percentage points. One malformed output and one token-cap hit
appeared after RL, versus zero before. The observed drop fails the strict
zero-drop gate; its broader effect remains uncertain.

A qualitative agent inspection of ten fixed task IDs found German thinking prose
in all ten episodes under both policies. Terminal answers were numeric-only
(3 before, 4 after) or absent (7 before, 6 after). Some thinking blocks remained
unclosed. There was no independent human or native-speaker review. This supports
no claim of improved language choice, German fluency or reasoning quality.

## Evidence and saved artifacts

The verified summary replays all 146 training tool executions and checks sampled
IDs, complete conditioning contexts and masks for 18,200 assistant tokens and
3,413 external/framing tokens excluded from the loss. Twelve groups had reward
variation, all 16 steps had nonzero gradients and all 392 adapter tensors changed.
An independent elementwise check confirms that the merged BF16 checkpoint retains
changes in all 196 LoRA target tensors; the 115 non-target tensors are identical.

The ignored shared-storage directory `runs/german-qwen-agent-rl/pilot-02/` contains:

- `model-card.md`, `training/checkpoint/` and `merged-checkpoint/`;
- frozen specifications, input hashes and execution/training manifests;
- `baseline/`, `candidate/`, `retention/` and full training rollout evidence;
- `summary/summary.json`, `summary/summary.md` and independent artifact audits.

The GPU allocation has been released. The implementation passed 199 local tests,
Ruff and package build; the corresponding runtime and reporter CI runs are linked
in the aggregate JSON.

This is one small synthetic task family with overlapping numeric ranges, explicit
first-action scaffolding, one training seed and one stochastic trial per test
item. Item bootstraps omit training/generation-seed variance. MGSM scores also
reflect output format and token limits; pretraining exposure is unknown. The
container tag was recorded but its immutable digest was not captured. These
results qualify an executable training/evaluation path, not general agent ability
or deployment readiness. Further development needs fresh evaluation data before
additional model selection.