German Qwen agentic RL pilot results

The 15 October 2026 pilot completed real multi-turn GRPO on the German SFT Qwen3-1.7B checkpoint. It does not establish a quality improvement: strict agent success changed from 4/32 to 5/32, while MGSM German accuracy fell from 30/64 to 29/64 and failed the zero-drop retention gate. The candidate is not release eligible.

See the recipe and frozen protocol and aggregate evidence. Weights and raw trajectories remain on shared storage; no model was published.

Experiment

One A100-SXM4-80GB performed 16 optimizer steps with four on-policy rollouts per group, rank-16 LoRA, BF16, learning rate 0.00001 and KL coefficient 0.02. Training took 636.9 seconds and consumed 16 unique tasks from the 128-task pool, producing 64 episodes. The 32 development tasks were unused. Each task requires a hidden order lookup, a calculator call and a final integer answer.

The prompt supplies the exact first lookup JSON; hidden facts become available only through actual tool execution. Training rewards verified lookup/calculation milestones, strict completion and tool-error penalties. Its mean reward of 0.140625 is not a success rate: strict training success was 3/64. There was no supervised tool-format warmup.

Prompt and reward choices used training-only preflights. The 16-step budget was selected before held-out evaluation; an earlier prepared 32-step plan was never executed. Main training started from the original SFT checkpoint, without a preflight adapter. Runtime source commit: 1fe200a2ab3f3df38bca3e03e357e7e982ec249d.

Matched agent evaluation

Both checkpoints used the same 32 frozen test tasks, native tool parser, tokenizer, thinking mode, decoding settings, three tool-loop iterations and 768-token completion-stream budget. All episodes completed without infrastructure errors.

The paired result was one win and zero losses: +3.125 percentage points, with a descriptive paired item-bootstrap 95% interval of [0, +9.375] points. There is only one discordant item, so this does not establish a reliable improvement. The same five items had valid workflows under both policies. The extra success supplied a valid accepted final answer within that set; the baseline final was malformed or unparseable. This does not demonstrate improved arithmetic. Tool errors, token use and time increased.

Mathematical retention and language

The separate retention check reused the exact 64 frozen MGSM German questions, tokenizer and decoding protocol from the SFT pilot. Strict final-answer protocol accuracy changed from 30/64 (46.875%) to 29/64 (45.3125%): one paired win, two losses. The descriptive bootstrap interval for the difference was [−6.25, +3.125] percentage points. One malformed output and one token-cap hit appeared after RL, versus zero before. The observed drop fails the strict zero-drop gate; its broader effect remains uncertain.

A qualitative agent inspection of ten fixed task IDs found German thinking prose in all ten episodes under both policies. Terminal answers were numeric-only (3 before, 4 after) or absent (7 before, 6 after). Some thinking blocks remained unclosed. There was no independent human or native-speaker review. This supports no claim of improved language choice, German fluency or reasoning quality.

Evidence and saved artifacts

The verified summary replays all 146 training tool executions and checks sampled IDs, complete conditioning contexts and masks for 18,200 assistant tokens and 3,413 external/framing tokens excluded from the loss. Twelve groups had reward variation, all 16 steps had nonzero gradients and all 392 adapter tensors changed. An independent elementwise check confirms that the merged BF16 checkpoint retains changes in all 196 LoRA target tensors; the 115 non-target tensors are identical.

The ignored shared-storage directory runs/german-qwen-agent-rl/pilot-02/ contains:

• model-card.md, training/checkpoint/ and merged-checkpoint/;
• frozen specifications, input hashes and execution/training manifests;
• baseline/, candidate/, retention/ and full training rollout evidence;
• summary/summary.json, summary/summary.md and independent artifact audits.

The GPU allocation has been released. The implementation passed 199 local tests, Ruff and package build; the corresponding runtime and reporter CI runs are linked in the aggregate JSON.

This is one small synthetic task family with overlapping numeric ranges, explicit first-action scaffolding, one training seed and one stochastic trial per test item. Item bootstraps omit training/generation-seed variance. MGSM scores also reflect output format and token limits; pretraining exposure is unknown. The container tag was recorded but its immutable digest was not captured. These results qualify an executable training/evaluation path, not general agent ability or deployment readiness. Further development needs fresh evaluation data before additional model selection.