RESEARCH RECIPES · WORK IN PROGRESS
Reproducible workflows for reasoning adaptation, distillation and agentic reinforcement learning. Experiments record baseline identities, task splits, training settings and matched evaluation.
Current case study: German Qwen3-1.7B, with agent task success and MGSM mathematical retention assessed separately.
01 / METHODS
Fixed baselines, development-set selection and separate assessment of task success and retained capabilities.
Freeze the model, data and evaluation before you train.
Bind the checkpoint, tokenizer, prompts, task splits and decoding settings to a reproducible specification.
Use SFT, distillation or RL for the supervision you have.
Reuse pinned upstream trainers and a verifier that reflects success in the target task.
Evaluate the tool loop and retained model capabilities.
Select on development tasks. Keep final agent assessment and mathematical retention checks separate from training reward.
Save the inputs, outcomes and limits behind the claim.
Recorded reports include source and artifact identities, paired outcomes and explicit qualification boundaries.
02 / RECIPE COLLECTION
11 recipes & guides
Frozen specifications, adapters, reports and regression checks.
Pinned TRL recipes and a CPU wiring check.
Local teacher generation, answer verification and filtered SFT data.
Source-aligned German math adaptation and matched MGSM evaluation.
Native multi-turn tool training, agent evaluation and retention checks.
NeMo Gym with GPU-tested TRL; OpenEnv and Verifiers CPU qualification.
Native OpenEnv, shared DEV and MGSM tests, sequential GPU scheduling; quality results pending.
Trace contract and optional Lightning/verl launcher; Lightning GPU path unqualified.
German RAG with cited sources and business-agent workflows; expert tasks and benchmark validation remain open.
Domain-native starting points and proposed evaluation contracts.
Local checksummed bundles and model/dataset card templates.
No recipes match. Try another search or category.
CASE STUDY / GERMAN REASONING
A local Qwen3-1.7B checkpoint is adapted using German SFT and multi-turn agent RL, then compared with its baseline under matched evaluation settings.
A research pilot. No candidate is qualified for release.
03 / MEASURED OUTCOMES
German Qwen pilot results, with sample sizes and limitations. Integration checks are reported separately from model-quality assessments.
A fixed 64-item German MGSM pilot, one stochastic trial per item.
Descriptive 95% paired interval: -10.9 to +17.2 percentage points. Scores depend on answer format and token limits; the observed change does not establish better reasoning.
32 synthetic lookup/calculator episodes and the fixed 64-item German MGSM retention check; one stochastic trial per item.
Agent gain remains inconclusive. Descriptive 95% paired interval: +0.0 to +9.4 percentage points. Strict success includes final-answer format; it does not demonstrate improved arithmetic or tool workflow.
The candidate failed the observed no-drop MGSM gate. Descriptive 95% paired interval: -6.2 to +3.1 percentage points. This candidate is not qualified for release.
These small, fixed pilots do not establish broad German reasoning ability. Descriptive paired intervals exclude training-seed and generation-seed variance. No recorded candidate here is qualified for release. Inspect the comparison checks · Inspect agent evidence verification
Verified adapter update. No held-out quality or MGSM retention claim.
Read qualification scopeNative SDK and client exercised. No model inference or GPU training.
Read qualification scopeNative MCP tools exercised. Model inference and Prime-RL training remain unqualified.
Read qualification scopeLonger-run quality results require their own recorded assessment. This is saved evidence, not a live job or queue monitor.
04 / THE BENCHMARK PROGRAMME
Two open benchmark designs. Expert task authoring and benchmark validation remain outstanding.
Assess retrieval, citation support and abstention using German source material.
Assess final workflow state, permission constraints, approvals and action side effects.
These benchmarks are not released. The repository’s small synthetic development tracks test the evaluation pipeline.
05 / DOMAIN DIRECTIONS
Domain-specific starting points, kept as sketches until a team, dataset and metric are defined.
LOCAL SETUP
Run the evaluation demo with authored fixtures. No model, GPU or provider API key required.
Read the setup guidePrivate GitHub access and UV are required. The demo checks the pipeline; it does not measure a model.
git clone https://github.com/ChTauchmann/hessian-recipes.git
cd hessian-recipes
uv venv .venv --python python3.11
uv pip install --python .venv/bin/python -e .
.venv/bin/hessian-recipes demo --output runs/first-demo