hessian AI & JAIF

Reproducible post-training recipes and evaluation evidence for research teams: reasoning SFT/GRPO, offline sequence distillation, agent RL, and evaluation before release. The recipes reuse upstream trainers and environments, preserve input and artifact identities, and compare adapted models with frozen baselines.

The main reading surface is hessian AI & JAIF on Hugging Face, with workflow pages, setup guides, evidence and a runnable source snapshot. The website and its source downloads are public. The Space uses Protected visibility: its Hugging Face repository remains private, as does GitHub, which retains development history and tests. Share the direct website address above; the Hugging Face repository page still requires access. The Hub guide explains the layout and institutional examples. The website serves its approved source snapshot without login. HF repository downloads previously returned an account storage-quota error, so use the private GitHub checkout for collaborative development. Weights, downloaded data and raw run artifacts remain on the cluster.

Start with the CPU demo below, then choose a workflow. The documentation guide indexes every component, example and evidence report. Current validation separates completed experiments from remaining work. The programme status maps the post-training recipe, evaluation kit and two proposed benchmarks to what is implemented and pending.

First run: evaluation without a GPU

Use Python 3.11 or later and UV. Start from the private development checkout for the complete Git history:

On JAIF, use the existing cluster checkout rather than creating another checkout. The commands below are the portable setup for other teams.

git clone https://github.com/ChTauchmann/hessian-recipes.git
cd hessian-recipes
uv venv .venv --python python3.11
uv pip install --python .venv/bin/python -e .
.venv/bin/hessian-recipes --help
.venv/bin/hessian-recipes demo --output runs/first-demo
cat runs/first-demo/summary.json

See Run from Hugging Face for the public source-snapshot layout and the separate authenticated repository route. Recipe pages and linked source files are readable without login in the Protected App.

The demo needs no model, API key or GPU. It replays authored responses for two provisional synthetic development tracks and writes JSON/Markdown evidence. These fixture scores test the pipeline; they are not model capability results. Use a fresh output directory each time. Optional training and gym dependencies are installed separately for their respective workflows.

To inspect baseline/candidate comparison with the same fixtures:

.venv/bin/hessian-recipes eval benchmarks/reasoning_reliability/spec.json \
  --output runs/first-baseline
.venv/bin/hessian-recipes eval benchmarks/reasoning_reliability/spec-candidate.json \
  --output runs/first-candidate
.venv/bin/hessian-recipes compare \
  runs/first-baseline/report.json runs/first-candidate/report.json

The comparison fails on incompatible or regressing evidence. A passing fixture comparison does not authorize a release.

Cluster checkout in JAIF

The default project directory on our Determined workers is now /pfss/mlde/workspaces/mlde_wsp_JAIF/hessian_recipes, in the JAIF workspace:

cd /pfss/mlde/workspaces/mlde_wsp_JAIF/hessian_recipes
.venv/bin/hessian-recipes --help

Start training from this checkout after obtaining the required GPU allocation; opening the Hugging Face site does not start a training job. The operations guide describes the workspace mount and shell configuration. Other teams can clone into any directory; the recipe commands use paths relative to their checkout, with model and input paths supplied explicitly.

Keep credentials outside the repository. The local JAIF credential file is in the separate tauchmann/ directory and must not enter a source bundle, run artifact or Git commit. After relocating a checkout, recreate environments and prepare fresh run plans: virtual environments and frozen plans can contain absolute paths. Preserve historical manifests and result evidence unchanged.

Choose a workflow

For the German model path, start with SFT, then choose local agent RL or NeMo Gym training. Both recorded RL runs start from the original SFT checkpoint; the Gym qualification does not continue from the earlier RL candidate. Read the recorded results before starting a new run. Data preparation and planning can run on a CPU node; actual Qwen training and model evaluation require an allocated GPU. The guides give the exact commands and pinned environments.

The longer NeMo Gym experiment and matched OpenEnv arm preregister sealed native recovery, DEV-only checkpoint selection and full MGSM assessment. As of 21 October 2026, both 20 October GPU attempts stopped at the strict token audit during resume qualification; neither reached the 256-step run or held-out assessment. There is no new longer-run quality result. Prepare fresh JAIF plans, verify the recreated environments and pass each backend's actual GPU resume qualification before further long training. See operations for current submissions and the prerequisites of legacy sequential controllers.

What the completed runs establish

The repository is a working research foundation. The evaluation service is a future managed-execution layer over the local kit. The proposed benchmark topics are German RAG with cited evidence and business-communication agent sequences; the existing starter fixtures are not those benchmarks. Domain implementations remain sketches. This work has not created a hessian.AI Hugging Face organization. No recorded candidate is qualified for release. See the roadmap for the remaining acceptance work.

Environments and saved artifacts

Code, specifications, configurations and aggregate evidence are versioned here. Checkpoints, adapters, downloaded data, raw model outputs and full run logs stay in ignored runs/ directories on shared storage. The result guides identify the recorded run directories and model cards. A fresh clone contains the recipes and aggregate evidence, not the trained weights or the original cluster filesystem.

Design, reuse and development

The architecture explains the evidence contract and backend boundaries. The related-work map records academic and industrial sources, reuse decisions, exact integration pins and research-only extensions. The documentation guide links the full source and example layout.

Run the repository checks in the core environment:

uv pip install --python .venv/bin/python -e '.[dev]'
.venv/bin/python -m pytest -q
.venv/bin/python -m ruff check .
.venv/bin/python -m build

CI checks Python 3.11/3.12 and the optional Inspect adapter separately. A manually dispatched CPU ML smoke exercises the pinned trainer stack. Native gym checks use their isolated environments; GPU runs are recorded separately from CI. New recipes need pinned inputs, an executable verifier and measured evidence at the validation level they claim. A successful launch alone is insufficient.

The supplied survey motivates the scope: 13/43 teams want reasoning post-training, 13/43 distillation, 47/70 evaluation before release, and 36/70 would use an evaluation service. These are planning figures supplied for this project.

The GitHub and Hugging Face repositories are private; the website and its approved source snapshot are publicly accessible. No public source-code or data license has been selected. Access to source files does not itself grant a license. Review upstream code, model, dataset and teacher-output terms separately before distributing derivative artifacts. Model release bundles remain local candidates. Recipe hub publication explicitly targets the existing Protected Space.