Validation evidence

The repository now includes completed CPU checks and bounded A100 training runs. The initial JAIF CPU environment check on 21 October passed 500 tests, with eight optional checks skipped, using .[dev,inspect]. Subsequent deployment checks are included in the current CI runs. Ruff, the offline demo, package builds and the recipe website build also pass from the JAIF checkout. Earlier checks in the pinned ML environments passed the upstream scalar optimizer/RNG roundtrip and four saved BF16 tensor audit cases; five optional Verifiers tests passed separately. Those earlier checks do not qualify the recreated JAIF GPU environments. A fresh install and CPU demo were also checked from the earlier HF source snapshot. These counts describe software checks, not model accuracy or release readiness. Use the documentation index to find protocols, results and limitations.

Current measured scope

Completed work	Evidence and practical limit
Private Hugging Face reading experience, 21 October	The research overview, methods index, source-bound interactive results and article reader, using official hessian.AI branding, were checked in Chromium at 62 desktop/mobile page-viewports. Final automated accessibility checks report zero violations; targeted 320/768px reflow, keyboard scrolling, code copy, search/filtering, mobile navigation, contents links and no-JavaScript reading pass. These are UI checks, with no model inference. See interface evidence.
German Qwen SFT	LoRA SFT completed 128 optimizer steps on 1,024 German explanation examples, followed by merge and matched evaluation on 64 MGSM German items. Strict final-answer protocol accuracy changed from 28/64 to 30/64; the paired result is inconclusive. A ten-item agent inspection supports a language/format change, not German fluency or improved arithmetic. See results and aggregate evidence.
Local multi-turn agent RL	Native TRL GRPO completed 16 steps and 64 training rollouts with exact sampled-token/context checks and external-token loss masking. Matched strict workflow success changed from 4/32 to 5/32; MGSM retention fell from 30/64 to 29/64 and failed the zero-drop gate. This qualifies an executable path, not improved general agent ability or a release. See results and aggregate evidence.
Native NeMo Gym online training	The actual pinned resource server and remote client completed eight A100 GRPO steps and 32 rollouts from the original German SFT checkpoint. All 392 trainable adapter tensors changed; independent replay checked 71 tool observations and masks for 7,955 assistant and 1,648 external tokens. Strict training success was 5/32. No held-out agent or MGSM retention evaluation was performed for this integration qualification. See gym protocol and limits and aggregate evidence.
Native gym CPU checks	NeMo passed 11 service and nine client checks. Prime Verifiers passed seven scripted native lifecycle scenarios and five optional tests. The Verifiers fixture does not perform model inference or training; its Prime-RL GPU path remains unqualified. See gym evidence.
NeMo stress and recovery checks, 20 October	Actual pinned service: 32 concurrent isolated episodes, 260 sequential reset/solve episodes, the 256-active-session bound and cleanup recovery, service restart with sticky old-session failure and independent fresh recovery. Actual TRL tool dispatch was exercised with scripted outputs; invalid dispatch can retain partial reward while strict success fails. No model inference or learning. See aggregate stress evidence.
Native OpenEnv CPU qualification, 20 October	Twelve actual SDK/service checks passed, including 32 simultaneously open WebSocket sessions, rejection/recovery at the configured 32-session bound, 260 sequential resets, malformed requests and poisoned Python environment isolation. The actual SDK client passed strict tool/replay/isolation checks and sticky service-loss handling; 24 native tests and 27 client/wrapper tests pass separately. The 32-test matched driver and 21-test offline queue controller pass. The later 20 October GPU attempt stopped at the strict token audit; GPU resume equivalence and model assessment remain unqualified. See aggregate evidence and matched protocol.
Longer NeMo/OpenEnv attempts, status 21 October	Fixed-horizon native resume, deterministic DEV selection and CPU optimizer/scheduler/RNG checks pass. Both 20 October GPU attempts failed during the initial continuous resume-qualification run at the strict sampled-token/context/tool-mask audit. Neither established GPU resume equivalence or reached 256-step training and held-out assessment. New JAIF plans and runtime qualification are required before another attempt. See NeMo protocol and matched OpenEnv arm.

The three completed GPU pilots retain private weights and raw evidence under ignored runs/ directories; committed results contain aggregate measurements and evidence hashes. The NeMo qualification does not change the earlier agent-RL quality scores or resolve its retention regression. No adapter or model has been published, no Hugging Face organization has been established by this work, and no hosted evaluation service is implemented. The private personal Hugging Face recipe App and links-only collection are live; publication evidence records access and source checks, including the repository-download quota limit. No models or datasets are published by this workflow. The two development tracks remain provisional synthetic fixtures; robotics, Bio/Med and materials remain sketches.

Historical foundation checks

The initial foundation was checked at commit 8bc44ee on 15 October 2026, Linux x86-64, Python 3.11.9. Its 73-test snapshot includes actual Inspect mock execution, Ruff, package builds and dependency consistency checks. This historical count is retained in validation.json; it is not the current suite total or a record of the subsequent GPU runs.

Check	Evidence and practical limit
Core CLI and unit tests	Offline replay, coverage/error handling, integrity checks, compatible comparisons, category regression rejection, filtering, trajectory validation, and candidate-bundle verification.
Real Inspect execution	The actual pinned Inspect SDK executes the task with a local mock provider; controlled outputs score correctly. No external model API is called.
Teacher generation	Transformers generates locally from a random tiny GPT-2 checkpoint and saved tokenizer. Outputs and input identities are recorded; no teacher-quality claim.
Actual SFT step	TRL completes a step and saves/reloads a checkpoint; parameters differ from the initial model. Recorded loss approximately 2.72 and gradient norm 3.14.
Actual GRPO step	TRL samples 32 one-token completions with varied rewards; reward mean 0.03125, reward standard deviation approximately 0.1768, gradient norm approximately 1.528, and parameters change. The initial scalar loss is zero because group advantages average to zero; it does not imply a zero gradient.
Local evaluation	Real CPU generation from baseline and SFT checkpoints uses one frozen tokenizer. Complete execution and compatible evidence are checked. Tiny random-model outputs are not useful task results.
Agent Lightning pilot	Actual pinned upstream checkout passes source/API checks; the bounded external tool loop passes mocked tests. No gateway, Ray training, GPU stack, or distributed optimizer run was executed.
Packaging	Python source distribution and wheel build successfully; the CLI is installed in isolated environments.
Slurm script	Bash syntax is checked. No cluster job is submitted.

The machine-readable foundation summary records that historical checkpoint's counts, source identities and selected measurements. Model weights and generated logs are not added to Git. The dependency snapshots in requirements/training-cpu.txt and requirements/inspect.txt describe the separate environments, including developer tools. They are observed exact-version captures, not cryptographic package locks.

Repeat the checks

Use a clean environment from the repository root:

python -m pip install -e '.[dev]'
python -m pytest -q
python -m ruff check .
hessian-recipes demo --output runs/replay-check
python -m build
bash -n scripts/slurm-train.sbatch

The core suite skips the Inspect integration when its optional dependency is not installed. In a separate environment, install .[dev,inspect] and rerun the suite to exercise the actual mock-model integration. To recreate the observed Inspect environment, use pip install -r requirements/inspect.txt -e ..

For the actual CPU training and local inference path:

python -m venv .venv-training
.venv-training/bin/python -m pip install -r requirements/training-cpu.txt -e .
HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 \
  .venv-training/bin/python examples/training/cpu_smoke.py --output runs/cpu-check
HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 \
  .venv-training/bin/python examples/evaluation_cpu_smoke.py \
  --baseline runs/cpu-check/tiny-model \
  --candidate runs/cpu-check/sft/checkpoint \
  --tokenizer runs/cpu-check/sft/checkpoint \
  --output runs/evaluation-cpu-check

Choose fresh output paths. The local evaluation example permits a full score drop deliberately: it tests checkpoint/tokenizer compatibility and execution coverage, not a quality threshold. Production thresholds require representative tasks.

The commands above reproduce foundation checks. The actual GPU protocols and their separate dependency environments are documented in the German SFT use case, local agent-RL use case and RL agent gyms. Use their bounded budgets and fresh output paths when reproducing those runs.

Still unqualified: the generic Slurm/container recipes at a deployment site, actual model checkpoint recovery after interruption, multi-GPU or multi-node jobs, a real Lightning/team-harness training update, Prime-RL training through Verifiers, expert benchmark validity, native domain models and hosted evaluation services. The GPU runs record image tags but lack immutable image digests. Useful adaptation with retained capabilities and production release conditions remain acceptance targets in the roadmap.