Architecture and implementation boundaries

This repository adds repeatable recipes and inspectable evidence around existing training and evaluation projects. The package's core uses the Python standard library; ML and evaluation libraries are optional, independently pinned extras.

pinned data + recipe + model/tokenizer
    +--> local teacher / filtering --> curated SFT data
    +--> TRL SFT / answer-only GRPO --> checkpoint + run manifest
    +--> native tool harness / NeMo Gym HTTP resources
              --> TRL multi-turn GRPO --> checkpoint + rollout evidence

external callable --> exact-token trace contract / validation
external calculator harness --> optional Lightning / verl pilot
                                 (GPU execution unqualified)

frozen evaluation specification
    +--> replay / Transformers / callable --> outputs + report + comparison
    |                                                  |
    |                                          local release bundle
    +--> Inspect task --> native Inspect logs (no common-report importer)

Implemented foundation

The CLI provides local evaluation and comparison, offline distillation data preparation, optional teacher generation and TRL training entry points, trajectory validation, and local release-bundle assembly. The two sample benchmark tracks are synthetic development data. Replaying their saved answers tests the workflow and report format; it is not evidence about a trained model.

Training calls upstream SFT/GRPO implementations rather than implementing an optimizer. Each execution must record its configuration, model and dataset revisions, and environment. A dry run checks and displays the plan; it does not validate GPU compatibility or learning quality. See the training recipe files for the supported configuration surface.

The German Qwen SFT pilot and native multi-turn agent-RL pilot have recorded A100 runs with matched evaluation. The optional NeMo Gym HTTP bridge also completed an eight-step, 32-rollout GPU integration qualification while TRL retained generation, token auditing and optimization. That qualification supplies no new held-out quality result. Prime Intellect Verifiers has a separate native CPU fixture integration; it is not a qualified training backend here.

Evaluation uses one explicit specification per run. Results must preserve task coverage and execution failures so a failed or missing run cannot become a high score by dropping samples. Comparisons require compatible task and scorer provenance. Upstream evaluators retain their native semantics and raw output; an adapter must not relabel their scores as a different metric. Inspect has an optional native task adapter. LM Evaluation Harness is an optional dependency invoked through its own CLI; a common-report importer is not implemented.

The trajectory contract preserves per-call prompt/response token IDs and masks, episode/group/policy identity, reward provenance, and the distinction between assistant outputs and tool observations. Its validator and external callable example are implemented; importing that JSONL directly into a training backend is not implemented. A separate optional launcher wires an external calculator harness into the pinned upstream Lightning trainer; source/API checks and mock tests pass, while GPU training remains unqualified. Contract and pilot.

Extension boundaries

Area	Current boundary	Promotion criterion
Native TRL agent RL	German Qwen GPU pilot with exact sampled-token audits and matched agent/MGSM evaluation; its retention gate failed.	Broader task/harness coverage and a repeatable held-out comparison satisfying release criteria.
Agent gyms	NeMo Gym resource server and real TRL GPU update; Verifiers native CPU fixtures.	Held-out quality/retention evidence for the Gym path; real inference and a compatible training pilot for Verifiers.
External Agent Lightning harness	Validated neutral JSONL and an optional real upstream launcher with a bounded external tool harness.	Qualified Lightning GPU execution, exact captured tokens, checkpoint update and repeatable held-out comparison.
Cluster containers	Generic packaging and launch scaffolding.	A pinned image successfully trained and evaluated on the target cluster, with recorded resources and logs.
Evaluation service	Local specifications, reports and release comparison.	Job isolation, private-data handling, access control, queueing and a measured service pilot.
Two open benchmarks	Development tracks and task-construction conventions.	Confirm requested topics; expert-validated tasks, provenance, held-out protocol, and overlap review.
Domains	Sketches in domains.md.	One native baseline and artifact importer per adopted pilot.
Hugging Face	A local release candidate bundle.	Verify the actual namespace and permissions, then add an explicit publication workflow. No organization is created here.

Reproducibility model

Treat model, tokenizer and chat template as one policy identity. Freeze data and scorer versions before an adaptation comparison; record decoding and reasoning budgets. Capture exact sampled tokens when collecting RL traces. Store infrastructure failures separately from task outcomes, and retain raw native outputs with summaries. Release comparisons should include repeated trials and uncertainty where sampling or environment state changes the outcome.

A test passing on fixture data establishes a software invariant. Hardware compatibility, score validity, and actual adaptation gains require separate recorded experiments. The staged acceptance criteria are in roadmap.md.