# Architecture and implementation boundaries

This repository adds repeatable recipes and inspectable evidence around existing
training and evaluation projects. The package's core uses the Python standard
library; ML and evaluation libraries are optional, independently pinned extras.

```text
pinned data + recipe + model/tokenizer
    +--> local teacher / filtering --> curated SFT data
    +--> TRL SFT / answer-only GRPO --> checkpoint + run manifest
    +--> native tool harness / NeMo Gym HTTP resources
              --> TRL multi-turn GRPO --> checkpoint + rollout evidence

external callable --> exact-token trace contract / validation
external calculator harness --> optional Lightning / verl pilot
                                 (GPU execution unqualified)

frozen evaluation specification
    +--> replay / Transformers / callable --> outputs + report + comparison
    |                                                  |
    |                                          local release bundle
    +--> Inspect task --> native Inspect logs (no common-report importer)
```

## Implemented foundation

The CLI provides local evaluation and comparison, offline distillation data
preparation, optional teacher generation and TRL training entry points, trajectory
validation, and local release-bundle assembly. The two sample benchmark tracks
are synthetic development data. Replaying their saved answers tests the workflow
and report format; it is not evidence about a trained model.

Training calls upstream SFT/GRPO implementations rather than implementing an
optimizer. Each execution must record its configuration, model and dataset
revisions, and environment. A dry run checks and displays the plan; it does not
validate GPU compatibility or learning quality. See the training recipe files for
the supported configuration surface.

The [German Qwen SFT pilot](use-cases/german-reasoning.md) and
[native multi-turn agent-RL pilot](use-cases/german-agent-rl.md) have recorded A100
runs with matched evaluation. The optional [NeMo Gym HTTP bridge](agent-gyms.md)
also completed an eight-step, 32-rollout GPU integration qualification while TRL
retained generation, token auditing and optimization. That qualification supplies
no new held-out quality result. Prime Intellect Verifiers has a separate native
CPU fixture integration; it is not a qualified training backend here.

Evaluation uses one explicit specification per run. Results must preserve task
coverage and execution failures so a failed or missing run cannot become a high
score by dropping samples. Comparisons require compatible task and scorer
provenance. Upstream evaluators retain their native semantics and raw output; an
adapter must not relabel their scores as a different metric. Inspect has an
optional native task adapter. LM Evaluation Harness is an optional dependency
invoked through its own CLI; a common-report importer is not implemented.

The trajectory contract preserves per-call prompt/response token IDs and masks,
episode/group/policy identity, reward provenance, and the distinction between
assistant outputs and tool observations. Its validator and external callable
example are implemented; importing that JSONL directly into a training backend
is not implemented. A separate optional launcher wires an external calculator
harness into the pinned upstream Lightning trainer; source/API checks and mock
tests pass, while GPU training remains unqualified. [Contract and pilot](agent-harness.md).

## Extension boundaries

| Area | Current boundary | Promotion criterion |
|---|---|---|
| Native TRL agent RL | German Qwen GPU pilot with exact sampled-token audits and matched agent/MGSM evaluation; its retention gate failed. | Broader task/harness coverage and a repeatable held-out comparison satisfying release criteria. |
| Agent gyms | NeMo Gym resource server and real TRL GPU update; Verifiers native CPU fixtures. | Held-out quality/retention evidence for the Gym path; real inference and a compatible training pilot for Verifiers. |
| External Agent Lightning harness | Validated neutral JSONL and an optional real upstream launcher with a bounded external tool harness. | Qualified Lightning GPU execution, exact captured tokens, checkpoint update and repeatable held-out comparison. |
| Cluster containers | Generic packaging and launch scaffolding. | A pinned image successfully trained and evaluated on the target cluster, with recorded resources and logs. |
| Evaluation service | Local specifications, reports and release comparison. | Job isolation, private-data handling, access control, queueing and a measured service pilot. |
| Two open benchmarks | Development tracks and task-construction conventions. | Confirm requested topics; expert-validated tasks, provenance, held-out protocol, and overlap review. |
| Domains | Sketches in [domains.md](domains.md). | One native baseline and artifact importer per adopted pilot. |
| Hugging Face | A local release candidate bundle. | Verify the actual namespace and permissions, then add an explicit publication workflow. No organization is created here. |

## Reproducibility model

Treat model, tokenizer and chat template as one policy identity. Freeze data and
scorer versions before an adaptation comparison; record decoding and reasoning
budgets. Capture exact sampled tokens when collecting RL traces. Store
infrastructure failures separately from task outcomes, and retain raw native
outputs with summaries. Release comparisons should include repeated trials and
uncertainty where sampling or environment state changes the outcome.

A test passing on fixture data establishes a software invariant. Hardware
compatibility, score validity, and actual adaptation gains require separate
recorded experiments. The staged acceptance criteria are in
[roadmap.md](roadmap.md).