# Related work and reuse decisions

Primary sources checked on 15 October 2026. This is a build-versus-reuse map, not a
claim that a single framework or model is currently best. Most commit links below
are research snapshots; a link alone is neither a dependency lock nor installation
evidence. Explicitly integrated runtime pins live in `pyproject.toml`,
`requirements/` and the corresponding setup instructions. Qualification is stated
per integration, and a recipe needs a successful hardware run before promotion to
a validated cluster recipe.

The [OpenEnv/NeMo/FineEnvs comparison](environment-frameworks.md) and
[German RAG/workplace benchmark designs](benchmark-designs.md) add primary-source
checks from 20 October 2026. These references do not change the frozen runtime pins.

## The smallest useful stack

| Layer | Academic / community and industrial work | What we build on | Status here |
|---|---|---|---|
| Reasoning adaptation | [TRL](https://github.com/huggingface/trl/tree/9d76f2f8c0421fcf8a4a72113a9e226ce796e592), Hugging Face | Existing SFT and GRPO trainers; our contribution is the recipe, data provenance, and evaluation evidence. | CPU SFT/GRPO checks and real German Qwen A100 SFT/agent-RL pilots completed; [measured results and limits](use-cases/german-agent-rl-results.md). |
| Offline distillation | [OpenThoughts](https://github.com/open-thoughts/open-thoughts/tree/e86cf2dbbe69ae178aec302be33a3454a7183457), DataComp / Bespoke Labs collaboration | Transparent teacher-data curation and ablations. We implement a small generation/filtering path, not the complete OpenThoughts pipeline. | Implemented local preparation and optional generation; reference methodology. |
| Recipe collection | [NeMo Skills](https://github.com/NVIDIA-NeMo/Skills/tree/bcf059af55c20a89f797724598f9908d126153e6), NVIDIA | Synthetic data → training → evaluation and local/Slurm conventions. Upstream is moving rollout and benchmark implementations into Gym. | Architectural precedent; no dependency. |
| Existing agent harness RL | [Agent Lightning](https://github.com/microsoft/agent-lightning/tree/69f5a40b36d9bff646b8e7a5532e8b78bd41eaf9), Microsoft Research | Gateway capture with the deployment harness and a verl-backed trainer. | Optional real launcher and external calculator harness; source-checked and mock-tested, GPU execution unqualified. |
| Reusable environments | [NeMo Gym](https://github.com/NVIDIA-NeMo/Gym/tree/720b5f318caee339bfd29c554c59a97edcedc846), NVIDIA | Environment lifecycle and authoritative tool verification. | Actual CPU service/client checks and an eight-step A100 TRL bridge qualification completed; [setup, evidence and limits](agent-gyms.md). |
| Reusable environments | [OpenEnv 0.9.0](https://github.com/huggingface/OpenEnv/tree/17c1edd22168fcb76b086def1ed90a7168fb8749), Hugging Face | Typed stateful environments, persistent SDK WebSocket sessions and trainer adapters. The official [reasoning walkthrough](https://huggingface.co/docs/openenv/tutorials/end-to-end-walkthrough) explains Qwen plus GRPO. | Native service and SDK/TRL bridge CPU checks pass; [matched German arm](use-cases/german-openenv-comparison.md) retains the frozen TRL 0.29 API and protocol. GPU qualification and quality tests pending. |
| Reusable environments | [Verifiers 0.4.0](https://github.com/PrimeIntellect-ai/verifiers/tree/6e69354ffe1b445603b493b722d0ecffbca6686d), Prime Intellect | Native tasksets, harnesses, MCP tools and reward lifecycle. | Optional German environment in an isolated stack; scripted native execution is distinct from model training. |
| Distributed RL | [verl / HybridFlow](https://github.com/verl-project/verl/tree/5b550b0c3dad8c7bb7aab834a3fe195f8cdfc517), ByteDance / academic-community collaboration | Distributed optimizer and inference scheduling rather than a new trainer. | Extension backend through a future Lightning pilot. |
| System and custom evaluation | [Inspect](https://github.com/UKGovernmentBEIS/inspect_ai/tree/08f8600926ee7383c0b582faada2640c6c6f0740), UK AI Security Institute | Existing evaluation SDK and task implementations. | Optional adapter; actual pinned SDK mock execution passed. |
| Conventional model evaluation | [LM Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness/tree/d6de81643928d653435c431bae19945d41d32520), EleutherAI | Existing task catalog and model interfaces. | Optional dependency for native upstream CLI use; no report importer implemented. |
| Evaluation methodology | [HELM](https://github.com/stanford-crfm/helm/tree/63754d05db6f874e41a395880fb573890a13e791), Stanford CRFM | Transparent scenarios, multiple metrics, and reproducible reporting. | Methodological reference, no dependency. |

[OpenThoughts: Data Recipes for Reasoning Models](https://arxiv.org/abs/2506.04178)
is the principal academic data-recipe reference. The
[Agent Lightning v1.0 paper](https://arxiv.org/abs/2608.17528) is the closest
systems reference for preserving an existing harness. These works motivate
controlled baselines and data/trajectory contracts; their reported results do not
predict gains for participating teams.

## Agent Lightning versus NeMo Gym

Start with **Lightning + verl** when a team must retain an existing orchestration
loop. Its current upstream design separates the model gateway, rollout execution,
and GPU training. See the [architecture documentation](https://microsoft.github.io/agent-lightning/stable/05-basics/).
Use **Gym** when reusable task environments and verifiers are the
main shared asset. Its current repository documents environment servers, external
harnesses, token capture, and Slurm-oriented execution. These are upstream
capabilities. This repository now has a concrete native Gym resource server and
external TRL bridge; its narrower checks and limits are documented in
[RL agent gyms](agent-gyms.md). Token capture in that bridge remains the existing
TRL audit rather than an asserted upstream Gym capture feature.

Do not install both distributed stacks in the default environment. Freeze one
representative task, policy, reward, and compute budget; compare trace fidelity,
recovery after failures, environment isolation, and time to a first reproducible
update. The concrete pilot and its acceptance criteria are in
[agent-harness.md](agent-harness.md).

## Benchmarks: reuse before inventing

The supplied programme proposal now targets German RAG with cited evidence and
business-communication agent sequences. The included reasoning and tool-workflow
tracks remain synthetic development fixtures. See the [benchmark designs](benchmark-designs.md)
for existing German RAG meta-evaluation, citation metrics, workplace simulations
and tool-injection suites to reuse; neither new programme benchmark is implemented.

For interactive workflow design, study Sierra's
[τ²-bench](https://github.com/sierra-research/tau2-bench/tree/5ade54e282a4555e4d6c52d6b9b861ea8872afa6):
evaluate task state and user/tool interaction rather than answer text alone. For
physical materials predictions, use the native
[Matbench Discovery](https://github.com/janosh/matbench-discovery/tree/6ffd8070e0a7133a795a9bd4b42ec8dd9e543a46)
protocol rather than converting the problem into LLM question answering.

Each proposed public benchmark must justify a decision existing evaluations cannot
support, include expert review and contamination controls, separate public
development data from held-out evaluation, and measure reliability across repeated
trials. An upstream track is preferable when the gap fits an existing benchmark.

## Domain-native references

These are sketches only; there are no working domain adapters in this release.

| Domain | Reference snapshot | Reuse boundary |
|---|---|---|
| Robotics | [LeRobot](https://github.com/huggingface/lerobot/tree/297774fca4e3ad0aa34e7425fbf937111637e77b), Hugging Face | Robot datasets, policy adaptation, and native episode evaluation. |
| Biomedical agents | [Biomni](https://github.com/snap-stanford/Biomni/tree/400c1f366b96a35ca253e13c9b06c5076af41d65), Stanford | Tool-rich biomedical harness reference; not an RL optimizer. |
| Medical text / image adaptation | [MedGemma](https://github.com/google-health/medgemma/tree/a60a66024f6153496dfa9490dba12d4f1fbd092e), Google Health | Model-specific adaptation examples and intended-use terms. |
| Biological prediction models | [BioNeMo Recipes](https://github.com/NVIDIA-BioNeMo/bionemo-recipes/tree/e1563ee6e4c31ba03e3ffc09e6e46f76f19468d2), NVIDIA | Native biological model workflows and recipe packaging. |
| Materials / atomistic models | [MACE](https://github.com/ACEsuit/mace/tree/7913afb6fa729878d20886b84a2079523727d958), academic-community project | Equivariant interatomic potentials, native training and simulation evaluation. |

The proposed experiments, artifact contracts, and domain-specific evaluation
limits are in [domains.md](domains.md).

## What remains research-only

These additional references broaden the experimental choices without becoming
dependencies:

| Reference | Useful experiment or methodology | Status here |
|---|---|---|
| [AllenAI open-instruct](https://github.com/allenai/open-instruct) | Transparent SFT, preference optimization and RL with verifiable rewards. Its maintainers direct evaluation users to OLMES rather than the unmaintained internal evaluations. | Related work only. |
| [OpenThoughts-Agent](https://github.com/open-thoughts/OpenThoughts-Agent) | Data recipes and infrastructure for agent trajectories; inspect trajectory curation and controlled comparisons. This is evolving research code. | Candidate for agent-distillation research; not integrated. |
| [s1](https://github.com/simplescaling/s1) | Small curated reasoning-SFT baseline and test-time budget control. | Baseline reference; no current best-model claim. |
| [Open-R1](https://github.com/huggingface/open-r1) | Open reasoning adaptation/data recipes around the R1 approach; not a guarantee of reproducing the original private training process. | Baseline reference, not a dependency. |
| [SDPO](https://github.com/lasgroup/SDPO), [paper](https://arxiv.org/abs/2601.20802) | Learn from rich environment feedback through feedback-conditioned self-distillation; compare against scalar-reward RL at matched compute. | Research-only extension. |

On-policy distillation, rich-feedback self-distillation, asynchronous distributed
RL, and multi-agent credit assignment are extension topics. They are not synonyms
for the implemented offline sequence-distillation path. Require a matched SFT
baseline, frozen test protocol, and measured cost before promoting any additional
algorithm. New upstream claims and API changes must be checked at the pinned
revision rather than inferred from an evolving README.