Agent Training and Environments

Train completion, not a convincing account

An agent acts on a world through tools. A good-looking final response does not establish that the agent changed that world correctly. Training needs evidence from the resulting state: tests pass, a file is valid, a workflow finishes, or an independent checker confirms the requested change.

For a repository repair, the initial state includes a broken revision and its dependencies. The goal describes the required behavior. The verifier checks the modified repository. The agent can use several valid patches; training need not prescribe a single reference trajectory.

This makes environment construction part of the learning algorithm. A broken dependency can turn a valid patch into an apparent failure. A leaked answer can turn an invalid policy into an apparent success. Both change the reward distribution before the optimizer receives anything.

Separate task, policy, harness and environment

A harness is the agent program around the model. It can change the tools, observation order and context without changing model weights. A sandbox isolates execution; it does not itself define task correctness. Record these boundaries so an infrastructure failure is not mislabeled as a policy error.

For a CSV-script repair, one harness may offer edit_file(path, content), while another requires a shell patch. Separate an incorrect repair from a correct intended edit expressed through an unsupported tool. Better format validity can improve completion without the same gain in programming reasoning.

The policy-gradient machinery belongs to Reinforcement Learning. Here the focus is the evidence and execution contract that makes those gradients meaningful.

Construct a task and its evidence together

Start with a grounded seed: a real issue, a document bundle, a service specification or an observed capability gap. Construct the initial assets and task goal. Build the verifier alongside them, then validate the verifier independently. Co-generation makes the parts consistent; it can also create correlated mistakes.

Execute several agents against the task. Compare actual outcomes with the stated requirements. Reject unbuildable environments, tasks solved by leaked artifacts, and checkers that accept unintended shortcuts. Repair legitimate failure cases instead of retaining every generated example.

Separate task acceptance from trajectory acceptance. An excellent task may produce bad demonstrations. A successful trajectory may contain an unreliable turn that should not be an imitation target. Feed validated failures back into task generation to expand coverage rather than merely increasing dataset size.

Rebuild the state before the fix

Historical repository changes supply an executable before-and-after relation. Reconstruct the parent commit before the fix, install the dependencies, and expose the issue without the hidden reference patch. Package that state in an isolated environment with reproducible setup instructions.

The hidden patch helps validate the task factory. It is not an agent-visible answer. Remove accidental solution material from Git history, caches, build artifacts and instructions. A task whose tests already pass at the initial revision does not demonstrate the requested repair.

Keep repository identity, revision, dependency versions, setup failures and test commands in provenance. Split evaluation by repository or time where appropriate. Near-duplicate changes from the same project can otherwise make held-out tasks look independent when they are not.

Require both necessity and regression safety

Suppose a task fixes empty-input handling. The empty-input test must fail initially and pass after repair. A normal-input test must pass at both revisions. Checking only the first condition rewards a patch that fixes one case while breaking ordinary behavior.

Passing these tests is evidence under their coverage, not a proof of all intended semantics. Hidden edge cases and an independent inspector can find exception swallowing, hard-coded answers or unrelated rewrites. Preserve the distinction between an invalid task and a valid task with an incorrect solution.

Choose what the tool world makes reproducible

A model-simulated observation is not equivalent to executing the action. Validate schemas, state transitions and impossible outputs. Mock a live service when reproducibility matters, but test whether the mock preserves the failure modes relevant to the task.

Store service and tool versions. Timeouts, authentication failures and transient rate limits need an explicit classification. Do not silently score every failed request as a wrong model decision.

Local worlds can become stale; live worlds can change after a task is constructed. Store the capture time as well as the version. Choose the mixture for the behavior that must transfer, not simply the easiest world to scale.

Kimi K2: scale tool coverage without confusing simulation and execution

A tool schema describes an action. It does not prove that the action changed the world. Kimi K2 combines real MCP tool specifications with synthetic specifications, generates agents and rubric-backed tasks, then simulates stateful conversations. A judge filters trajectories. Real execution sandboxes complement simulation for coding and software engineering.

For example, a booking tool can return valid JSON without reserving a seat. A useful verifier checks the reservation state, duplicate bookings and failure handling. This is an illustrative test design, not a reported Kimi experiment.

Keep three outcomes distinct: verified success, verified task failure, and invalid execution. INTELLECT-3 masks coding completions when their sandbox fails instead of treating the failure as a wrong solution.

Infrastructure failures can depend on trajectory length or tool choice. Masking protects the label but can still bias accepted data. Log invalid rates by task family and rerun independently where practical. For simulated tools, audit state invariants and test held-out real tools. These are proposed checks, not measured guarantees from either report.

Kimi K2 §3.1.1: agentic data synthesis ↗ · INTELLECT-3 §3.1.2: code verification ↗ · Choose imitation targets →

Compose dependencies, not a tool inventory

Model Context Protocol (MCP) specifications and API descriptions can provide tool schemas. Useful tasks require meaningful dependencies among tools. Searching for a customer, reading that customer's orders and updating the selected order is a workflow; listing hundreds of unrelated endpoints is not.

Construct tasks that require earlier outputs as later arguments. Validate entity identity, parameter constraints and state consistency across calls. Include recoverable errors, pagination and ambiguous matches when those behaviors are grounded in the interface.

Nanbeige4.2 describes thousands of MCP specifications, implemented through local Python, live MCP services and model-simulated components. That count describes available specifications, not independently validated tasks or universal tool competence.

Verify the delivered artifact

Office tasks begin with actual document, spreadsheet, slide or PDF assets. A request can combine them: read a source sheet, compute a summary and place it in a presentation. The deliverable is the file bundle, not an explanation of what the agent intended to produce.

Check structure and content separately. A spreadsheet formula can exist but have stale displayed values. A slide deck can contain the right words but clip its labels. Recalculate numerical outputs, inspect rendering and verify agreement across files.

Keep input assets and expected invariants together. A generated artifact may serve as a later input only after independent validation. Provenance should identify the source bundle, transformation and checks that established acceptance.

For synthesis, collect artifacts across domains, cluster by topic and structure, and sample coherent bundles. Let the task generator inspect the bundle before writing the goal and rubric. Then run the agent in isolation and judge both the delivered files and the useful trajectory. Input-file diversity and executable checks serve different purposes.

Inspect what the rendered result does

Visual agent work introduces another observation channel. The agent writes or edits, renders or runs the result, inspects what appears, receives a judgment and revises. A successful source-code edit can still produce an unusable visible interface.

GLM-5.3 describes frontend RL around real user flows, layout, interaction and GUI-based feedback. The report scope does not disclose enough to infer environment counts, exact reward formulas or verifier accuracy. Visual quality claims need the evaluation harness and judge conditions, not only the checkpoint name.

Recycle outputs without recycling hidden errors

Validated outputs can enlarge the initial-asset pool. A completed report becomes an input for a consistency task; a repaired workbook becomes a base for a new analysis. This expands workflows without generating every asset from nothing.

Recycling also propagates hidden defects. Independent checks, provenance and human-source anchoring limit that drift. Deduplicate related descendants and prevent an output or its hidden solution from entering both training and evaluation.

Judge diversity as well as volume. Repeated synthetic formatting and assumptions can teach a narrow agent style. Real documents, grounded specifications and adversarial failure cases help reveal that narrowing.

Expand tasks through grounded dependencies

Kimi K3 describes hierarchical task construction through a knowledge graph. Retrieve grounded material from a seed, expand it into atomic requirements, reuse validated nodes, and combine dependencies into larger tasks. A directed acyclic graph makes prerequisite relationships explicit.

The graph should encode actual dependence, not topic proximity. If step B needs the result of A, record that edge. If A and B merely discuss related subjects, forcing an order invents a prerequisite. Preserve source links and deduplicate reused nodes.

Combined tasks need fresh end-to-end validation. Correct atomic nodes do not guarantee a valid composite: their file versions, assumptions or tool states may conflict. Task difficulty can grow through dependencies while final verification remains independent.

Expansion stops at sufficiently atomic concepts. Reuse grounded nodes, then sample linked concepts across levels. A composite can require general and specialized knowledge, text and vision, or several dependent tools. This is inspectable coverage, not a guarantee that the generated prerequisite graph is correct.

Allow many valid paths; constrain final verification

K3's Autonomous Execution Tasks specify a goal, an initial environment, available operations, budgets and independent verification. They need not provide a reference execution path. Several action sequences can reach a correct final state.

Separate public diagnostics from hidden final checking. Public feedback can explain an invalid file or missing field. Hidden criteria evaluate completion without becoming an unrestricted answer oracle. Limit submissions and record diagnostic exposure.

Budgets are part of the task definition. An unlimited agent with unlimited checker access solves a different problem from a bounded agent. Avoid rewarding repeated hidden-check probing as ordinary progress.

Mine capability gaps, not maximum difficulty

Mine failures from seed agents and classify the missing capability. Useful axes include horizon, tool depth, parameter complexity, ambiguity, cross-artifact consistency, recovery and verification. Length alone is not a complete difficulty measure.

Increase difficulty where the policy has enough competence to produce useful contrast. A task that nearly every attempt fails can produce little outcome differentiation. A task that every attempt passes can still teach quality through a suitable grading signal.

Compact agents may benefit from shorter solvable tasks rather than the hardest available environments. Nanbeige4.2 uses relatively high pass@8 and short trajectories in its agent RL selection. Keep those conditions attached to that recipe; they do not define a universal curriculum.

Do not discard every recovery trace to make the data look clean. Perfect-only demonstrations remove examples of responding to realistic tool errors.

Keep failures as context, not targets

A successful trace can include an unsupported tool call followed by a valid correction. Retain the unsupported action and its error observation in the input history. Mask the bad assistant action from supervised targets, then train the corrected action under that real history.

The distinction is between conditioning tokens and target tokens. Prompt and tool-observation tokens can affect the next prediction without receiving assistant imitation loss. Deleting the mistake changes the conditional problem and may hide the recovery skill.

GLM-5 retains recoverable unsuccessful turns in long-context SFT data with selective masking. Nanbeige4.2 filters malformed, looping, redundant or truncated action targets while keeping useful context. Decide which turns are unreliable; do not mark the entire successful trajectory correct by default.

Vary the interface while keeping the goal fixed

The same model can behave differently in different agent programs. Tool names, system instructions, context folding, memory, skills and subagents alter what the model observes and how an action is interpreted. Multi-harness training varies these interfaces while sharing policy weights.

K3 uses configurable white-box modules for tools, system prompts, context, skills, memory and subagents. Controlled variation helps identify whether a failure comes from task reasoning or interface adaptation. Version the modules; a silent prompt or tool edit changes the training distribution.

Different scaffolds can expose broad exploration, localized search, direct editing, test-first debugging and different recovery paths. The desired transfer is task competence across interfaces. A stronger average over the training harnesses does not guarantee competence on each harness or an unseen one.

Separate task transfer from interface transfer

An unseen task is not automatically an unseen harness. A model trained with all evaluation interfaces can improve on held-out tasks without demonstrating transfer to a new agent program.

Hold the task pool, budgets and verifier fixed when comparing harnesses. Record prompt templates, tools, context policy and stopping rules. Otherwise a score difference may reflect more opportunity rather than better interface robustness.

Capture the action the model actually sampled

A training proxy can intercept model requests from an unmodified agent program. It translates API dialects, sends the effective prompt to an inference engine, and records the engine's exact prompt IDs, generated IDs and behavior log probabilities. The capture boundary must include the prompt the engine actually used.

Decoding generated IDs and re-encoding the text need not recover the sampled sequence. Tool-call reformatting can change both tokenization and the conditioned history. Token-in/token-out (TiTo) training preserves the action sequence instead of reconstructing a plausible transcript.

Prompt tokens and tool observations do not become policy targets merely because they are in a captured call. Declare completion masks and retained branches. An endpoint lacking the required token evidence can support evaluation without supplying a valid training capture.

OpenEnv's Harbor capture documentation ↗ describes this boundary.

Keep the collection distribution and version

Temperature, top-k and top-p processing can change the distribution that sampled the action. Record the processed sampling probabilities and support. Raw-logit probabilities are not interchangeable with probabilities after those transformations.

Asynchronous collection adds policy versions. A long episode may cross a synchronization boundary or resume under newer weights. Track the version associated with each segment. Replay and importance corrections cannot recover evidence that was never stored.

See RL's sampling and stale-policy contracts for estimator details. An environment transcript alone is insufficient to reconstruct a trustworthy policy update.

Track branches and rewritten contexts

Agent programs can branch, retry or compact context. Captured calls therefore form a graph, not necessarily one uninterrupted conversation. Exact prefix matches can link calls; a rewritten context can begin a new root.

The graph records what was conditioned on and which completions remain in a selected learning path. It does not automatically decide reward propagation. An abandoned branch needs an explicit credit rule rather than inheriting the final answer's success by convenience.

Keep the original IDs even when presenting a shorter readable transcript. Context compaction may preserve semantic information while replacing the exact policy-conditioning state.

Compare like opportunities within a group

Group-relative training compares several attempts under a declared common opportunity. FineEnvs' example uses the same task and harness within each group, with diversity between groups. Mixing arbitrary tasks inside one mean changes the comparison.

Store task, harness, seed, budget and group identity. A slow incomplete group requires an explicit policy: wait, pad under defined conditions, or discard. Do not silently substitute an unrelated completed response to fill the group.

Harness variation within a group is possible as a different design, but unequal interfaces can confound quality and opportunity. Establish comparable action budgets and observations before interpreting relative rewards.

Define the unit of contribution

Equal episode counts do not imply equal token counts, training rows or loss contribution. A long harness can produce more completion targets per episode. The chosen loss reduction decides whether those extra tokens change its total scalar coefficient mass.

Even equal coefficient mass does not establish equal gradient influence. Advantages, clipping, masks and gradient directions can differ. Log the unit you balance and several resulting distributions instead of calling the mixture equal without qualification.

Mix harness families explicitly

Harness weighting is a coherent experimental extension. Group related interfaces into families, select a family, then select a harness within it. This separates coverage across interaction styles from redundancy among nearly identical wrappers.

Choose whether the weight applies to task selection, completed episodes, target tokens or the objective coefficient. Applying weights at both sampling and loss stages changes the effective mixture again. Availability and latency can prevent the requested sampling weights from matching accepted training shares.

Potential experiments include uniform families, capability-gap weights, novelty weighting and bounded adaptive allocation. Keep a minimum exposure floor and evaluate held-out interfaces. These are proposed adaptations, not verified FineEnvs results or a demonstrated best weighting rule.

Do not let latency choose the curriculum

An asynchronous learner consumes completed work. With equal worker counts, a faster harness completes more episodes before an update. The accepted mixture can therefore differ from the dispatched mixture even when task selection is balanced.

Use completion quotas, deficit-aware scheduling or adaptive concurrency when a target mixture matters. Track dispatched, completed, filtered and accepted shares separately. Token budgets add another axis: equal completed episodes may still give unequal generated-token exposure.

Latency bias and policy staleness are distinct. A balanced set of slow episodes can still be too old. A fresh set of fast episodes can still omit important task families.

Reward efficiency only after correctness

A fast wrong answer should not receive a tool-efficiency reward. Gate any bounded bonus by task correctness.

This is a teaching formula, not the exact FineEnvs implementation. Define efficiency relative to the task, budget and tool costs. Fewer calls can mean better planning, but also skipped verification or poorer exploration.

Compare cost at matched success rates and inspect common successful tasks separately. A mean tool-call count over all episodes can fall simply because the policy gives up earlier.

What the FineEnvs comparison does and does not show

FineEnvs' historical example trains LFM2.5-2.6B with asynchronous GRPO across OpenCode, Claude Code, Codex and Mini-SWE-Agent. Groups contain eight attempts under one task and harness. At step 1,000, mean pass@1 moves from 42.2% to 54.2%; OpenCode-only training reaches 52.3%.

The 1.9 percentage-point difference between training regimes is reported as within noise. The experiments use one seed, unequal data exposure and different compute. All four evaluation harnesses are included during multi-harness training. This comparison supports improvement over the base, not held-out-interface transfer or a universal mixture ranking.

Tool calls fall by about 31% on tasks that both the base and multi-harness models solve. That common-success condition matters. No LFM ablation isolates the efficiency bonus. The result does not show that fewer calls cause higher correctness.

FineEnvs multi-harness guide ↗ · Experiment section ↗ supplies the experiment. Its preserved comparison does not establish equal per-task completion counts or a harness-ranking weight scheme.

A solved task can still have useful quality differences

Binary correctness is necessary but coarse. For rewards [1,1,1,1], centered group advantages are all zero. The solutions may still differ in edge-case coverage, unwanted side effects, maintainability and unnecessary work.

MiMo's Groupwise Reward Synthesis (GRS) creates quality-sensitive rewards under a correctness gate. Groupwise Advantage Redistribution (GAR) reallocates existing positive binary advantage. These solve related but different problems: GAR cannot create missing binary credit in an all-pass group.

Synthesize criteria before online scoring

GRS spends grader effort offline. Several varied attempts expose task-specific failure modes and quality differences. A synthesis model builds solution rubrics and behavior rubrics from the task specification, environment evidence and those attempts.

Online scoring applies the rubric to each new solution and process. Solution criteria concern the delivered change; behavior criteria concern evidence gathering, checking and unnecessary actions. Multiply their scores by executable correctness so a failed task remains zero.

For four passing attempts with combined quality [1,.8,.4,.2], the mean reward is .6 and centered advantages are [.4,.2,−.2,−.4]. Quality creates contrast despite identical binary outcomes. Attempts guide criteria; they do not become ground truth. Reject incidental style rules and criteria contradicted by the specification.

Redistribute existing positive credit

GAR instead inspects the current group online. The grader compares patches, test evidence and traces. It scores passing solution quality using suitability, precision, minimality, side effects and craftsmanship. Allow ties when evidence does not justify an ordering.

A confirmed exploit first changes the effective task reward to zero. Recompute the whole group's mean and advantages after that correction. Subtracting weight only from the hacked response would leave every other response's baseline inconsistent.

For [1,1,0,0], binary advantages are [.5,.5,−.5,−.5]. Passing quality [.2,1] gives uncapped $\lambda=\frac53$ and redistributed advantages $\left[\frac16,\frac56,-0.5,-0.5\right]$. The positive budget remains one, but more credit reaches the better passing solution.

Naive downweighting reduces positive mass while leaving negative mass unchanged. That alters the reinforcement balance. Excessive entropy growth is a possible failure mechanism described by the authors, not a theorem about every optimizer or batch.

A cap changes the conservation guarantee

Uncapped normalization preserves total positive mass by construction. A λ cap prevents extreme amplification but can break that equality. Subtracting the new group mean restores zero sum; it does not generally restore the original positive budget.

In the previous group, a cap of one gives [.1,.5,−.5,−.5] before centering. Subtracting the mean −.1 yields [.2,.6,−.4,−.4]. Passing mass is now .8 rather than one. Treat capped conservation as approximate or conditional, not exact.

All-pass and all-fail binary groups have no positive mass to redistribute. Avoid a zero denominator or invented ranking in that case. GRS can supply new quality contrast; GAR needs existing outcome contrast. Report-specific cap constants must come from the implementation, not this toy.

Locate process faults without inventing credit

Sequence outcomes can hide localized process defects. For a positive trajectory, mask flagged bad-action targets and redistribute the declared positive credit across clean tokens. Retain the observations needed to understand recovery.

For a negative trajectory, amplify flagged likely-fault tokens by κ>1, then normalize the weights to the declared negative budget. A plausible local cause is still a grading judgment, not a causal proof.

With positive mass one and flags [true,false,false,false], clean tokens receive $\frac13$ each and the flagged token zero. With negative mass −1 and $\kappa=2$, scores [2,1,1,1] give weights [−.4,−.2,−.2,−.2]. If every token is flagged, positive redistribution has no clean recipient: drop or report unallocated mass. Uniform negative amplification cancels after normalization.

Outcome and process rewards are another independent choice. Outcome asks whether the task finished. Process scores can concern tool validity, correct parameters, information gain, state progress, repetition and recovery. A generic additive form is below; it is not MiMo’s GRS or GAR formula.

Control excessive length after competence

Length regularization should follow sufficient competence. Penalizing long attempts on a task the policy cannot solve can suppress exploration rather than remove waste. MiMo conditions length control on a group pass-rate threshold.

Among successful trajectories, a chosen length percentile sets the reference. A tolerance permits necessary variation. Only excess length receives a smoothly increasing capped penalty. The threshold, percentile, tolerance and cap are recipe parameters, not universal constants.

Separate explicit length penalties from loss reduction. Taking a token mean within each response controls its coefficient mass; it does not directly penalize generating more tokens. Compare accuracy, recovery and verification frequency alongside length.

Finalize grading before accepting a learning update

An asynchronous grader can store heavy trajectory payloads separately from lightweight control metadata. The collector submits a trace; grading later returns outcomes, rubric scores or corrected exploits. That overlap reduces waiting without making provisional scores final.

Finalize rewards, masks and group membership before computing accepted advantages. A late confirmed hack changes the whole group's statistics. Do not allow a race where a provisional reward trains the policy and the correction arrives afterward.

Define a failure policy for grader timeouts or invalid outputs. A report can fall back to the original executable reward rather than inventing a score. Record which reward path was used and compare fallback rates by source. Expensive grading is useful only if its criteria and latency improve accepted signal.

Keep sample, sequence, context and segment distinct

A context rewrite and a resumed segment are different events. The first changes conditioning tokens. The second may continue the same context under another policy version. Track both instead of flattening all records into one text transcript.

Keep large logs, screenshots and artifacts in payload storage. Driver memory should hold control references rather than every complete rollout. Preserve immutable identifiers so resumed work and graders refer to the same evidence.

Resume execution without erasing policy history

Long episodes can hold resources while waiting for tools. Keep environment state outside a preemptible GPU pool. Suspend generation at a safe token boundary, preserve cache and route state where supported, and resume with explicit segment lineage.

K3 partial rollouts pause unfinished work once a fraction completes and continue it in a later iteration. The available description does not disclose a universal fraction. Newer-policy resumption introduces staleness; it is not one ordinary uninterrupted on-policy sample.

DeepSeek describes time-shared GPU phases with persisted key-value state, expert routes and recorded policy versions. Route replay supports numerical consistency; it does not erase the behavioral change introduced by different weights. Reconstruct the trajectory according to its actual segments.

Treat the verifier as an attack surface

The verifier is part of the attack surface. An agent can optimize reward by reading leaked patches, modifying tests, exploiting exception handling or manipulating grader-visible files. A passing test command is not enough when the command or its environment can be attacked.

Clean build logs, patch caches, bytecode, post-base Git references and answer-bearing artifacts. Restrict network access according to the task. Use independent inspectors, adversarial probe agents and fresh audits on new trajectories.

A sandbox crash or infrastructure fault needs an explicit failure signal. Distinguish invalid execution from a valid wrong action. MiMo reports confirmed hacking below 2% in its final run under its detection process; that is not an upper bound on undetected exploits.

Keep evaluation infrastructure separate and hardened. Do not train toward known checker loopholes while treating the resulting score as task competence.

GLM: recovery data and operational feedback

GLM-5 combines long-context SFT recovery data with asynchronous agentic RL. Its maximum SFT context is 202,752 tokens. More than 1,000 simultaneous rollouts use a TiTo gateway, direct double-sided importance sampling and filtering for stale or invalid runs.

Incomplete groups can be padded when enough valid attempts remain, or dropped. Periodic weight synchronization and optimizer resets affect the training state. Rollout-ID consistent hashing preserves cache locality; FP8, multi-token prediction and prefill/decode separation serve execution efficiency, not task verification.

Environment construction spans more than 10,000 software tasks from roughly a thousand repositories in nine languages: Python, Java, Go, C, C++, JavaScript, TypeScript, PHP and Ruby. Thousands of Docker/Harbor terminal tasks report build success above 90%. A search knowledge graph covers over two million pages. Slide tasks combine static, runtime and perceptual checks.

Context management changes measured behavior. Folding old tool observations while retaining five recent rounds moves BrowseComp from 55.3 to 62.0. A 32K discard rule gives 75.9 in the stated setup. These are harness/context results, not isolated architectural effects. GLM-5 report ↗ provides the recipe.

Nanbeige: a compact agent curriculum

Nanbeige4.2 constructs repository tasks with hidden patches and both F2P and P2P tests. Claude Code, OpenHands, SWE-agent and Codex generate scaffold-diverse traces. Its tool worlds mix local, live and simulated components; validated office artifacts support recycling.

These are target-token shares, not task counts. Longer context stages allocate more supervised exposure to agentic content. The table does not isolate the causal contribution of context length from data mixture.

For the compact 3B model, agent RL favors relatively high pass@8 and shorter trajectories. The report summary records an approximately 20% reduction in normalized single-turn error; do not read that as 20 percentage points without the original metric units. Validation rises from 66.0 to a peak of 71.0, not necessarily the final checkpoint score.

Nanbeige4.2 report ↗ gives the context. Compact Model Recipes owns its architecture and broader schedule.

MiMo: heterogeneous tasks and comparative grading

MiMo-V2.6's task mixture is 68% coding, 12% general tool use, 13% visual/aesthetic work, 3% context tasks and 4% cybersecurity. Coding includes real issues, pull requests, repository/specification workflows and expanded horizons. Tools include deterministic files, mocked APIs, CLI, GUI and MCP workflows.

Cybersecurity tasks use verifiable environments such as OSS-Fuzz with sanitizer evidence. They belong to grounded execution and grading here, not a recreated inventory of individual challenge write-ups.

MiMo samples groups of 16. Frequent all-pass groups motivate GRS; quality differences in mixed groups motivate GAR. Ablation conditions include changing turn and token distributions, so the comparison does not establish a perfectly compute-matched isolated component effect.

Training through four coding interfaces raises the reported held-out-harness mean from roughly 50% to 66%. Unlike the FineEnvs all-seen-interface comparison, this addresses interface transfer under the report's stated task/evaluation conditions. MiMo report ↗ contains the methods and results.

DeepSeek: separate construction and inspection

DeepSeek-V4.1-Flash uses voluntarily returned hard or failed sessions, deduplicated, and selected repositories as task seeds. Repository-star thresholds and some selection settings are not disclosed. A planner specifies the target behavior, buildability, regressions and external-resource needs.

A builder installs dependencies and constructs the initial working tree, instructions and tests. It validates its setup, then removes leaked reference material before packaging the container layer. Several solvers attempt the task. An independent inspector checks consistency, unintended solutions and exploitability.

Repair addresses invalid tasks and adjusts difficulty. Task constructors are themselves trained with difficulty and correctness signals. Fresh solver trajectories re-audit tasks when they enter later RL runs; construction is ongoing maintenance rather than one initial dataset export.

General workflows include mocked SaaS and enterprise backends built from observed API behavior, schemas and constraints. Failure feedback improves them. Mocked fidelity remains a validation question, not a consequence of successful execution. DeepSeek report, §5.1 ↗ is the source.

DeepSeek: scalable isolated execution

DeepSeek's DSec infrastructure supports demand for millions of concurrent sandbox instances. That is not a count of unique validated tasks or dedicated cores. For comparable workloads, live-container density increases from about 1,000 to over 2,500 per node, not 2,500 simultaneous repository builds.

Scale units and relaxed scheduler replicas reduce coordination cost; final node admission still protects resource limits. Sub-NUMA binding improves locality. Background work uses SCHED_IDLE and priority isolation, with hyperthread-aware placement where described. AppArmor and eBPF enforce per-sandbox restrictions.

Crashes produce explicit failure information. Preserve the task's environment while GPU generation is preempted. Versioned Claude Code, OpenCode, Pi and DeepSeek scaffolds expand interface conditions. Sample-level dispatch need not wait for previous group stragglers; it does not justify mixing unrelated prompts inside advantage groups.

Completion quotas, optional early rejection of short work, waiting bounds and masks for too-old tokens regulate collection. The report does not disclose universal numerical thresholds for these controls. Throughput gains require both mixture and staleness management.

Mix sources without losing numerical lineage

MiMo's 25 sources vary by roughly 90× in generated tokens and 66× in rollout duration. Fixed concurrency alone cannot maintain a target mixture. Adaptive concurrency, deficit-aware scheduling, predictive cache dispatch and bounded replay manage that heterogeneity.

Numerical execution also matters. MiMo uses MXFP4 alignment with quantize-dequantize after updates, expert-route replay and top-p candidate replay. These address specific rollout/training mismatches; using the same checkpoint identifier alone does not establish the same sampled distribution.

Predictive memory estimates can become stale. Restart paths and cache-memory pools need explicit accounting. Hardware or network failures should not be attributed to policy decisions without evidence. Link numerical details to Precision and Quantization, routing to Mixture of Experts and update semantics to RL.

Two small contracts to implement first

First implement task acceptance independently of the solver. This conceptual kernel requires both test families and distinguishes invalid initial states from failed repairs. The production checker must also protect its own files and execution path.

def valid_repair(before, after):
    target_was_broken = not before.target_pass
    target_is_fixed = after.target_pass
    regression_was_valid = before.regression_pass
    regression_is_safe = after.regression_pass
    return (target_was_broken and target_is_fixed
            and regression_was_valid and regression_is_safe)

Next preserve aligned captured arrays [T] for generated IDs, behavior log probabilities and masks. An entire context has separate prompt/observation IDs. The example assumes every returned log probability corresponds to the processed distribution that generated that ID.

capture = engine.generate(effective_prompt_ids, sampling)
assert len(capture.ids) == len(capture.behavior_logp)
record = {
    'prompt_ids': effective_prompt_ids,
    'completion_ids': capture.ids,
    'behavior_logp': capture.behavior_logp,
    'policy_version': capture.version,
    'sampling': sampling,
    'target_mask': retained_assistant_mask,
}
validate_and_store(record)

Validation checks finite probabilities, masks, lineage, support and task identity before training acceptance. Neither kernel is a complete training loop. Keep reward finalization and group statistics outside the environment's action handler.

Measure capability, interface robustness and cost

Report task success and confidence intervals by task family and harness. Include held-out interfaces when claiming interface transfer. Evaluate cost, latency, calls, target length, recovery and invalid-environment rates under matched budgets.

A useful ablation changes one contract at a time: task pool, harness mixture, rubric, length rule or scheduler. Record compute and accepted exposure. Keep independent inspectors and selected human review for failures that automated metrics miss.

How the engineering problems connect

Agent training extends demonstrated tool use into executable, outcome-grounded tasks. Repository history supplies initial states and regression tests. Synthetic factories expand tool and artifact workflows. Recovery masking teaches correction without copying every failed action.

As policies and workloads grow, interface transfer and heterogeneous collection become separate problems. TiTo capture preserves the sampled evidence. Multi-harness training varies the agent program; scheduling controls which completed work reaches the learner.

Binary success then saturates on solvable tasks. GRS supplies task-specific quality contrast; GAR redirects existing mixed-group credit. Long-horizon interruption, adversarial inspection and environment isolation keep the resulting evidence valid. These are connected engineering branches, not one sequence in which each new report replaces every earlier method.

Sources and connected topics

Kimi K3 report ↗ · Nanbeige4.2 report ↗ · MiMo-V2.6 report ↗ · DeepSeek-V4.1 report ↗ · GLM-5 report ↗

FineEnvs multi-harness guide ↗ · OpenEnv Harbor ↗ · Token-In, Token-Out ↗ · TRL asynchronous GRPO ↗

Reinforcement Learning · SFT and Adaptation · Distillation · Training Data · Frontier Recipes

The visual models use invented token IDs, lengths, durations and quality values. Their outputs follow exact declared calculations. They are explanations, not measured training results or replicas of a full production scheduler.