Model specialization and training agents

Updated 23 Oct 2026

At SynthLabs, I implemented document-adaptation, supervised fine-tuning and on-policy distillation pipelines for open-weight models. I then built agents to prepare, run and investigate that work. A training agent turned a specialization request into a model specification, data, assessments and checkpoints. An outer researcher agent inspected those episodes and revised the training agent’s reusable skills and tools. I connected generated tasks to teacher demonstrations, student training and checkpoint assessment, and implemented on-policy updates from a student’s own tool-using trajectories.

The underlying question was practical: what should a model learn, and what should its surrounding system supply? A company-specific assistant might need to recall a stable convention, retrieve a changing specification, call a tool, write code, or follow a particular workflow. Those are different capabilities. Providing the same pile of documents to every component does not define how to train or evaluate any of them.

This article describes that January–May 2026 work, followed by a related May–June team application. It precedes my distributed training and evaluation platform and complements the later repository-specific agent construction workflows. The connection is deliberate: model weights, executable environments and reusable agent behavior have to fit together, but they remain independently adjustable parts of the system.

Choosing what to specialize

An assistant can become more useful without changing its weights. Better instructions can direct its investigation. Searchable resources can expose organization knowledge. Tools can give it the operations that the work requires. Training can then change how the model uses those affordances, or internalize information that would otherwise have to be supplied repeatedly.

I treated these as separate optimization choices rather than different names for fine-tuning.

For example, training a closed-book model to answer questions about company documentation tests what it can answer unaided after weight updates. Training an agent to locate the right document tests a different behavior. Giving the model the answer-bearing document at evaluation time changes the question again.

The target also extends beyond factual question answering. A specialist might have to produce a script, make a recommendation under company policy, transform structured data, or perform a sequence of actions. Each calls for representative work and an assessment appropriate to that output. A question-answer benchmark cannot stand in for all of them.

That distinction shaped both the early experiments and the later training-agent design: define the behavior and deployment conditions before choosing the training procedure.

Compiling document knowledge into weights

PostHog’s public handbook supplied organization-specific engineering and operating guidance for a concrete specialization testbed. Nathan built document-to-data generation machinery; I developed and exercised the training and evaluation pipelines around these sources.

One actual training example asked when an engineer would choose Celery rather than Temporal or Dagster. The supporting engineering-stack section described PostHog’s adopted division of work: tiny, low-latency, fire-and-forget jobs for Celery, reliable complex workflows for Temporal, and scheduled data transformations for Dagster. The target was the organization’s particular engineering guidance, rather than a generic comparison of the three systems.

The central experiment was intentionally demanding: could a model answer domain questions without receiving the supporting document at inference time? That separated knowledge in weights from the immediate benefit of supplying context.

I built a shared data representation containing a question, supporting context and a reference answer. Different training procedures consumed different views of the same information. Storing them in one record did not expose the context to the question-only student.

The implemented routes included:

• Documentation adaptation: train on packed source documents with their file and section information.
• Synthetic question-answer SFT and answer distillation: generate supervision from source material, including answers from a context-grounded teacher, then train on question-answer pairs. General instruction-following examples could be mixed into that supervision.
• On-policy context distillation: let the student generate an answer, then use a context-privileged teacher to score those generated tokens.
• Context annealing: progressively reduce the source context available to the student during training.

These alternatives ask different things of the optimization process. Training on documents exposes content directly. Question-answer supervision rehearses ways that content is used. A contextual teacher can supply a stronger answer than a closed-book model. On-policy distillation directs supervision toward what the student currently produces, rather than only answers produced by another model.

I compared data volumes, teacher-generated supervision, initial training states and distillation settings. Evaluation tracked domain performance separately from general instruction following. That second measurement matters: acquiring specialized knowledge is less useful if the resulting model stops reliably following the instructions through which that knowledge is requested.

The larger supervision sets reached higher recorded scores. Within each fixed set, the recorded scores changed less with further training. More distinct examples also meant more updates for each training-set equivalent, so this comparison does not isolate data quantity from compute. It nevertheless exposed a useful decision: whether to spend the next training budget repeating the existing supervision or expanding its coverage.

The mixture retained higher recorded instruction-following accuracy at the shared measured checkpoints. Domain-only training usually scored higher on the domain questions. Tracking both made that trade-off visible instead of hiding it behind a single specialization score.

These are reconstructed exploratory historical measurements, with one training run per condition. The preparation protocol split by question rather than source document: the target was answering about the supplied corpus, not transfer to unseen documents. The QA score comes from a source-grounded model judge, not independently verified task success, and the historical dataset revision was not pinned.

The broader experiments included SFT data-volume sweeps, teacher comparisons, documentation-adaptation comparisons, distillation from an instruction-tuned starting model, SFT-warm distillation, and subsequent instruction-following training. The value was an experimental framework in which these choices could be changed and measured, rather than a fixed one-off training script.

A teacher with information the student does not have

The SFT sweeps varied the supervision supplied to the learner. The on-policy route changed where that supervision came from: the learner’s own current responses became the trajectories on which the teacher supplied feedback.

Context distillation makes the information boundary explicit. The teacher sees a question and its supporting context. The student sees the question alone. The objective is to transfer some of the teacher’s context-conditioned behavior into the student’s parameters.

In the on-policy route I implemented, the student first samples a response. The teacher scores that same response under its richer input. For a sampled response token, the training signal uses the difference between its teacher-conditioned and student-conditioned log probabilities. Tokens are selected by the student; the teacher evaluates those tokens with additional information.

For question $q$, context $c$ and response prefix $a_{<t}$, the sampled token-level signal has the form

This is a sampled reverse-KL-style distillation signal. It is not a comparison of two independently written answers, and it does not require the teacher to generate a replacement response for every student attempt. The signal is positive when the contextual teacher gives a sampled token more probability than the rollout policy did, and negative in the opposite case.

The implementation had to preserve the teacher’s extra context, align response tokens and masks, obtain the relevant log probabilities, and turn the resulting signal into Tinker training updates. The updated model’s sampler then supplied the next student responses. Model identity and checkpoint identity were part of the experiment, rather than inferred from a run’s display name.

The log-probability difference was supplied as a detached per-token advantage to the learner, alongside the captured sampling log probabilities and response masks. Tinker’s importance-sampling training loss then applied the student update. Keeping those roles separate matters: the teacher score supplies supervision; the loss determines how that supervision changes the student’s weights.

There are two important operational distinctions here. First, the teacher’s information advantage is separate from model size: the same model family can be used on both sides with different visible context. Second, answer distillation and token-level distillation impose different compatibility requirements. Text answers can be retokenized for a different student. My direct context-distillation path instead rendered the teacher prefix separately, appended the student’s sampled response IDs, and aligned returned scores to masked response positions. This path requires compatible token-ID semantics.

Teacher privilege also has a behavioral boundary. The teacher can consult extra evidence to improve its supervision, but the behavior the student is trained to imitate must still fit its deployed job. An answer that tells the student to inspect a hidden solution file or call an unavailable tool teaches a different contract. This is not a ban on the teacher’s extra information—that advantage is the point. I incorporated the distinction into the later training agent’s data-generation and method-selection skills, for both teacher-generated SFT answers and on-policy distillation.

I also explored the starting distribution. An SFT warm start can make relevant answers more accessible before on-policy training, while later instruction-following distillation can address another capability. The order of these stages is a substantive experimental choice, not interchangeable launch syntax.

Making training an agent task

The next problem was repeatability. Running a known recipe is useful, but many specialization requests arrive before the dataset, evaluation and deployment behavior have been specified.

I built Claude Code-based training agents that could work through that preparation as well as execute the training. These agents operated the training workflow; the specialist model checkpoint was their output. An episode began with a goal and available resources, not a completed configuration file. The agent inspected the material, clarified what the resulting model was supposed to do, proposed a plan, prepared data, ran checks, launched training and exposed the resulting checkpoints and evaluation artifacts.

One March assignment asked for a PostHog technical assistant trained from Qwen3.5-4B and the public product documentation. The request named example jobs such as configuration, feature-flag behavior and HogQL queries, but did not provide the finished training recipe. The agent had to turn that request into a concrete preparation and training plan. This was a broader product-support task than the earlier handbook-knowledge experiments.

The workflow separated decisions that are easy to conflate:

1. The intended model behavior. What inputs will it receive? Which tools and resources will it have? What counts as a useful response or completed action?
2. The training distribution. Where do examples come from, what do they cover, and how is their correctness established?
3. The training procedure. Which supported method, model, data rendering and update settings fit that behavior?
4. The assessment. How will the resulting model be compared and inspected without treating successful job completion as sufficient evidence of quality?

I introduced a model specification that represented the intended behavior alongside its instructions, tool definitions and resources. Creation and validation tools checked the referenced artifacts. A renderer could translate the specification into a harness configuration for synthetic-data generation. That gave the training agent a concrete object to build and inspect, rather than leaving the desired model as an informal paragraph.

The agent’s own tools mattered just as much. They needed to expose real training parameters, distinguish failed launches from completed jobs, provide logs and status, and return usable checkpoint and sampler handles. A tool accepting a parameter internally was insufficient if the CLI through which the agent acted never forwarded it. I worked on those interfaces together with the skills that taught the agent how to use them.

Onboarding had a persistent handoff: a versioned JSON plan for downstream stages and a readable view for approval. It recorded deployment tools, expected queries, data sources, the training procedure and assessment criteria. The profile called for a small generation probe before bulk work, with a fresh-context reviewer checking relevance and coverage against the intended query distribution, then the user’s approval of that direction. These were agent workflow instructions, distinct from the structural validation performed by code.

The orchestration agent retained the goal and decisions while data-intensive preparation, independent quality checks, training and evaluation could be delegated to specialist agents. A trained model came with inspectable artifacts, so the next decision did not depend on trusting a summary that training had succeeded.

There were two distinct assessments. Task verification checked whether the training workflow had fulfilled its execution and artifact contract. Checkpoint evaluation examined the resulting model’s behavior. Completing the first did not answer the second.

These components were integrated through Claude Code plugins, training tools, Harbor tasks and sandbox execution. I added plugin-directory support to our Harbor adapter so the same training-agent configuration could be exercised as an executable task rather than only in a developer’s interactive session.

A researcher for the training workflow

A training agent can complete an episode while still preparing weak data, using a poor experiment, or spending its effort in the wrong place. That makes the agent’s reusable process another object worth improving.

I built an outer researcher around the training agent. The training agent attempted the specialization task and produced data, model checkpoints and artifacts. The researcher inspected what happened and could revise the training agent’s plugin: instructions, skills, tools, data preparation or experiment procedure. It then launched further episodes and assessed their outputs.

The researcher had operations for launching and managing sandboxes, inspecting status and logs, retrieving artifacts, sending follow-up messages, and reloading or restarting the inner agent when appropriate. A trace parser exposed relevant segments instead of forcing every investigator to read an entire conversation. Persistent journals retained observations, hypotheses, changes and comparisons across sessions.

For research episodes, the launcher supplied a focus task and assigned the outer agent the run–inspect–revise–restart loop, including the operator role at the inner agent’s routine approval turns.

The distinction between the two loops is important. The inner episode optimizes or constructs a specialist model. The outer loop changes how that episode is conducted. Its useful mutations can be source-selection rules, better checks, correct tool interfaces or revised instructions, rather than another learning rate.

Turning a run’s discovery into a reusable skill

One concrete example came from generating training data from documentation. Some MDX files contained imports and component wrappers rather than the facts their filenames suggested. A character-count filter could accept them as substantial documents, leaving the generator without the underlying information.

The training agent diagnosed that distinction during an episode, changed its content check and regenerated the data. The resulting data passed a subsequent quality check, and a trained checkpoint was evaluated repeatedly. The researcher then incorporated the lesson into the shared data-generation skill, including checks for wrapper-only files and references to externally loaded content.

Other investigations concerned consistency between training and inference instructions, the relationship between dataset size and the actual number of optimizer steps, and parameters that did not survive the path from the agent’s CLI to its training backend. These examples demonstrate why optimizing a training agent involves its entire working environment.

The researcher was implemented and exercised, with retained source changes, experiment journals, checkpoint identities and repeated model comparisons. Those comparisons guided development; repeatedly used evaluation prompts did not provide independent confirmation of improvement. The record makes the research process inspectable, including which reusable component changed and which model was subsequently assessed.

Shorter loops for data-generation research

Not every question needed a complete training-agent episode. In April I also separated task-specification extraction, instance generation and diagnostic comparison into a smaller artifact-based workflow. These task specifications defined the learning work; the earlier model specification defined the specialist, and the plan its construction. One agent derived a task specification from source material and a standalone guide. Another was assigned the specification and a general generation recipe. A separate agent inspected a shuffled, unlabeled mixture of generated and comparison examples, returning classifications and reasons. The comparison labels were stored in a separate file.

The reusable components were independently editable: the specification guide, extraction instructions, generation recipe and comparison preparation. The recipe gained explicit output-style checks and representation normalization; the retained handoffs and generated artifacts made it possible to inspect what each stage had actually received and produced. This gave me a shorter route to investigating data-generation behavior before paying for a downstream training comparison.

The discriminator was a diagnostic, not a task-correctness oracle. Matching presentation or making two sets harder to distinguish does not by itself show that the generated work is correct or that training on it improves a model.

Building the training distribution

Beyond operating a training recipe, I worked on constructing the learning distribution: representative tasks and feedback suited to the target capability. A desired capability is not a training set, and a benchmark’s examples are not a recipe for producing more of the right work. Alongside the training-agent and researcher stack, I built a separate task/data pipeline connecting task specification, selected source material, generation, executable verification and policy training. The factory—not just the training run—was an object of research and engineering.

I used MMTU, a table-understanding and reasoning benchmark, as a concrete testbed. Its task families include question answering, SQL generation, table transformation, schema matching and relationship detection. They do not all share the same inputs, answer format or definition of correctness. That made it a useful setting for building beyond synthetic Q&A.

Specifications and generation are different objects

A task specification recorded the action, visible input, output contract, source material, selection rules, ground-truth derivation and scoring. Source allocation specified which task families each source would supply and what variation in column subsets, row counts and operations to cover. The procedure that turned those decisions into instances was independently adjustable: improve the generator without silently changing the task, or change the distribution without rewriting execution machinery.

The factory assembled a working directory for each task-family/source generation job. Each job could produce multiple learning instances. It rendered instructions from task-specific contracts and shared templates, connected the source files, and attached validation. This helped both coverage and parallel execution. A long generation session can converge on repetitive examples; many isolated calls can return the same obvious example. Each unit of work therefore carried an explicit allocation instead of asking every worker to invent its own interpretation of “diverse.”

I separated the controller from bulk generation. The controller could use a full coding-agent environment to research sources and revise the procedure. Repeated generation used an API model with a lightweight file-and-tool harness. It could inspect a CSV’s schema, sample relevant rows, use Python for computation, and write output to disk. It did not have to load every source row into its conversation or regenerate a long response to repair a small file error.

An asynchronous sampler ran these jobs concurrently, retained tool trajectories alongside token and estimated-cost records, and returned validation errors to the same working conversation. The generator could correct its existing artifacts while keeping the context of how they had been produced. Resume logic checked saved output before deciding to skip a job.

Follow one generated SQL task through the factory

One generated SQL task combined three named tables: patients, medical and billing, linked by PatientID. Its reference joined those tables, filtered a medical condition and selected the corresponding name, doctor and billing fields in name order:

SELECT p."Name", m."Doctor", b."Billing Amount"
FROM "patients" p
JOIN "medical" m ON p."PatientID" = m."PatientID"
JOIN "billing" b ON p."PatientID" = b."PatientID"
WHERE m."Medical Condition" = 'Asthma'
ORDER BY p."Name";

The packaged task contained a SQLite database with those same named tables and an expected-result CSV with matching output columns. The assembly code cleaned the reference SQL, executed it, and stored the database and reference output in the task’s verification information. During assessment, the verifier executes the candidate’s SQL and compares returned values. Correctness is not defined as reproducing one reference SQL string: different queries could express the same solution.

Other task families required different assembly: normalizing labels, carrying reference files for Python transformations, removing duplicate prompts and creating stratified training and evaluation splits. Structural validation, reference execution and source boundaries each addressed a different part of quality; none alone established that every generated task was correct.

Connect generated work to the learner

The policy being trained had its own table-parsing, Python and SQL tools, separate from the generation harness. A teacher could solve the generated tasks with those tools; I rendered its assistant actions and tool observations into SFT records while preserving which tokens were training targets. For reinforcement learning, the student attempted the tasks and the verifier scored its output. The task data, policy plugin and verifier were combined with the selected model or SFT checkpoint in the training entry point.

I exercised the connection with a Haiku 4.5 teacher using the policy tools. The sampling stage saved its conversations and task-scorer results; a filtered trajectory set then fed SFT with assistant actions and tool observations rendered under explicit trainable-token boundaries. A Qwen3.5-4B run reached final trainer-state and sampler checkpoints. I then assessed the final sampler on tasks through the policy tools, exercising the complete path from generated work through tool-using demonstrations and student training to checkpoint assessment.

Distilling tool-using policies

Knowledge-only training leaves out much of what makes an agent useful. An agent may have to inspect files, execute code, interpret tool output and decide what to do next. Training on those trajectories introduces a different integration problem from training on plain question-answer pairs.

In April, I implemented a tool-trajectory on-policy distillation path alongside the coding-agent training work. Unlike the teacher-generated SFT demonstrations, the student supplied the trajectory by acting in a sandbox. Its record retained the model-native tokens and sampling log probabilities, and the teacher scored the same student-generated actions under a causal view of the interaction. Tinker performed the parameter updates, after which the sampler was refreshed for subsequent rollouts.

Here the teacher received the causal conversation and tool history. In the retained runs, a larger fixed teacher supplied a different policy over that same history. Its advantage was not extra document context or access to future outcomes before scoring an earlier action. That is a different teacher–student relationship from the earlier knowledge-compilation experiments.

The response mask was central. A trajectory contains model output, prompts and tool observations. The model should learn from the tokens it generated, not be credited with producing a compiler message or a file’s contents. Teacher scoring also has to respect next-token alignment and the information available at each action. A final transcript rendered as convenient text is not automatically a correct training record.

The teacher term was added to verifier-reward advantages in the training integration. That made it important not to infer the response mask from nonzero advantages. If every attempt received the same verifier reward, the relative advantages could all be zero while teacher feedback still supplied a learning signal. I retained an independent mask for generated tokens and shifted it with the targets and captured probabilities. That kept teacher supervision on the model’s actions without accidentally applying it to observations—or suppressing it altogether.

I joined task results to their generating trajectories, checked token and mask alignment, and retained update and sampler identities so an update could be traced back to its training evidence.

In May, I trained specialist policies with GRPO using grouped sandbox rollouts and checkpointed updates. One target was Promptfoo, an LLM-testing and red-teaming framework. I evaluated a trained checkpoint and its starting model on work involving configuration, provider integrations, diagnosis and red-team design. The analysis joined actor trajectories and separate verifier reports to task-specific criteria, completion state, tool use, elapsed time and token usage. That supported investigation of how a policy carried out the work, rather than treating a shorter response or one aggregate reward as sufficient evidence of improvement.

This connects the specialization research to the execution and trace machinery described in the platform article.

A related application in Nemoral

In the Stanford CS194 team project Nemoral, I worked on a user-facing application of related ideas: a prompt and budget leading to a fine-tuned model with an inspectable training and experiment history.

The application connected data acquisition and curation, approval of a plan, fixed training and validation data, Tinker LoRA SFT, and budgeted experiment selection. Candidate changes were evaluated and either kept or reverted, with checkpoint and metric artifacts available through the application.

I implemented the Tinker LoRA/SFT runner and budgeted experiment-loop integration. Candidate selection used forward-only validation on the finalized split, with cross-entropy aggregated over assistant-target tokens. I expressed validation intervals in processed training examples and translated them to steps for each candidate’s batch size, keeping the measurement contract explicit as training recipes changed.

Nemoral was a separate Stanford team project, with collaborators owning substantial components. The final project documentation and my training and AutoResearch integration contributions provide public source material.

What carries forward

This work made the intended capability, training distribution, tool environment, sampled actions and checkpoint state explicit parts of one engineering problem. A useful improvement could come from a better source allocation, an aligned teacher, a working tool interface, or a reusable investigation procedure—not only a change to the optimizer. The training agent could carry out that work; the outer researcher could improve how it was carried out.

That experience shaped the later distributed platform, which generalized the execution and learning machinery. The repository-specific construction workflows applied the same attention to information, tools and reusable procedure without requiring a weight update. Together, the projects span both ways of building a specialist: changing what the model has learned and changing the system through which it does the work.

Sources and contribution

This account is based on the original training implementations, versioned agent plugins, experiment records, source-derived datasets and contemporaneous design discussions. The PostHog material came from public documentation and the handbook; this work does not imply a PostHog customer relationship.

I led the training-method implementations, specialization experiments, training-agent and researcher integration, and the later executable-task/data factory described here. The early PostHog work built on Nathan’s document-to-data generation machinery and the team’s wider tooling. I used coding agents extensively in developing and operating these systems. Nemoral was a separate Stanford team collaboration. The underlying SynthLabs training implementations remain private; this article explains the mechanisms without releasing proprietary source or original conversations.

For the underlying training foundations, see the public Tinker Cookbook and Thinking Machines’ explanation of on-policy distillation.