From real software work to specialized agents

Updated 21 Oct 2026

At SynthLabs, I built systems that turn software-engineering work into two reusable artifacts: tasks that reconstruct the job, and agent profiles that specialize the worker. Task factories captured specialist assignments, partitioned ML-framework transformations and compiled historical review opportunities. A separate construction workflow researched repository code and engineering history and returned a runnable, repository-specific reviewer.

The two sides solve different problems. Factories make work repeatable across agents and model versions. Construction specializes the worker through repository knowledge and investigation procedures, without changing model weights. Their separation lets a workflow produce the assignment or the worker, then evaluate and improve each independently.

The reviewer application makes this concrete: repository URLs go into construction; a reviewer profile comes out. A second workflow takes that profile and a PR URL, reconstructs the change, and returns a review and its artifacts.

The reviewer targets contract-sensitive changes: producer–consumer guarantees, authorization and isolation, resource ownership, accounting and lifecycle, and supported compatibility. The scope requires a concrete guarantee and an affected execution path—not merely a change touching an API.

These workflows share task and agent-profile abstractions with my distributed agent platform. Here I focus on constructing the assignments, specialists and application workflows that use those contracts.

Work and behavior are separate artifacts

A review opportunity is more specific than a PR number. It includes the source being compared, the available context, the responsibility assigned to the reviewer, and the evidence that can establish whether a finding is correct. Likewise, an agent is more specific than a model name: its tools, instructions, setup, context and execution behavior all affect the work it can do.

I separated those responsibilities deliberately. The task supplies the starting environment and assignment. The profile supplies the complete agent-side recipe: model, native engine, files, setup, invocation, input handling and required outputs. The task defines the assignment; the profile controls how it reaches the native agent. Task packages therefore do not have to encode one harness’s invocation conventions.

That separation makes comparisons meaningful. I can hold a task fixed while changing the model, investigation method or repository knowledge. I can retain an answer while changing the judge. And a workflow that constructs an agent can return a profile that another workflow executes, rather than a prose description that someone must manually translate into a working agent.

Three ways to construct executable work

Software work does not arrive in one form. A parent agent delegates a problem from an evolving workspace. A framework migration needs a semantic plan and a division of ownership. A historical review needs the exact change and evidence available at the chosen opportunity.

I built distinct task-production workflows for these sources. Each produces a reconstructable assignment with explicit source, scope, artifacts and assessment context. The important difference is how the assignment is defined—not merely how its files are packaged.

Capture specialist assignments as the work evolves

In one workflow, a parent agent works through a source-grounded engineering scenario and delegates bounded assignments to a specialist. The capture point is immediately before the specialist runs: the current workspace and request preserve the problem as it was posed, before its answer exists, so another actor can attempt the same assignment.

I built scenario materializers that separate the domain contract, starting files and runtime configuration. The domain defines the work’s vocabulary and evidence requirements; the scenario defines a concrete starting state; the runtime layer defines model access, resources and execution limits. Composing these layers produces the parent task; its specialist calls then supply the captured child tasks. Scenario authoring and child-task capture remain separate responsibilities.

The shared skeleton reused capture mechanics across source packs with different proof obligations, such as generated-schema consistency and query-result equivalence. Domain-specific evidence could change without redesigning the capture and replay loop.

Original solver traces and results remain separate from the promoted task’s starting files, while their lineage is retained for assessment and analysis.

The assignments must stand on observable files and a self-contained request. “Continue from the previous response” is not a replayable problem. The request instead names the objective, allowed edit surface, acceptance checks and expected evidence. An audit can legitimately return a reproduction without a repair; an implementation task has a different completion contract.

This supported evolving work on model rendering, data contracts, asynchronous lifecycle, checkpoint and sampler state, and training-workflow integration. A checkpoint assignment, for example, had to distinguish resumable trainer state from the sampler used for generation: the two handles describe different capabilities, not interchangeable model paths.

In the replay workflow, a worker attempts the captured problem in a separate workspace, a judge assesses its artifacts, and the resulting changes and feedback return to the parent. Subsequent assignments can build on the new file state without depending on the previous worker’s transcript.

My work covered the source-grounded scenarios, normalized materialization, task contracts, corpus curation and replay integration, building on the team’s specialist-capture and actor/judge runtime foundations.

Partition a repository transformation before materializing it

For ML-framework migration work, I used a different route: a technical translation plan, a complete file-level action inventory, a partition into bounded worker assignments, and mechanical task materialization.

For example, transforming an RL framework toward Tinker-backed computation is not an import-renaming exercise. The plan must distinguish algorithmic semantics worth preserving—advantage calculation, losses, response masking, experience records and checkpoint intent—from the distributed execution substrate being replaced. Ray actors, local inference engines and optimizer ownership do not map to the new backend in the same way as a pure PPO loss function.

I made that distinction explicit in transformation plans for frameworks including verl and OpenRLHF. A file inventory assigns each source path an action and an owner. Structured worker specifications declare existing and new paths, deletions, boundaries, validation and expected evidence. The materializer checks that the assigned source paths cover the inventory without overlapping ownership.

Each task starts from the same pinned repository state and owns a coherent slice of the transformation. The resulting assignments can be replayed and attempted in parallel; integrating their patches is a separate operation. Shared package surfaces have explicit ownership. Instructions are rendered from the structured specification rather than repaired after the fact from arbitrary prompt text.

One SFT assignment illustrates the semantic boundary. It had to preserve response loss masking, gradient accumulation, evaluation hooks, checkpoint cadence and final-save behavior while replacing the training backend with Tinker clients. The useful work was preserving the training contract, not merely getting the new imports to resolve.

The semantic planning and partitioning are where domain judgment belongs. Once those decisions are made, packaging the workspace, instruction, judge assets and lineage should be deterministic.

Compile historical review opportunities from source and evidence

For code review, the work already exists in repository history. I built a producer that surveys submitted review events, applies explicit selection rules, captures the associated records, resolves source revisions, and compiles runnable review tasks and dataset manifests.

The compiler exports complete pinned source trees, the comparison base and the actual diff. It can include companion repository snapshots where a contract crosses a repository boundary. Actor context and private assessment evidence are projected into separate views before packaging.

That distinction matters for historical work. A later fix or review discussion may help assess an answer, but must not become the actor’s answer key. The source object database is not simply copied into the review environment, where future commits could defeat that boundary. Evidence retains its source identity, publication and observation information; unavailable historical versions are not silently replaced with their current text.

Independent capture and compilation use bounded parallel workers. Each case depends on its own opportunity and inputs, not the other cases selected beside it. Immutable identities connect the visible work, private assessment view, compiled package and uploaded task version. Changing the actor or judge does not require recollecting the underlying problem.

Hold the work fixed, change the evaluator

Separating the task, actor and judge creates useful experimental controls. An actor can produce work without any automated score. Fresh attempts can be paired with a task verifier or a selected judge profile. Retained outputs can also be reassessed without asking the original actor to solve the work again.

For review tasks, I built a reusable judge profile containing its own rubric, evidence-prompt preparation, structured judgment checks and deterministic reward calculation. Each task supplies the case-specific private references in its verifier environment. The model produces a structured judgment; the profile’s Stop hook validates it and computes the reward. The execution runtime launches the recipe and collects the result; it does not need a hidden code-review-specific grading policy.

This lets a judge serve a task family while its model, instructions or rubric change independently of the actor. Repeated assessments of fixed outputs can expose judge variability; comparing different judges does not require paying to recreate the original work. The platform article explains the retained-output reassessment mechanism.

For historical-reference tasks, this judge scores agreement with qualified recorded evidence. Comparing reviewers asks a broader question: which claims are supported by source, including valid defects absent from the recorded review? I developed a separate local workflow with coding agents around a per-opportunity evidence ledger, preserving the original candidate and recorded Codex/CodeRabbit reviews and linking judgments to pinned source and affected consumers. The same underlying issue received one identity across different wording and output sections; repeated descriptions did not multiply credit.

Diagnoses and proposed remedies were assessed separately. Known misses, incomplete leads and alternative interpretations remained explicit. New source evidence could revise an earlier judgment while preserving its history. Deterministic tooling checked custody and whether every emitted item had an assessment, then aggregated accepted judgments over paired opportunities. Defect detection, misleading assertions and harmful advice stayed separate in reporting. Source investigation determined whether a claim was true; historical reviews were comparators, not exhaustive answer keys.

Separating the actor from assessment also matters outside experiments. Deployment can use the task factory and chosen actor profile without attaching a judge to every work item. Evaluation and supported weight training remain separate choices.

A creator that builds the reviewer

The next step was to make specialization itself repeatable. I built a construction workflow that starts from repository URLs and authorized read access, not a hand-written organization specification or a previous reviewer to repair.

The workflow collects two distinct sources of engineering knowledge. Git provides code evolution and exact revisions. GitHub provides PR descriptions, reviews, threaded discussion and decisions that the code alone may not explain. The collector builds Git bundles and a searchable SQLite history, with recorded refs, collection times, coverage and content identities.

Joining those sources takes more than cloning the default branch. A review may refer to a force-pushed revision no longer reachable from current advertised refs. I added an audit of authenticated review commit pointers against bundled ancestry, followed by recovery and pinning of fetchable missing revisions. The resulting history retains source that would otherwise disappear precisely where a historical discussion needs it.

A factory turns that material into a creator task: restored Git mirrors, searchable discussion history, the shared reviewer base, trusted assembly tools and writable research space. The creator is an ordinary agent operating on that task. Its output is another agent’s repository-specific knowledge and runnable profile.

The creator’s job is semantic research. It discovers architecture, guarantees, producer–consumer relationships, adopted decisions, intentional exceptions and useful investigation procedures. It must distinguish adopted decisions from discarded proposals and unresolved interpretations: a historical review comment is not automatically a rule. The shared base owns the stable execution machinery: model and engine settings, tools, review discipline, output format, validators and launch behavior.

Trusted code assembles those parts. The creator does not have to reinvent launcher conventions, executable helpers or plugin metadata, and generated knowledge cannot silently replace them. The assembler validates the structured data, references and relationships and produces the runnable profile. At export, the controller independently rebuilds it from the approved base, generated knowledge and history manifest, and requires an exact match.

This division puts inference where it adds value—discovering and expressing repository knowledge—and deterministic code where it can enforce a precise contract. Structural validation and subsequent quality assessment are different operations.

The separation also makes maintenance cheaper. An approved shared-base update can be assembled with the exact existing specialization without repeating the creator’s research. The new profile records both inputs rather than silently changing what an older profile identity means.

Knowledge that an agent can navigate

Construction happens infrequently; review happens on every relevant change. That asymmetry justifies deeper repository research during construction, but only if the reviewer can use its results efficiently.

I organized specialization as a local knowledge library rather than flattening it into one read-once prompt. The profile carries architecture, contracts, repository-specific instructions, native skills, limitations, provenance and navigation routes.

Each contract has more than a description. It records applicability, triggers, producer and consumer paths, exceptions, investigation checks, counterevidence, related skills and source references. A local lexical router supports orientation, search and complete-record lookup. Behavioral terms, symbols and changed paths return compact routes; the reviewer can then open the full relevant record. Setup reconstructs these named files from a verified canonical representation: transport compression does not summarize away their contents.

The creator also exercises the assembled library with path and behavioral queries, checking which records are found and whether their full applicability conditions, exceptions and related routes remain available. This makes retrieval usability part of construction, not a formatting concern postponed until deployment.

An actual generated OpenFGA contract concerned ordering-sensitive iterator pruning. Two sorted inputs do not make their concatenation globally sorted; an algorithm using ordering to prune candidates must receive the guarantee it actually relies on. The generated record connected storage iterators and their wrappers to the set-operation consumers using that guarantee. Its procedure covered ordering flags, SQL ordering and collation, wrapper forwarding and pruning guards. A related skill covered iterator lifetime and cleanup as well as ordering.

That is more useful than “check iterator correctness.” It tells the reviewer where a seemingly local wrapper change can break a downstream algorithm, and which counterconditions could make the suspicion wrong. Construction-time knowledge guides the investigation; the current task’s source still decides whether the defect exists.

Turning context into investigation

A review factory resolves the requested PR snapshot and prepares the source the agent will actually investigate. The reviewer gets complete files or enclosing source scopes, numbered locations and continuation routes for long contexts, alongside the diff and relevant repository knowledge.

The shared base separates two jobs: discover plausible problems broadly, and investigate each candidate deeply enough to support a useful claim. Parallel workers inspect complementary source paths; source-first discovery can proceed without inheriting the parent’s leading allegation. The parent must reconcile the actual producer, controlling guards and consumer behavior, rather than count how many agents agree.

I also built a whole-draft challenge stage. A trusted helper snapshots the actual structured review, including its proposed remedies, alternatives, reassurances and coverage claims, and resolves its references to exact source. A fresh challenger receives that packet rather than a favorable summary written by the author of the finding. Its response and the draft it inspected remain linked. This is an actor-side investigation step, separate from a judge selected to assess the completed review.

The output distinguishes defects, resolved concerns, trade-offs, specific open questions and coverage. These categories have no quotas. A useful review can explain why a suspicious path is safe or which condition changes a trade-off, without disguising either as a bug. Findings and proposed fixes are separate: a correct diagnosis does not automatically make every suggested remedy safe.

The parent and its delegates share an execution deadline and budget. An early valid output protects the work, while remaining time goes into further source investigation and refinement. The purpose of parallelism is more useful evidence within the review window, not more agents or a longer final report.

Improve the responsible component

I developed these profiles by examining completed reviews, their source evidence and their execution traces. A missed candidate and an incorrectly classified candidate call for different interventions: one is a discovery problem, the other a verification problem.

The component boundary determines where a reusable improvement belongs. Missing or misleading source presentation belongs in the task factory. Missing repository knowledge belongs in construction and retrieval. Weak investigation or evidence integration belongs in the shared reviewer behavior. An unreliable assessment belongs in the assessment method, rather than being treated as an actor defect.

This kept development focused on transferable changes to the creator, base and supporting machinery. Reserved comparison work tested a frozen composition rather than becoming another repository-specific repair instruction. The external application below exercises that distinction: the same reusable construction method builds a new specialist from the other organization’s own sources.

Construction and use as cloud workflows

I exposed two cloud development workflows through GitHub Actions: build a reviewer and review a PR. Construction collects the repository inputs, prepares a creator task, runs the creator, validates and assembles its output, and stores a content-addressed reviewer bundle. Use downloads a selected profile, resolves a PR’s source and context, constructs a review task and returns the review and artifacts.

The October workflows run complete profiles through standalone Harbor and native Claude Code, with direct Anthropic inference and disposable Daytona environments. They preserve the profile’s setup, files, skills, tools, hooks and output semantics. This is the same agent-profile format used by the platform, with a different inference and orchestration route for this application.

Source acquisition stays on the control side, using repository-scoped read authority. Execution credentials are injected separately rather than embedded in the profile or generated knowledge. Repository content and discussion are research inputs, not trusted configuration. Tools and launch settings come from the shared base; assembly and export checks prevent generated knowledge from replacing that executable contract.

The workflows retain exact profile and source identities, outputs, traces, model usage, reported inference cost and timing. Profiles have durable private storage separate from temporary diagnostic artifacts. An authorized operator can return to the result without essential profile files or a controller left running on my laptop.

Follow-up review adds another kind of state. The implementation carries prior findings alongside the old and new source, with stable finding origins and versioned events for persistence, change, resolution and recurrence. Uncertain identity remains explicit rather than merging allegations by guesswork. The full current review stays intact; a separate private-history delta highlights new, changed and resurfacing findings.

I also implemented on-demand PR invocation with operator, repository and profile authorization and source pins established at admission. Request identity and finding identity solve different problems: request deduplication prevents duplicate execution, while finding identity tracks the same issue across source changes. Source and authorization are checked again before publication, which anchors the review to the admitted commit. These integration paths were exercised through cloud workflows and PR publication.

These are implemented and exercised development workflows, not a generally available service. Their interfaces make construction and use remotely operable while keeping repository, operator and execution authority explicit.

Following a contract across components

I used the reusable construction method beyond SynthLabs to build an OpenFGA-specific reviewer from programmatically acquired source and engineering history preceding the comparison period. The generated profile then reviewed later historical changes without an OpenFGA-specific repair packet or changes to the shared creator and reviewer base.

OpenFGA is an authorization service; its stores group authorization models and relationship data. One consequential example came from a historical snapshot of OpenFGA’s MySQL read-replica work. The change routed a store lookup to a replica. The important consequence appeared in a consumer of that lookup: the command that deletes a store.

A store could be committed on the primary before replication made it visible to the secondary. The delete command first performed the replica-backed lookup. “Not found” was treated as successful idempotent deletion, so the command could return success without issuing the actual delete against the primary. Once the replica caught up, the supposedly deleted store remained visible.

The issue was not merely that a replica can return stale data. A stale read was being used as authority to skip a mutation, violating the operation’s intended postcondition. Establishing that required tracing creation, consistency routing, the lookup’s error translation, the command’s early return and the primary-bound delete. The generated reviewer identified this path; source-backed assessment confirmed it, and it was absent from the recorded baseline review.

In the historical comparison, the constructed reviewer found more source-confirmed actionable and consequential defects than the recorded first-pass CodeRabbit reviews. The assessments were internal and performed with coding agents, checking pinned source and affected consumers. They compared complete reviewers; they did not isolate the contribution of generated knowledge from the model, shared harness or source preparation. The example describes the evaluated historical snapshot, not a claim about a current OpenFGA vulnerability.

The value of the system is broader than the individual finding: a repeatable construction method produced a repository-specific agent, and a separate task factory let that same agent investigate real changes under controlled source conditions.

How this connects to learning

Tasks, profiles and assessment create independent improvement levers. Better source preparation changes what the actor can inspect. A better creator changes the reusable knowledge it receives. A better base changes how it investigates. A better judge changes how outcomes are assessed. Weight training changes the model itself.

The reviewer construction described here specializes knowledge and harness behavior without updating model weights. The distributed platform supplies the separate execution, evaluation and supported weight-training primitives: assessed task experience can produce a model checkpoint that returns to the same harness.

My contribution was connecting these layers into reusable engineering workflows: task production, independently configurable actors and judges, repository research, knowledge assembly, agent investigation and remote operation. I built this with the SynthLabs team and extensive use of coding agents, using Harbor, native coding harnesses, model providers and sandbox infrastructure as foundations.

The resulting capability is to turn existing software work and engineering knowledge into executable tasks and reusable specialists—and to improve the work, the worker and the assessment without rebuilding the entire system each time.

Related articles

• Model specialization and training agents — the earlier work on model updates, executable training distributions and agents that conduct training.
• The distributed execution and learning platform