Ponder Fix-it: learning from a tool call that went wrong

An agent calls the right function with the wrong argument. It reads the error, fixes the call, and finishes the task. Tomorrow, a fresh session makes the same mistake.

Ponder Fix-it connects that experience to a learning workflow. It captures a bounded record of the failed tool call and the attempts that follow, lets you review a useful recovery, and prepares it for specialist training. Your main agent can then consult a selected model for advice while keeping its own tools and permissions.

The distinction between those steps matters. Capturing a correction does not update a model. Finishing training does not prove that the update helped. We learned that directly in our first small SFT experiment.

Start with a setup prompt

On OpenPond, choose Get API Key, then Copy prompt. Give the prompt to your coding agent. It points to our maintained setup guide, reuses the existing OpenPond account and home, installs missing capabilities for that client, and verifies access with harmless reads.

Basic setup does not authorize uploads or training. In your Model's Overview, Set up tool learning supplies the separate prompt for the selected workspace, Model, and client. Review the tools and context you want to share before enabling capture. Follow the current guide for supported versions and the exact commands rather than copying an installer from a blog post.

Our completed staging qualification used the native OpenPond runtime 0.2.63 and Claude Code 2.1.292 with OpenPond CLI 0.2.66. It included actual failed commands, successful corrections, fresh sessions, and lifecycle checks. A different client version needs its own qualification; installing a hook alone does not prove that it observes the right events.

Keep the failure and the evidence together

Consider this deliberately small Python example:

Python rejects the unexpected count argument. The correction is to use quantity=3, preserving the requested amount. The important evidence includes the signature, the actual error, the attempted correction, and its observed result.

A later command exiting successfully is only a recovery candidate. It might have skipped the task or changed the requested value. The owner reviews the proposed target before it enters a supervised batch. Capture credentials cannot approve that target or authorize training spend.

The capture policy selects tool names and permitted fields and bounds retained events. Full conversations and native permission state are outside this capture path. You can disable an installation, purge hosted raw events, or revoke its credential. Reviewed Dataset material has separate retention; purging raw events does not remove an already-trained model's weights.

Train a specialist, then test it on different work

Supervised fine-tuning teaches a model from reviewed input and target pairs. For Fix-it, the target can advise a concrete revision, say that the call is already correct, or ask for missing context. A LoRA adapter stores the trained update while retaining the selected base model.

Our first experiment used Qwen3-8B, a rank-16 LoRA adapter, one approved recovery, and eight optimizer steps on one NVIDIA A100-SXM4 with 80 GB of memory. The run applied real parameter updates and produced an adapter. Training loss fell from about 3.94 to 0.47.

The held-out result went the other way. On three separate controls, the base passed one and the candidate passed none. The candidate also tried to revise an already-valid call. We kept it inactive.

We then added two reviewed, actually executed Python fixtures: a valid call that should be left alone, and a call missing information that the agent should request. Together with the captured recovery, that made three training examples. We kept the same separate regression panel.

On one warm A100, an eight-step run scored one out of three, matching the base. Allowing all eight passes over the three examples—24 optimizer steps—raised the candidate to two out of three. Both runs completed training, checkpoint loading, evaluation and canonical adapter publication, and reused the same GPU.

The remaining failure is instructive. The task already supplies the intended recipient, but the specialist asks for it again. It now handles the valid call and a genuinely missing-context case, yet still needs to distinguish “not passed to the function” from “not available in the task.”

This is an inspected, reused three-case regression panel, not a public benchmark or a broad quality estimate. The two added training examples were manually authored fixtures, not production captures. A loss chart and a completed training job cannot replace checking the behavior the update is meant to improve.

Bring qualified advice back to the agent

The consultation integration exposes openpond_consult for an observed tool failure. Advice comes back to the agent; the agent decides what to do next under its existing permissions and verifies the result. The specialist does not execute the tool or bypass an approval.

We have qualified fresh-session consultation against an unchanged selected model. A learned route additionally needs a completed training run, a fully passing selected comparison, and a matching served adapter. The new candidates are published artifacts, but neither is activated for learned consultation. Closing the remaining regression and verifying the exact learned adapter in fresh client sessions are still ahead.

Understand the cost before scaling it

Installation is free. Hosted consultation uses the selected model's input, cached-input, and output token rates. Training and serving have separate usage meters and require their own authorization.

The shared GPU session for the two newer runs lasted about 20 minutes, including model loading and idle time. Its native compute-and-disk cost estimate was about $0.60; the posted provider bill was still pending when we recorded that result. Job charges, optional warm-session charges and provider expenses are separate accounting views, so they should not be added together as one customer bill. This tiny qualification workload does not establish a price per successful correction or a production cost forecast.

Batching eligible work can spread GPU startup and idle time across more examples. Reusing warm capacity can also avoid repeatedly loading the model. Neither removes training tokens, evaluation work, serving capacity, or tenant isolation requirements. Automatic nightly Fix-it training and concurrent multi-user scheduling are follow-up work; this experiment does not establish their economics.

You can explore the Fix-it workflow and pricing and use the setup guide to connect your agent. Start with a correction you can verify, then measure whether the resulting update helps on work it did not train on.