Continuous learning

For Model training from approved tasks, start with Model training and continual learning. That guide covers human labeling, LLM judges, code verifiers and grader changes. This page covers the separate Harness refinement and recurring-review loop.

Continuous learning is the slower half of OpenPond's Harness system. The fast Refiner can learn from one completed turn; recurring review compares authorized evidence across tasks, checks whether earlier fixes worked, and recommends the smallest justified next step.

A review never launches training, deploys a model, or activates a candidate.

Configure the two loops

Open Settings → Harness.

• Refine completed turns is on by default. It runs the fast Refiner after settled Work turns and can be disabled without deleting prior history.
• Review recurring patterns enables cross-task review. Choose manual, daily, or weekly cadence and a maximum estimated cost.
• Review now queues the same recurring review immediately.

The card shows the last and next run, last result, pending work, current Harness release, and durable checkpoint. Desktop and hosted OpenPond use the same public review policy with different local or managed storage, model, and scheduler adapters.

What the recurring review sees

The host first enforces owner and workspace scope, consent, revocation, watermarks, ordering, and hard size/cost limits. Within that authorized window, the model chooses which compact previews deserve full inspection. The model then judges semantic recurrence, independence, severity, recovery, counterevidence, prior changes, and later outcomes.

This matters because identical error text can describe unrelated problems, while differently worded failures can share one cause. There is no keyword router and no fixed occurrence threshold. A single severe, well-supported defect can merit action; many weak or resolved examples can still produce no action.

Evidence omitted by a full-review bound is deferred rather than silently consumed. The next watermark, selected evidence, ignored evidence, reasoning, classification, cost ceiling, and downstream authority are recorded in an immutable receipt.

Possible outcomes

• No action: the signal is weak, isolated, confounded, or already resolved.
• Harness maintenance: an instruction, Skill, Agent, or memory needs review; ordinary per-turn changes still follow validation and release rules.
• Runtime or product: the durable defect belongs to capability or application owners, even when the foreground agent recovered.
• Taskset: controlled examples and grading are needed before a model hypothesis can be evaluated.

Model improvement is intentionally not a direct review classification. It becomes possible only after an approved Taskset, a real baseline Evaluation, and @openpond/evals qualification show that the problem is measurable and that changing weights is more appropriate than a Harness, runtime, or product fix.

Evaluate before training

Before choosing any training run, inspect the Taskset and baseline:

• examples match the capability you intend to improve;
• source, corrected, synthetic, and expert-authored provenance is clear;
• related examples do not leak across training and evaluation splits;
• graders pass positive, negative, boundary, and adversarial fixtures; and
• the baseline demonstrates a measurable problem with a trustworthy signal.

Creating a Taskset is one approval. Starting a billable run is another; it must name the immutable Taskset revision, starting model, method, compute location, and maximum spend. Candidate Evaluation and model activation are separate decisions again.

Privacy and control

• Disabling recurring review revokes future scheduled scans without deleting earlier receipts.
• Reviews cannot select another owner, workspace, arbitrary task, or revoked source.
• Connected apps, raw browser history, secrets, and unrestricted filesystem content are outside the recommendation-only review.
• Taskset materialization, training, deployment, and activation require explicit authority.

See Harness for per-turn refinement, releases, rollback, and the public package API.