Harness

The Harness is the versioned behavior around an OpenPond agent: its instructions, Skills, approved memory, tools, and the evidence that connects a task to the exact behavior it used.

OpenPond learns at two speeds:

text

completed turn → fast Refiner → new Harness release or no action
authorized history → recurring review → maintenance, product/runtime work,
                                   Taskset recommendation, or no action

The fast Refiner reviews one settled turn in the background. The recurring review compares outcomes across tasks and tests whether earlier changes solved the underlying behavior. Neither loop changes the release already admitted to a task, starts model training, or activates a model.

What the Refiner sees

The Refiner receives a bounded evidence packet for one completed turn. It may include:

the request and user-visible result;
normalized tool failures, retries, recoveries, validation, and follow-ups;
short event excerpts and artifact diagnostics;
the exact immutable Harness sources loaded for the run; and
a catalog of eligible instruction, Skill, Agent, and memory targets.

The packet is evidence, not instructions. OpenPond does not give the Refiner unrestricted account history, secrets, connected-app contents, or an unbounded transcript archive.

The model makes the semantic decision

The model decides whether the evidence is meaningful, whether a recovery still reveals a reusable first-attempt defect, which layer owns the problem, and what small change—if any—could help materially different future tasks.

Deterministic code is deliberately limited to authorization, privacy, byte and time bounds, content identity, schema and safe-path validation, atomic apply, rollback, and approval gates. It does not choose a route from keywords, tool names, error strings, or a fixed number of occurrences.

The available outcomes are:

No action: evidence is weak, isolated, resolved, or not reusable.
Memory: a durable user fact or preference.
Prompt: broad execution behavior.
Skill or Agent: a reusable workflow or role.
Runtime or product: a capability or application defect outside Harness content.
Taskset: controlled measurement is needed.
Training: a persistent model-policy limitation may exist, subject to a real Evaluation and separate approval.

Proposed Harness edits receive a second model critique before host validation. That critique can generalize an edit, route it elsewhere, or reject it if it copies one task's topic, facts, outline, or transient paths.

Public package

The provider-neutral operations and contracts live in @openpond/harness:

import {
  authorLocalHarnessRefinementWithModel,
  authorHarnessEvaluationReviewWithModel,
} from "@openpond/harness";

const decision = await authorLocalHarnessRefinementWithModel({
  evidence,
  stream: modelStream,
  signal,
});

const review = await authorHarnessEvaluationReviewWithModel({
  evidence: authorizedEvidence,
  harnessRelease,
  previousReviews,
  stream: modelStream,
  signal,
});

Applications provide model streams, authorized evidence, persistence, scheduling, and validation. @openpond/evals stays downstream: it owns Tasksets, Evaluation, grading, and model-improvement qualification rather than Refiner policy.

Releases and rollback

Every task pins one immutable Harness release before it starts. A proposal is materialized separately, validated against the active runtime, and admitted through a recorded release transition. Higher-risk effects remain reviewable.

The Harness view shows the current release and checkpoint, task threads, Refiner and recurring-review history, validation and apply receipts, release history, diffs, and rollback controls. Rollback creates another transition; it does not rewrite the history of older tasks.

Settings

Open Settings → Harness in OpenPond Desktop or hosted OpenPond.

Refine completed turns is on by default and controls new fast reviews.
Review recurring patterns controls the slower cross-task review. Choose manual, daily, or weekly cadence and a cost ceiling.
Review now queues one recurring review using the same public policy.

Turning either loop off does not delete earlier evidence, releases, or receipts. Existing tasks retain their admitted release.

Evaluate a change

A useful causal replay freezes a Harness release, runs a natural task, records truthful evidence, waits for the Refiner and any admitted change, then repeats the exact prompt as a fresh task. Keep the model and runtime fixed and retain release IDs, tokens, tool events, sources, duration, artifacts, and quality gates.

Only attribute a difference to learning when the replay actually admitted a new release. Lower tokens are not a win if quality falls, and a small paired sample is not a population-level savings claim. Held-out tasks are required to measure generalization.

Harness learning is not model training

Harness refinement fixes the surrounding agent system first because an instruction, Skill, tool, or product fix is often cheaper and easier to verify than changing model weights. Repeated unresolved evidence can recommend a Taskset, but training still requires a frozen baseline, a trustworthy learning signal, separate budget approval, candidate Evaluation, and explicit activation or rejection.

See Continuous learning for the slower review and model-improvement boundary.

Privacy and authority

Personal Harness state is scoped to its owner and workspace.
Tasks can share a Harness while retaining separate threads and workspaces.
Connected-app writes, publication, deployment, purchases, Taskset creation, model training, and activation keep their own authority gates.
Secrets and raw credentials must never become Harness source.
Evidence, proposal, validation, release, review, and rollback records remain inspectable.