Tasksets

A Taskset is an immutable, reviewable collection of tasks and expected evidence. It is the bridge between a useful real-world example and a measured improvement decision.

What a task records

A task records its input, expected outcome or grading rule, provenance, split, and any attachments needed to reproduce the evaluation. Keep source evidence and approval separate: collecting an example does not automatically authorize it for training.

Typical flow

1. Capture a representative task with its outcome and provenance.
2. Review/redact it and choose its split.
3. Add a grader and run an evaluation.
4. Freeze the reviewed version before using it for a candidate model.

Tasksets support evaluation first. A training run is a later decision with its own budget, candidate, and activation gate.

Next: Graders.