Tasksets

A Taskset is an immutable, reviewable collection of tasks and expected evidence. It is the bridge between a useful real-world example and a measured improvement decision.

What a task records

A task records its input, expected outcome or grading rule, provenance, split, and any attachments needed to reproduce the evaluation. Keep source evidence and approval separate: collecting an example does not automatically authorize it for training.

Typical flow

1. Capture a representative task with its outcome and provenance.
2. Review/redact it and choose its split.
3. Add a grader and run an evaluation.
4. Freeze the reviewed version before using it for a candidate model.

Tasksets support evaluation first. A training run is a later decision with its own budget, candidate, and activation gate.

Review tasks for learning

Human review applies the attached scoring criteria to a response. Keep that label separate from whether the task is suitable for training: a failed response can identify a useful task for the model to attempt again.

See Review and label tasks and Model training and continual learning for the paths from task intake to evaluation, grader improvement and training.

Next: Graders.