Datasets
Create or import versioned tasks, review their sources and splits, and connect them to experiments and Model learning.
Open Datasets in Evaluation. A Dataset contains task inputs, reference information, provenance, grading dependencies and any execution environment needed to reproduce an attempt. Published Taskset releases are the immutable evaluation contract behind these datasets.
Choose a source
New Dataset opens a full page with source cards and the selected source's controls below them. Choose Upload file, Hugging Face, SDK or agent ingestion or Manual. Switching sources preserves your inputs in the current browser document; choosing a source neither imports tasks nor saves a Dataset. The same source controls appear in a Model's Dataset section, which also offers Existing Dataset to attach a saved record and its graders.
| Source | Setup and result |
|---|---|
| Manual | Write task inputs and reference answers, then name and save the Dataset. |
| Hugging Face | Enter a public, ungated repository or URL and choose Import tasks. Up to 25 verified sample tasks populate the page in place. Ambiguous input/reference columns require a choice; source options select revision, configuration and upstream split. |
| Upload file | Select a UTF-8 CSV, JSON array or JSONL file, up to 512 KiB and 1,000 rows. Confirm the input/reference mapping and choose Import tasks. |
| SDK or agent ingestion | Choose Application SDK, Agent activity or Coding-agent setup prompt. Save the Dataset destination before copying application instructions with its exact ID. Agent activity connects to its Imported conversations Project; it does not redirect that collector into the selected Dataset. |
| Existing Dataset, inside Model setup | Attach a saved workspace Dataset and its exact revision and graders. This uses the existing record rather than creating a duplicate. |
Web file and Hugging Face source captures are limited to 512 KiB. The reviewed import must also fit its complete save-size limit, including retained source evidence. Use the official CLI or SDK for sources beyond the web importer limits; those APIs retain their own request limits.
Hugging Face imports are bounded, one-time samples. They do not download the whole repository or subscribe to future changes. An agent collector's import schedule is also separate from any Model training schedule. Marketplace acquisition is described in Marketplace.
Review the draft before publication
Use a readable Dataset name. The setup generates an editable identifier; Details holds optional fields, including the imported Dataset's purpose. Imported Tasks start collapsed; expand them when you want to inspect, edit or remove examples. Importing stays on the page with its source controls. After changing an imported source or mapping, choose Update import before saving so the tasks match the current source settings.
For imported tasks, review the mapped input, reference answer and family identity where available. The default import role is validation. Choose the train, validation or holdout role deliberately. Check the input, expected outcome, permitted context, source and split for each task. Keep reference answers and private grader state separate from what the model sees. Label corrected, synthetic and source-derived examples clearly.
Choose Save Dataset to retain the working Dataset and open its Tasks view. Saving a Dataset does not publish a release, admit tasks for training or enable a schedule. Attach checked graders, review source permissions and publish a version when an evaluation or learning policy requires an immutable release.
The page retains source evidence and requires source-permission review before publication. Source switching preserves the draft within the current browser document. Reloading drops unsaved fields; save before leaving. Retry a saved import using its retained operation rather than starting a duplicate import.
Add application tasks
The TypeScript openpond-sdk and Python ponder-sdk clients submit structured
tasks to an editable Dataset. In SDK or agent ingestion → Application SDK,
select TypeScript or Python and copy the destination-specific instructions.
A task needs a readable name and useful input; a correct reference answer and
source provenance are optional. Keep credentials server-side and retain a stable
source event/idempotency key across retries.
A successful receipt includes the saved task ID and server upload time. The row appears in ordinary Tasks with its source and review state. Adding it does not authorize training, publish a version or enable collection. Reusing the same key and task returns the original receipt; changing the content with that key is a conflict. New appends do not rewrite previously published releases.
Optional OpenTelemetry span exporters require an explicit projection into these structured tasks. This endpoint is not an OTLP receiver and does not automatically capture arbitrary logs. See SDK task submission for packages and the maintained API reference.
Collect conversations from a served Model
Successful saved chat-model calls automatically add tasks to that Model's existing accessible Dataset, whether called from Chat, its Playground or an external application. Each task retains the actual model-visible messages, recorded response and served-version evidence. Supported Chat attachments retain their original file references; JSON responses preserve the original text and parsed content when valid. Chat shows Saved to Dataset only after durable task intake succeeds.
Recorded answers are not automatically reference answers. Review, grading, publication and Model learning admission remain separate. Appends do not rewrite previous releases. For a file-backed Dataset, Tasks shows collected conversations alongside imported source rows; those collected tasks do not silently alter the imported file or an existing training artifact.
Inspect the Dataset
| Tab | What to inspect |
|---|---|
| Tasks | Stable task references, upload time, Model learning state, outcomes and individual grader results. |
| Graders | Exact attached scoring releases, compatibility and required checks. |
| Experiments | Associated experiments and their retained evidence; training history is also accessible under Models → Runs. |
| Versions | Published releases and the exact revision used by an experiment or Model. |
| Continual learning | Related Models and their canonical learning policies, readiness counts and run history. |
Check the selected version when viewing tasks or graders: an edit to a working draft does not rewrite a previous release or experiment. Attach the Graders needed to measure success. If tasks need tools or a simulator, inspect their Environment and readiness. Publish the reviewed Dataset so an experiment can pin its release.
Connect a Model's continual learning
Open Continual learning, select a compatible Model, and inspect or configure its saved policy. The Model's own learning view edits the same policy; one Dataset can feed several Models. The first configuration creates the Model's policy; open that existing policy for later edits.
Review the bound Dataset release, minimum approved-task count, schedule and timezone, next check, spending limits and candidate behavior. Review/ready/used counts describe the selected Model's evidence, not automatic approval of all Dataset rows. Follow the review, current iteration and run-history links for the retained records. Pause future cycles separately from stopping current work.
Ordinary Model learning needs an explicit human admission for submitted evidence in its Tasks → Needs review queue. Saving, publishing or adding rows does not automatically enroll every row for review or training. The selected release is pinned; the policy does not automatically follow every future revision. Conversation policies have their own qualified verification and admission path, with exceptions and candidate decisions in Inbox.
If the selected Dataset representation cannot resolve a canonical Model policy, follow its available Model entry point; an unavailable relationship does not mean learning is enabled. See Continual learning for the schedule, admission and candidate gates.
Inspect tasks and manage settings
The first column uses task-YYYYMMDD-HHMMSS from the task’s first recorded save
time in UTC. Retries keep that reference. Same-second references can repeat;
selection and links retain each task’s distinct canonical ID. Historical tasks
without a retained timestamp show an unavailable date.
Choose a Model and Latest evaluation per task, or an exact run. Each row’s grader values come from one retained attempt with its original grader versions. Use the inspector to choose a seed/fixture attempt, read content, output and feedback, or open the retained trace. A task absent from a selected run says Not in this run; missing or pending grades are distinct from zero.
Filter combines checkbox groups for Task use and Learning. Choices within a group match any selected value; the two groups apply together. Clearing a group restores all values. Columns reveals input previews, output and optional usage fields. Input and reference content stay in the task inspector by default.
Learning state belongs to the selected Model. Ready means reviewed data; Queued for run requires an actual reservation. Included in training requires completion evidence, and remains separate from accepting or serving a candidate. Overview and Dataset inventory show the saved Next learning check with its Model/timezone, plus honest paused, manual or waiting states.
Use Settings for Dataset management, upload credentials and marketplace listing. Listing still requires an eligible exact release and the appropriate workspace permission. The Tasks header keeps Add task and Set Up Continual Learning.
Submit published tasks to Model review
For an ordinary Model policy, use the explicit handoff in Continual learning:
- Select a related Model with a current human-admission policy bound to this exact published Dataset release.
- Choose Submit Dataset tasks for review. Inspect the paged training rows and select up to 200 tasks for one submission.
- Choose Submit selected tasks for review. Retain the receipt and follow its link to the Model's Tasks review queue.
- Review and admit the submitted evidence there. Submission alone grants no training permission and starts no training job.
Submission requires the Model owner and workspace permission to control managed training. The saved Model, current policy, enabled direct source, task format and graders must agree on the exact release. If they have changed, review the Model settings before submitting; a newer Dataset version is not adopted silently. Only the published training split is selectable for this handoff.
The receipt maps selected tasks to their canonical evidence. Retrying the same submission reuses its retained result. Unchanged task content retains its evidence, admission and consumption identity; a semantically changed task needs fresh evidence and human review. Training, validation and evaluation family boundaries remain protected. The handoff creates no global Inbox assignments, new learning policy or automatic subscription to future releases.
Evaluate before training
Start an Experiment from the Dataset and review the selected task population, target, scoring and limits. An imported benchmark is not automatically executable in every environment.
Human review and approval for training remain separate. A failed response can identify a useful training task without becoming a demonstration to copy. Keep related examples out of both training and held-out splits, then use the approved tasks in Ponder Training.