Versions and splits

A version is a frozen copy of the approved working copy. Editing examples afterwards never changes it, and its digest is a function of its content alone, so a report that names a version digest names exactly the examples that were evaluated. Every example lives in one of three splits — development, validation, holdout — and the holdout is held apart by governance so nothing that produced a candidate ever saw it.

Status (25 October 2026). The datasets component is implemented and its requirements EV-010 to EV-018 are marked implemented but unverified in the requirements ledger; verification is in progress.

Freezing

Datasets → (dataset) → Versions → Freeze (or POST /api/v1/datasets/:id/versions) copies every approved, live example — payloads and digests, split, family, tags, metadata, attachments, fixture — in stable-key order into rows that are never updated. The version records:

• version (a counter per dataset) and digest: sha256 over the ordered list of content digests, so the same content always has the same digest whatever order it was written in;
• the schema revision and its digest; the kind; whether the dataset was protected;
• example_count, counts per split, and how many pending and rejected examples were left out;
• optional notes (up to 4,000 characters), who froze it and when.

Freezing content identical to the latest version, under the same schema, creates nothing and answers that version with created: false. A working copy with no approved examples cannot be frozen (NOTHING_TO_FREEZE), and a version holds at most 100,000 examples. Every freeze queues a leakage scan against every frozen version in the project (Slices and leakage).

What a consumer gets

Other components read a version through two calls: getDatasetVersion (a number, or null for the latest frozen version; NO_FROZEN_VERSION when there is none) and iterateExamples, which pages through a version — optionally one split or one slice — in pages of up to 500. Each example carries its stable key and position, input and expected output with their digests, has_expected and expected_origin, metadata, tags, split, family key, attachments, repository fixture, provenance, and a synthetic block: whether it is synthetic, its validation status and whether it counts as ground truth. Suites pin a dataset version this way; see Suites and thresholds.

Splits

Split	Role
development	the working set; the only examples a synthetic generator is shown as references
validation	held back from development; still a curator's to change
holdout	governed: adding, removing or altering a member is a change set, no model ever suggests its expected outputs, and exporting it needs dataset.protected.write

The split policy decides where a new example goes; existing examples keep their split.

• Manual: the row names its split, or the import's default applies. Examples sharing a family_key must share a split, and a conflicting row is refused.
• Hash (deterministic): the split is sha256 of the dataset's id and the example's family key (or its stable key when it has none), mapped onto 0–99 and cut by the dataset's percentages for development, validation and holdout. The same key always lands in the same split under the same percentages, and a family cannot straddle splits because the family is the hash basis. A row that names a different split is refused with the split it belongs in.

A protected dataset's split policy and percentages change only through governance (Protected data and governance).

Comparing versions

Versions → Compare diffs two versions, or a version and the approved working copy, by stable key: what was added, removed and changed, field by field (input, expected, split, family key). The same is GET datasets/api/versions/diff.

Can a consumer use every example?

A version may reference files — images, audio, video, documents — and repository fixtures. Versions → Inspect → Check (checkModalities) takes what a consumer says it can take — modalities, content types, whether it has a repository — and answers supported, or lists the examples it cannot use and why. An unsupported modality fails explicitly instead of being dropped or garbled. The modalities and their content types:

Modality	Content types
text	text/plain, text/markdown, text/csv, application/json
image	image/png, image/jpeg, image/gif, image/webp
audio	audio/mpeg, audio/wav, audio/x-wav, audio/ogg, audio/webm, audio/flac
video	video/mp4, video/webm, video/quicktime
document	application/pdf, text/html, Word, Excel and PowerPoint (OOXML)

Reading a version

The Versions page lists each version with its digest, example count, what was left out, schema revision and notes; Inspect shows the digest, schema, modalities and the leakage verdict (Holdout contaminated or not), and browses the frozen examples by split. Bodies need dataset.read_body. Through the API, GET /api/v1/datasets/:id/versions/:version/examples?split=&cursor=&limit= pages the frozen examples, bodies included, under the key's read scope.