Datasets

Examples and schemas, immutable versions and splits, slices and leakage detection, import and export, curation and synthetic data, protected data under governance.

• Datasets and examplesA dataset's schema, its working copy of examples with stable keys, splits, tags, metadata and review status, the rules every write obeys, and the SDK endpoints.
• Versions and splitsFreezing an immutable, digested version of the approved working copy, the three splits and the two split policies, comparing versions, and checking what a consumer can use.
• Slices and leakageNamed subsets of a dataset with versioned definitions and deterministic samples, required slices that suites must report on their own, and the exact and near-duplicate scans that find leakage across splits and versions.
• Import and exportImporting JSON, JSONL and CSV with a dry run, diagnostics for every row and a resumable apply, and exporting a frozen version as reproducible JSONL or CSV with a manifest.
• Curation and synthetic dataCandidates curated from trace spans, reviews, proposals and incidents, synthetic generation under a token budget, AI-suggested expected outputs, and why none of it is ground truth until a person approves it.
• Protected data and governanceProtected datasets and holdout membership change only through governed change sets with a digest, applied when an approval for that exact digest arrives, and never authorized by EvalBook itself.