The EvalBook manual.
Every page is written from the application’s contracts and code. Where an area is still being finished or not yet verified, the page says so at the top in a Status note. Search for a task, or start from a category.
- Pages
- 54
- Categories
- 11
- With a Status note
- 43
- Verified areas
- 0
M-01 · 6 pages
Getting started
Sign in with Bookbag, organizations and projects, environments and targets, API keys, and the project roles.
M-02 · 6 pages
Traces
Ingestion over the SDK, HTTP and OTLP, the explorer, redaction and capture policy, sampling and backpressure, retention, export and deletion.
M-03 · 6 pages
Datasets
Examples and schemas, immutable versions and splits, slices and leakage detection, import and export, curation and synthetic data, protected data under governance.
M-04 · 6 pages
Scorers
The registry and versions, deterministic scorers, executable verification in a runner, model judges and their calibration, pairwise comparison.
M-05 · 6 pages
Evaluations
Suites and thresholds, experiments, immutable reports, release gates, the playground, and simulation versus replay versus rerun.
M-06 · 4 pages
Review
Review queues and annotation, adjudication and agreement, online scoring policies, and alerts.
M-07 · 5 pages
Improvement
Failure clusters, investigations, improvement proposals, remediation through CodeBook, and release channels.
M-08 · 3 pages
Impact and usage
The metric dictionary and source events, impact contracts with baselines and cohorts, and usage and cost allocated from SSO.
M-09 · 4 pages
Integrations and peers
The capability register, peer links, CodeBook as a peer (station gates, traces, outcomes, proposals) and MadeBook as a peer (evidence, governance).
M-10 · 5 pages
Administration
Platform admin, readiness and operations, the security model, self-observability, and implementation status.
M-11 · 3 pages
Local development
Running EvalBook locally, the local end-to-end SSO, and deploying it.
A page with this mark carries a Status note: its area is being finished or is not yet verified.