API keys

Machines reach EvalBook with one of two credentials. A project API key (evb_…) is for SDKs, OpenTelemetry collectors, CI jobs and anything that calls the public API under /api/v1/. A runner token (evr_…) is for the separately installed runner that executes scorers on your machine. People never use either; they sign in with their Bookbag account. Peers (CodeBook, MadeBook) use neither; they use the peer secret from Bookbag SSO.

Creating a key

Settings → API keys needs the apikey.manage capability, which only a project_admin holds. A key has a name, a set of scopes, an optional environment binding and an optional expiry.

The key is shown once, in the create answer, and never again. EvalBook stores only its sha256 hash and a 12-character prefix for listing. If the create request is retried with the same Idempotency-Key, the answer is token: null, token_already_shown: true; make a new key if you lost it.

Scopes are a ceiling

A key acts as the person who created it, and its scopes are a ceiling that is intersected with that person's project roles at the time of use. A key never reaches outside its organization and project, and no scope reaches apikey.manage, member.manage or runner.manage.

Scope	Lets the key
ingest	Send traces: POST /api/v1/traces, POST /api/v1/otlp/v1/traces
read	Read what its creator may read: traces, datasets, scorers, evaluations, reports
datasets:write	Create and edit datasets and examples, import
evaluate	Start and cancel evaluations and experiments through /api/v1/evaluations
admin	Everything its creator may do, minus the three excluded capabilities

Two consequences follow. If the creator loses a role, every key they made loses what that role allowed, at once. If the creator leaves the organization, their keys authorize nothing (403 NOT_A_MEMBER). Make keys from an account the team keeps, and name them after the system that holds them.

Environment binding

A key bound to an environment can only ingest batches for that environment. Bind collectors: a production exporter holding a production-bound key cannot write staging traces by mistake. See Environments and targets.

Revoking

Revoking a key is immediate and audited. Lists show the prefix, the name, the scopes, the environment, the last use and the expiry. There is no way to recover a revoked key's secret, because EvalBook never had it.

Using a key

curl -X POST "$EVALBOOK_URL/api/v1/traces" \
  -H "Authorization: Bearer $EVALBOOK_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: b-2026-10-25-0001" \
  --data @batch.json

The tenant is never taken from the body: the organization and project are the key's. A batch naming another organization or project is refused whole with 403 TENANT_MISMATCH. See Ingestion.

Runner tokens

A runner token (evr_…) is created on Scorers → Runners by a project admin (runner.manage), is shown once, and authorizes exactly one thing: the runner's claim, heartbeat, complete and fail calls, plus a request for a short-lived read-only repository credential for the one repository a job needs. Like an API key, it acts within its creator's authority: if they lose runner.manage or leave the organization, the token authorizes nothing. See Executable verification.

What a key is not

• Not a peer credential. CodeBook and MadeBook authenticate with the shared peer secret and an HMAC signature over the raw body; see Peer links.
• Not a way around roles. A key cannot read a span body its creator may not read (trace.read_body), nor activate a suite version (config.activate).
• Not stored in plain text anywhere, including EvalBook's own database.

Next

• Roles and access
• Ingestion