Security model

EvalBook authenticates through Bookbag SSO and keeps a session of its own. Every routed action is authorized on the server. Credentials are hashed, secrets at rest are encrypted, peer calls are signed, and every tenant-owned row names its organization and project.

Two rules shape the rest. EvalBook never authorizes: MadeBook decides, and a standalone project applies separation of duties. EvalBook never runs customer code outside the separately installed, outbound-only runner; the API and worker execute no customer code.

Sign-in and sessions

Sign-in is SSO's authorization-code flow with PKCE S256. The flow cookie evalbook_sso_flow is lax, lives 10 minutes and is sealed with a key derived from the app token. The session cookie evalbook_login is httpOnly and sameSite strict, per device, idle for 12 hours and absolute for 30 days with sliding renewal. It is bound to the SSO session id and checked against SSO on every request (10-second cache), so signing out anywhere in Bookbag ends it here. An account is matched by its SSO subject. No token travels in a URL. EvalBook has no password form. See Sign-in and organizations.

API keys and runner tokens

A project API key (evb_…) and a runner token (evr_…) exist in the clear only in the answer that creates them; only the sha256 and a 12-character prefix are stored, and an Idempotency-Key replay answers token_already_shown: true.

A key acts as its creator, with its scopes (ingest, read, datasets:write, evaluate, admin) as a ceiling intersected with the creator's project roles at the time of use. No scope reaches member.manage, apikey.manage, runner.manage, link.manage or settings.privacy: a leaked CI key cannot mint a key, change who is on the project, unlink governance or widen capture. A key never reaches outside its organization and project. A runner token acts within the authority of the person who created it. See API keys.

Authorization

Every routed action authenticates and asks the guard for a capability, with a project or an entity. A static check fails the build on any unguarded action, and every entity is mapped to its organization and project.

Organization roles come from SSO; owners and admins administer every project. Everyone else holds project roles, which are separated duties, not a ladder: a reviewer cannot write a suite, a scorer_author cannot edit a dataset. Activating a suite, scorer or judge version needs config.activate. When the project is linked to MadeBook, EvalBook asks MadeBook for a decision bound to the action's digest and activates only on allow; unreachable is defer, which fails closed. Standalone, the author of a protected change cannot activate it; a different member with release_admin must. See Roles and access and, for the platform role, Platform admin.

Peer calls

CodeBook, MadeBook and EvalBook call each other with the peer secret from SSO's peer directory as a bearer (the previous secret is accepted for one hour after rotation), X-Peer-App naming the caller, and a required HMAC-SHA256 X-Peer-Signature over the raw body. A call acting for a person has that person's organization role read live from SSO. Outbound messages go only to apps the directory names, under their API base. A consumer verifies a report by fetching it over the authenticated channel and recomputing its digest; a callback never carries the verdict acted on. See Peer links.

Secrets and outbound calls

EVALBOOK_ENCRYPTION_KEY (32 bytes) encrypts credentials EvalBook stores on someone's behalf with AES-256-GCM, stored as v1.<iv>.<tag>.<ciphertext>. There is no plaintext fallback: without the key, storing a secret throws and readiness fails. The kernel holds no provider key, no Git credential (repository access goes through SSO) and no peer secret at rest.

Every call to an address a customer supplied (an HTTP scorer, a target's health URL, an import source, a webhook) goes through one egress guard: http and https only, no credentials in the URL, every resolved address checked against loopback, private, link-local and metadata ranges, the connection pinned to the checked address so DNS rebinding cannot pass, at most 3 same-origin redirects, a timeout and a body cap. EVALBOOK_PRIVATE_HOSTS, set by the operator and never by a user, admits a named private host, never a metadata address.

Customer data and customer code

The capture policy (full, redacted, metadata_only) is applied before anything is stored; see Redaction and capture. Retention is per content class and deletion leaves tombstones; see Retention, export and deletion.

Executable scorers run only in the runner's container driver: no network by default, read-only root, non-root user, dropped capabilities, process and resource limits, the protected harness mounted read-only outside the candidate's tree. The process driver is for development only and marks every result development_unisolated, which a release-gate suite refuses. See Executable verification.

Audit

Every write made on someone's behalf is recorded in the same transaction, in an append-only log hash-chained per organization. A write that cannot be audited does not happen.