Roles and access
Organization roles come from Bookbag SSO; project roles are EvalBook's own and are separated duties, not a ladder. The capabilities each role holds, and how they combine.
Access in EvalBook is decided on the server for every action, from two sources. Organization roles come from Bookbag SSO: owner, admin, member, viewer. Project roles are EvalBook's own, held by a project member, and they are separated duties, not a ladder: a reviewer cannot write a suite, a scorer author cannot edit a dataset, a release admin cannot run an experiment unless they also hold that role. A member with several roles holds their union.
Status (25 October 2026). Project roles, the capability matrix and the server-side guard are in the application; requirement EV-052 (fine-grained project access) is in progress in the requirements ledger and being verified now. The authorization gate in the application's test suite reports every routed action as guarded. Nothing on this page is marked verified.
Organization roles
Organization owners and admins administer every project of their organization without a project membership. They are listed separately on a project's Members page because no row is needed. Members and viewers at the organization level need a project membership to see a project at all.
Platform administrators are a third thing: SSO's per-product role for EvalBook, re-read at every sign-in, granting platform capabilities only. See Platform admin.
Project roles
Each row below is "a viewer's reads, plus this". The "+" never means "the row above".
| Role | Can |
|---|---|
viewer |
Read project metadata, reports, aggregate results, suite, scorer and dataset metadata, and impact. No raw bodies, no usage. |
trace_reader |
+ read traces and the span bodies the capture policy allows, save queries, read investigations. |
dataset_curator |
+ create and edit datasets and examples with their bodies, import, export. |
scorer_author |
+ create and edit scorers, rubrics and draft suite versions. Never activate. |
experiment_runner |
+ start and cancel experiments and evaluations, use the playground, open investigations, write proposals. |
reviewer |
+ review queues: read, annotate, score, adjudicate when assigned. |
release_admin |
+ activate suite, scorer and judge versions and protected-dataset changes (subject to MadeBook or separation of duties), read example bodies, export reports, dispatch proposals, read usage and cost. |
project_admin |
Every capability: members, environments, targets, API keys, runners, privacy and retention, links, trace ingestion, export and deletion, review, online and alert management, impact writes. |
Capabilities
The role-to-capability matrix lives in one file in the application and nowhere else. The capabilities are:
project.read project.admin member.manage apikey.manage runner.manage link.manage
trace.read trace.read_body trace.ingest trace.export trace.delete query.save
dataset.read dataset.read_body dataset.write dataset.export dataset.protected.write
scorer.read scorer.write suite.read suite.write config.activate
experiment.read experiment.run report.read report.export
review.read review.annotate review.adjudicate review.manage online.manage alert.manage
investigation.read investigation.write proposal.write proposal.dispatch
impact.read impact.write usage.read settings.privacy
Two splits matter most. trace.read is a trace's structure, timings and attributes; trace.read_body is its input and output, and the capture policy can still withhold them. dataset.read is dataset metadata (names, versions, digests, counts, splits, tags, provenance); dataset.read_body is example content.
How a request is decided
Every routed action authenticates, then asks the guard for a capability against a project, or against an entity by id. The guard answers itself when it refuses, so a controller cannot forget to. Every tenant-owned row carries organization_id and, where it has one, project_id; a row of another tenant asked for by id is 404 NOT_FOUND, not 403. 403 NOT_A_MEMBER appears only when an organization itself is named, for a peer's assertion about a person, or for an API key whose creator left.
The application's authorization gate statically checks every routed action is guarded and fails the build otherwise; its cookie-isolation test proves a session for one product cannot be presented to another.
API keys and runner tokens
An API key acts as its creator with its scopes as a ceiling; a runner token acts within its creator's runner.manage. Neither can exceed the person. See API keys.
Separation of duties
Activating a suite version, changing a threshold, activating a scorer or judge version, editing a protected dataset or a holdout, and ending a governance link are governed actions. In a project linked to MadeBook, MadeBook decides. In a standalone project, EvalBook's own rule applies: the author of the change cannot activate it; a different member with release_admin must. A standalone mode can never be chosen to weaken a linked project. See MadeBook as a peer and Protected data and governance.