Usage and cost

EvalBook never bills, invoices or charges. Plans, usage and the token ledger belong to Bookbag SSO. What EvalBook keeps is an allocation: this charge belongs to that unit of work, so that cost per accepted change can be computed from figures someone else is accountable for. Every figure on the Usage & cost page is either SSO's own, or an allocation labelled "allocated from SSO usage" with its ledger reference, or a cost labelled by its own source, or unknown.

Status (25 October 2026). The impact component is built in the application (Impact and Usage & cost pages are live in the shell); its requirements IMP-001 to IMP-022 and EV-049 are not yet marked implemented in the requirements ledger and are being verified now.

What SSO reports

The first panel shows the organization's AI allowance as SSO answers apps/api/ai/usage: used, limit and remaining tokens for the current window, whether the organization is over, and EvalBook's runs by feature. These are the authoritative figures, shown for context. When SSO cannot be reached the panel says so rather than showing a number.

The page needs usage.read, which the release admin and the project admin hold; a viewer does not see usage. See roles and access.

EvalBook's own model calls

Every platform-originated model call goes through one service with a purpose: judge, pairwise_judge, dataset_generation, investigation, optimization or summary. EvalBook records an AiCall row per attempt with the logical request id, attempt number, purpose, model asked, provider and model answered, the usage SSO returned, and the outcome, including outcome_unknown for a network failure after the request was sent. A retry reuses the same idempotency key, so SSO charges one logical request however many attempts it took.

The second panel groups these calls by purpose: logical requests, attempts, outcome unknown, refused or failed, and billed tokens where SSO answered. Judges are described under LLM judges, generation under curation and synthetic data and the improvement purposes under investigations. Online judge calls draw on a daily budget per policy (see online scoring); every call reserves against a budget first and a run stops at its bound.

Allocations

An allocation ties a charge to a unit of work:

Field	Values
unit.kind	work_unit, contract, evaluation, evalbook_ai, run, team, service
category	development_ai, evaluation_ai, improvement_ai, external_runtime, review_labor, rework_labor, operating, implementation_amortization
ledger_kind	sso_ai, provider_report, invoice_line, estimate, manual
source	ledger (SSO charges only), provider_report, estimate, unknown
amount, unit	a number in an ISO 4217 currency or billed_tokens; null when unknown, never shown as 0
share	the fraction of the charge allocated to this unit, in (0, 1]

One charge is allocated once. The same reference for the same unit is a duplicate; the same reference with different figures is ALLOCATION_CONFLICT and the first allocation stands; shares of one reference that would exceed the whole charge are refused with ALREADY_ALLOCATED. A rollup that lists the child references it covers contributes to a total only while none of them is allocated as a leaf, so a parent and its children are never both counted.

SSO charges

Reconciliation turns the project's AiCall rows into allocations, one per idempotency key, in the category of their purpose: judge, pairwise judge, generation and summary calls are evaluation_ai; investigation and optimization calls are improvement_ai. The reference is sso:<idempotency key> and the amount is the billed tokens SSO answered, or unknown for a call whose outcome is unknown. Each such allocation has a ledger status: unconfirmed until SSO's ledger (apps/api/ai/ledger) confirms it, then confirmed with the ledger's figure, void when the ledger voided it, or missing when the ledger has no such key. Check SSO's ledger asks now; an SSO without the ledger route leaves allocations unconfirmed with that reason.

Other costs

Record a cost allocates review or rework labour, external runtime, operating or implementation costs by a reference of your own (an invoice line, a timesheet id), with how the amount is known, or with no amount when it is not. The same reference is never counted twice. Recording needs impact.write.

How totals are shown

Allocations are summed by category and unit. Where any item in a category is unknown the known total is shown as a lower bound, marked ≥, and the unknown items are listed under Unknown costs; they keep cost per accepted change incomplete rather than making it look cheaper. Test allocations are kept out unless the test view is on, and never mixed in.

The dictionary metrics cost_per_accepted_change and billed_tokens_per_accepted_change read these allocations against accepted work units; a contract that requires cost categories nobody has allocated reports its cost as incomplete (see impact contracts and metrics and sources). Allocations fall under the project's cost_records retention class: those incurred before the cutoff are deleted, and results computed earlier keep their numbers (see retention, export and deletion).