Metrics and sources
Where impact events come from, the event vocabulary, signed webhooks and CSV imports, the versioned metric dictionary, and how every figure keeps its source and denominators.
Impact is computed only from events your systems send. A metric is a versioned definition over an event vocabulary; a source is a connection that delivers events; a figure is a computation over those events as of a moment, with its unit, window, sample count, missingness and maturity. Where no source provides what a metric needs, the figure is not measured, never zero. EvalBook shows team, service and factory views only and never scores individual people. In the app: Impact → Overview, Metric dictionary and Sources.
Status (25 October 2026). The impact component is built in the application (Impact and Usage & cost pages are live in the shell); its requirements IMP-001 to IMP-022 and EV-049 are not yet marked implemented in the requirements ledger and are being verified now.
The event vocabulary
One schema applies to every event however it arrives: an id unique per source, a type from a closed list, occurred_at, the key its type needs (work_unit.key, deployment.key or incident.key), and optionally dimensions (team, service, environment, factory, repository), a measure, links with a quality (verified, reported, inferred, unknown), attributes, provenance (measured, reported, estimated, finance_verified), corrects and test. The types cover work items (work_item_completed is declared done, not acceptance), pull requests, deployments, incidents, review effort, interventions, evaluation_completed and business_event.
An event is at most 64 KB and is stored once: a repeat is a duplicate, and the same id with a different body is a recorded conflict that is never applied. A correction is a new event naming the one it corrects. Only a verified or person-confirmed link counts as a defect link, and finance_verified provenance is accepted only from a finance-verified source.
Sources
A person connects a signed_webhook or a csv_import source. EvalBook makes the others itself: evalbook_internal for its own evaluation results and proposal states, and codebook_peer when CodeBook's first outcome event arrives over an active link.
A source declares the event types it provides, which is what "not measured" is judged against. It has a status (active, paused, revoked), a trust level (standard, or finance_verified with a written attestation of who in finance verifies the figures), and a test mode whose events are shown only in the labelled test view. Its health is derived: receiving, awaiting_first_event, stale after 7 days, not_connected. Names a source spells for teams, services, repositories and environments are reviewed under Names to review and never guessed. A CSV of at most 5 MB is checked first (dry run) and applied only when no row is invalid.
Signed webhooks
A webhook source receives POST /api/v1/impact/webhooks?connection=<source id> with a body { "events": [ … ] } of at most 500 events. The sender signs the raw body:
X-EvalBook-Signature: t=<unix seconds>,v1=<hex HMAC-SHA256(secret, "<t>.<raw body>")>
Deliveries outside the tolerance (30–900 seconds, default 300) are refused. The secret is shown once; a rotated secret's predecessor stays valid for 24 hours. There is no session and no API key: the delivery is authenticated by the signature and authorized as the person who connected the source, who must still hold impact.write. The capability register lists webhook.source.receive as in_progress with assurance observation_only.
CodeBook outcomes
CodeBook's outcome.observed event (merged, reverted, reopened, deployed, production issue and the like, on work unit codebook:card:<id>) is a proposed amendment to the peer protocol. The receiver exists; CodeBook does not send it yet, so the CodeBook source shows "not connected" and nothing is inferred from runs or tokens. The factory funnel on the overview (declared done → accepted → deployed → observed success) is EvalBook's own count from these events (CodeBook's account: https://codebook.work/docs/integrations/linked-products). See CodeBook.
The metric dictionary
Each project's dictionary is seeded from the system dictionary and may grow custom business metrics. A definition is a key, name and category (factory, quality, delivery, cost, business, experience); every change to what it counts is a new immutable version with its own digest, and a result keeps the version it was computed with. A version has a computation, unit, direction, maturity_days (0–365), terms (role, event type, counted once per work unit, deployment, incident or external id), exclusions, eligible source kinds, parameters and a caveat shown with every result.
System metrics: accepted_throughput, first_pass_acceptance, cycle_time, human_effort, rework_rate, escaped_defect_rate, intervention_rate, cost_per_accepted_change, billed_tokens_per_accepted_change, evaluation_pass_rate, and the delivery set deployment_frequency, change_lead_time, change_fail_rate, failed_deployment_recovery_time, deployment_rework_rate. Delivery metrics read deployment and incident sources only; a merged pull request never counts as a deployment.
A result carries a state (value, not_measured, insufficient_data and the like), its denominators, sample_count, maturity, a confidence interval with its method and seed, missingness by reason and a label such as measured, estimated or finance_verified. Export calculation lineage exports the definition, parameters and every record a result read, with a re-run that states whether the export reproduces the value. Results feed impact contracts and cost metrics read usage and cost.
Reading needs impact.read; connecting, importing, mapping and reviewing need impact.write. See roles and access.