CodeBook as a peer

CodeBook runs the work; EvalBook measures it. When a CodeBook workspace is linked to an EvalBook project (a workspace administrator does this under CodeBook's Workspace settings → Linked products), four things cross the boundary, and nothing without the link. CodeBook's side shipped and is deployed; the message shapes below are the ones it sends and serves, recorded in both products' peer protocol documents. CodeBook's own page is at codebook.work/docs/integrations/linked-products.

Status (25 October 2026). CodeBook's half is deployed and verified against an EvalBook stand-in, and requirement EV-038 is verified on CodeBook's side. EvalBook's side — serving evaluation requests, suites, reports and CodeBook spans, and receiving outcomes and progress — is routed in the application but not yet marked implemented in the requirements ledger; it is being finished and verified now. The two products have never talked to each other for real.

A station can require a suite

In a station's settings, a CodeBook administrator picks an EvalBook suite from the linked project (CodeBook asks POST …/project/api/peer/suites for the project's suites with their active version, required checks and mode) and marks it required or advisory. When a run reaches the station, CodeBook asks EvalBook to evaluate the exact candidate:

{ "organization_id": 12, "sub": "7", "project_id": 31,
  "origin": { "app": "codebook", "workspace_id": "44" },
  "idempotency_key": "codebook:job:9912:evaluator:5",
  "mode": "release_gate",
  "suite": { "id": 5, "version": null },
  "subject": { "kind": "repository_change", "resource_id": "card:880", "revision": "job:9912",
               "environment_id": "development", "artifact_set_digest": "sha256:…",
               "repository_full_name": "acme/api", "commit_sha": "<40 hex>", "base_commit_sha": "<40 hex>" },
  "context": { "factory_id": 3, "workflow_id": 9, "station_id": 41, "worker_id": 15, "run_id": 9912, "trace_id": "cb-12-job-00009912" },
  "deadline": "2026-10-15T12:00:00Z",
  "callback_url": "https://app.codebook.work/api/workspace/api/peer/evalbook/events" }

suite.version: null pins the suite's active version at admission, and the answer names it. EvalBook answers 202 with the evaluation's id, status queued, the subject_digest it computed, the pinned suite and dataset versions, the required check ids and the deadline. The run waits in CodeBook as Waiting for EvalBook until the report arrives or the two-hour deadline passes.

When the evaluation ends, EvalBook sends evaluation.completed (or .failed, .cancelled) to the callback. CodeBook does not act on that message's fields. It fetches the report (POST …/project/api/peer/report), recomputes the digest over the canonical body, and decides: a pass only when the digest matches, the report is this organization's, this exact candidate's, this suite's and version's, current, not expired, with complete evidence and every required check passed, and the pull request has not moved since. EvalBook's fail, error or inconclusive, a check reported missing, stale or unsupported, a superseded, expired or tampered report, a passed deadline or an unreachable EvalBook are all "cannot tell", with the reason, and block a required evaluator as a fail would. Advisory suites are recorded and measured only. Cancelling the run cancels the evaluation (POST …/peer/cancel); the answer says whether it is cancelling or cancelled.

One authoritative gate per criterion. A station whose required evaluators include both a legacy agent judge and an EvalBook suite on the same criterion cannot be saved until a workspace administrator records a cutover saying which decides; the other runs as a shadow. See Release gates and Reports.

Runs can be exported as traces

Per link, a CodeBook administrator switches trace export on, with or without the runs' bodies. Every finished run goes to the project as one signed, idempotent batch (POST …/project/api/peer/traces, batch_id cb-job-<job>-v<n>) in the span schema: the run as a run span (trace_id cb-<org>-job-<8-digit job>), each model attempt as an llm span with its usage, each tool call as a tool span, each evaluator verdict as an evaluator span with purpose: evaluation, and links to the card, the run, its artifacts and the pull request. CodeBook spans carry codebook.sso_call_key_prefix instead of a per-call ledger reference, because CodeBook keeps no per-call SSO key. Bodies follow the CodeBook factory's retention and the EvalBook project's capture policy, whichever is stricter. A batch EvalBook could not take gives up visibly in CodeBook with Send again; export never holds a run. See Ingestion.

Outcomes are told to EvalBook

Everything CodeBook records about what became of finished work goes to each linked project as one signed, idempotent outcome.observed event: a card finished or dropped, its pull request merged, closed or reverted, a reopen, a deploy, a production issue, a person's override, a withdrawn report. The key is outcome.observed:<outcome id>:project:<EvalBook project id> (a workspace linked to two projects sends each its own), and EvalBook deduplicates observations by outcome.id. EvalBook decides what each means (a completion is self-declared done, not acceptance) and records them as impact source events. CodeBook's factory Outcomes page asks EvalBook back (POST …/project/api/peer/impact/factoryOutcomes) for What EvalBook observed: declared done, accepted, deployed, observed success, in EvalBook's words; a project EvalBook cannot answer for is said so, not shown as zeros. See Metrics and sources.

Proposals arrive as cards

An EvalBook improvement proposal (POST {codebook}/workspace/api/peer/improvements, key evalbook:proposal:<id>) carries a title, the problem, a hypothesis, the failure cluster and the reports it was drawn from, the subject, the workflow, allowed_paths, protected_paths, max_attempts, a deadline, a budget, a reproduction and a link back. CodeBook makes one card per proposal (a duplicate answers the same card), enforces the bound through its run limits rather than asking a model, and treats the protected paths as a refusal list that every repository-writing tool checks. Progress comes back as improvement.updated (queued, in_progress, in_review, blocked, done, dropped, reopened, with a revision that supersedes lower ones and an optional pull request). See Proposals and Remediation.

Phase two: CodeBook's existing evaluators

CodeBook's station evaluators and factory scorers stay authoritative for the stations that use them until a per-factory cutover, which is an administrator's act. A later phase imports CodeBook's scorer definitions and historical results into EvalBook with legacy_reported provenance and their original scales, never relabelled as verified, for shadow comparison. That phase is not started.

Related

• Peer links
• MadeBook as a peer
• The capability register