Spending, reservations and reconciliation

Q keeps provider expenditure separate from the prices charged to API customers. ADR 0028 explains the decision. Fair capacity still bounds concurrent work and tokens; monetary limits add a separate check.

Dispatch and limitsCopy Anchor Link

Every completion and embedding attempt through the gateway first commits a PostgreSQL reservation. REST, MCP, A2A and Temporal workers share the ledger. No cached allow verdict authorises spending. Database failure rejects dispatch with spend_store_unavailable; there is no Redis or in-memory monetary fallback.

Global Q_TENANT_DAILY_BUDGET_USD and Q_TENANT_MONTHLY_BUDGET_USD retain their legacy zero-means-disabled environment convention. Explicit tenant JSON zero means a zero-dollar cap. Configure existing q_tenant_feature_access.features through the authorised tenant control plane:

{
  "budget": {
    "daily_usd": "10.00",
    "monthly_usd": "200.00",
    "run_usd": "2.00",
    "user_budgets": {"daily_usd": "5.00"}
  },
  "api_key_budgets": {"synthetic-key-id": {"daily_usd": "3.00"}}
}

A root run and all its children share run_usd, the root's authenticated principal and persisted user identity. Daily/monthly periods are UTC. Unknown reservations carry into current periods until resolved, including after cancellation or process death. Limits use provider costs plus outstanding holds; customer tariffs never consume the provider budget a second time. Older GuardrailConfig.cost_limit_per_tenant checks are advisory; configure the shared caps above for atomic enforcement. Q_BUDGET_FAIL_CLOSED is deprecated: monetary admission always fails closed.

Price qualificationCopy Anchor Link

Set SPEND_MODEL_PRICES_JSON to an exact-model map, for example this synthetic profile:

{
  "synthetic": {
    "account": "fixture-account",
    "reservation_usd": "0.10",
    "input_per_token": "0.001",
    "output_per_token": "0.002",
    "per_call": "0"
  }
}

reservation_usd is a conservative total ceiling for one call, including the largest permitted input/output, multimodal inputs and any provider tool charges. These fixture rates are not live prices. Token rates are optional and only suitable where complete usage counts cover the tariff. Without them, even a complete SDK-reported cost retains its ceiling until provider reconciliation. SDK estimates cannot discount the qualified snapshot tariff. Explicit zero-priced profiles may represent qualified free calls; an unknown model is never assumed free. Custom endpoints require an exact api_base match in the profile. Snapshots retain a configuration digest and account label, never endpoint credentials. Fallbacks require their own eligible profiles.

SPEND_LEDGER_ENABLED defaults true and may only be disabled in the isolated test environment. SPEND_TRANSACTION_SECONDS defaults to three seconds. Set identical accounting settings on API and worker processes. Temporal queues must only accept Q's authenticated workflow submissions. Unscoped internal maintenance uses the separately visible __system__ lane.

Machine-readable totalsCopy Anchor Link

GET /v1/agents/{agent_id}/runs/{run_id} includes usage_json.spending for authorised callers. It reports own/root attempt counts, estimated_usd (original reservations), held_usd, provisional_usd, actual_usd, unresolved/unpriced counts, root_run_id and budget_stop_reason. Monetary amounts are decimal strings. Replay/polling does not dispatch or charge again. Another real provider attempt, including a fallback, reserves and accounts separately.

GET /v1/billing/spend now reports provider expenditure in total_cost_usd and provider-only breakdown. Its additive accounting object identifies the basis, currency, opening confirmation, provider actuals, provisional costs, reservations, unresolved attempt count and separate customer charges. The period report selects calls by dispatch time; admission additionally carries unresolved holds from earlier periods. An unresolved unpriced attempt may have no dollar amount, so always inspect unresolved counts. All period fields come from one SQL statement snapshot.

Provider SDK/LiteLLM cost calculations and token-priced costs are provisional. actual_usd means an operator-matched provider statement (or verified opening balance). An incomplete stream never releases its hold merely because some usage arrived. Budget stops use stable reasons including spend_day_limit, spend_month_limit, spend_key_limit, spend_user_limit, spend_root_limit, spend_price_unknown, spend_opening_balance_required and spend_reservation_overrun. A monetary stop is not permission to automatically retry.

Operator cutover and reconciliationCopy Anchor Link

Pause provider dispatch, back up accounting data, migrate, qualify prices and accounts, then deploy API and workers together. Historic usage_records and API tariff rows overlap and cannot be imported as exact expenditure. Before resuming an existing tenant, confirm its opening UTC day/month provider totals using a restricted operator environment:

python -m scripts.billing.reconcile --tenant synthetic-tenant opening \
  --day-usd 0.20 --month-usd 0.20 --reference verified-opening

Opening balances are accepted once, before any new attempt. The month total includes the day total. A first-of-month opening requires equal totals. Tenant-wide openings cannot infer historic per-key/user splits. Until the next UTC day, the entire day opening conservatively counts against every key/user cap. Quiesce older active runs before cutover; new root-run caps start with new executions. Existing rows with ambiguous export history are marked legacy_delivery_unknown and are not automatically sent to billing.

Match final provider statement lines to the original attempt ID, exact provider, account, model and provider reference using trusted provider evidence. The bounded input file contains a JSON array such as:

[{
  "attempt_id": "00000000-0000-0000-0000-000000000001",
  "provider": "fixture",
  "account": "fixture-account",
  "model": "synthetic",
  "receipt_id": "fixture-statement-line",
  "actual_usd": "0.04",
  "provider_reference": "fixture-response"
}]

python -m scripts.billing.reconcile --tenant synthetic-tenant statement verified-statement.json

The command only writes Q's ledger; it does not download statements, call providers, bill customers or issue refunds. The operator verifies statement completeness and finality. Identical receipts are no-ops; conflicting receipts, account/model/reference mismatches and cross-tenant attempts reject. Files are bounded to 1,000 rows/2 MB; each line commits independently, making an interrupted import safely resumable. Amount corrections after a confirmed final line require investigation; there is no silent last-writer-wins update.

Delivery and guaranteesCopy Anchor Link

API usage rows are durable export intents. A stable server event ID binds tenant, surface and request/run correlation; insert retries cannot create a second charge. Changed usage for one identity rejects. Calls without any correlation get one ID per recording operation, retained across its database retries; callers replaying independent operations must supply the original correlation. Diagnostic usage records also deduplicate repeated provider response observations and ambiguous commit retries; they are not the authoritative ledger.

The existing billing flush loop now drains the database outbox before its legacy Redis queue. Five-row leases, bounded receiver calls and owner-fenced acknowledgements allow crash recovery. Receiver timeouts retain the rows and stable IDs. Receiver event-ID idempotency is necessary for exactly-once charging; delivery itself is at least once. Rows recorded with Vellon disabled stay export_disabled and are not retroactively billed by enabling it. Legacy Redis entries retain the earlier delivery behaviour; drain and audit them during cutover. No external delivery was performed during local qualification.

Admission has zero estimated overshoot: existing accounted spend plus reservations plus a new ceiling must fit every cap. Real spend fits this bound only if the qualified ceiling and tariff cover actual provider charges. Any excess is recorded in full; the current request stops on a detected overrun, and later admission checks the corrected total. Q cannot prevent provider overcharges or retroactive price corrections. Maximum correction exposure is the sum of positive actual-minus-previously-accounted differences, not a claimed fixed live-dollar guarantee.

Legacy admin diagnostic charts and the old unified_spend view are not authoritative reconciliation reports. Production still needs provider statement comparison, receiver idempotency qualification, cutover/opening checks, realistic monetary-lock load measurements and monitoring of unknown holds. Sprint 41 records local verification; Q6.1 follows with the MCP profile.