Platform metrics contract

ContractCopy Anchor Link

These are the installation-wide Prometheus metric names, types, labels, and aggregation rules. To connect a collector, see Collect metrics.

How values are producedCopy Anchor Link

• Computed from the database at scrape time. The metrics role (bun run apps/control-host/src/main.ts metrics) is stateless. Each request reads the database and converts the results into metrics. Counters therefore do not reset to 0 when the Pod restarts.
• One target. Values are installation-wide totals. Two replicas would report the same values twice. The chart fixes replicas at 1 and uses Recreate.
• Read-only connection. Database connections start with default_transaction_read_only=on, so the database rejects accidental writes. This is a session setting, not a database privilege. The DATABASE_URL account has write permissions like other roles, and can write if the setting is reverted in the connection. The setting is sent through the startup parameter options; a pooler rejecting options (such as PgBouncer without options in ignore_startup_parameters) prevents only the metrics role from connecting.
• Concurrent scrapes share one read. Requests arriving during a read receive the same result. A failed read returns 503 without stale values (a failed scrape for the collector; Prometheus up = 0).
• /healthz checks only that the process responds. The probe does not read the database.

Every scrape reads and aggregates database data. Shorter intervals increase database load. Consider increasing the interval if collection takes more than half of it.

MetricsCopy Anchor Link

The prefix is werft_, defined in one place in the code.

tool_calls is a gauge because tool outcomes can change later. A late result changes unknown to ok or failed; an earlier result arriving later can replace the existing outcome. A decrease in a counter would look like a reset and distort increments. Use tool_calls_total for call count increases and tool_calls for outcome ratios.

The same reasoning excludes outcome_unknown turns from turns_total. Operator recovery changes them to completed or cancelled and updates the end time. They enter turns_total and the histogram at that point.

Session statesCopy Anchor Link

The first matching row determines state. Each session matches exactly one row, so the sum by state equals the total session count.

needs_input is not stored in the database. A running session or a session with a parked turn reads as needs_input when a person can answer a pending request. Metrics use the same rule. ended keeps growing because closed sessions are not deleted. Exclude ended when assessing current load.

Tool outcomes and denialsCopy Anchor Link

• ok, failed, and unknown come from tool_calls.outcome (the tool result block's is_error).
• denied means a person answered a permission request with deny. pending_requests (kind = 'permission', answer.decision = 'deny') joins tool_calls on (session_id, tool_use_id). The provider's unique tool_use id also joins a parked request answered by the next attempt to the same call. Any deny answer for that tool_use makes it denied, even if it was later allowed and executed.
• Agent tool policy deny, expired/cancelled requests, and denials from older workers without tool_use_id follow the result block (failed, etc.), rather than counting as denied.

Costs and tokensCopy Anchor Link

• Cost sums provider_usage.cost_usd, calculated when the call was recorded using platform prices. Metrics neither reprice calls nor maintain separate price settings.
• priced_by records the pricing basis: table (listed model), fallback (unlisted model, using the highest rates), or unmetered (profile billing: none, cost 0). Track usage for zero-cost calls such as self-hosted models through tokens with priced_by="unmetered".
• Time windows use the call recording time (provider_usage.recorded_at). Use counter increases such as increase(werft_provider_cost_usd_total[1h]) for period costs.
• There are three differences from GET /v1/usage/summary. Results match if all sessions were created and charged within the period.
  • summary selects sessions created in the period (sessions.created_at) and sums their entire costs. Costs incurred during the period by older sessions appear only in metrics.
  • summary sums sessions.cost_usd, capped by the session cost field (99,999,999.999999 USD). Per-call costs are not capped there.
  • summary includes session costs from before per-call records existed (before migration 0169, session_pre_metering_cost); metrics do not.
• Tokens sum per-call values from provider_usage. cache_creation includes all cache writes, including 1-hour cache writes.

QueuesCopy Anchor Link

• Launch queue: sessions the scheduler can launch in the next pass, counted using the same SQL conditions: launchable admission, no execution reclamation/takeover, within cost/token limits, restore retry time elapsed, and no launch holding a slot. Limits come from the same config.limits as the scheduler. Wait time starts at unassigned_sessions.signaled_at.
• Input queue: inputs in queue_messages not taken by any worker whose visibility time has passed. This matches Compose's werft.queue.oldest_age_seconds.

Label rulesCopy Anchor Link

• Session, turn, user, API key, agent, and attempt ids are never labels. Use GET /v1/usage/* for per-agent costs.
• state, status, admission, outcome, priced_by, kind, and le have fixed value sets.
• tool and model come from data. MCP tools mcp__<server>__<tool> collapse to mcp__<server>. The first 100 tool names and 50 model names keep their names in order of appearance; the rest become other. Rows are not deleted, so named values usually remain. However, once the cap is full, a late projection row (tool) or late-committed row (model, where recorded_at is transaction start time) with an earlier timestamp can swap a name with other. The displaced series stops, the newly named series starts, and other may jump or look like a reset.
• Tests: the cardinality test checks label names, absence of id-shaped values, and caps.

Mapping to Compose metricsCopy Anchor Link

The Compose Datadog overlay produces similar values through PostgreSQL custom queries (Datadog guidance, infra/datadog/conf.d/postgres.d/conf.yaml). Kubernetes installations can use the metrics here.