Monitoring

BirdHouse separates matching rules from observation. Your game defines ticket constraints; monitoring shows the activity those rules produce.

Queue and match activity

Read queue depth alongside creation rate, match rate, and wait time. A growing queue can indicate incompatible party sizes, restrictive rules, missing player traffic, or insufficient processing capacity.

Signal	What to look for
Pending tickets	Tickets waiting in the queue; interpret alongside in-progress matching work.
Active and status counts	Retained tickets, including completed tickets during their retry window.
Created and matched ticket rates	Incoming demand versus successful matching throughput.
Published matches	Completed groups; one match can contain several tickets and players.
Match duration	Time from ticket creation to successful match publication.
Workers online	Approximate number of workers with live leases.

Retained matched tickets are not searches still waiting to match. The service preserves their results for ten minutes so lost responses can be recovered.

Live snapshots

The monitor exposes tRPC procedures at /api/trpc:

Procedure	Purpose
stats	Latest snapshot with up to 120 history points; may be null before a sample is available.
statsSubscription	Current snapshot followed by live updates over server-sent events.
tickets	Read-only ticket search using a query string.
ticket	Read one retained ticket by id.

The default fast refresh interval is one second. Status and metadata aggregates refresh on a slower, ten-second interval. Subscription buffering keeps the latest update for a stalled consumer rather than an unbounded backlog.

This is the monitoring transport, separate from the matchmaking API. An application integrating directly with tRPC should use the monitor’s typed router contract instead of inventing REST paths for these procedures.

Environment activity

The environment-aware service supplies status counts, lifetime counter totals, and sampled rates per canonical environment. Rates come from counter changes; they are not reconstructed from retained ticket documents.

Environment status aggregates have their own freshness timestamp. Different panels can therefore represent samples taken at slightly different times. Per-environment counters begin with the environment-aware release and cannot reconstruct earlier expired activity.

Health endpoints

Endpoint	Service	Behavior
GET /livez	API service and worker	Responds when the process is alive; independent of the data-store connection.
GET /readyz	API service and worker	Returns 503 when the shared data-store connection is not ready.
GET /metrics	API service, worker, monitor	Prometheus metrics for the owning component.
GET /healthz	Monitor	Basic process response.

A healthy process can be temporarily unready. An unavailable dependency should not be mistaken for a process that needs to restart.

Metrics and freshness

Worker metrics include matches_published_total, matched_players_total, and ticket_match_duration_seconds. These are worker-local: aggregate rates and histogram buckets across workers.

Global queue and lifecycle gauges come from the monitor. Avoid summing duplicate samples of the same global quantity.

workersOnline is a lease-based approximation. After a hard worker failure, the value can remain stale for almost 41 seconds, including lease expiry, the slow discovery scan, and the next snapshot. Treat connection freshness and data freshness as separate signals.

Investigate a waiting search

Retain the session’s TicketId. Check its environment, party size, required metadata, and constraints. Verify that enough compatible tickets exist and that the search is still sending successful confirmations.

Use the read-only explorer to inspect the retained ticket. If it is absent after an outage, it may have expired. If it is matched, recover the shared result within the fixed retention window rather than creating a duplicate search.