Skip to content

Enterprise · Edge resilience

Monitoring and outages

You alert on config age instead of health codes, and you have a tested runbook for the day the backend goes down.

Video coming soon

Monitoring and outages

Before you start

  • Your edges run with snapshot.dir set (Local snapshots).
  • You have a Prometheus-compatible scraper, or another way to poll HTTP endpoints.
  • Everything on this page is Core.

Four places that show freshness

WhereWhat you seeNeeds the backend
Response headersFluxGate-Config-Source and FluxGate-Config-Age on every evaluationNo
GET /health/detailStream state, source, age, last sync, feature count, queues, relay stateNo
GET /metricsThe same values as Prometheus seriesNo
Settings, Edge fleetEvery edge with role, via, source, age, version and last seenYes

None of the edge endpoints needs authentication, and none holds secrets or flag data.

Response headers

Check an evaluation response
curl -si -X POST http://localhost:8081/ofrep/v1/evaluate/flags/express-checkout \
  -H "Authorization: Bearer $FLUXGATE_SDK_KEY" \
  -H "Content-Type: application/json" \
  -d '{"context":{"targetingKey":"user-42","country":"CA"}}' | grep -i '^fluxgate-config'
http
FluxGate-Config-Source: stream
FluxGate-Config-Age: 0

Both headers are listed in Access-Control-Expose-Headers, so browser apps can read them too. The headers are absent while the edge has nothing loaded.

Health endpoints

EndpointUse it forStatus
GET /health/liveLiveness probe200 while the process serves HTTP
GET /health/readyReadiness probe, load balancer check200 when the edge can evaluate flags, else 503
GET /healthSame as /health/ready, for older probesSame
GET /health/detailOperators and dashboardsAlways 200

An edge is ready when it holds a complete snapshot of its team's flags, from the stream, a local file or a bootstrap file, also while the backend is down. With ready_max_age_secs set, it reports 503 with "status": "not_ready" once its config age passes the limit.

GET /health/detail during an outage
{
  "status": "ready",
  "streamConnected": false,
  "configSource": "stream",
  "configAgeSecs": 412,
  "lastSyncedAt": "2026-10-25T15:01:12Z",
  "snapshotGeneratedAt": "2026-10-25T14:58:40Z",
  "featureCount": 10,
  "team": "fff5ab64-4f7c-4820-9d6e-a1bf17dce30f",
  "queues": {
    "evaluationEvents": { "depth": 318, "dropped": 0 },
    "assignments": { "depth": 12, "dropped": 0 }
  },
  "relay": null,
  "version": "2.0.0"
}

"status": "ready" with "streamConnected": false is the expected state during an outage: the edge serves its cache, and the queues fill until the backend is back.

Metrics and alerts

SeriesTypeMeaning
fluxgate_edge_config_age_secondsgaugeHow old the served flags are. Absent while nothing is loaded
fluxgate_edge_upstream_connectedgauge1 while the stream to the backend is open, else 0
fluxgate_edge_config_source{source}gauge1 for the current source (stream, local or bootstrap)
fluxgate_edge_featuresgaugeCached features of the edge's team
fluxgate_edge_evaluation_events_dropped_totalcounterEvaluation events dropped because their queue was full
fluxgate_edge_assignments_dropped_totalcounterSticky assignments dropped because their queue was full
fluxgate_edge_relay_childrengaugeOpen child streams. Only on a relay

Do not alert on the health status code. It stays 200 during an outage by design, so that a readiness probe does not remove every edge at once. Alert on the age and the stream instead:

Prometheus alert rules
groups:
  - name: fluxgate-edge
    rules:
      - alert: FluxGateEdgeDisconnected
        expr: fluxgate_edge_upstream_connected == 0
        for: 5m
        annotations:
          summary: "Edge {{ $labels.instance }} has no stream to the backend"
      - alert: FluxGateEdgeConfigStale
        expr: fluxgate_edge_config_age_seconds > 1800
        annotations:
          summary: "Edge {{ $labels.instance }} serves flags older than 30 minutes"
      - alert: FluxGateEdgeServingFromFile
        expr: fluxgate_edge_config_source{source=~"local|bootstrap"} == 1
        for: 10m
      - alert: FluxGateEdgeDroppingEvents
        expr: increase(fluxgate_edge_evaluation_events_dropped_total[10m]) > 0
      - alert: FluxGateEdgeEmpty
        expr: fluxgate_edge_features == 0

For relays, also alert when fluxgate_edge_relay_children falls below the number of children you run.

The fleet view

Open Settings, then Edge fleet. The Fleet tab lists every edge node the backend saw in the last 5 minutes: its node ID and client, its role (edge or relay), the relay it connects through (Via), its config source and age, its version, when it was last seen, and its state. Each edge reports these values in its heartbeat.

Edge fleet settings, Fleet tab, listing five edge nodes with role, via, config source stream, config age, version 2.0.0, last seen time and the state Connected (opens full size in a new tab)
The Fleet tab of the Checkout install: two direct edges, one relay, and two children of the relay.

The fleet view needs the backend, so use it before and after an outage, not during one. The same list is at GET /api/v1/edge/fleet.

Backend outage runbook

Check

  1. GET /health/detail on a few edges: streamConnected, configSource, configAgeSecs, lastSyncedAt, queues.
  2. The response headers on any evaluation: FluxGate-Config-Source and FluxGate-Config-Age.
  3. /metrics: fluxgate_edge_upstream_connected is 0 and fluxgate_edge_config_age_seconds grows.
  4. Relays: the relay block of /health/detail, and fluxgate_edge_relay_children.
  5. Edge logs: Failed to connect to backend gRPC, Ignoring bootstrap snapshot ..., and the single error line when a local file is unusable.

Act

  • An edge is not ready after a restart. It has no usable local file. Put a fresh signed export at its bootstrap location. The edge retries after 5, 10, 20 and 40 seconds, then every 60 seconds, and applies the file without a restart.
  • A security incident during the outage. You cannot revoke a client on the edges while the backend is down. Block the client at the network layer, and revoke it when the backend is back.
  • Old flags must not be served. Set resilience.ready_max_age_secs before the next outage, so a load balancer drops edges whose flags are older than you accept. The default 0 keeps them in.
  • Do not change flags and expect them to arrive. Nothing reaches the edges until the backend is back, and kill switches cannot fire.

Recover

When the backend is back, each edge's stream delivers a complete snapshot. The source becomes stream, the age returns to 0, flags the backend no longer has are removed, and queued events flush. Check the fleet view: every edge should show stream and an age near 0 within a minute.

Run an outage drill

Run this once per environment, and again after you change a topology:

  1. Start an edge with snapshot.dir while the backend is up. Wait for "configSource": "stream".
  2. Stop the backend. Evaluate a flag: it still answers, and FluxGate-Config-Age grows.
  3. Restart the edge. It is ready again with "configSource": "local".
  4. For bootstrap: delete the .fgs file, restart the edge, and check "configSource": "bootstrap".
  5. Start the backend. Check that the source returns to stream and the age to 0.

Next

Read the configuration reference for every edge setting and the Edge server API for the full health and metrics fields.