Signed bootstrap snapshots

Before you start

• Your license has the edge_resilience entitlement (Enterprise). Only creating the first signing key checks it.
• The backend has FLUXGATE_ENCRYPTION_KEY set. It seals the private signing key. Without it, creating a key fails with 400 encryption_key_missing.
• You can sign in as admin System admin. Only system admins manage signing keys.
• Your edges already use a local snapshot. A bootstrap file is for the edges that do not have one yet.

When you need a bootstrap file

A local file covers an edge that restarts. It does not cover an edge that has never run, or one whose disk is empty:

• a new edge that an autoscaler starts during an outage;
• a Kubernetes pod rescheduled to a node without its volume;
• a site that has no path to the backend at times.

A bootstrap file is a snapshot of one team that the backend signs. Any edge that trusts the signing key can load it, check that it is genuine and recent, and start serving.

Create a signing key

A signing key is an Ed25519 key pair. Its ID is esk_ followed by the first 16 hex characters of the SHA-256 hash of the public key. At most one key is active at a time. Retired keys stay in the list and never sign again. The private key never leaves the backend.

(opens full size in a new tab) (opens full size in a new tab)

The same actions are in the REST API. Every response holds public data only.

{
  "keyId": "esk_c4cbce9f094eb4cd",
  "publicKey": "<base64 Ed25519 public key>",
  "status": "active",
  "createdAt": "2026-10-25T14:55:12+00:00",
  "retiredAt": null,
  "trustedKey": "esk_c4cbce9f094eb4cd:<base64 Ed25519 public key>"
}

trustedKey is the exact value for the trusted_keys setting of an edge.

Key errors

Export a snapshot

GET /api/v1/teams/{team_id}/edge-snapshot returns the signed snapshot of one team as a binary file (application/octet-stream, file name edge-snapshot-<team_id>.bin). The file holds:

• every feature of the team, exactly as the edge stream sends it;
• the time the backend generated the snapshot;
• the enabled clients of the team, each with its client info and the SHA-256 hash of its SDK key.

Disabled and deleted clients are left out, so revoking a client takes effect at the next export. The file holds no plaintext SDK keys and no users. The signature covers the text fluxgate-edge-snapshot-v1 followed by the SHA-256 hash of the payload.

Who may export:

• system admins;
• members of the team;
• a system client of the team whose token has the scope Edge snapshot (edge:snapshot). This scope allows the export and nothing else. System clients with admin:read can also export, because that scope reaches every GET.

The export needs no entitlement. It works in every license state while a key is active.

Export on a schedule

FLUXGATE_URL=https://flags.junipermarket.example/api/v1
FLUXGATE_TOKEN=<snapshot-export system client token>
TEAM_ID=<Checkout team ID>

#!/bin/sh
set -eu
. /etc/fluxgate/export.env
out=/srv/snapshots/team-checkout.bin
curl -fsS -H "Authorization: Bearer $FLUXGATE_TOKEN" \
  -o "$out.partial" "$FLUXGATE_URL/teams/$TEAM_ID/edge-snapshot"
fluxgate edge snapshot verify "$out.partial" --trusted-key "$(cat /etc/fluxgate/trusted-key)"
mv "$out.partial" "$out"
aws s3 cp "$out" s3://juniper-snapshots/eu/team-checkout.bin

*/15 * * * * deploy /usr/local/bin/fluxgate-snapshot-export

set -e stops the script when the export or the check fails, so the last good file stays published. Exporting more often than your flags change is harmless: an edge ignores a file that is not newer than what it has.

The FluxGate CLI can also download the file. It writes <file>.partial and renames it only when the download is complete:

fluxgate edge snapshot export --team checkout --out team-checkout.bin

Verify a file offline

fluxgate edge snapshot verify checks a file without a login or network access. It exits 0 and prints the team ID, the generation time, the number of features and the key ID. It exits 1 when the file is not a snapshot, no --trusted-key matches its key ID, or the signature is not valid. Repeat --trusted-key for more than one key.

fluxgate edge snapshot verify team-checkout.bin \
  --trusted-key "esk_c4cbce9f094eb4cd:<base64 public key>"

Verify checks the signature only. The edge also checks the team, the rollback floor and the age.

Configure the edge

[snapshot]
dir = "/var/lib/fluxgate/edge"
bootstrap = "https://juniper-snapshots.s3.eu-west-1.amazonaws.com/eu/team-checkout.bin"
trusted_keys = ["esk_c4cbce9f094eb4cd:<base64 public key>"]
startup_wait_secs = 5
max_age_secs = 604800

The edge checks trusted_keys at startup, even when bootstrap is empty, so a bad key shows at deploy time and not during an outage. A malformed entry, a key ID that does not match its key, a duplicate, or bootstrap without keys stops the edge with Invalid snapshot configuration: .... The edge reads trusted_keys only at startup: restart the edge after you change it.

How the edge reads the file

The edge waits startup_wait_secs after it starts. If the stream has not completed a snapshot by then, and the stream is disconnected, the edge reads the bootstrap. If the stream is open and still sending its snapshot, the edge waits and tries again. The bootstrap runs once after startup; a later outage does not read it again.

• A path is read from disk. A URL must answer within 10 seconds, body included.
• The edge follows no redirects, and refuses a text/html answer, which is usually a login page.
• Files and answers larger than 64 MiB are refused.
• Logs show the source without user, password, query and fragment, so a presigned URL stays out of the logs.

The checks, in order

The first check that fails rejects the file, and the edge logs one line: Ignoring bootstrap snapshot <source>: <reason>.

1. The signature is Ed25519, its key ID is in trusted_keys, and it is valid.
2. The file names a team and a generation time.
3. The team equals the team the edge already knows, from its local file or an earlier answer of the backend.
4. The file is newer than the rollback floor. An older or equal file is ignored with an info line.
5. With max_age_secs set, the file is not older than that. A file from the future counts as new.
6. An edge that does not know its team yet accepts the file only if it lists the edge's own client, in the file's team, with the SHA-256 hash of its configured client_secret.

The last check stops a new edge from serving another team's flags. It also means: export again after you create a client for an edge, rotate its SDK key, or enable it again. Otherwise a new edge with that client refuses the file with the reason does not list this edge's client.

The rollback floor

An old file must never replace newer flags. The floor is the backend time of the last change the edge applied from the stream. Every flag change, heartbeat and full snapshot carries it, and children of a relay get it through the relay. After the edge applies a bootstrap file, the floor is that file's generation time.

The floor lives in the local snapshot file. An edge without snapshot.dir, or with an empty disk, has no floor after a restart, so it accepts any older file within max_age_secs. Set max_age_secs on those edges.

Retries

If the read fails or the file is rejected and nothing is loaded, the edge tries again after 5, 10, 20 and 40 seconds, then every 60 seconds, until a bootstrap is applied or a stream snapshot completes. You can therefore fix the cause during an outage, publish a new file, and the edge picks it up without a restart.

If a local file is already loaded, the edge makes three attempts in total for a failed read, and never retries a rejected file.

After the edge accepts the file

• The edge serves the flags at once. FluxGate-Config-Source is bootstrap, and the age counts from the export time.
• It removes flags of a loaded local file that the newer bootstrap does not contain.
• With snapshot.dir, it keeps the file as its local file, with the original generation time and the source bootstrap. A restart then loads it again and still reports bootstrap, and the same bootstrap is not applied twice.
• The next complete stream snapshot replaces it, and the source becomes stream.

Team correction

The team in a file is provisional. When the backend is reachable, the edge asks it for the team of its client. If the backend names another team, the backend wins: the edge logs an error, drops the loaded flags, assignments and snapshot state, deletes the local file, and serves the backend's team.

Client keys from a bootstrap file

While the backend is unreachable, the edge hashes each presented SDK key with SHA-256 and compares it, in constant time, with the hashes in the applied file.

• A match serves the exported client info. An unknown client, a wrong key, or a client not in the file gets 502.
• resilience.client_stale_if_error_secs governs this too. The window counts from the export's generation time, so an export older than the window authenticates nothing. 0 turns it off.
• A live answer of the backend always wins. An accepted key replaces the exported entry; a rejection removes it.
• A key that was rotated or disabled after the export keeps working while the backend is unreachable, until the backend answers or the window ends.

Rotate a signing key

Rotation retires the old key at once, and edges trust public keys. Update them in this order to avoid a gap:

If a private key may be exposed, rotate at once. Then replace trusted_keys with the new key only, restart the edges, and export again. An edge that no longer trusts the old key refuses every file signed with it. Retire works in every license state, including ReadOnly, so you can always take a key out of service.

Retiring the only active key leaves the backend without a signing key. Exports then fail with 409 snapshot_key_missing until you create a key again, and that needs the entitlement.

Treat snapshot files as confidential

• Keep the bucket private, and restrict reads to the edges and the people who run them. Use https://.
• A presigned URL must still be valid when an outage happens. Use a long expiry, or a private bucket path with a network rule. Keep presigned URLs out of logs and tickets.
• Store the edge:snapshot token like a credential: it can export every day.
• Everyone who may export receives the key hashes: team members, system clients with admin:read or edge:snapshot, and admins.

Audit entries

The export writes edge_snapshot_exported, at most once per user or system client and team per hour, with the team, key ID, feature count, client count and the kind of caller. Key changes write edge_snapshot_key_created, edge_snapshot_key_rotated and edge_snapshot_key_retired in the same transaction as the change. A refused first key writes license_deny. See Audit log.

Troubleshooting

An edge ignores the bootstrap file

Read the reason in the Ignoring bootstrap snapshot line. The usual causes:

• signed with an untrusted key: the edge does not trust the key that signed the file. Add the trusted key and restart, or export again with a trusted key.
• The file is for another team: point bootstrap at the export of the edge's team.
• The file is not newer than the edge's flags: nothing to do, the edge already has newer flags.
• The file is older than max_age_secs: publish a fresh export.

A new edge says the file does not list its client

The edge has no local file, so it needs proof that the file belongs to its team, and that proof is its own client in the file. Check these causes:

• The file belongs to another team.
• The client was created, rotated or enabled again after the export. Publish a new export.
• The edge uses the wrong client_id or client_secret.

The edge retries the read, so a new file applies without a restart.

The export fails with 409 or 500

409 snapshot_key_missing: no key is active. Create one. 500: the backend cannot open the active key, because FLUXGATE_ENCRYPTION_KEY is missing or changed. Restore the old value, or rotate the key.

Next

Cut the number of streams per region with Relay edges.