Knowledge base · Backups

Verified restores explained

What the weekly verification actually does, what each check means, why a verification can fail, and what the badge shows.

A backup you've never restored is just a hope. Verification is VaultKeep restoring your backup, for real, on a schedule, and telling you what happened.

When it runs

Every Sunday, in the hour after your backup window, for every active project that has weekly verification enabled and at least one successful database backup. You can also click Verify on any successful backup to run one immediately.

What it does

1. Downloads the latest database backup from wherever it lives (managed storage or your own destination).
2. Checks integrity: the ciphertext SHA-256 must match what was recorded at backup time; then the artifact is decrypted (AES-256-GCM, which fails loudly on any tampering) and the plaintext SHA-256 must match the manifest.
3. Boots a throwaway Postgres cluster of the same major version inside the isolated job runner.
4. Runs a full pg_restore of the backup into it, capturing every error.
5. Compares with the manifest captured at backup time:

• every table must exist (table_count);
• tables with exact counts in the manifest must restore with exactly that many rows (row_counts);
• tables with estimated counts must be within ±10% (estimate_sanity);
• the content of every table must hash to the same value it did at backup time (content_hashes);
• every sequence must exist and hold a value at or above the one recorded at backup time (sequences);
• every index and constraint recorded at backup time must exist by name (schema_objects).

1. Destroys the cluster and records the checks with the result.

Past row counts

Counts are the one thing a restore gets right while the data is wrong: a truncated text column, a broken encoding or a reset sequence all leave the row count intact. The last three checks exist for that.

Content hashes. At backup time, inside the same snapshot the dump uses, VaultKeep hashes each row as text and folds the per-row hashes into one value per table. Tables up to 50,000 rows are hashed in full, in an order-independent way that needs no primary key. Larger tables are sampled: the first and last 500 rows by primary key, which catches the common failure of a column that came back wrong for every row. Very large tables with no primary key, or with a text primary key (whose order depends on the cluster's collation), are recorded as skipped and never fail a verification. Hashing stops after two minutes per backup; anything left is skipped too. Every row hash is salted with the backup's id, so the manifest holds one-way hashes and cannot be used to look up row contents. Date, float, interval and bytea output formats are pinned on both sides so the same row prints the same bytes.

Sequences. The dump reads each sequence after the snapshot is taken, so a restored value must be at or above the one in the manifest. A lower value means the sequence was reset. Sequences owned by a serial or identity column are also compared with the column's maximum. A sequence that restored at or above its backup value yet sits below its column was already below it on the source database, usually because an application called setval or inserted explicit ids past the sequence. The restore is a faithful copy, so such sequences are listed in the check's details for information and do not fail it; fix them on the source, where the next insert would collide.

Schema objects. Index and constraint names are recorded at backup time and checked after restore. A CREATE INDEX or ADD CONSTRAINT that fails shows up twice: as a pg_restore error and as a named missing object here. Objects that exist only on the restored side (platform shims) are listed for information and do not fail the check.

Backups taken before these checks existed carry the first five checks only; the next backup after the upgrade carries all of them.

Reading the result

On the project page each backup shows its verifications with a passed or failed badge; expand one to see every check with expected and actual values. A failed verification emails owners and admins and turns the public badge red.

Typical reasons for a failure:

• the backup was taken with a role that couldn't read some tables (row counts short);
• an extension needed by the schema isn't available in the verification cluster (restore errors);
• the backup file was corrupted in storage (integrity check);
• a very large table's estimate drifted more than 10% between backup and restore (usually harmless; run it again);
• a content hash differs after the verification cluster moved to a newer Postgres major that prints a type differently (rare; check the listed tables, then take a fresh backup).

Treat a failed verification as "this backup is unreliable until the next one passes", and check the next backup and verification.

The badge

Each project can expose a public SVG badge at https://vaultkeep.dev/badge/<token>.svg:

• green: restore verified · N days ago
• red: restore verified · check failing
• grey: no verification yet or badge disabled

It reveals only the verification status and recency, never names, data or credentials. The URL contains an unguessable token; you can disable the badge or rotate the token under the project's Status badge tab. Put it in your README:

[![restore verified](https://vaultkeep.dev/badge/<token>.svg)](https://vaultkeep.dev)

Still stuck? Email support@vaultkeep.dev with your project name (never your connection string) and we'll take a look.