The restore drill nobody runs
Everyone has backups. Almost nobody has a restore. The difference only shows up on the worst possible day, which is a poor time to discover that the dump has been zero bytes since a path changed in March.
A backup you have never restored is not a backup. It is a hypothesis about a backup.
The drill
Once a quarter I block forty minutes and do exactly this, working from the documentation only — no memory, no shortcuts, no "I'll just check the history":
- Spin up a throwaway machine with nothing on it.
- Fetch the most recent backup using only credentials stored in the password manager, not the ones already on my laptop.
- Restore it. Time the whole thing, honestly, including the waiting.
- Run three queries whose answers I know by heart.
- Write down everything that was wrong in the instructions.
Step five is the point of the exercise. The restore almost always works; the instructions almost never do.
What it caught the first time
Three things, none of which would have surfaced any other way. The nightly dump had been silently excluding one schema since a refactor. The documented restore command referenced a flag removed two major versions earlier. And the whole thing took fifty-five minutes, not the fifteen I had been telling people.
$ time pg_restore -d app_restore -j4 latest.dump real 54m38s user 8m11s sys 1m02s
That last number changed a business decision. Fifteen minutes of downtime is an inconvenience you absorb; an hour is an incident with a phone call attached, and it needed to be budgeted for honestly rather than hoped away.
Making it stick
The drill survives because it is on the calendar as a recurring event with the checklist attached, and because it is short enough that skipping it feels lazy rather than reasonable. Anything longer than an hour gets postponed forever, and a drill you postpone is just a backup you have never restored.