Why should you care?

PostgresVerse checkout down at midnight — playbook beats random restarts that lose in-flight payments.

See it live — copy this example

Run in pgAdmin or psql.

-- Incident snapshot script (run read-only on primary)
SELECT now() AS ts, count(*) FILTER (WHERE NOT pg_is_in_recovery()) AS primaries;
SELECT count(*) AS total, count(*) FILTER (WHERE state='active') AS active FROM pg_stat_activity;
SELECT pg_size_pretty(pg_database_size('PostgresVerse')) AS db_size;
SELECT * FROM pg_stat_database_conflicts;

What happened?

• Quick health pack: confirm connected to primary, session counts, database size growth, standby conflict counters.
• Save output to incident ticket before changes.

Practice next

1. Identify symptom user-facing vs internal only.
2. Run snapshot script; no blind RESTART.
3. If disk full — find WAL or log bloat; if locks — cancel offender.
4. Maintain one-page incident cheat sheet in repo docs.
5. Practice tablespace full scenario in staging quarterly.

Remember

Collect evidence before mutate. Mitigate user pain then fix root cause. Blameless postmortem with timestamps.

PostgresVerse Sev-1 playbook

Disk 98% from WAL slot; drop unused slot; archive catches up; no restart needed.

Outcome: Checkout restored 22 minutes; postmortem adds slot monitoring alert.