Zero-downtime database migration
Moved a decade-old monolith off a single overloaded database onto a managed cluster, with no maintenance window and no lost writes.
- Year
- 2023
- Role
- Engineer
- Stack
- PostgreSQL, Python, Terraform
Placeholder content — replace with the real write-up.
The problem
The primary database had become the single point of failure for the whole product. It was running on hardware nobody wanted to touch, backups were untested, and the schema had accumulated fifteen years of decisions made under deadline.
What I built
A dual-write and backfill pipeline: writes went to both the old and new database while a background job copied history across, with a continuous consistency checker comparing row counts and checksums. Reads moved over one query pattern at a time behind a flag, so any regression affected a single endpoint rather than the site.
The cutover itself was a config change. The interesting work was everything that made it boring.
What I’d do differently
The consistency checker should have existed from day one rather than week three — for the first fortnight we were migrating on optimism.