Your App Deploys in Seconds. Your Schema Change Takes the Site Down.

You have spent months building a solid CI/CD pipeline. Deploys are automated, tested, and fast. Your team ships multiple times a day with confidence. Then someone needs to add a column, rename a field, or drop a table — and the whole deployment process grinds to a halt.

The deploy itself finishes in seconds. But the database migration locks a table, queries start timing out, connections pool up, and your users see errors. Suddenly, a routine change has become an incident.

This is one of the most common gaps in otherwise mature engineering teams. The application layer is automated, reproducible, and safe. The data layer is still a manual, hold-your-breath operation. And the cost compounds: teams start batching schema changes, shipping them less often, and accumulating risk with every delay.

Zero-downtime database migration is not a luxury reserved for companies running thousands of nodes. It is a discipline — a set of patterns and habits that any team shipping daily can adopt. The soil work happens in how you design your migrations, not in how fast your hardware is.

Why Schema Migrations Are Different From Code Deploys

Code deploys are stateless in the ways that matter most. You build a new artifact, route traffic to it, and the old version disappears. If something goes wrong, you roll back to the previous artifact. The blast radius is contained because you control both sides of the transition.

Database schema changes are fundamentally different. The database is shared, stateful, and persistent. When you alter a table, the old code and the new code must both be able to work with the schema during the transition window. There is no clean swap. There is only a period of coexistence.

Most migration failures come from ignoring this coexistence window. A migration that works perfectly in a test environment with no traffic will fail under load because locks block reads, because the old application code cannot handle the new schema, or because a rollback path was never planned.

The Lock Problem

In Postgres — and this applies broadly across relational databases — many DDL operations acquire an ACCESS EXCLUSIVE lock on the table. That lock blocks every other operation, including SELECT. On a table with millions of rows, an ALTER TABLE ... ADD COLUMN ... DEFAULT value in older Postgres versions would rewrite the entire table while holding that lock. Even in modern Postgres (11+), where adding a column with a volatile default is handled more gracefully, other operations like creating indexes without the CONCURRENTLY flag will still lock the table.

The first postgres migration best practice is understanding which operations acquire which locks, and for how long. This is not theoretical knowledge. It is the difference between a smooth deploy and a page at 2 a.m.

The Expand and Contract Pattern

The most reliable way to deploy database changes safely is the expand and contract pattern. It separates every schema change into at least two phases, and sometimes three:

Phase 1: Expand

Add the new structure alongside the old one. This might mean adding a new column, creating a new table, or introducing a new index. Critically, the existing application code continues to work without modification. Nothing is removed, renamed, or constrained in a way that breaks backward compatibility.

During this phase, you also begin backfilling data if needed. If you are migrating data from an old column to a new one, a background process — not a blocking migration — handles the copy. For large tables, this backfill runs in batches to avoid locking and to keep transaction sizes manageable.

Phase 2: Migrate Application Code

Once the new schema is in place and data is populated, you deploy application code that writes to both the old and new structures (dual-write) or reads from the new one. This is the coexistence window. Both the old and new application versions work against the current schema.

This phase is where most teams skip steps. The temptation is to jump straight from adding a column to dropping the old one. Resist it. The dual-write or dual-read period is what gives you a safe rollback path. If the new code has a bug, you roll back the application deploy — not the schema — and the old code still works.

Phase 3: Contract

Only after the new code is stable, verified, and the old column or table is confirmed unused, do you remove the old structure. This is the cleanup step. Drop the column, remove the old index, drop the table. It is a separate migration, deployed on its own, often days or weeks after the expand step.

The expand and contract pattern turns one dangerous, atomic change into three safe, incremental ones. Each step is independently deployable, independently reversible, and independently testable.

A Database Schema Migration Checklist

Whether you are working in Postgres, MySQL, or another relational store, this checklist applies to any team that wants to ship schema changes without downtime.

Before Writing the Migration

  • Classify the change. Is it additive (new column, new table, new index), transformative (rename, type change, data move), or destructive (drop column, drop table)? Additive changes are almost always safe. Destructive changes are almost always the contract phase of a two-step migration. Transformative changes need to be decomposed into expand-then-contract steps.
  • Check the lock behavior. For your specific database engine and version, verify what locks each DDL statement acquires. In Postgres, use CREATE INDEX CONCURRENTLY instead of CREATE INDEX. Understand that ALTER TABLE ... ADD COLUMN with a non-volatile default is fast in Postgres 11+, but adding a NOT NULL constraint on an existing column without a default still requires a full table scan in some configurations.
  • Estimate the table size and row count. A migration that runs in milliseconds on a dev database with 500 rows can take minutes on a production table with 50 million rows. Size determines whether you can run inline or need a background job.
  • Plan the rollback path. For every migration, write down what happens if you need to undo it. If the answer is we cannot undo it without data loss, you have not decomposed it enough.

Writing the Migration

  • One logical change per migration file. Do not bundle unrelated schema changes. Each migration should be independently understandable and reversible.
  • Set a lock timeout. In Postgres, set lock_timeout to a short duration (a few seconds) so that if the migration cannot acquire the lock quickly, it fails fast instead of queuing behind long-running queries. You can retry safely. You cannot un-queue a blocked connection pool.
  • Use advisory locks or migration-specific guards to prevent two instances from running the same migration simultaneously during a rolling deploy.
  • Avoid long-running transactions. If a migration involves a data backfill across millions of rows, do not wrap it in a single transaction. Batch it. Long transactions hold locks, bloat the write-ahead log, and increase the blast radius of a failure.

Deploying the Migration

  • Run migrations before code deploys, not during. Decouple migration execution from application startup. If your application runs migrate on boot, every instance in a rolling deploy races to apply the same migration. Separate the concern. Run migrations as a distinct pipeline step.
  • Deploy to a canary or staging environment that mirrors production scale. Not schema — scale. A migration that passes on a copy of the schema with no data has not been tested.
  • Monitor during and after. Watch query latency, lock wait times, connection pool saturation, and error rates. Your observability stack should surface problems before your users report them.

After the Migration

  • Verify data integrity. If you backfilled data, run validation queries. If you added a constraint, confirm it holds.
  • Schedule the contract step. Do not leave orphaned columns indefinitely. Technical debt in the schema is as real as technical debt in code. Put the cleanup migration on your board with a date.
  • Document what you did and why. The next person to touch this table — which might be you, six months from now — needs context.

Postgres-Specific Patterns Worth Knowing

Postgres has specific behaviors that reward careful engineering. A few patterns that matter most for zero-downtime database migration:

Adding a NOT NULL Column Safely

Do not add the column with NOT NULL DEFAULT value in one step on large tables in older Postgres versions. Instead: add the column as nullable, set the default, backfill existing rows in batches, then add the NOT NULL constraint using NOT VALID followed by a separate VALIDATE CONSTRAINT step. The validation acquires a weaker lock (SHARE UPDATE EXCLUSIVE) and does not block reads or writes.

Renaming a Column

Do not rename it. The expand and contract pattern here means: add the new column, dual-write from the application, backfill, cut over reads to the new column, stop writing to the old column, then drop the old column in a later migration. A direct ALTER TABLE ... RENAME COLUMN breaks every query referencing the old name instantly.

Index Creation

Always use CREATE INDEX CONCURRENTLY. The non-concurrent version locks the table against writes for the duration of the index build, which on a large table can mean minutes of downtime. The concurrent version takes longer but does not block writes. Note that CREATE INDEX CONCURRENTLY cannot run inside a transaction, so your migration tooling must support non-transactional migrations.

Enum Type Changes

Adding a value to a Postgres enum is safe in Postgres 10+ (ALTER TYPE ... ADD VALUE). But removing or renaming an enum value is not supported directly. If you need to change an enum, consider migrating to a text column with a check constraint, or create a new enum type and use the expand and contract pattern to migrate.

The Discipline Behind the Pattern

Zero-downtime database migration is ultimately a discipline, not a tool. You can adopt the best migration framework available, but if your team treats schema changes as an afterthought — something to figure out right before deploy — the tooling will not save you.

The teams that ship daily without fear have built a few habits into their engineering culture:

  • Schema changes are designed as part of the feature, not bolted on at the end.
  • Every migration is reviewed for lock behavior and rollback safety, just like application code is reviewed for correctness.
  • The expand and contract pattern is the default, not the exception.
  • Observability is in place before the migration runs, not configured after something breaks.

This is foundational work. It is not glamorous. But it is the kind of engineering that lets a team scale — not just the infrastructure, but their confidence in shipping.

When the Foundation Needs an Architect

If your team is shipping application code confidently but still treats database migrations as a manual, high-risk event, the gap is not in your talent — it is in the process and architecture around the data layer. Building a migration discipline that holds under pressure takes experience with the failure modes, not just the happy paths.

At Figtree Development, we design CI/CD pipelines and infrastructure-as-code environments where schema migrations are a first-class citizen, not an afterthought. With over fifteen years of experience engineering and managing production cloud environments, we bring principal-level judgment to the kind of soil work that lets your team ship without holding their breath.

If your infrastructure needs to scale with you — not crack under the next schema change — book a free 20-minute discovery call and let us look at what you are working with. No sales pitch, just an honest conversation about where the roots need strengthening.

Ready to Build?

Let's Plant Something Real.

Every project starts with a free 20-minute discovery call — no pitch, just a real conversation about what you're building and where the friction is.

Book a Discovery Call → ← Back to Blog