SQL Practice Logo

SQLPractice Online

Database Migration Strategies: Performance

Module: Database-Specific Features

Migration performance considerations: (1) Data transfer time: 100GB = 1-2 hours (1Gbps network), 1TB = 10-20 hours. Use parallel export/import for faster transfer. (2) Downtime: Big bang = hours to days, blue-green = seconds to minutes. Choose strategy based on downtime tolerance. (3) Replication lag: Incremental sync has lag (seconds to minutes), monitor lag before cutover. (4) Performance regression: New database may be slower (different query planner, missing indexes), test thoroughly. (5) Resource usage: Migration uses CPU/memory/disk, schedule during low-traffic periods. (6) Network bandwidth: Data transfer saturates network, use dedicated migration network or off-peak hours. Real-world: GitLab migration took 6 months (planning + execution), Uber migration improved performance 2-3x (MVCC), Stack Overflow had zero downtime (blue-green).

Parallel export/import: Use multiple threads for faster data transfer (pgloader --workers 8)

Disable indexes during import: Drop indexes, import data, recreate indexes (10x faster)

Batch inserts: Use COPY (PostgreSQL) or LOAD DATA (MySQL) instead of INSERT (100x faster)

Network optimization: Use dedicated migration network, compress data transfer, off-peak hours

Incremental sync: Use CDC (Debezium, AWS DMS) for continuous sync, minimal downtime

Performance testing: Load test new database before cutover, ensure no regression

Index optimization: Analyze query patterns, create missing indexes, drop unused indexes

Connection pooling: Use PgBouncer (PostgreSQL), ProxySQL (MySQL) to handle connection spikes

Insufficient testing: Test on production-like data, not small sample (catches edge cases)

No rollback plan: Always have backup and rollback procedure (Murphy's law applies)

Ignoring encoding: MySQL latin1 → PostgreSQL UTF8 requires conversion (data corruption)

Missing indexes: New database may be missing indexes, causing performance regression

Replication lag: CDC has lag (seconds to minutes), monitor before cutover