Database Migration Strategies: Performance
Module: Database-Specific Features
Migration performance considerations: (1) Data transfer time: 100GB = 1-2 hours (1Gbps network), 1TB = 10-20 hours. Use parallel export/import for faster transfer. (2) Downtime: Big bang = hours to days, blue-green = seconds to minutes. Choose strategy based on downtime tolerance. (3) Replication lag: Incremental sync has lag (seconds to minutes), monitor lag before cutover. (4) Performance regression: New database may be slower (different query planner, missing indexes), test thoroughly. (5) Resource usage: Migration uses CPU/memory/disk, schedule during low-traffic periods. (6) Network bandwidth: Data transfer saturates network, use dedicated migration network or off-peak hours. Real-world: GitLab migration took 6 months (planning + execution), Uber migration improved performance 2-3x (MVCC), Stack Overflow had zero downtime (blue-green).
Parallel export/import: Use multiple threads for faster data transfer (pgloader --workers 8)
Disable indexes during import: Drop indexes, import data, recreate indexes (10x faster)
Batch inserts: Use COPY (PostgreSQL) or LOAD DATA (MySQL) instead of INSERT (100x faster)
Network optimization: Use dedicated migration network, compress data transfer, off-peak hours
Incremental sync: Use CDC (Debezium, AWS DMS) for continuous sync, minimal downtime
Performance testing: Load test new database before cutover, ensure no regression
Index optimization: Analyze query patterns, create missing indexes, drop unused indexes
Connection pooling: Use PgBouncer (PostgreSQL), ProxySQL (MySQL) to handle connection spikes
Insufficient testing: Test on production-like data, not small sample (catches edge cases)
No rollback plan: Always have backup and rollback procedure (Murphy's law applies)
Ignoring encoding: MySQL latin1 → PostgreSQL UTF8 requires conversion (data corruption)
Missing indexes: New database may be missing indexes, causing performance regression
Replication lag: CDC has lag (seconds to minutes), monitor before cutover