replication lag
/rep-lih-KAY-shun lag/ · noun · Development · Origin: 2000
Definitions
The delay between when data is written to a primary database and when it appears on its read replicas. Replication lag causes the classic 'write then immediately read' bug where a user creates something, the read hits a stale replica, and they think their action failed. Measured in seconds or milliseconds, but even small lag can cause user-visible inconsistencies.
In plain English: The delay between when information is saved to the main database and when copies of that database catch up — causes bugs where you save something but then can't see it.
Example: Users kept reporting that their profile updates 'didn't save' because the confirmation page was served from a replica with 200ms of replication lag.
Etymology
- 1970s
- Distributed database research at IBM and UC Berkeley explored data replication. The inherent delay between primary writes and replica updates was a known theoretical challenge.
- 2000
- MySQL's built-in replication (introduced in v3.23) brought replication lag into the daily experience of web developers. Read replicas could fall seconds or minutes behind the primary.
- 2010s
- Cloud databases (Amazon RDS, Aurora, Cloud SQL) made read replicas trivial to provision but did not eliminate lag. 'Eventual consistency' became a term developers had to reason about carefully.
- 2020s
- Technologies like CockroachDB and Spanner aimed for strong consistency across regions. Replication lag remains a fundamental tradeoff in distributed systems, governed by the CAP theorem.
Origin Story
The Lag That Haunts Distributed Databases
Replication lag is the delay between when data is written to a primary database and when that change appears on its replicas. The problem became a significant engineering concern in the early 2000s as internet companies scaled beyond single-server architectures. When sites like Amazon, eBay, and early social networks began using read replicas to distribute query load, they discovered an uncomfortable truth: replication is never instant. A user might update their profile on the primary server, then immediately read from a replica that has not yet received the change, seeing stale data. This 'read-your-own-writes' problem became a classic distributed systems challenge. MySQL's asynchronous replication, PostgreSQL's streaming replication, and similar systems all exhibit some degree of lag, ranging from milliseconds under normal conditions to minutes or hours during heavy load or network issues. Replication lag forced the industry to grapple with the CAP theorem in practical terms: you cannot have perfect consistency, availability, and partition tolerance all at once. Solutions range from semi-synchronous replication to application-level techniques like sticky sessions and causal consistency tokens.
Context: Emerged as a recognized problem in the early 2000s as web companies adopted read replicas for horizontal scaling.
Fun fact: During peak traffic events like Black Friday, some e-commerce platforms have reported replication lag spikes exceeding 30 seconds, causing bizarre user experiences like items appearing 'in stock' on one page and 'sold out' on another simultaneously.