8. Transactions
8.8 Production failure catalog for this chapter
| Symptom | Underlying mechanism |
|---|---|
| Two increments, counter went up by one | Lost update — read-modify-write race |
| User sees a new email but a zero unread count | Dirty read (or missing multi-object isolation) |
| Sale awarded to one buyer, invoice sent to another | Dirty write across two tables |
| $100 appears to vanish between two account reads | Read skew under read-committed |
| A backup taken over 3 hours contains inconsistent data | Backup without snapshot isolation |
| Both doctors went off call | Write skew — snapshot isolation is not enough |
| Two bookings for the same room at the same time | Phantom — nothing existed to lock |
| Two accounts registered with the same username | Write skew — fixable with a uniqueness constraint |
| Balance went negative despite a check | Write skew on an aggregate — double-spending |
| User's work discarded on a transient error | ORM doesn't retry aborted transactions |
| Retrying made the outage worse | Retrying a contention/overload error instead of backing off |
| Email sent twice | Side effect outside the database on a retried transaction |
| Everything hangs; one query holds a whole-table lock | 2PL with no suitable index for a range lock |
| Latency p99 collapses under contention | 2PL blocking; one slow transaction stalls the system |
| Massive abort rate after enabling serializable | SSI under high contention, or predicate lock escalation |
| Vacuum can't keep up; table bloats indefinitely | A long-running transaction pinning the MVCC horizon |
| Database refuses writes entirely | Transaction ID wraparound |
| Rows locked for 20 minutes, then forever | In-doubt 2PC transaction; coordinator crashed or lost its log |
| Two systems permanently disagree | Heuristic decision on an in-doubt XA transaction |
| Committed transactions get aborted on recovery | Coordinator lost the most recent part of its log |
| A node shuts itself down for no visible reason | Clock offset exceeded the configured maximum (Spanner/CockroachDB) |
| Cluster throughput collapses at 1,000 writes/s | Cross-shard transactions in a serial-execution system |