1. Trade-Offs in Data Systems Architecture
1. Trade-Offs in Data Systems Architecture
Chapter 1 of Designing Data-Intensive Applications — 12 sections.
"There are no solutions; there are only trade-offs." — Thomas Sowell
Thesis of the chapter: every architectural choice in a data system is a trade-off along four axes. This chapter names the axes and gives you the vocabulary for the rest of the book.
The four axes:
- Operational (OLTP) vs Analytical (OLAP) — what the data is for
- Cloud vs Self-hosted — who builds it and who runs it
- Distributed vs Single-node — how many machines, and why
- Business needs vs User rights — law, ethics, and data minimization
Sections
- 1.01The mental model for the whole bookThe hard part is never one block. It's choosing between blocks with different characteristics, and gluing blocks together when no single tool does the job.
- 1.14Operational vs Analytical SystemsKey observation: analysts and scientists both read data that users and backend services generated, and they do not modify it (they may create derived datasets).
- 1.22Cloud vs Self-HostingTwo separable decisions: who builds the software and who deploys it.
- 1.3Distributed vs Single-Node SystemsA distributed system = several machines communicating over a network.
- 1.4Data Systems, Law, and SocietyArchitecture is shaped by human and legal needs, not just technical ones.
- 1.5Deep divesTechnology deep divesFor each: what problem, why wasn't the alternative enough, how it works internally, deployment, monitoring, scaling, backup, what breaks in production.
- 1.6Failure catalogCross-cutting production failure catalog for this chapter
- 1.7Decision sheetDecision cheat sheetYes if any of: analysts need to join across ≥2 operational systems; analytical queries would compete with user traffic; analysts need ad-hoc SQL; or dataset is heading past a few…
- 1.8TerminologyTerminology introduced here (used for the rest of the book)
- 1.9Worked examplesWorked examplesAnalysts need to join across 3 of them. Data volume says "no warehouse needed" — but the silo argument says yes, because you cannot join across three separately-owned OLTP databas…
- 1.10Self-testSelf-test
- 1.11Forward linksForward links
Designing Data-Intensive Applications
Chapter-by-chapter notes on Designing Data-Intensive Applications, 2nd edition — concepts, diagrams, technology deep dives, failure catalogs, and self-tests.
1.0 The mental model for the whole book
The hard part is never one block. It's choosing between blocks with different characteristics, and gluing blocks together when no single tool does the job.