13. A Philosophy of Streaming Systems
13. A Philosophy of Streaming Systems
Chapter 13 of Designing Data-Intensive Applications — 12 sections.
"If the highest aim of a captain was to preserve his ship, he would keep it in port forever." — St. Thomas Aquinas (paraphrased)
This chapter is MORE OPINIONATED than previous chapters, presenting A DEEP DIVE INTO ONE PARTICULAR PHILOSOPHY rather than comparing multiple approaches.
The arc:
- Data integration — how do you get data into all the right places?
- Unbundling databases — the whole organization’s dataflow as One huge database
- Observing derived state — write path vs read path, extended to the end-user device
- Aiming for correctness — integrity Without coordination; the end-to-end argument
Sections
- 13.13Data Integration
- 13.22Unbundling Databases
- 13.31Designing Applications Around Dataflow
- 13.42Observing Derived State — the write path and the read path
- 13.54Aiming for Correctness
- 13.6Deep divesTechnology deep dives
- 13.7Failure catalogProduction failure catalog for this chapter
- 13.8Decision sheet2Decision cheat sheetPrecompute (write path) when the query set is small and known; compute on read when it's large or unbounded.
- 13.9Worked examplesWorked examplesThis is §1.5's "asynchrony is what makes systems based on event logs robust," as a number.
- 13.10Self-testSelf-test
- 13.11TerminologyTerminology introduced here
- 13.12Backward links — where each idea came from