12.10 Self-test

Why can't MapReduce's sort start producing output before reading all input? What does that imply about unbounded data?

Define an event. What are the streaming counterparts of "file" and "filename"?

Why does polling get relatively more expensive the more frequently you do it?

State the two questions that differentiate messaging systems, and the three options for the first.

When is message loss acceptable, and what is the trap even then?

Give three direct-messaging approaches and the shared assumption that limits all of them.

Give four differences between a message broker and a database.

Distinguish load balancing from fan-out. How do Kafka consumer groups provide both?

Draw the sequence in which load balancing + redelivery reorders messages. How do you avoid it, and what do you give up?

Describe the poison-message loop. Why does strong ordering make it worse, and what fixes it?

Contrast the "transient messaging" mindset with the database mindset. What two capabilities does the log-based hybrid recover?

What is a partition offset, and what ordering guarantee does it give — and not give?

Give the two downsides of assigning whole shards to consumers. State the rule for choosing between log-based and JMS-style brokers.

Why does a log-based broker only need to record periodic offsets rather than per-message acks? What does that cost you on failure?

Do the 20 TB / 250 MB/s calculation. What does it tell you about how long you have to fix a slow consumer?

Give three operational advantages of a slow consumer in a log-based broker versus a traditional one.

Why is consuming a log "more like reading a file"? What does that enable that AMQP cannot?

Draw the dual-write race. Why won't you notice it? Name the second, independent problem with dual writes.

What does CDC fundamentally do to the topology of your systems?

Why is an initial snapshot needed, and what must it be tied to?

Explain log compaction. What determines the disk space of a compacted log, and what does it let you do that a snapshot otherwise would?

Give three differences between CDC and event sourcing. Which one makes log compaction impossible, and why?

Explain the CDC-schema-as-public-API problem and how the outbox pattern addresses it. What two costs does the outbox add?

State the integrate/differentiate relationship between state and event streams. Quote the Gray–Reuter line about databases.

Give the accounting analogy for handling mistakes in an immutable log.

Give an example of information present in an event log but absent from current state.

Why does "normalized vs denormalized" become "largely irrelevant" with an event log?

Give one way event sourcing worsens and one way it simplifies concurrency control.

Give three limitations of immutability. Explain crypto-shredding and its two structural limits.

What is the one crucial difference between batch and stream processing, and give two consequences.

Explain the query/data role reversal in CEP. Where else in the chapter does the same reversal appear?

Why is it wrong to say stream processing is inherently approximate?

Why do analytics-oriented frameworks fit materialized-view maintenance badly?

Give the two drawbacks of REFRESH MATERIALIZED VIEW, and explain what IVM does instead.

Contrast event time with processing time. Use the Star Wars analogy. Draw what a redeploy does to a processing-time rate metric.

What is a straggler? Give the two handling options and the problem with watermark-style "no more messages before t" signals.

Give the three-timestamp scheme for untrusted device clocks and the two assumptions it makes.

Define tumbling, hopping, sliding, and session windows. Which two have unbounded state cost, and why?

For the search/click join: why isn't embedding search details in the click event equivalent? What does the join emit when no click arrives?

Why is a stream–table join really a stream–stream join? What is the window on the table side?

Write the timeline query as a table–table join and explain the product rule (u·v)′ = u′v + uv′ in that context.

What is the time-dependence problem in joins? Give the tax-rate example, the SCD fix, and what the fix costs.

Why can't batch's fault-tolerance approach be used directly for streams?

Compare microbatching and checkpointing. What implicit window does microbatching impose?

State precisely where exactly-once semantics stop working, and list the four things that must commit atomically.

Why do stream-internal atomic commits work where XA failed?

Give the four assumptions that idempotence-based exactly-once relies on. Which one requires a log-based broker?

Give four ways to recover operator state after a failure, and one case where no replication is needed at all.