3.12 Quotas and throttling

"Kafka brokers have the ability to limit the rate at which messages are produced and consumed. This is done via the quota mechanism."

Three quota types

Type	Limits	Unit
produce	rate at which clients can send data	bytes per second
consume	rate at which clients can receive data	bytes per second
request	percentage of time the broker spends processing client requests	% of broker time

Scope: can be applied to all clients (defaults), specific client-ids, specific users, or both.

"User-specific quotas are only meaningful in clusters where security is configured and clients authenticate."

Static configuration (broker config file) — and why it's bad

# default for all clients
quota.producer.default=2M

# per-client overrides (while not recommended)
quota.producer.override="clientA:4M,clientB:10M"

"Quotas specified in Kafka's configuration file are static, and you can only modify them by changing the configuration and then restarting all the brokers. Since new clients can arrive at any time, this is very inconvenient."

Dynamic configuration — the usual method

Via kafka-config.sh or the AdminClient API:

# limit clientC (by client-id) to produce only 1024 B/s
bin/kafka-configs --bootstrap-server localhost:9092 --alter \
  --add-config 'producer_byte_rate=1024' \
  --entity-name clientC --entity-type clients

# limit user1 (by authenticated principal): produce 1024 B/s, consume 2048 B/s
bin/kafka-configs --bootstrap-server localhost:9092 --alter \
  --add-config 'producer_byte_rate=1024,consumer_byte_rate=2048' \
  --entity-name user1 --entity-type users

# limit ALL users to consume 2048 B/s, except users with a more specific
# override — this is how you dynamically modify the DEFAULT quota
bin/kafka-configs --bootstrap-server localhost:9092 --alter \
  --add-config 'consumer_byte_rate=2048' --entity-type users

How throttling actually works — two mechanisms

· In most clients this automatically reduces the request rate (because the number of in-flight requests is limited) → client traffic falls to the allowed level.
· Muting “protects the broker from misbehaved clients that keep sending additional requests while being throttled.”

Figure 3.12.1How throttling actually works — two mechanisms

Mechanism 1 is elegant: it exploits the client's own in-flight limit as a natural rate limiter, so well-behaved clients need no throttling logic at all. Mechanism 2 is the enforcement backstop for clients that ignore the hint.

Throttling metrics exposed to clients

 produce-throttle-time-avg    produce-throttle-time-max
 fetch-throttle-time-avg      fetch-throttle-time-max

The average and maximum time a produce/fetch request was delayed due to throttling.

"Note that this time can represent throttling due to produce and consume throughput quotas, request time quotas, or both. Other types of client requests can only be throttled due to request time quotas, and those will also be exposed via similar metrics."

⚠️ WARNING — the buffer-exhaustion cascade

This is the most important operational warning in the chapter. Trace it:

“It is therefore important to plan and monitor to make sure that the broker capacity over time will match the rate at which producers are sending data.”

Figure 3.12.2This is the most important operational warning in the chapter. Trace it

"It is therefore important to plan and monitor to make sure that the broker capacity over time will match the rate at which producers are sending data."

And note the echo of Ch. 1's ActiveMQ lesson: step 4 is the producer being stalled by the broker. Async sends do not protect you — they only move the stall point from "every send" to "when the buffer fills."