Are you an LLM? Read llms.txt for a summary of the docs, or llms-full.txt for the full context.
Skip to content

Performance Tuning

Most shove performance questions resolve to four levers: prefetch count, worker count, the concurrent-processing flag, and (negatively) transactional mode. Knowing which lever to pull first — and which to ignore — is what this page is for. The charts below are generated from a committed results document, and each carries its own provenance (shove version, generation date, hardware) in the caption; always measure on your own hardware and workload.

Benchmarks

Two conventions hold across every chart. A backend that cannot do a flow (HasBroadcast / HasCoordinatedGroups gate them at compile time) appears as an explicit "not supported" marker, never as a zero. And a backend the results document does not contain is never plotted, so absence from a chart means "not measured here", not "slow".

The document these charts render from measures all six backends: in-process, Kafka, NATS, RabbitMQ, Redis and SQS. The SQS rows measure LocalStack, which is non-representative of the AWS service by design, and run a smaller corpus than the other backends; the rows and the charts record both (see the benchmark runbook's "The SQS corpus deviation"). Every run was measured on the same host: the harness refuses to merge runs from different hosts into one document.

Throughput vs consumer count

How drain rate scales with consumer count, per backend.

Throughput vs consumer count, per backend

Throughput vs payload size

The cost of moving 64 B, 1 KiB and 64 KiB messages through the same flow.

Throughput vs payload size, per backend

Parallel vs sequenced

What ordering costs: the same workload consumed in parallel and through sequenced (FIFO) delivery.

Parallel vs sequenced throughput, per backend

Dispatch latency

Per-message dispatch percentiles (p50/p95/p99). When the results document cannot support the claim, the chart refuses on its face instead of plotting a misleading number — the accounting for each backend is rendered in the chart itself.

Dispatch latency percentiles, per backend

Framework overhead

What shove itself costs, in nanoseconds per message per flow — measured on the in-process backend so no broker round-trip is included.

Framework overhead per flow, nanoseconds per message

The throughput levers

In order of impact:

  • prefetch_count (largest lever): how many unacked messages the broker pre-delivers to a consumer. Higher values mean more in-flight work and fewer round-trips waiting for acks.
  • Worker count (linear scaling on handler-bound workloads): more concurrent handlers means more throughput when handlers are CPU- or I/O-bound. Scaling is close to linear until you hit a broker or partition limit.
  • concurrent_processing flag: true allows multiple handler invocations to run concurrently within a single worker, bounded by prefetch_count. false (default) processes one message at a time per worker.
  • Transactional mode (negative lever): the rabbitmq-transactional feature reduces throughput roughly 10–15× per channel compared to non-transactional. The trade-off is routing safety — every ack and publish land in the same AMQP transaction.

prefetch_count deep-dive

prefetch_count tells the broker how many unacked messages it may deliver to a single consumer before waiting for acks. With prefetch_count = 1, the broker waits for an ack before sending the next message — every handler invocation pays a full network round-trip. With higher values, the consumer has a local buffer of pre-delivered messages and can begin processing the next one immediately after finishing the previous.

Why too high is also a problem: if one consumer holds a large unacked backlog, other consumers in the same group receive less work — distribution becomes uneven. Very high values also increase per-consumer memory usage and slow graceful shutdown (all unacked messages must drain).

Recommended starting point: 10–40. Tune based on handler latency: short-latency handlers benefit more from high prefetch; long-latency handlers (where the bottleneck is the handler itself) need high worker counts instead.

Worked example. Handler average latency 10 ms, network RTT 1 ms:

  • prefetch_count = 1: every message pays the full 11 ms cycle — 10 ms of handler work plus a round-trip idle waiting for the next delivery. The round-trip tax caps throughput below what the handler alone could sustain.
  • prefetch_count = 20: the consumer always has the next message buffered, so the cycle is the 10 ms of handler work — the handler runs at close to its own maximum rate.

Workers — when to add them

Add workers when the topic backlog grows faster than the consumer drains it. That's the signal that the handler is the bottleneck.

A single shove consumer rarely saturates the network or broker on its own. The bottleneck is almost always handler latency × concurrency. Each additional worker adds a parallel handler goroutine; throughput scales linearly until you hit:

  • Broker quota — e.g. RabbitMQ per-connection channel limits.
  • Partition count (Kafka) — a 4-partition topic can only be consumed by 4 group members simultaneously.
  • Sub-queue count (RabbitMQ sequenced) — the number of consistent-hash shards limits sequenced consumer parallelism.

with_concurrent_processing(true)

When concurrent_processing is true, a single worker can have multiple handler invocations in-flight simultaneously, limited by prefetch_count. The benefit is full utilization of the pre-fetched buffer: while one handler is awaiting a database response, another can start.

Use when: handlers are pure I/O-bound — waiting on a database query, an HTTP call, an external API. The async runtime multiplexes tasks efficiently.

Don't use when: handlers are CPU-bound. Concurrent tasks without additional OS threads don't add parallelism — they compete for the same thread pool and can increase latency without improving throughput. For CPU-bound work, increase worker count instead.

Measurement methodology

The published charts render from a committed results document, benches/results/bench-results.json, generated by the stress harness one backend at a time — --results-file merges into the same document by backend key:

cargo run --release --example inmemory_stress --features inmemory -- \
    --flow all --payload all --tier moderate --handler zero \
    --consumers 1,2,4,8 --results-file benches/results/bench-results.json

Repeat per backend with its example target and feature flag — note redis_stress needs --features redis-streams and sqs_stress needs --features aws-sns-sqs (the full table is in Choosing a backend). The harness sweeps (flow, payload, tier, handler, consumers) and stamps provenance — shove version, generation date, hardware — into the document; every chart caption renders it.

The charts regenerate from the committed document:

cargo run --no-default-features --example chartgen -- \
    --input benches/results/bench-results.json --out-dir docs/public/bench

tests/chartgen.rs byte-compares the committed SVGs against the committed results document, so a results file updated without regenerating the charts — or a hand-edited chart — fails the test suite.

Hardware, broker version, network setup, and OS scheduler all affect results. The charts are a starting reference — measure on your own setup before committing to a configuration.

What NOT to tune first

Ranked by typical impact:

  • Serde format — JSON serialization overhead is negligible compared to network round-trip. Switching to bincode or MessagePack rarely moves the needle for normal message sizes.
  • Connection pool size — a single AMQP channel or Kafka producer moves messages far faster than a typical handler consumes them. Pooling is for fault-tolerance (connection failover), not raw throughput.
  • TLS — modern TLS handshake and record overhead is negligible for the message sizes and rates shove handles. Disable only if profiling shows it in a hot path.
  • Tracing — structured logging at info level adds less than 1% overhead. Disable only at debug level if profiling shows it dominates.

What's next