Windowed High Cardinality
Same tumbling 30s window and count + avg(latency_ms) aggregates as the low-cardinality bench, but grouped by user_id (~100k keys) — one open window per active user. Stresses the keyed-state store rather than framework overhead.
Last updated: 2026-09-10 · same-machine, relative comparison.
Bench characteristics
In-process computation — Tumbling 30s event-time window on the Kafka message timestamp, 1 s grace, GROUP BY user_id (≈ 2 × rate keys) → count + avg(latency_ms); one pane per key per window close.
| product | delivery | processing | engine | scaling on one partition | decode | state |
|---|---|---|---|---|---|---|
| turbine | at-least-once (idempotent, acks=all) | batch (Arrow columnar) | in-process · Rust core + free-threaded CPython 3.14t | intra-partition shards (parallelism=N, hash of partition_key, in-process) |
schema projection (6 of 19 fields, Arrow) | own RocksDB store (merge operator for windows) |
| flink | at-least-once (checkpointed) | batch (pipelined, JVM code-gen SQL) | in-process · JVM (PyFlink local MiniCluster) | keyed exchange to N slots (parallelism.default, in-process local channels; source stays 1 subtask) |
schema projection (4 of 19 fields, Jackson); connector-default fetch | RocksDB (embedded, on-disk LSM) |
| quix | at-least-once (idempotent, acks=all) | row (per-record StreamingDataFrame) | in-process · pure-Python over librdkafka (C client) | none (one consumer per partition, GIL) | full record (19 of 19 fields, orjson dict) | RocksDB (embedded); changelog topic OFF for the bench |
| risingwave | at-least-once | batch (vectorized SQL) | external server · Rust (single-node: meta+compute+frontend) | hash exchange to N stream actors (SET streaming_parallelism, in-process) |
schema projection (4 of 19 fields) | server-managed (Hummock LSM store) |
Input is a single Kafka partition (lz4). Each product may spread it over several cores through its own in-process mechanism (column scaling on one partition; what it actually used is cpu_avg); an in-process exchange is not a broker hop — the extra rec/rec column counts records that DID cross the broker on other topics. All products run at-least-once, windows are tumbling 30 s event-time on the Kafka message timestamp.
Consumer fetch: Turbine's core hard-codes librdkafka fetch.min.bytes=64 KB, fetch.wait.max.ms=10, max.partition.fetch.bytes=50 MB and a 500k-msg / ~2 GB prefetch queue (not adjustable), so the SAME client gets the SAME settings: quix and RisingWave (librdkafka) run with them, and so does FastStream (aiokafka, whose equivalent knobs lifted it ~14%). Flink runs at its Java connector's defaults: an A/B (2026-09-10) showed the identical numbers throttled the Java consumer (373k/s vs 543k/s at the same ~285k rows/s per core) — identical numbers are not identical behaviour on a different client. Memory at saturation is mostly this prefetch queue, for every librdkafka product.
Throughput / efficiency (live)
| product | offered/s | consumed/s | regime | lag drift | cpu_avg % | rows/s per core | rows/s per GB | µs CPU / record | panes/s | extra rec/rec | run |
|---|---|---|---|---|---|---|---|---|---|---|---|
| turbine 0.2.0a1 | 188,021 | 186,802 | keep-up | +0.7% | 122.5 | 152,514 | 84,460 | 6.56 | 12,582.36 | 0 | 2026-09-10 |
| flink 2.2.1 | 118,737 | 117,953 | keep-up | +0.7% | 104.5 | 112,921 | 107,400 | 8.86 | 7,975.80 | 0 | 2026-09-10 |
| quix 3.23.6 | 44,532 | 44,271 | keep-up | +0.6% | 72.1 | 61,408 | 70,975 | 16.28 | 2,978.44 | 0 | 2026-09-10 |
| risingwave v2.8.4 | 247,448 | 247,580 | keep-up | -0.1% | 350.9 | 70,559 | 63,031 | 14.17 | 16,476.81 | 0 | 2026-09-10 |
consumed/s is what the product actually drained (its consumer-group offset or engine counter), offered/s what gendata produced. regime: keep-up = consumed ≈ offered with a flat backlog (a sustainable point); saturated = fed beyond capacity, so consumed/s IS the ceiling and latency is unbounded; draining = still eating warmup backlog (consumed > offered; CPU is catch-up, not steady state — lengthen the warmup, or raise the rate if the ceiling is what you want). lag drift = backlog change over the window ÷ records consumed. cpu_avg is the whole process tree, 100 = one core busy. rows/s per core = consumed ÷ (cpu_avg/100); rows/s per GB = consumed ÷ memory; extra rec/rec = records written to topics other than the input and output per record consumed (0 = no broker hop).
panes/s is the window-emission cadence (≈ keys ÷ window size), a liveness signal, not a throughput figure — it only moves when windows close, which needs the consumer to keep up.
Latency (close latency — pane published minus window end)
| product | regime | p50 ms | p90 ms | p99 ms | p99.9 ms | max ms | records | setup floor p50 / p99 ms | added p50 / p99 ms |
|---|---|---|---|---|---|---|---|---|---|
| turbine 0.2.0a1 | keep-up | 11,298.9 | 19,030.7 | 20,789.2 | 21,301.7 | 21,506.3 | 1516874 | 105.7 / 204.2 | 11,193.2 / 20,585.0 |
| flink 2.2.1 | keep-up | 3,924.4 | 8,047.4 | 8,736.6 | 8,987.1 | 8,987.1 | 959884 | 106.0 / 204.0 | 3,818.4 / 8,532.6 |
| quix 3.23.6 | keep-up | 1,435.7 | 1,855.4 | 2,106.2 | 2,310.6 | 2,310.6 | 359964 | 106.0 / 203.8 | 1,329.7 / 1,902.4 |
| risingwave v2.8.4 | keep-up | 8,023.5 | 10,008.5 | 10,106.8 | 10,924.7 | 11,128.1 | 1992612 | 106.2 / 204.5 | 7,917.3 / 9,902.3 |
Measured by a probe consumer tailing the output topic (C client, 10 ms max fetch wait — the probe adds ≲10 ms, the same for every product). Only shown for keep-up runs: under saturation the figure is the backlog's age and grows without bound. setup floor is a harness producer writing one small record every 10 ms (acks=all, 5 ms linger) to its own topic during the same run, measured by the same probe: the broker's produce→visible latency at the products' output shape (on Redpanda, small produces surface on its flush cadence), which every product inherits. added = product − floor, the part the product is responsible for (fetch, batching, commit/epoch cadence, produce).
Resources (raw)
| product | cpu_avg % | cpu_p95 % | mem_avg MB | mem_max MB | samples | instrument |
|---|---|---|---|---|---|---|
| turbine 0.2.0a1 | 122.5 | 173.3 | 2,264.8 | 2,649.5 | 116 | /proc Δ, pss |
| flink 2.2.1 | 104.5 | 192.5 | 1,124.6 | 1,131.3 | 117 | /proc Δ, pss |
| quix 3.23.6 | 72.1 | 101.9 | 638.7 | 666.8 | 118 | /proc Δ, pss |
| risingwave v2.8.4 | 350.9 | 500.5 | 4,022.2 | 5,213.4 | 118 | /proc Δ, rss |
CPU is summed over the whole process tree (JVM, UDF servers, the RisingWave container's processes) as utime+stime deltas per 1 s sample — 100 = one core. Memory is PSS (shared pages counted once) where readable, RSS otherwise (instrument). Rows marked ⚠ legacy ps were measured with the old ps %CPU lifetime average and are not comparable: re-run them.
flink — PyFlink local MiniCluster, parallelism=1, RocksDB keyed-state backend (EmbeddedRocksDBStateBackend, on-disk LSM) — apples-to-apples with quix/Turbine/RisingWave, NOT Flink's default heap (RAM) backend (which sustained 350k here, 3.5× more, but kept all ~700k window keys in the JVM heap at ~3.5 GB). At-least-once checkpointing. CPU/RSS include the JVM child (see the stateless note). Single-partition knee ≈ 100k (consume ≈ offered, RSS bounded ~1.2 GB off-heap); 120k decouples. Flink-RocksDB (100k) lands above quix's RocksDB (27k) but below turbine/RW (200k, different backends).
Sanity — are we benching the same thing?
| product | panes/s | expected | sample |
|---|---|---|---|
| turbine 0.2.0a1 | 12,582.36 | 12,666.73 | 500 |
| flink 2.2.1 | 7,975.80 | 6,667.00 | 500 |
| quix 3.23.6 | 2,978.44 | 1,785.00 | 500 |
| risingwave v2.8.4 | 16,476.81 | 13,333.00 | 500 |
Aggregated output: checked against the expected pane cadence (expected.output_rate in bench.yaml), not per-record selectivity.
Verdict: ✓ matches theory — panes/s within 2× of the expected cadence for every row.
Host: AMD Ryzen 5 7600X 6-Core Processor · 6 cores / 12 threads · 31241 MB · kernel 6.12.107+deb13-amd64.
Redpanda: rpk version: v26.1.6. Redpanda, gendata and the product under test share this host.