Model Inference — exactly-once
Score every record with the SAME frozen 4-16-1 PyTorch MLP (models/, trained once by models/make_model.py). No filter, so selectivity is 1.0 — this prices what each framework costs around a forward pass: decode, feature extraction, the call itself, encode, produce. The model is deliberately tiny so the frameworks, not the matmul, are what is measured. Benched at each product's max sustainable single-partition rate (its knee).
Last updated: 2026-11-10 · same-machine, relative comparison.
flowchart LR
src[("bench-input<br/>1 partition")]
s0["<b>map</b><br/>4 features scaled to [0, 1]"]
s1(["<b>model</b><br/>MLP 4-16-1 → risk"])
sink[("bench-out")]
src --> s0
s0 --> s1
s1 -->|"exactly-once"| sinkBench characteristics
In-process computation — Per record: four gendata metrics (latency_ms, queue_depth, cpu_usage, error_count) divided by their documented maxima into [0, 1], fed to a frozen 4-16-1 MLP (Linear/ReLU/Linear/Sigmoid) → risk. Stateless; every record is scored and republished.
| product | delivery | processing | engine | scaling on one partition | decode |
|---|---|---|---|---|---|
| turbine-eos | exactly-once (Kafka transaction per batch; crash recovery: replay) | batch (Arrow columnar; 100k rows / 500 ms) | in-process · Rust core + free-threaded CPython 3.14t | intra-partition shards (parallelism=N, hash of partition_key, in-process) |
schema projection (9 of 19 fields, Arrow) |
| arroyo-eos | exactly-once (checkpoints every 5 s; Kafka transaction per checkpoint, 2PC sink) | vectorized SQL around a Rust UDF on ONNX Runtime (one ORT run per record) | in-process · Rust (arroyo cluster: API + compiler + controller + one worker process) |
none for a per-record job (chained into the subtask reading the partition) | schema projection (7 of 19 fields) |
| flink-eos | exactly-once (EXACTLY_ONCE checkpoints every 5 s; Kafka transaction per checkpoint, 2PC sink) | batch (pipelined, JVM code-gen SQL) | in-process · JVM (local MiniCluster; Python only submits the job) | rebalance to N slots (parallelism.default, in-process local channels; source stays 1 subtask) |
schema projection (7 of 19 fields, Jackson); connector-default fetch |
| quix-eos | exactly-once (Kafka transaction per commit, every 5 s) | row (per-record StreamingDataFrame) | in-process · pure-Python over librdkafka (C client) | none (one consumer per partition, GIL) | full record (19 of 19 fields, orjson dict) |
Input is a single Kafka partition (lz4). Each product may spread it over several cores through its own in-process mechanism (column scaling on one partition; what it actually used is cpu_avg); an in-process exchange is not a broker hop — the extra rec/rec column counts records that DID cross the broker on other topics. Every line runs exactly-once: its output becomes visible to a read_committed consumer at its transaction's commit (column delivery).
Consumer fetch: Turbine's core hard-codes librdkafka fetch.min.bytes=64 KB, fetch.wait.max.ms=10, max.partition.fetch.bytes=50 MB and a 500k-msg / ~2 GB prefetch queue (not adjustable), so the SAME client gets the SAME settings: quix and RisingWave (librdkafka) run with them, and so does FastStream (aiokafka, whose equivalent knobs lifted it ~14%). Flink runs at its Java connector's defaults: an A/B (2026-09-10) showed the identical numbers throttled the Java consumer (373k/s vs 543k/s at the same ~285k rows/s per core) — identical numbers are not identical behaviour on a different client. Memory at saturation is mostly this prefetch queue, for every librdkafka product.
Drain ceiling (--prefill: static backlog, no producer running)
| product | backlog | drained/s | cpu_avg % | rows/s per core | rows/s per GB | µs CPU / inference | output/s | run |
|---|---|---|---|---|---|---|---|---|
| turbine-eos 0.3.5a1 | 120M | 721,199 | 307.6 | 234,477 | 391,807 | 4.26 | 721,214 | 2026-11-10 |
| arroyo-eos 0.15.0 | 120M | 323,138 | 210.6 | 153,465 | 160,375 | 6.52 | 344,135 | 2026-11-06 |
| flink-eos 2.2.1 | 60M | 379,999 | 446.7 | 85,067 | 237,415 | 11.76 | 386,007 | 2026-11-06 |
| quix-eos 3.23.6 | 60M | 30,561 | 102.7 | 29,771 | 33,735 | 33.59 | 30,899 | 2026-11-06 |
How to read this table
gendata wrote the backlog and exited BEFORE the consumer started, so the window measures a drain with no producer competing for the box — the product's consume ceiling free of gendata's CPU and 10 Hz burstiness. Not comparable with a live table (different regime); ⚠ exhausted = the backlog ran out mid-window, rate understated.
Cost of exactly-once
| line | vs | regime | rows/s per core | µs CPU / inference | consumed/s | latency p50 ms | measured |
|---|---|---|---|---|---|---|---|
| turbine-eos · drain | turbine | drain → drain | +2.8% | 4.38 → 4.26 | -12.6% | - | interleaved |
| arroyo-eos · drain | arroyo | drain → drain | -6.5% | 6.09 → 6.52 | -9.6% | - | separately |
| flink-eos · drain | flink | drain → drain | +14.8% | 13.50 → 11.76 | +6.4% | - | separately |
| quix-eos · drain | quix | drain → drain | +2.7% | 34.49 → 33.59 | +2.7% | - | separately |
How to read this table
Each exactly-once line against its at-least-once line — the same product, workload, mode and cores, the newest batch of each. rows/s per core: the change in efficiency (the inverse of µs CPU per record, shown at-least-once → exactly-once); consumed/s: the change in capacity, only when both runs were fed beyond it (drain or saturated — at a keep-up point it is the offered rate); the latency p50 when both kept up — an exactly-once line's output becomes visible to the probe, a read_committed consumer, at its commit (a batch's transaction for Turbine, a checkpoint's for Flink and Arroyo, a commit's for Quix: the interval is in the characteristics' delivery and processing columns — on the latency bench every exactly-once line commits at Turbine's batch timeout, a transaction per batch, the others through BENCH_COMMIT_INTERVAL_MS (Arroyo every 1 s, the shortest checkpoint interval it accepts), which their at-least-once lines, publishing without waiting for a commit, do not set: ≠ config there). measured: interleaved = both lines in one interleaved batch (one host, run by run); separately = two sessions, the difference then carrying the host-to-host spread too (a few %; a Flink VM once read 30 % slower, before the session gate). · ≠ config / ≠ host / ≠ bench: the two runs differ there too — read the figures with it. Output records/s read from offsets include one transaction marker per transaction that wrote to the output — noise on the per-record workloads (≲ 1 per 10k records), up to ~1 in 16 panes on windowed-lowcard's 0.53 panes/s.
Resources (raw)
| product | cpu_avg % | cpu_p95 % | mem_avg MB | mem_max MB | samples | instrument |
|---|---|---|---|---|---|---|
| turbine-eos 0.3.5a1 | 307.6 | 349.8 | 1,884.9 | 2,014.3 | 59 | /proc Δ, pss |
| arroyo-eos 0.15.0 | 210.6 | 257.1 | 2,063.2 | 2,488.1 | 59 | /proc Δ, pss |
| flink-eos 2.2.1 | 446.7 | 458.6 | 1,639.0 | 1,653.6 | 59 | /proc Δ, pss |
| quix-eos 3.23.6 | 102.7 | 109.7 | 927.7 | 941.3 | 59 | /proc Δ, pss |
How to read this table
CPU is summed over the whole process tree (JVM, UDF servers, the RisingWave container's processes) as utime+stime deltas per 1 s sample — 100 = one core. Memory is PSS (shared pages counted once) where readable, RSS otherwise (instrument). Rows marked ⚠ legacy ps were measured with the old ps %CPU lifetime average and are not comparable: re-run them.
Sanity — ✓ matches theory
Are we benching the same thing?
| product | selectivity (out/consumed) | expected | ✓ |
|---|---|---|---|
| turbine-eos 0.3.5a1 | 1.0006 | 1.0000 | ✓ |
| arroyo-eos 0.15.0 | 1.0607 | 1.0000 | ✓ |
| flink-eos 2.2.1 | 1.0158 | 1.0000 | ✓ |
| quix-eos 3.23.6 | 1.0114 | 1.0000 | ✓ |
Verdict: ✓ matches theory — every row's selectivity is within 25% of the expected value (output ÷ consumed).
Turbine releases
| line | version | bench | regime | offered/s | consumed/s | rows/s per core | µs CPU / inference | rows/s per GB | p99 ms | Δ efficiency | Δ capacity | run |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| turbine-eos · drain | 0.3.5a1 | 1 | drain | — | 721,199 | 234,477 | 4.26 | 391,807 | - | +6.5% | +1.9% | 2026-11-10 |
| turbine-eos · drain | 0.3.4a1 | 1 | drain | — | 707,587 | 220,158 | 4.54 | 386,785 | - | - | - | 2026-11-10 |
How to read this table
The Turbine lines measured on several releases, newest first: a regression check, not part of the cross-product comparison above (which shows the pinned release only). Δ compares each release with the one below it: efficiency = rows/s per core (the inverse of µs CPU per record), capacity = consumed/s, only when both runs were fed beyond capacity (saturated or drain — at a keep-up point consumed/s is just the offered rate). ⚠ = worse by more than 5%, about the single-run noise: confirm with --repeat 3. Two releases measured in two sessions also differ by the host-to-host spread — same flavor, another physical host: −0.7 % consumed / −3.5 % rows/s per core in the c3 control of 2026-10-07 —, which leaves little headroom under the 5 %: re-measure both releases in ONE session (suite.py run … --version X) before acting on a ⚠. No delta is computed across a change of bench version (bench, from bench.yaml), host or run config; ⚠ deprecated rows were measured on an older bench definition — re-run that release with suite.py run turbine WORKLOAD --version X.
Host: ovh:c3-32 (ovh:c3-32@DE1) · AMD EPYC-Milan Processor (2400 MHz) · 16 cores / 16 threads · 31345 MB · kernel 6.12.100+deb13-cloud-amd64.
Redpanda: rpk version: v26.2.2. Redpanda, gendata and the product under test share this host.