Fraud Scoring Latency (iso-rate) — exactly-once
The online-scoring case — fraud detection, where a payment waits for its score and must get exactly one: the inference app of every exactly-once line, the same frozen MLP on every record, run in the engine by Turbine (its infer= operator, ONNX Runtime once per batch), Flink (a Java UDF) and Arroyo (a Rust UDF) — ONNX Runtime once per record for those two — and as eager PyTorch once per record in Quix's Python. Every line is fed the SAME modest rate, every batch closes on the latency setting's timeout and every transaction commits at it (Arroyo every 1 s, the shortest it accepts); a read_committed probe consumer tails the output topic: latency = score visible − gendata's timestamp (p50 / p99 / p99.9). Only meaningful in the keep-up regime. Not comparable with the latency page: its app republishes ~31 % of the records (~1.3 MB/s), which surface on the broker's 100 ms flush timer (its setup floor); this one republishes every record (~3.5–5 MB/s), which surfaces sooner — below that floor.
Last updated: 2026-11-12 · same-machine, relative comparison.
flowchart LR
src[("bench-input<br/>1 partition")]
s0["<b>map</b><br/>4 features scaled to [0, 1]"]
s1(["<b>model</b><br/>MLP 4-16-1 → risk"])
sink[("bench-out")]
probe>"latency probe<br/>arrival − timestamp"]
src --> s0
s0 --> s1
s1 -->|"exactly-once"| sink
sink -.-> probeLatency (end-to-end — record published minus gendata's timestamp)
| product | regime | p50 ms | p90 ms | p99 ms | p99.9 ms | max ms | records | setup floor p50 / p99 ms | added p50 / p99 ms |
|---|---|---|---|---|---|---|---|---|---|
| turbine-eos 0.3.5a1 | keep-up | 33.3 | 40.0 | 44.6 | 50.1 | 52.5 | 3610000 | 107.4 / 205.9 | below / below |
| arroyo-eos 0.15.0 | keep-up | 1,029.7 | 1,586.9 | 1,736.5 | 1,881.6 | 1,884.2 | 3586998 | 106.8 / 205.2 | 922.9 / 1,531.3 |
| flink-eos 2.2.1 | keep-up | 27.2 | 32.3 | 37.5 | 44.2 | 46.9 | 3640000 | 107.9 / 206.0 | below / below |
| quix-eos 3.23.6 | keep-up | 147.2 | 165.8 | 168.1 | 186.5 | 195.8 | 3610000 | 107.0 / 205.6 | 40.2 / below |
How to read this table
Measured by a probe consumer tailing the output topic (C client, 10 ms max fetch wait — the probe adds ≲10 ms, the same for every product). Only shown for keep-up runs: under saturation the figure is the backlog's age and grows without bound. setup floor is a harness producer writing one small record every 10 ms (acks=all, 5 ms linger) to its own topic during the same run, measured by the same probe: the broker's produce→visible latency at the products' output shape (on Redpanda, small produces surface on its flush cadence), which every product inherits. added = product − floor, the part the product is responsible for (fetch, batching, commit/epoch cadence, produce). The floor is a low-volume topic's (100 small records/s, surfacing on Redpanda's 100 ms flush.ms): an output topic carrying several MB/s — every record scored and republished — surfaces sooner: a percentile under the floor's reads below, added does not apply. Colours: each percentile against the best (lowest) of its column — green under 2×, yellow from 2×, orange from 5×, red from 10× (the ratio on hover).
Bench characteristics
In-process computation — Same as inference (four gendata metrics scaled to [0, 1] → the frozen 4-16-1 MLP → risk, every record scored and republished); the output additionally carries gendata's timestamp for the probe.
| product | delivery | processing | engine | scaling on one partition | decode |
|---|---|---|---|---|---|
| turbine-eos | exactly-once (Kafka transaction per batch; crash recovery: replay) | batch (Arrow columnar; 10k rows / 10 ms) | in-process · Rust core + free-threaded CPython 3.14t | intra-partition shards (parallelism=N, hash of partition_key, in-process) |
schema projection (9 of 19 fields, Arrow) |
| arroyo-eos | exactly-once (checkpoints every 1 s; Kafka transaction per checkpoint, 2PC sink) | vectorized SQL around a Rust UDF on ONNX Runtime (one ORT run per record) | in-process · Rust (arroyo cluster: API + compiler + controller + one worker process) |
none for a per-record job (chained into the subtask reading the partition) | schema projection (8 of 19 fields) |
| flink-eos | exactly-once (EXACTLY_ONCE checkpoints every 10 ms; Kafka transaction per checkpoint, 2PC sink) | batch (pipelined, JVM code-gen SQL) | in-process · JVM (local MiniCluster; Python only submits the job) | rebalance to N slots (parallelism.default, in-process local channels; source stays 1 subtask) |
schema projection (8 of 19 fields, Jackson); connector-default fetch |
| quix-eos | exactly-once (Kafka transaction per commit, every 10 ms) | row (per-record StreamingDataFrame) | in-process · pure-Python over librdkafka (C client) | none (one consumer per partition, GIL) | full record (19 of 19 fields, orjson dict) |
Input is a single Kafka partition (lz4). Each product may spread it over several cores through its own in-process mechanism (column scaling on one partition; what it actually used is cpu_avg); an in-process exchange is not a broker hop — the extra rec/rec column counts records that DID cross the broker on other topics. Every line runs exactly-once: its output becomes visible to a read_committed consumer at its transaction's commit (column delivery).
Consumer fetch: Turbine's core hard-codes librdkafka fetch.min.bytes=64 KB, fetch.wait.max.ms=10, max.partition.fetch.bytes=50 MB and a 500k-msg / ~2 GB prefetch queue (not adjustable), so the SAME client gets the SAME settings: quix and RisingWave (librdkafka) run with them, and so does FastStream (aiokafka, whose equivalent knobs lifted it ~14%). Flink runs at its Java connector's defaults: an A/B (2026-09-10) showed the identical numbers throttled the Java consumer (373k/s vs 543k/s at the same ~285k rows/s per core) — identical numbers are not identical behaviour on a different client. Memory at saturation is mostly this prefetch queue, for every librdkafka product.
Throughput / efficiency (live) — capped by the machine: gendata produces ~430k/s at most, so read capacity in the drain table
| product | offered/s | consumed/s | regime (lag drift) | cpu_avg % | rows/s per core | rows/s per GB | µs CPU / inference | output/s | extra rec/rec | run |
|---|---|---|---|---|---|---|---|---|---|---|
| turbine-eos 0.3.5a1 | 19,790 | 19,780 | keep-up (+0.0%) | 12.7 | 155,824 | 72,819 | 6.42 | 19,817 | 0 | 2026-11-12 |
| arroyo-eos 0.15.0 | 19,791 | 19,698 | keep-up (+0.5%) | 18.4 | 106,952 | 33,963 | 9.35 | 19,700 | 0 | 2026-11-12 |
| flink-eos 2.2.1 | 19,794 | 19,794 | keep-up (+0.0%) | 72.8 | 27,186 | 16,878 | 36.78 | 19,877 | 0 | 2026-11-12 |
| quix-eos 3.23.6 | 19,794 | 19,778 | keep-up (+0.0%) | 75.6 | 26,167 | 69,928 | 38.22 | 19,931 | 0.08 | 2026-11-12 |
How to read this table
consumed/s is what the product actually drained (its consumer-group offset or engine counter), offered/s what gendata produced; a dotted underline marks the median of a repeated batch — hover it for the run count and the min – max. regime (lag drift): keep-up = consumed ≈ offered with a flat backlog (a sustainable point); saturated = fed beyond capacity, so consumed/s IS the ceiling and latency is unbounded; draining = still eating warmup backlog (consumed > offered; CPU is catch-up, not steady state — lengthen the warmup, or raise the rate if the ceiling is what you want). The lag drift in parentheses = backlog change over the window ÷ records consumed — the evidence for the regime (≈ 0 at keep-up, positive when saturated, negative while draining). cpu_avg is the whole process tree, 100 = one core busy. rows/s per core = consumed ÷ (cpu_avg/100); rows/s per GB = consumed ÷ memory; extra rec/rec = records written to topics other than the input and output per record consumed (0 = no broker hop).
Resources (raw)
| product | cpu_avg % | cpu_p95 % | mem_avg MB | mem_max MB | samples | instrument |
|---|---|---|---|---|---|---|
| turbine-eos 0.3.5a1 | 12.7 | 18.6 | 278.1 | 280.2 | 59 | /proc Δ, pss |
| arroyo-eos 0.15.0 | 18.4 | 22.5 | 593.9 | 612.9 | 59 | /proc Δ, pss |
| flink-eos 2.2.1 | 72.8 | 104.8 | 1,200.9 | 1,224.6 | 59 | /proc Δ, pss |
| quix-eos 3.23.6 | 75.6 | 81.2 | 289.6 | 289.7 | 59 | /proc Δ, pss |
How to read this table
CPU is summed over the whole process tree (JVM, UDF servers, the RisingWave container's processes) as utime+stime deltas per 1 s sample — 100 = one core. Memory is PSS (shared pages counted once) where readable, RSS otherwise (instrument). Rows marked ⚠ legacy ps were measured with the old ps %CPU lifetime average and are not comparable: re-run them.
Product notes
turbine-eos — A Kafka transaction per batch (output + input offsets): at this rate every batch closes on its timeout, so a transaction commits at that timeout — the interval the other exactly-once lines are given.
arroyo-eos — sink.commit_mode = 'exactly_once': a Kafka transaction per checkpoint and sink subtask, committed once the checkpoint completes — checkpoints every 1 s (BENCH_COMMIT_INTERVAL_MS), the shortest Arroyo accepts ("between 1 second and 1 day"), not Turbine's batch timeout the others get; 5 s on the other per-record workloads.
flink-eos — Its two-phase-commit Kafka sink commits a transaction per checkpoint per sink subtask, once the checkpoint completes: checkpoints at Turbine's batch timeout (execution.checkpointing.interval, BENCH_COMMIT_INTERVAL_MS — 10 ms is Flink's minimum; aligned barriers, as EXACTLY_ONCE requires), 5 s on the other per-record workloads.
quix-eos — A Kafka transaction per commit, the consumed offsets inside it: commit_interval at Turbine's batch timeout (BENCH_COMMIT_INTERVAL_MS; quixstreams' default 5 s, kept on the other workloads).
Sanity — ✓ matches theory
Are we benching the same thing?
| product | selectivity (out/consumed) | expected | ✓ |
|---|---|---|---|
| turbine-eos 0.3.5a1 | 1.0027 | 1.0000 | ✓ |
| arroyo-eos 0.15.0 | 1.0001 | 1.0000 | ✓ |
| flink-eos 2.2.1 | 1.0041 | 1.0000 | ✓ |
| quix-eos 3.23.6 | 1.0076 | 1.0000 | ✓ |
Verdict: ✓ matches theory — every row's selectivity is within 25% of the expected value (output ÷ consumed).
Host: ovh:c3-32 (ovh:c3-32@SBG5) · AMD EPYC-Milan Processor (2400 MHz) · 16 cores / 16 threads · 31345 MB · kernel 6.12.100+deb13-cloud-amd64.
Redpanda: rpk version: v26.2.2. Redpanda, gendata and the product under test share this host.