Conviva has detailed an attempt to replace memory-mapped file access with io_uring in its Rust analytics engine, reporting that the first rewrite was 60% slower despite sharply reducing major page faults. The result offers a caution for engineering teams considering asynchronous direct I/O: removing one bottleneck does not guarantee that an initial replacement design will be faster.
The company’s engine processes large Arrow IPC files stored on local NVMe drives. A typical query reads several columns across eight files and touches roughly 13GB for each day of data. The system uses DataFusion, Arrow, Rust, Rayon and Tokio on servers with 192 CPU cores and about 750GB of memory. Memory mapping originally fit Arrow’s layout well because disk and memory representations match, enabling zero-copy random access.
Problems emerged under concurrent production load. Conviva found that long queries filled the host page cache, forcing processes to compete over shared kernel state. In a controlled comparison on one machine, a single pod outperformed four pods on 14-day queries: the advantage reached 41% at the maximum and exceeded 20% at the 95th percentile. Performance profiling showed kernel lock contention, repeated page eviction and reinsertion, and millions of minor faults per second.
The pressure also produced more than two million context switches per second in a stressful run, compared with about 14,000 on a warm-cache run. Conviva measured peak mmap throughput of 3.44GB per second on hardware whose 32-drive setup reached about 21GB per second in an fio test. The company concluded that page-cache behavior and kernel contention, rather than the raw drives, were limiting query performance.
Engineers then built an alternative around io_uring, O_DIRECT and the Rust-native Compio library. The design bypassed the page cache and submitted a future for every Arrow column read, with 40 column operations issued concurrently for the test workload. Early development on macOS helped validate batching and compilation even though that operating system does not provide io_uring.
The change produced the expected reduction in page faults: major faults fell by nearly 70 times in the early test. Total runtime nevertheless increased, leaving the first implementation substantially slower. Conviva says its initial design did not realize several anticipated advantages of io_uring and that building an effective implementation required more deliberate coordination of I/O, scheduling and decoding.
The finding does not show that io_uring is inherently slower than mmap. It shows that production performance depends on the complete access pattern and software design. Conviva’s measurements also apply to its specific file sizes, concurrency, kernels and hardware. Even so, the published investigation provides a useful reminder to benchmark under representative load and to treat lower-level I/O primitives as tools, not automatic performance upgrades.
