Docs/Deploy & operatev0.0.1-preview
io_uring and the WAL
How TulaDB's write-ahead log uses io_uring on Linux, which distributions allow it, how to enable it on RHEL 9, and how to check which backend a node runs.
On Linux, TulaDB’s write-ahead log (WAL) uses io_uring to take fdatasync off the writer thread’s critical path. The writer submits a group’s fsync to the kernel and goes on writing the next group while the kernel flushes. A transfer is still acknowledged only after its group is durable on disk: io_uring changes when the flush is waited for, not whether it is. Release builds always include io_uring support, and the package installs liburing as a dependency.
When the kernel refuses io_uring, the WAL uses blocking fdatasync instead. The node is fully correct either way; io_uring buys throughput and tail latency on a durability-bound workload.
Which machines run it
| Distribution | Kernel | io_uring |
|---|---|---|
| Ubuntu 22.04 | 5.15 | on |
| Ubuntu 24.04 | 6.8 | on |
| Debian 12 | 6.1 | on |
| RHEL 9, Rocky 9, AlmaLinux 9 | 5.14 | off by default: the kernel ships kernel.io_uring_disabled = 2 |
| macOS | — | not applicable: the WAL uses fsync |
Inside a container, the runtime’s seccomp profile may block the io_uring system calls. The node then uses fdatasync, exactly as on RHEL.
Check which backend a node runs
The runtime metrics snapshot names it:
grep -o '"wal_backend": "[a-z_]*"' /var/lib/tuladb/runtime/metrics.json
# "wal_backend": "io_uring" or "wal_backend": "sync"
When the kernel refuses io_uring, the node logs this once at start:
WARN [wal] iouring_disabled_by_kernel: the kernel refuses io_uring (EPERM: kernel.io_uring_disabled is set; RHEL 9 ships 2). The WAL uses fdatasync. To allow it for this service only: kernel.io_uring_disabled = 1 and kernel.io_uring_group = the tuladb group's gid (sysctl.d)
Prometheus has two more metrics:
tuladb_wal_backend_fallbackis1when a configuration asked forio_uringand gotsync.tuladb_wal_iouring_degradedis1while the io_uring backend cannot take fsyncs (a full submission queue, say). Groups then fall back to a blocking flush, still durable, andtuladb_wal_iouring_submit_fallbacks_totalcounts them.
Enable io_uring on RHEL 9
RHEL disables io_uring for every process, because of its history of kernel vulnerabilities. Allow it for the TulaDB service only. Setting kernel.io_uring_disabled = 1 permits io_uring for members of kernel.io_uring_group, so make that the tuladb group:
gid=$(getent group tuladb | cut -d: -f3)
printf 'kernel.io_uring_disabled = 1\nkernel.io_uring_group = %s\n' "$gid" \
| sudo tee /etc/sysctl.d/60-tuladb-io-uring.conf
sudo sysctl --system
sudo systemctl restart tuladbd
grep -o '"wal_backend": "[a-z_]*"' /var/lib/tuladb/runtime/metrics.json # "io_uring"
Every other process keeps io_uring disabled. To undo it, remove the file, set kernel.io_uring_disabled = 2 and restart tuladbd. Weigh the change with your security team: it widens the kernel surface reachable by the tuladb user.
Configuration
| Key | Default | |
|---|---|---|
wal_backend |
auto |
auto uses io_uring when the kernel allows it; io_uring asks for it (and reports tuladb_wal_backend_fallback when refused); sync never uses it |
iouring_ring_depth |
256 (512 in the production profiles) |
queue depth of the WAL’s ring; deeper lets more groups’ flushes overlap on NVMe |
The shipped profiles use wal_backend = "auto", so there is nothing to set: the node takes io_uring wherever it is allowed.
Getting the most from it
io_uring overlaps flushes; it cannot make a slow disk fast. Put the WAL on its own local NVMe device (--wal-dir at install, or wal_dir in node.toml). A WAL sharing a disk with the data, or living on a network disk, is the most common cause of tail latency whichever backend runs. See Durability for what an acknowledgement promises, and Configuration for the WAL sizing keys.