* [Store] Add lock-free MmapArena allocator for buffer mmap path
Replace per-allocation mmap() syscalls in allocate_buffer_mmap_memory()
with a lock-free atomic bump allocator (MmapArena). Pre-allocates a
configurable pool (default 64GB) and serves allocations via CAS loop,
reducing allocation latency from ~1us (mmap syscall) to ~50ns (atomic).
Allocation lifecycle is static: all callers (ClientBufferAllocator,
global segments in RealClient::setup_internal) allocate at startup and
free at shutdown. The arena outlives all allocations, so the bump-only
(no individual free) design is correct for this usage pattern.
Feature-flagged via gflags:
--use_mmap_arena_allocator (default: true)
--mmap_arena_pool_size (default: 64GB)
Falls back to direct mmap() when arena is disabled, fails to init,
or is exhausted.
Cherry-picked from flow-ipc-poc branch (utils.cpp perf path only).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* [Store] Fix three correctness issues in MmapArena
1. Honor caller's alignment contract: allocate() now accepts a
per-call alignment parameter and uses max(arena default, caller
request). allocate_buffer_mmap_memory() forwards its alignment
argument to the arena. Previously, the caller's alignment was
silently ignored — the arena always used 64-byte alignment
regardless of what the caller requested.
2. Remove MAP_POPULATE from arena pool mmap: the default pool is
64GB but callers typically use only a fraction (e.g. 4GB of
segments). MAP_POPULATE would pre-fault all 64GB of pages upfront,
causing seconds of startup delay and potentially triggering OOM
on machines with less physical memory. Pages now fault on demand.
3. Make alignment_ atomic and store it BEFORE the CAS on pool_base_:
previously alignment_ was a plain size_t written AFTER the release
CAS, so the store was not in the happens-before relationship
established by the acquire-release pair on pool_base_. Now both
alignment_ and pool_size_ are stored before the CAS with the
release fence guaranteeing their visibility to readers.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
3ac7de25ac