C++ Memory Pools — How Exhaustion Crashed a Trading Engine
Fixed-capacity pool of 100,000 slots returns null on exhaustion, causing crashes.
20+ years shipping performance-critical C and C++ systems. Written from production experience, not tutorials.
- ✓Deep production experience
- ✓Understanding of internals and trade-offs
- ✓Experience debugging complex systems
- A memory pool pre-allocates a large block and serves fixed-size slots in O(1) time
- Heap fragmentation disappears because all allocations are same size from a contiguous region
- Pool allocators eliminate per-allocation lock contention using thread-local caches
- Typical allocation latency: ~10-20ns vs 100-300ns for malloc (varies by implementation)
- Production failure: pool exhaustion causes silent crashes when objects outlive pool lifetime
- Biggest mistake: using pool for variable-size objects, causing internal fragmentation and wasted space
A memory pool allocator (or fixed-size allocator) pre-allocates a large contiguous block of memory and divides it into slots of equal size. When you request an allocation, it returns a pointer to the next free slot — no searching, no coalescing. When you free, it marks the slot as available, often by pushing it onto a free list.
You'll find pool allocators in game engines, trading systems, embedded firmware, and custom STL allocators. They shine when you allocate many objects of the same type (e.g., particles, network packets, database rows) and you control their lifetimes tightly.
The key difference from general-purpose allocators is that pools don't handle arbitrary sizes. That's a feature, not a bug — by giving up generality, you gain speed and predictability.
Imagine you run a parking lot. Instead of letting cars park anywhere they want — creating gaps and making it hard to fit new ones — you pre-paint numbered spaces before the day starts. Every car goes into a pre-assigned slot instantly, no searching, no gaps. A memory pool allocator does exactly this for your program's RAM: it grabs a big chunk of memory upfront, slices it into fixed-size slots, and hands them out in microseconds — no rummaging through the heap, no fragmentation.
In a game engine rendering 120 frames per second, a trading system processing 10 million orders a minute, or an embedded firmware loop that must respond in under 50 microseconds, one thing will kill you faster than a logic bug: unpredictable memory allocation latency. The default new and delete operators are general-purpose tools — they handle any size, any time, with synchronization locks baked in. That generality has a cost, and in performance-critical C++ that cost is often unacceptable.
The standard heap allocator, whether it's glibc's ptmalloc, jemalloc, or TCMalloc, must maintain bookkeeping metadata, walk free-lists of variable sizes, coalesce adjacent freed blocks, and acquire locks in multithreaded contexts. Every call to new can trigger a system call, invalidate cache lines, and introduce allocations that are milliseconds apart in memory yet logically adjacent — fragmenting your address space until reallocation itself becomes a bottleneck. Memory pool allocators solve this by inverting the model: instead of asking the OS for memory on demand, you reserve a large contiguous arena upfront and serve allocations from it yourself, with full knowledge of your objects' sizes and lifetimes.
By the end of this article you'll understand how pool allocators work at the bit level, how to implement a thread-safe fixed-size pool and a more flexible slab-style allocator from scratch, how to plug them into STL containers via a custom allocator, and exactly when the tradeoffs make sense — and when they don't. You'll also leave with the mental model interviewers are probing for when they ask about custom allocators in systems programming interviews.
What is a Memory Pool Allocator?
A memory pool allocator (or fixed-size allocator) pre-allocates a large contiguous block of memory and divides it into slots of equal size. When you request an allocation, it returns a pointer to the next free slot — no searching, no coalescing. When you free, it marks the slot as available, often by pushing it onto a free list.
You'll find pool allocators in game engines, trading systems, embedded firmware, and custom STL allocators. They shine when you allocate many objects of the same type (e.g., particles, network packets, database rows) and you control their lifetimes tightly.
The key difference from general-purpose allocators is that pools don't handle arbitrary sizes. That's a feature, not a bug — by giving up generality, you gain speed and predictability.
// TheCodeForge — Fixed-size pool allocator #include <cstddef> #include <cassert> #include <new> namespace io::thecodeforge { class FixedPool { struct Slot { union { Slot* next; // free list pointer char data[1]; // placeholder, actual size at runtime }; }; Slot* pool_start_; Slot* free_head_; size_t slot_size_; size_t slot_count_; public: FixedPool(size_t slot_size, size_t slot_count) : slot_size_(slot_size), slot_count_(slot_count) { if (slot_size < sizeof(Slot*)) slot_size_ = sizeof(Slot*); // ensure we can store free ptr pool_start_ = static_cast<Slot*>(::operator new(slot_size_ * slot_count_)); free_head_ = pool_start_; for (size_t i = 0; i < slot_count_ - 1; ++i) { Slot* current = reinterpret_cast<Slot*>( reinterpret_cast<char*>(pool_start_) + i * slot_size_); current->next = reinterpret_cast<Slot*>( reinterpret_cast<char*>(pool_start_) + (i + 1) * slot_size_); } // last slot points to null Slot* last = reinterpret_cast<Slot*>( reinterpret_cast<char*>(pool_start_) + (slot_count_ - 1) * slot_size_); last->next = nullptr; } void* allocate() { if (!free_head_) throw std::bad_alloc(); Slot* slot = free_head_; free_head_ = slot->next; return slot; } void deallocate(void* ptr) { Slot* slot = static_cast<Slot*>(ptr); slot->next = free_head_; free_head_ = slot; } ~FixedPool() { ::operator delete(pool_start_); } }; } // namespace
- Pre-allocated memory = stack of trays
- free_head_ = pointer to the next tray
- allocate = pop the tray (O(1))
- deallocate = push it back (O(1))
Slab Allocator: Handling Multiple Sizes
A slab allocator maintains several fixed-size pools (slabs) for different size classes. When you request 24 bytes, it returns a chunk from the 32-byte slab. This trades external fragmentation for some internal fragmentation per slab, but still avoids the overhead of walking free lists of variable sizes.
The Linux slab allocator popularised this approach. It also caches recently freed objects in a per-CPU cache to avoid locking entirely.
Implementing a simple slab allocator means creating an array of FixedPool objects, each with a size from a predefined size table (usually powers of two plus some mid-range entries: 16, 32, 64, 128, ...). The allocator rounds up the requested size to the next slab size.
In production, slab allocators work well for networking stacks and kernel-level memory management.
// TheCodeForge — Slab allocator with fixed size classes #include <array> #include <cstddef> #include <new> #include <algorithm> namespace io::thecodeforge { template <size_t... Sizes> class SlabAllocator { static constexpr std::array<size_t, sizeof...(Sizes)> kSizes = {Sizes...}; struct Slab { size_t slot_size; FixedPool* pool; Slab(size_t sz) : slot_size(sz), pool(nullptr) {} void ensure_pool(size_t count) { if (!pool) pool = new FixedPool(slot_size, count); } }; Slab slabs_[sizeof...(Sizes)]; // one per size class size_t slab_count_ = sizeof...(Sizes); public: SlabAllocator(size_t slots_per_slab = 256) { size_t idx = 0; ((slabs_[idx++] = Slab(Sizes)), ...); // C++17 fold expression } void* allocate(size_t bytes) { for (auto& slab : slabs_) { if (bytes <= slab.slot_size) { slab.ensure_pool(256); return slab.pool->allocate(); } } // no matching slab — fallback to malloc return ::operator new(bytes); } void deallocate(void* ptr, size_t bytes) { for (auto& slab : slabs_) { if (bytes <= slab.slot_size) { slab.pool->deallocate(ptr); return; } } ::operator delete(ptr); } }; using SmallObjectAllocator = SlabAllocator<8, 16, 32, 64, 128, 256, 512, 1024>; } // namespace
Thread-Safety Considerations
The simple pool above uses a single free list protected by nothing. That's safe only in a single-threaded context. In multi-threaded environments, you can't share the free_head_ pointer without protection — threads will race, corrupt the list, and crash.
Three common strategies: 1. Global mutex around allocate/deallocate — simple but kills performance under contention. 2. Thread-local pools — each thread has its own pool, no sharing. Objects must be freed on the same thread that allocated them, or you need a transfer mechanism. 3. Lock-free free list — use atomic compare-and-swap (CAS) to manipulate the head pointer. This is the fastest option for moderate contention.
Let's implement a lock‑free pool using std::atomic<Slot*>.
// TheCodeForge — Lock-free fixed-size pool #include <atomic> #include <cstddef> #include <new> namespace io::thecodeforge { class LockFreePool { struct Slot { std::atomic<Slot*> next; }; Slot* pool_start_; std::atomic<Slot*> free_head_; size_t slot_size_; size_t slot_count_; public: LockFreePool(size_t slot_size, size_t slot_count) : slot_size_(slot_size), slot_count_(slot_count) { if (slot_size < sizeof(std::atomic<Slot*>)) slot_size_ = sizeof(std::atomic<Slot*>); pool_start_ = reinterpret_cast<Slot*>(::operator new(slot_size_ * slot_count_)); for (size_t i = 0; i < slot_count_; ++i) { Slot* s = reinterpret_cast<Slot*>( reinterpret_cast<char*>(pool_start_) + i * slot_size_); s->next.store(i + 1 < slot_count_ ? reinterpret_cast<Slot*>(reinterpret_cast<char*>(pool_start_) + (i + 1) * slot_size_) : nullptr, std::memory_order_relaxed); } free_head_.store(pool_start_, std::memory_order_relaxed); } void* allocate() { Slot* old_head = free_head_.load(std::memory_order_acquire); Slot* new_head; do { if (!old_head) return nullptr; // pool exhausted new_head = old_head->next.load(std::memory_order_relaxed); } while (!free_head_.compare_exchange_weak(old_head, new_head, std::memory_order_acq_rel, std::memory_order_acquire)); return old_head; } void deallocate(void* ptr) { Slot* slot = static_cast<Slot*>(ptr); Slot* old_head = free_head_.load(std::memory_order_acquire); do { slot->next.store(old_head, std::memory_order_relaxed); } while (!free_head_.compare_exchange_weak(old_head, slot, std::memory_order_acq_rel, std::memory_order_acquire)); } ~LockFreePool() { ::operator delete(pool_start_); } }; } // namespace
Plugging Into the STL: Custom Allocators
The STL containers like std::vector and std::map accept custom allocators via the template parameter. This lets you make std::vector<MyParticle, PoolAllocator<MyParticle>> use your pool instead of new/delete.
A custom allocator must satisfy the Allocator concept: it needs allocate()deallocate()rebind struct, and equality operators. The C++17 std::pmr::memory_resource provides a higher-level interface, but we'll show the classic approach.
Here's an adaptor that wraps our FixedPool into an STL allocator.
// TheCodeForge — STL-compatible pool allocator #include <memory> #include <cstddef> #include "io/thecodeforge/fixed_pool.h" namespace io::thecodeforge { template <typename T> class PoolAllocator { public: using value_type = T; PoolAllocator(FixedPool& pool) noexcept : pool_(&pool) {} template <typename U> PoolAllocator(const PoolAllocator<U>& other) noexcept : pool_(other.pool_) {} T* allocate(std::size_t n) { if (n != 1) throw std::bad_alloc(); // pool is single-slot only return static_cast<T*>(pool_->allocate()); } void deallocate(T* p, std::size_t n) noexcept { if (n == 1) pool_->deallocate(p); } template <typename U> struct rebind { using other = PoolAllocator<U>; }; // equality: must share the same pool friend bool operator==(const PoolAllocator& a, const PoolAllocator& b) { return a.pool_ == b.pool_; } friend bool operator!=(const PoolAllocator& a, const PoolAllocator& b) { return a.pool_ != b.pool_; } private: FixedPool* pool_; // allow cross-type copy template <typename U> friend class PoolAllocator; }; } // namespace
rebind to allocate internal nodes (e.g., std::list nodes are different from value_type). Your allocator must support rebinding or the container will fall back to std::allocator. Also, std::vector may request n > 1 — you can't assume single-object allocation.bad_alloc or if exceptions disabled, undefined behavior.reserve()std::pmr::memory_resource in C++17 is a cleaner abstraction for runtime polymorphic allocators.std::vector and want it to use your poolPoolAllocator<T> adapter. But note: std::vector may request multiple items.std::map or std::list on the poolrebind — it will be used for nodes.When Pool Allocators Lose: Trade-offs and Failure Modes
Pool allocators are not a silver bullet. They fail badly when object sizes vary widely, lifetimes are unpredictable, or memory must be returned to the OS. Here are the problems that catch teams in production:
- Internal fragmentation — Fixed-size slots waste space if objects are smaller than the slot. A 64-byte pool for 16-byte objects wastes 75% of memory.
- Memory blow-up under load — Pools never shrink. If an allocator reserves 1MB for 1024 objects of 1KB, a single peak usage event pins that 1MB forever.
- No way to return memory to OS — The
::operator newinside the pool is freed only when the pool destructs. Long-lived pools are effectively memory leaks from the OS perspective. - Debugging difficulty — Use-after-free errors inside a pool are hard to detect because the slot memory still belongs to the pool and appears valid. Standard tools like ASan often can't catch intra-pool violations unless you poison freed slots.
For these reasons, pool allocators are best for hot paths where allocation counts are high and object sizes are uniform. For cold paths or unpredictable workloads, stick with general-purpose allocators.
// TheCodeForge — demonstrating pool pitfalls #include <iostream> #include <memory> #include <vector> struct Particle { int x, y, vx, vy; }; // 16 bytes // Suppose pool is sized for 1000 particles at 64 bytes each struct LargePool { char data[64 * 1000]; size_t next = 0; void* alloc() { if (next >= 1000) return nullptr; return data + (next++ * 64); } // ... no free() - pool never releases memory }; int main() { LargePool pool; std::vector<Particle*> vec; // worst-case: each particle uses only 16 bytes but pool wastes 48 for (int i = 0; i < 1000; ++i) vec.push_back(static_cast<Particle*>(pool.alloc())); std::cout << "Wasted space: " << (64-16)*1000 << " bytes\n"; // pool memory never freed until program exit return 0; }
malloc replacement with built-in leak detection.Why the Default Allocator Betrays You Under Load
You don't notice malloc fragmentation until your latency spikes 200x. The heap is a shared resource. Every thread's deallocation can dirty a cache line another thread needs. Pool allocators solve this by treating memory as a flat array of fixed-size slots. No fragmentation, no coalescing, no kernel calls. You reserve a contiguous block once, then hand out slots with a pointer bump or a free list. When every allocation is O(1) and every deallocation just pushes a pointer back onto a stack, your allocator stops being a bottleneck. The real win is CPU cache behavior: sequential allocations land on adjacent cache lines. The WHY is simple: malloc pays for generality you don't need. A pool pays for exactly your workload.
// io.thecodeforge #include <cstddef> #include <cstdint> #include <new> template<typename T, size_t PoolSize = 4096> class PoolAllocator { alignas(T) char pool_[PoolSize * sizeof(T)]; T* next_; public: PoolAllocator() : next_(reinterpret_cast<T*>(pool_)) {} T* allocate() { if (next_ >= reinterpret_cast<T*>(pool_ + PoolSize * sizeof(T))) throw std::bad_alloc(); return new (next_++) T(); } void deallocate(T*) noexcept { // Fixed pool: no per-slot free. Reset pool_ pointer to start for bulk reuse. } void reset() { next_ = reinterpret_cast<T*>(pool_); } };
reset() leaks destructors. Use a stack-based free list for per-object lifetimes.Stack Allocator: The Arena Pattern for Transient Work
Your game loop or request handler allocates thousands of small objects, then frees them all at once. Default allocators thrash the heap for no reason. The arena (or stack allocator) is your weapon: grab a large contiguous buffer, bump a pointer for each allocation, and reset the pointer when the frame or request ends. No deallocation calls during the hot path. No fragmentation. No thread contention if each thread owns its arena. The WHY: your allocation pattern is LIFO or bulk-free. You don't need individual frees. This is how game engines handle per-frame allocations and how HTTP servers handle per-request buffers. The cost is one: you cannot free a single object—only the whole arena. That constraint forces you to write code that batches lifetimes.
// io.thecodeforge #include <cstddef> #include <cstdint> class Arena { char* const start_; char* current_; const size_t size_; public: Arena(size_t bytes) : start_(static_cast<char*>(::operator new(bytes))) , current_(start_), size_(bytes) {} ~Arena() { ::operator delete(start_); } void* alloc(size_t sz, size_t align) { // Align pointer up, then check capacity uintptr_t raw = reinterpret_cast<uintptr_t>(current_); uintptr_t aligned = (raw + align - 1) & ~(align - 1); if (aligned + sz > reinterpret_cast<uintptr_t>(start_) + size_) return nullptr; current_ = reinterpret_cast<char*>(aligned + sz); return reinterpret_cast<void*>(aligned); } void reset() { current_ = start_; } };
reset(), or skip if POD.The Trading Engine That Crashed at Market Open
malloc when the pool is full, with a warning log. Also added a free-list validation (xor-linked list or magic numbers) to catch double-frees immediately. The pool size was adjusted to handle 3x the 99.9th percentile burst.- Never assume your pool will be large enough — always have a fallback allocator.
- Check every allocation return — pools aren't guaranteed to succeed.
- Instrument pool usage: log depletion warnings and monitor free-list integrity.
g++ -fsanitize=address -g myapp.cpp -o myapp && ./myappvalgrind --tool=memcheck --leak-check=full ./myappgrep -rn "pool.allocate" src/ | wc -l # check all call sitestail -f /var/log/app/error.log | grep "pool_full"perf record -g -F 99 -- ./myapp && perf reportvalgrind --tool=callgrind ./myapp # to see lock overhead| Aspect | Pool Allocator | General-Purpose Allocator (malloc) |
|---|---|---|
| Allocation latency | ~10-20 ns (pointer swap) | ~100-300 ns (free list traversal) |
| Memory fragmentation | None external, some internal (slot waste) | External fragmentation over time |
| Memory return to OS | Only at pool destruction | Immediate on free of large blocks |
| Thread safety complexity | Must be explicitly built (lock-free/thread-local) | Built-in (but pay lock overhead every call) |
| Debugging ease | Harder (intra-pool corruption) | Easier (tools like ASan/Valgrind) |
| Best workload | Fixed-size, high frequency, hot path | Variable sizes, low frequency, cold path |
| File | Command / Code | Purpose |
|---|---|---|
| io | namespace io::thecodeforge { | What is a Memory Pool Allocator? |
| io | namespace io::thecodeforge { | Slab Allocator |
| io | namespace io::thecodeforge { | Thread-Safety Considerations |
| io | namespace io::thecodeforge { | Plugging Into the STL |
| io | struct Particle { int x, y, vx, vy; }; // 16 bytes | When Pool Allocators Lose |
| pool_allocator.cpp | template | Why the Default Allocator Betrays You Under Load |
| arena.cpp | class Arena { | Stack Allocator |
Key takeaways
Common mistakes to avoid
5 patternsMemorising syntax before understanding the concept
Skipping practice and only reading theory
Using a fixed-size pool for variable-length objects
Forgetting to handle pool exhaustion
allocate(). In production, fall back to std::malloc with an error log. Monitor pool utilization and alert before exhaustion.Sharing a non-thread-safe pool across threads
Interview Questions on This Topic
Describe how a fixed-size pool allocator works internally. What are the trade-offs compared to malloc?
How would you make a pool allocator thread-safe? Compare mutex, lock-free, and thread-local approaches.
std::mutex. Simple, but under contention the lock becomes a bottleneck — overhead can be 500ns per operation.
2. Lock-free: Use std::atomic <Slot*> for the free head. Use CAS to pop/push. Avoids OS-level locks but is vulnerable to the ABA problem. Works well for moderate contention (~4-8 threads).
3. Thread-local pools: Each thread has its own pool. No locks, no ABA. But objects allocated on thread A cannot be freed on thread B unless you implement a transfer queue. On NUMA systems, memory is allocated near the thread's CPU, improving cache locality.
I'd choose thread-local whenever the allocation pattern allows thread-affine free. For cross-thread frees, I'd use a global lock-free pool combined with a per-thread free cache.Explain the ABA problem in lock-free data structures and how it affects pool allocators.
next pointer, and pushes it back. When T's CAS compares the head, it sees P again — but the head points to a different next node. T will successfully pop P, but now the list is corrupted because P's next no longer points to the correct successor.
Solutions: 1) Use tagged pointers (attach a version counter to the pointer). 2) Use hazard pointers (mark nodes as in-use before reading them). 3) Use RCU (read-copy-update). For pool allocators, a simpler fix is to use a double-CAS on the head and next pointer, or restrict the pool size so that ABA is statistically improbable.What is the difference between a pool allocator, an arena allocator, and a slab allocator?
Frequently Asked Questions
A memory pool allocator is like a pre-stocked tray of glasses at a bar. Instead of washing a glass (allocating) each time someone orders, the bartender grabs a clean glass from the stack instantly. When the glass is returned, it goes back onto the stack. No searching, no washing — just a swap.
Avoid pools when: 1) Objects have varying sizes (internal fragmentation hurts). 2) Peak usage is far above steady state (memory is pinned forever). 3) Allocations are rare (the complexity isn't worth it). 4) You need reliable leak detection (pools mask leaks). 5) Objects outlive the pool's lifetime (use-after-free risk).
Yes — write a custom STL allocator that wraps your pool. But be careful: std::vector::resize(n) allocates n objects at once. Your pool must support multi-slot allocation or you'll get bad_alloc. Prefer reserve()push_back()
Enable poison values: fill freed slots with a known pattern (0xDEADBEEF or 0xCD). Also fill allocated slots with a different pattern. When you crash, the value tells you if the pointer is freed or not (use-after-free check). Run with AddressSanitizer (ASan) — it can catch intra-pool violations if you tell it about the pool memory region via _._asan_poison_memory_region()
A pool allocator allows individual objects to be freed and reused individually. An arena (or bump allocator) only allows freeing all objects at once by resetting a pointer. Arenas are faster (~2ns vs 10-20ns for pools) but less flexible. Use arenas for temporary work (e.g., per-frame allocations in a game) and pools for long-lived objects that need reclamation.
20+ years shipping performance-critical C and C++ systems. Written from production experience, not tutorials.
That's C++ Advanced. Mark it forged?
4 min read · try the examples if you haven't