Memory Management Thrashing — 120GB Working Set on 64GB

Every program you run — a browser, a game, a database — needs memory to breathe. Without a fair, structured way to hand out that memory, one misbehaving app could read your bank app's data, a crashed process could corrupt the entire system, and you'd never be able to run more than one program at a time. Memory management is the silent contract that makes modern computing safe and multi-tasking possible.

The problem it solves is deceptively deep. Physical RAM is finite and shared. Process A shouldn't be able to peek into Process B's address space. The OS needs to allocate memory fast, reclaim it when a process exits, and give each program the illusion that it owns all the memory in the world — even when RAM is nearly full. Without a memory manager, none of that is possible.

By the end of this article you'll understand exactly how the OS partitions memory, why paging replaced older schemes, how virtual memory lets your laptop run 40 browser tabs on 8 GB of RAM, and what questions about memory management reveal in a system-design or OS interview. Let's dig in.

What is Memory Management in OS?

Memory management is the OS subsystem responsible for allocating and deallocating memory to processes, tracking which parts of memory are free or in use, and providing isolation so that one process cannot access another's data. At its core, it solves three problems: sharing of finite physical RAM, protection between processes, and translation from virtual to physical addresses. Every modern OS—Linux, Windows, macOS—implements memory management as part of the kernel, using hardware support from the CPU's MMU (Memory Management Unit). Without it, a bug in a browser could corrupt a password manager in memory, and multiprogramming would be impossible.

#include <stdio.h>
#include <unistd.h>

int main() {
    int var = 42;
    printf("Virtual address of var: %p\n", (void*)&var);
    printf("Process PID: %d\n", getpid());
    printf("Check /proc/%d/maps for virtual memory layout\n", getpid());
    // Run: cat /proc/PID/maps to see the address space
    return 0;
}

Memory Allocation Strategies: Contiguous vs Non-Contiguous

Early operating systems used contiguous allocation: a process gets a single block of physical memory. That led to fragmentation and overcommit problems. Modern OSes use non-contiguous allocation via paging. But within a process, memory allocation requests (malloc) are served by the heap manager, which uses a mix of contiguity and segmentation. The two main strategies are:

1. Contiguous (Fixed Partitioning): Each process gets a fixed-size block; simple but wastes memory (external fragmentation). Not used in general-purpose OSes.
2. Non-Contiguous (Paging): Physical memory is split into fixed-size frames; processes get virtual pages that can map to any frame. This eliminates external fragmentation and enables virtual memory.

Additionally, Segmentation allows variable-sized logical chunks (code, data, stack) but suffers from external fragmentation unless combined with paging. Most modern systems (Linux, Windows) use paged segmentation where segments are further broken into pages.

#include <stdio.h>
#include <stdlib.h>

// Simulate a page table for a process with 4 virtual pages
#define NUM_PAGES 4

struct page_table_entry {
    int valid;      // 1 if page is in physical memory
    int frame_num;  // physical frame number (if valid)
    int dirty;      // modified flag for swapping
    int referenced; // for LRU approximation
};

int main() {
    struct page_table_entry pt[NUM_PAGES];
    pt[0] = (struct page_table_entry){1, 10, 0, 0};
    pt[1] = (struct page_table_entry){1, 5, 1, 0};
    pt[2] = (struct page_table_entry){0, 0, 0, 0}; // page fault will occur
    pt[3] = (struct page_table_entry){1, 8, 0, 0};

    int virtual_page = 2;
    if (pt[virtual_page].valid) {
        int physical_addr = pt[virtual_page].frame_num * 4096; // 4KB pages
        printf("Physical address for vaddr %d: 0x%x\n", virtual_page, physical_addr);
    } else {
        printf("Page fault: must load from swap\n");
    }
    return 0;
}

Paging: The Mechanism Behind Virtual Memory

Paging divides virtual memory into fixed-size blocks called pages (typically 4KB on x86_64) and physical memory into frames of the same size. Each process has a page table that maps virtual page numbers to physical frame numbers. The MMU uses this page table to translate every memory access from virtual to physical address. When a process accesses a page not currently in physical memory, a page fault occurs, and the OS loads the page from disk (swap) or from the file system (demand paging).

Page tables themselves are hierarchical (e.g., 4-level page tables on x86_64) to avoid needing a flat table with billions of entries. The Translation Lookaside Buffer (TLB) caches recent translations to speed up address translation. On a context switch, the TLB must be flushed, which is why repeated context switching hurts performance.

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/resource.h>

int main() {
    struct rusage usage_before, usage_after;
    getrusage(RUSAGE_SELF, &usage_before);

    // Cause a page fault by accessing a page that hasn't been touched
    int *big = malloc(1024 * 1024 * 100); // allocate 100 MB
    for (int i = 0; i < 100 * 1024 * 1024 / sizeof(int); i += 4096 / sizeof(int)) {
        big[i] = i; // touch every page boundary
    }

    getrusage(RUSAGE_SELF, &usage_after);
    long page_faults = usage_after.ru_majflt - usage_before.ru_majflt;
    printf("Major page faults: %ld\n", page_faults);
    free(big);
    return 0;
}

Virtual Memory: The Illusion of Infinite Memory

Virtual memory extends the concept of paging: each process gets a full virtual address space (e.g., 2^48 on x86_64), but only the parts actively needed are backed by physical memory. The rest sits on disk (swap). When a process accesses a virtual address that is not in RAM, the OS loads the corresponding page from disk into a freed frame—this is demand paging. If no free frames exist, the OS evicts a page to disk using a replacement algorithm (LRU, Clock, etc.). Virtual memory enables: - Running programs larger than physical RAM. - Sharing libraries and memory-mapped files. - Copy-on-write (COW) forking.

The key components: page table, swap space (disk area), page replacement algorithm, and the page fault handler.

from collections import OrderedDict

def simulate_lru(pages, num_frames):
    frames = OrderedDict()
    page_faults = 0
    for page in pages:
        if page in frames:
            frames.move_to_end(page)  # mark as recently used
        else:
            page_faults += 1
            if len(frames) >= num_frames:
                frames.popitem(last=False)  # remove LRU
            frames[page] = True
    return page_faults

# Access pattern: start of loop, then large working set
accesses = [1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5]
print(simulate_lru(accesses, 3))  # Output: 8 (most page faults)

Page Replacement Algorithms: Which Pages Get Evicted?

When physical memory is full and a page fault occurs, the OS must evict a victim page to disk. The replacement algorithm determines which page to remove. The goal is to minimize future page faults by evicting pages unlikely to be used soon.

LRU (Least Recently Used): Evict the page not accessed for the longest time. Requires hardware support (reference bits) or software approximation (e.g., Clock algorithm).

Clock (Second Chance): Use a circular list with a reference bit. Sweep through, clearing bits; if bit is already clear, evict. Efficient approximation of LRU.

Working Set Model: Estimate the set of pages a process is actively using; only keep that set in RAM. Prevents thrashing by adjusting degree of multiprogramming.

Other algorithms: FIFO (simple but suffers from Belady's anomaly), Optimal (unimplementable, used as comparison).

#include <stdio.h>
#include <stdlib.h>

#define NUM_FRAMES 4

struct frame {
    int page;
    int referenced; // 0 or 1
};

int main() {
    struct frame frames[NUM_FRAMES] = {{1,0},{2,0},{3,0},{4,0}};
    int hand = 0; // clock hand
    int new_page = 5;

    // Clock algorithm: find a victim
    while (1) {
        if (frames[hand].referenced == 0) {
            printf("Evicting page %d from frame %d\n", frames[hand].page, hand);
            frames[hand].page = new_page;
            frames[hand].referenced = 0;
            break;
        } else {
            frames[hand].referenced = 0; // give second chance
            hand = (hand + 1) % NUM_FRAMES;
        }
    }
    return 0;
}