Python Performance Optimisation — O(n*m) Loop Cost $180K

Python is fast enough — until it isn't. The moment your data pipeline crawls past its midnight SLA, your API starts timing out under load, or your ML preprocessing loop becomes the bottleneck before a single model even trains, 'fast enough' stops being a philosophy and becomes a liability. The language's dynamic, expressive nature comes with measurable overhead at every layer: attribute lookup, memory allocation, the Global Interpreter Lock, and bytecode interpretation all stack up in ways that bite production systems in ways that are genuinely hard to predict without measurement.

The real problem is not that Python is slow. The real problem is that most developers optimise blindly. They reach for NumPy before profiling, rewrite loops as C extensions before measuring allocations, and add workers before understanding whether their bottleneck is CPU-bound or I/O-bound. I have watched teams spend a week parallelising code that had a quadratic algorithm inside it. More workers, same fundamental problem, more cloud spend. Blind optimisation is how you spend three days speeding up code that accounts for 2% of your runtime while the real bottleneck sits untouched.

This guide moves past surface-level advice into CPython internals: how the bytecode interpreter executes your code, why attribute lookup is more expensive than it looks, how __slots__ changes memory layout at the C struct level, and exactly when to reach for multiprocessing versus asyncio versus NumPy versus Cython. These are the patterns that separate a developer who knows Python from one who can own a high-performance Python system in production and explain every decision they made.

What is Python Performance Optimisation?

Python Performance Optimisation is the discipline of systematically measuring, understanding, and eliminating bottlenecks in Python applications. It is not guesswork and it is not heroics — it is a structured process that starts with measurement and only then moves to action.

The discipline spans multiple layers, and the order matters. Algorithmic complexity is the first layer — an O(n²) algorithm running on a million records is not a Python problem, it is a correctness problem in disguise. Fixing it in NumPy instead of fixing the algorithm gives you a faster wrong answer. Memory layout and allocation patterns come next — every Python object carries overhead that adds up when you are creating millions of them. Concurrency model selection follows — the right model depends entirely on whether your bottleneck is CPU-bound or I/O-bound, and choosing the wrong one makes things worse. Native acceleration (NumPy, Cython, Numba) sits at the innermost layer and should only be reached after the outer layers are correctly addressed.

CPython's dynamic nature means that performance intuition is systematically wrong in ways that are hard to reason about from first principles. Attribute lookups involve dictionary traversal. Integer arithmetic involves reference count manipulation. Function calls involve frame allocation on the heap. These costs do not exist in statically compiled languages, and they stack up in ways that only profiling reveals. The first rule — the one rule that matters more than every optimisation technique in this article — is always profile before you optimise.

import cProfile
import pstats
from io import StringIO


def io_thecodeforge_process_records(records: list[dict]) -> list[dict]:
    """
    Process and enrich records — but where is the actual bottleneck?
    This looks innocent at development scale. At 10M records it is not.
    """
    enriched = []
    for record in records:
        # dict.get() is a method call on every iteration
        # The comparison is a LOAD_ATTR + COMPARE_OP in bytecode
        # Neither is expensive alone — at 10M iterations, it adds up
        if record.get("status") == "active":
            record["score"] = record["value"] * 1.15
            enriched.append(record)
    return enriched


# Always profile on a representative sample — not 10 records
# Small samples hide algorithmic complexity issues entirely
profiler = cProfile.Profile()
profiler.enable()

data = [{"status": "active", "value": i} for i in range(100_000)]
result = io_thecodeforge_process_records(data)

profiler.disable()

# Sort by cumulative time to find the real bottleneck
# cumulative = total time in the function including everything it called
# tottime = time spent only in this function, not in subfunctions
stream = StringIO()
stats = pstats.Stats(profiler, stream=stream).sort_stats("cumulative")
stats.print_stats(10)
print(stream.getvalue())

# What this output tells you:
# - ncalls: how many times each function was called
# - cumtime: where total wall time is spent (the bottleneck indicator)
# - percall: average time per call (helps spot expensive individual calls)
# Look for the function with the highest cumtime — that is where to start

Python Data Structure Time Complexity (Big O) — List, Dict, Set Operations

Choosing the right data structure is the most impactful performance decision you can make — it operates at the algorithmic layer and often yields 100x or 1000x speedups without any other optimisation. When you understand the time complexity of Python's built-in data structures, you can avoid the O(n*m) pattern that cost $180K in the production incident above.

The table below summarises the average-case and amortised worst-case time complexities for the most common operations on list, dict, set, and tuple. These are CPython's implementation-specific complexities — they are not language guarantees, but they are stable across Python 3.x releases and extremely unlikely to change.

``text Operation | List | Dict | Set | Tuple -----------------------------------|-------------------|-------------------|-------------------|------------------- Index lookup | O(1) | N/A | N/A | O(1) Append / insert at end | O(1) amortised | N/A | N/A | N/A Pop from end | O(1) | N/A | N/A | N/A Insert at arbitrary index | O(n) | N/A | N/A | N/A Delete by index | O(n) | N/A | N/A | N/A Membership check (in) | O(n) | O(1) average | O(1) average | O(n) Get item (key access) | N/A | O(1) average | N/A | N/A Set item (key assignment) | N/A | O(1) average | N/A | N/A Delete key | N/A | O(1) average | O(1) average | N/A Iteration | O(n) | O(n) | O(n) | O(n) Slicing [i:j] | O(k) (k = slice length) | N/A | N/A | O(k) Copy (list.copy, dict.copy) | O(n) | O(n) | O(n) | O(n) ``

The crucial takeaway: membership checks (x in list) are O(n) — the list must be scanned linearly from the start until the element is found (or the list ends). For a 200K-entry list, the worst case is 200K comparisons. When you perform that check for each of 50M records, you get the $180K incident. Switching to a set or dict for the lookup reduces membership checks to O(1) — a single hash computation and lookup regardless of container size.

This is not a subtle optimisation. It is the difference between a pipeline that runs in minutes and one that runs for 14 hours. Always profile to confirm that the membership check is the bottleneck, but in data processing pipelines with large reference tables, the fix is almost always to use a set or dict for the lookup side.

import time

# Demonstrate O(n) membership check on list vs O(1) on set
# This is the exact pattern from the $180K incident

# Create a reference table with 200K entries
reference_list = list(range(200_000))
reference_set = set(reference_list)

# Simulate 50M records (we'll use 500K for this benchmark)
test_records = list(range(500_000))

# List membership (O(n)) — this is what the team had
def enrich_with_list(records, ref):
    enriched = []
    for r in records:
        # This 'in' check is O(n) — scans entire list in worst case
        if r in ref:
            enriched.append(r * 1.15)
    return enriched

# Set membership (O(1)) — the fix
def enrich_with_set(records, ref):
    enriched = []
    for r in records:
        # This 'in' check is O(1) — single hash lookup
        if r in ref:
            enriched.append(r * 1.15)
    return enriched

# Benchmark only on a smaller sample to keep demo fast
sample = test_records[:10000]

start = time.perf_counter()
result_list = enrich_with_list(sample, reference_list)
time_list = time.perf_counter() - start

start = time.perf_counter()
result_set = enrich_with_set(sample, reference_set)
time_set = time.perf_counter() - start

print(f"List membership: {time_list:.3f}s")
print(f"Set membership:  {time_set:.3f}s")
print(f"Speedup:         {time_list / time_set:.0f}x")
print(f"Results equal:   {result_list == result_set}")

The Speed of Built-ins: map(), filter(), zip() vs List Comprehensions

Python provides several built-in functional tools — map(), filter(), zip() — that can sometimes be faster than the equivalent list comprehension, but the performance difference is rarely large enough to justify sacrificing readability. The table below compares these constructs across several common patterns, including execution time, memory behaviour, and readability.

``text Pattern | Approach | Time (10M items) | Memory | Readability ----------------------------------|------------------|------------------|---------------|---------------- Transform x -> f(x) | map(f, iter) | ~0.40s | lazy (iter) | Good with simple f | [f(x) for x in] | ~0.45s | O(n) list | Excellent | (f(x) for x in) | ~0.41s | lazy (gen) | Good Filter x -> bool(x) | filter(pred, iter)| ~0.38s | lazy (iter) | Fair | [x for x in if p]| ~0.42s | O(n) list | Excellent | (x for x in if p)| ~0.39s | lazy (gen) | Good Zip two iterables | zip(a, b) | ~0.27s | lazy (iter) | Excellent | [(a[i], b[i])..] | ~0.55s | O(n) list | Poor | ((a[i], b[i])..) | ~0.50s | lazy (gen) | Poor ``

In modern Python (3.10+), list comprehensions are preferred for readability and debuggability. The only cases where map/filter/zip provide a meaningful performance advantage are: 1. The function is a built-in written in C (e.g., map(int, data)). 2. You need lazy evaluation without consuming memory — map and filter return iterators, list comprehensions produce full lists. 3. Combining multiple operations (zip(map(f, a), b)) where intermediate lists would waste memory.

When in doubt, use a list comprehension. If profiling shows the comprehension is a bottleneck, test the map/filter alternative, but measure with production data before committing the change.

import time

# Benchmark map() vs list comprehension with different functions

data = list(range(10_000_000))

# Case 1: Built-in function (int -> float conversion)
def case_builtin():
    # map with built-in
    start = time.perf_counter()
    result = list(map(float, data))
    t1 = time.perf_counter() - start
    
    # list comprehension
    start = time.perf_counter()
    result = [float(x) for x in data]
    t2 = time.perf_counter() - start
    
    print(f"Built-in float: map()={t1:.3f}s, listcomp={t2:.3f}s, map is {t2/t1:.1f}x faster")

# Case 2: Lambda function (simple arithmetic)
def case_lambda():
    start = time.perf_counter()
    result = list(map(lambda x: x * 1.5, data))
    t1 = time.perf_counter() - start
    
    start = time.perf_counter()
    result = [x * 1.5 for x in data]
    t2 = time.perf_counter() - start
    
    print(f"Lambda: map()={t1:.3f}s, listcomp={t2:.3f}s, listcomp is {t1/t2:.1f}x faster")

# Case 3: zip vs manual indexing
def case_zip():
    a = data
    b = [x * 2 for x in data]
    
    start = time.perf_counter()
    result = list(zip(a, b))
    t1 = time.perf_counter() - start
    
    start = time.perf_counter()
    result = [(a[i], b[i]) for i in range(len(a))]
    t2 = time.perf_counter() - start
    
    print(f"Zip: zip()={t1:.3f}s, indexing={t2:.3f}s, zip is {t2/t1:.1f}x faster")

# Run all cases
case_builtin()
case_lambda()
case_zip()

Memory Layout: __slots__, Object Overhead and Allocation Patterns

Every Python object carries overhead that most developers do not think about until it bites them in production. A plain class instance stores its attributes in a dynamic __dict__ — a hash table that can hold arbitrary key-value pairs. Before you store a single field of your own data, that __dict__ costs 104-232 bytes depending on the Python version and the number of entries. The object header itself adds another 48 bytes. At one million instances, you are paying 150MB in pure overhead before your data even enters the picture.

__slots__ eliminates the __dict__ entirely. When you declare __slots__ = ('field1', 'field2'), CPython allocates a fixed C struct with exactly those fields at known memory offsets. Attribute access becomes a direct struct field lookup rather than a dictionary traversal. Per-object memory drops by 40-60%, and attribute access in tight loops gets measurably faster because LOAD_ATTR can resolve to a direct C offset rather than a hash table lookup followed by a reference dereference.

The trade-off is real and you need to know it before using __slots__ in production. You cannot add arbitrary attributes at runtime — code that does obj.unexpected_field = value raises AttributeError instead of silently creating the field. Multiple inheritance becomes restricted in specific ways that require careful design. Subclasses must also declare __slots__ or they silently get a __dict__ back and lose the memory benefit. Most ORMs and some serialisation libraries expect __dict__ on instances.

Beyond __slots__, generator pipelines are the other high-impact memory pattern. A list comprehension over 10M records materialises all 10M results in heap memory simultaneously. A generator expression over the same data yields one result at a time — constant memory regardless of dataset size. For data pipelines that process records larger than available RAM, generators are not an optimisation, they are a correctness requirement.

import sys


class io_thecodeforge_Record:
    """
    Standard class — every instance has a __dict__.
    The __dict__ is a hash table that can hold any attribute you assign.
    Flexible, but expensive when you create millions of these.
    """
    def __init__(self, record_id: int, name: str, score: float):
        self.record_id = record_id
        self.name = name
        self.score = score


class io_thecodeforge_OptimisedRecord:
    """
    Same data, __slots__ instead of __dict__.
    CPython allocates a fixed C struct with three fields.
    No hash table. Direct memory offset access.
    Trade-off: cannot add arbitrary attributes at runtime.
    """
    __slots__ = ("record_id", "name", "score")

    def __init__(self, record_id: int, name: str, score: float):
        self.record_id = record_id
        self.name = name
        self.score = score


# Memory comparison at the individual object level
standard = io_thecodeforge_Record(1, "benchmark_record", 95.0)
optimised = io_thecodeforge_OptimisedRecord(1, "benchmark_record", 95.0)

print(f"Standard object size:      {sys.getsizeof(standard)} bytes")
print(f"Standard __dict__ size:    {sys.getsizeof(standard.__dict__)} bytes")
print(f"Standard total overhead:   {sys.getsizeof(standard) + sys.getsizeof(standard.__dict__)} bytes")
print(f"Optimised object size:     {sys.getsizeof(optimised)} bytes")
print(f"Has __dict__:              {hasattr(optimised, '__dict__')}")
print(f"Memory saved per object:   ~{(sys.getsizeof(standard) + sys.getsizeof(standard.__dict__)) - sys.getsizeof(optimised)} bytes")
print(f"At 10M objects:            ~{((sys.getsizeof(standard) + sys.getsizeof(standard.__dict__)) - sys.getsizeof(optimised)) * 10_000_000 // (1024**2)} MB saved")

# Generator pipeline — O(1) memory instead of O(n)
# The list comprehension below would allocate ~2-3 GB for 10M records
# The generator version uses constant memory regardless of dataset size
def io_thecodeforge_process_stream(records):
    """
    Process records without materialising the full result list.
    Each yield produces one result, consumed immediately, then discarded.
    Memory usage is bounded by the size of one record, not n records.
    """
    for record in records:
        if record["status"] == "active":
            # yield instead of append — caller consumes one at a time
            yield {**record, "score": record["value"] * 1.15}


stream = io_thecodeforge_process_stream(
    ({"status": "active", "value": i} for i in range(100_000))
)
# sum() consumes the generator without storing results — O(1) memory
result_count = sum(1 for _ in stream)
print(f"\nProcessed {result_count:,} records in constant memory")

The GIL, Concurrency and When to Go Parallel

CPython's Global Interpreter Lock is a mutex on the Python interpreter itself. Only one thread holds the GIL at any instant, which means only one thread executes Python bytecode at any instant. This is what makes CPython's memory management (reference counting) thread-safe without fine-grained per-object locking — but it is also what makes threading useless for CPU-bound parallelism.

The critical nuance is that the GIL is not held continuously. It is released during I/O operations — network calls, file reads, database queries, socket operations. Any time Python code does os.read() or socket.recv(), the GIL is released while the OS handles the I/O, allowing other threads to execute. This is why threading works well for I/O-bound workloads and why it is entirely wrong to say 'Python threading does nothing' — it does nothing for CPU-bound work, but it helps with I/O-bound work.

The GIL is also released by native extensions that explicitly drop it during computation. NumPy releases the GIL during most array operations, which is why NumPy computations can run in a thread without blocking other threads. Cython code that uses the nogil context manager does the same. This is a meaningful distinction: a thread running a NumPy computation and a thread handling I/O can execute genuinely in parallel, even though Python bytecode execution is serialised.

The practical decision framework is straightforward. I/O-bound work — API calls, database queries, file reads — gets asyncio or threading. CPU-bound work that is pure Python gets multiprocessing. CPU-bound work that is NumPy or Cython can use threading with the GIL released. Mixed workloads use asyncio for orchestration with run_in_executor(ProcessPoolExecutor) for the CPU-heavy segments.

The most expensive mistake I see is threading for CPU-bound pure Python work. Adding threads to a CPU-bound Python workload adds GIL contention overhead without any parallelism benefit. Two threads competing for the GIL on CPU-bound work run slower than one thread with no contention. This is not a subtle degradation — it is measurable and sometimes dramatic.

import asyncio
from concurrent.futures import ProcessPoolExecutor
import time


def io_thecodeforge_cpu_heavy(data: list[float]) -> float:
    """
    CPU-bound work — pure Python arithmetic.
    The GIL is held the entire time this runs.
    Threading this gives you contention with no parallelism.
    Multiprocessing this gives you a separate interpreter per process.
    """
    return sum(x * x for x in data)


async def io_thecodeforge_io_task(service_name: str) -> str:
    """
    I/O-bound work — asyncio handles this natively.
    await releases the event loop to run other coroutines during the wait.
    No threads. No processes. Cooperative multitasking on one thread.
    """
    await asyncio.sleep(0.1)  # Simulates a 100ms network call
    return f"Response from {service_name}"


async def io_thecodeforge_mixed_workload():
    """
    Production pattern for workloads with both I/O and CPU components.
    asyncio orchestrates everything.
    ProcessPoolExecutor handles the CPU-bound segment in a separate process.
    Both run concurrently — the I/O completes while the CPU work runs in the pool.
    """
    loop = asyncio.get_running_loop()

    # I/O-bound: three service calls, concurrent via gather()
    # Total time ~100ms, not ~300ms
    io_tasks = [
        io_thecodeforge_io_task("auth-service"),
        io_thecodeforge_io_task("inventory-service"),
        io_thecodeforge_io_task("pricing-service"),
    ]

    # CPU-bound: offload to ProcessPoolExecutor (separate process, separate GIL)
    # This runs concurrently with the I/O calls above
    data = [float(i) for i in range(500_000)]

    start = time.perf_counter()

    with ProcessPoolExecutor(max_workers=2) as pool:
        # submit CPU work to the process pool before awaiting I/O
        # run_in_executor returns an awaitable — the loop can schedule other work
        cpu_future = loop.run_in_executor(pool, io_thecodeforge_cpu_heavy, data)

        # await both concurrently — I/O and CPU overlap
        io_results, cpu_result = await asyncio.gather(
            asyncio.gather(*io_tasks),
            cpu_future,
        )

    elapsed = time.perf_counter() - start
    print(f"I/O results: {len(io_results)} service responses")
    print(f"CPU result:  {cpu_result:.0f}")
    print(f"Total time:  {elapsed:.2f}s (I/O and CPU ran concurrently)")


asyncio.run(io_thecodeforge_mixed_workload())

Vectorisation: From Python Loops to Native C Speed

Vectorisation is the practice of replacing Python-level loops with operations on typed, contiguous arrays executed by optimised C or Fortran routines. The performance difference is not incremental — it is architectural. A Python loop processing 10 million floating-point numbers involves reference counting on every value, type checking on every operation, and dynamic dispatch on every arithmetic expression. NumPy pushes the entire computation into a tight C loop that operates on raw typed memory with no per-element Python overhead. The gap is typically 100-500x for simple arithmetic on large arrays.

The reason NumPy is fast is not mysterious: it is because the loop runs in C on contiguous memory that fits in CPU caches. Python loops have poor cache behaviour because Python objects are pointer-chased heap allocations scattered across memory. NumPy arrays are contiguous blocks of typed bytes — the CPU prefetcher can predict the access pattern and keep the data in L1 cache. Modern CPUs can also SIMD-vectorise operations on contiguous numeric arrays, applying the same operation to multiple elements per clock cycle. None of this is available to Python loops.

Vectorisation applies most cleanly to numeric workloads — data that is homogeneous in type and fits the array paradigm. For complex per-element logic with conditional branching, recursive structure, or string manipulation, vectorisation either does not apply directly or requires careful reformulation using np.where(), np.select(), or boolean masking. The rule of thumb: if the loop body is more than five lines with complex conditionals, consider Numba's @jit decorator before NumPy — Numba can JIT-compile arbitrary Python numeric functions to native machine code without requiring you to reformulate the logic as array operations.

For tabular data in Pandas, the vectorisation principle applies to the method selection: use built-in Pandas methods (groupby, rolling, str.contains) rather than .apply() wherever possible. DataFrame.apply() falls back to a Python-level loop, which eliminates the Pandas performance advantage. When you must use .apply(), Numba can sometimes accelerate it via the engine='numba' parameter.

import time
import numpy as np


def io_thecodeforge_python_loop(data: list[float]) -> float:
    """
    Pure Python implementation.
    Every iteration: dereference pointer, check type, unbox value,
    compute, box result, update reference count, repeat.
    10M iterations of this overhead is the bottleneck.
    """
    total = 0.0
    for x in data:
        total += x * x + x * 0.5
    return total


def io_thecodeforge_numpy_vectorised(data: np.ndarray) -> float:
    """
    NumPy implementation of the identical computation.
    The entire operation executes in a single C loop over contiguous memory.
    No per-element Python overhead. SIMD-eligible on modern CPUs.
    """
    return float(np.sum(data ** 2 + data * 0.5))


def io_thecodeforge_numpy_inplace(data: np.ndarray) -> float:
    """
    Memory-optimised variant using in-place operations.
    Avoids creating intermediate arrays for data**2 and data*0.5.
    Relevant when data is large enough that intermediate arrays
    exceed L3 cache and cause cache pressure.
    """
    result = data.copy()
    result **= 2
    result += data * 0.5
    return float(result.sum())


N = 10_000_000
data_list = [float(i) for i in range(N)]
data_array = np.arange(N, dtype=np.float64)

# Warm up NumPy (first call includes JIT overhead in some contexts)
_ = io_thecodeforge_numpy_vectorised(data_array[:100])

# Python loop benchmark
start = time.perf_counter()
result_py = io_thecodeforge_python_loop(data_list)
time_py = time.perf_counter() - start

# NumPy vectorised benchmark
start = time.perf_counter()
result_np = io_thecodeforge_numpy_vectorised(data_array)
time_np = time.perf_counter() - start

print(f"Python loop:  {time_py:.3f}s")
print(f"NumPy:        {time_np:.3f}s")
print(f"Speedup:      {time_py / time_np:.0f}x")
print(f"Results match: {abs(result_py - result_np) < 1e6}")
print(f"\nData type matters: float32 is often 2x faster than float64")
print(f"float32 array size: {data_array.astype(np.float32).nbytes / 1024**2:.0f} MB")
print(f"float64 array size: {data_array.nbytes / 1024**2:.0f} MB")

Production Patterns: Caching, Lazy Evaluation and Profiling in CI

The most impactful performance optimisation in production is often not making code faster — it is making it run less. Caching eliminates redundant computation entirely. A function that takes 50ms and is called a thousand times per second with the same arguments takes 50ms once and zero milliseconds 999 times with a properly sized cache. That is the highest possible return on investment for any performance work.

functools.lru_cache is the in-process standard for pure functions with repeated inputs. It stores results in a hash table keyed by the function arguments and returns cached results in O(1). The critical production detail is maxsize — lru_cache without maxsize is an unbounded memory allocation that will grow until the process is OOM-killed in a long-running service. Always set maxsize and monitor cache_info().currsize and cache_info().misses to verify the cache is sized correctly for the key space.

Local variable caching in hot loops is the other pattern that gives disproportionate returns for negligible risk. Python bytecode uses LOAD_FAST for local variables — a direct C array index into the frame's local variable table. LOAD_ATTR for instance attribute access is a dictionary lookup: hash the attribute name, find the slot in __dict__, dereference the pointer. LOAD_GLOBAL for module-level names is similar. In a loop that iterates a million times, caching obj.method as a local variable before the loop replaces a million dictionary lookups with a million C array accesses. The speedup is typically 10-30%, the code change is one line, and the risk is zero.

Embedding profiling into your CI pipeline is the pattern that prevents incidents rather than responding to them. A cProfile report generated on every PR that modifies a data processing function, compared against a stored baseline, catches performance regressions in code review. A function that processes a million records in 0.4 seconds regressing to 4 seconds is caught before it merges — not six weeks later when the dataset has grown and the SLA is missed at midnight.

import functools
import time
from typing import Any


# Pattern 1: LRU cache with explicit maxsize and monitoring
# maxsize=1024 means up to 1024 unique argument combinations are cached
# Always set maxsize — unlimited cache is a memory leak in production
@functools.lru_cache(maxsize=1024)
def io_thecodeforge_get_config(service_name: str) -> dict:
    """
    Expensive config service call — 50ms per call without caching.
    With caching: 50ms on first call, ~0ms on every subsequent call
    for the same service_name within the cache capacity.
    """
    time.sleep(0.05)  # Simulates config service network call
    return {"service": service_name, "timeout": 30, "retries": 3}


def io_thecodeforge_check_cache_health():
    """Monitor cache performance — low hit ratio means wrong maxsize or key space."""
    info = io_thecodeforge_get_config.cache_info()
    hit_ratio = info.hits / max(1, info.hits + info.misses)
    print(f"Cache: hits={info.hits}, misses={info.misses}, ratio={hit_ratio:.1%}, size={info.currsize}/{info.maxsize}")
    if hit_ratio < 0.8:
        print("WARNING: hit ratio below 80% — cache too small or key space too large")


# Pattern 2: Local variable caching in hot loops
def io_thecodeforge_process_events_slow(events: list[dict]) -> list[Any]:
    """Naive implementation — LOAD_ATTR on every iteration."""
    results = []
    for event in events:
        results.append(str.upper(event.get("type", "unknown")))
    return results


def io_thecodeforge_process_events_fast(events: list[dict]) -> list[Any]:
    """
    Optimised with local variable caching.
    'append' and 'upper' are looked up once, stored as LOAD_FAST locals.
    Inside the loop, LOAD_FAST replaces LOAD_ATTR — direct array index vs dict lookup.
    10-30% faster for tight loops. Zero risk. One line of change.
    """
    results = []
    # Cache as locals BEFORE the loop — these assignments happen once
    _append = results.append      # LOAD_FAST instead of LOAD_ATTR
    _upper = str.upper            # LOAD_FAST instead of LOAD_GLOBAL + LOAD_ATTR
    _get = dict.get               # LOAD_FAST instead of LOAD_ATTR per iteration

    for event in events:
        _append(_upper(_get(event, "type", "unknown")))
    return results


# Pattern 3: Generator for lazy evaluation — O(1) memory pipeline
def io_thecodeforge_stream_large_file(filepath: str):
    """
    Yield lines lazily — memory usage is bounded by one line, not the file size.
    A 10GB log file processed with readlines() requires 10GB of RAM.
    This generator requires constant memory regardless of file size.
    """
    with open(filepath, "r") as f:
        for line in f:
            stripped = line.strip()
            if stripped:  # Skip empty lines
                yield stripped


# Benchmark: local variable caching impact
N = 1_000_000
sample_events = [{"type": f"event_{i % 50}"} for i in range(N)]

start = time.perf_counter()
result_slow = io_thecodeforge_process_events_slow(sample_events)
time_slow = time.perf_counter() - start

start = time.perf_counter()
result_fast = io_thecodeforge_process_events_fast(sample_events)
time_fast = time.perf_counter() - start

print(f"Naive (LOAD_ATTR):  {time_slow:.3f}s")
print(f"Cached locals:      {time_fast:.3f}s")
print(f"Speedup:            {time_slow / time_fast:.1f}x")

# Cache demo
io_thecodeforge_get_config("auth-service")  # first call — cache miss
io_thecodeforge_get_config("auth-service")  # second call — cache hit
io_thecodeforge_get_config("payment-service")  # different key — cache miss
io_thecodeforge_check_cache_health()

functools.lru_cache vs Custom Caching: When to Use Which

Caching in Python production systems falls into two broad categories: the built-in functools.lru_cache and custom caching solutions (in-memory dicts, Redis, Memcached, database-driven). The right choice depends on your requirements for invalidation, distribution, memory management, and data freshness.

The table below compares the different caching strategies across the dimensions that matter in production.

``text Requirement | functools.lru_cache | In-Memory Dict Cache | Redis/Memcached -------------------------------------|---------------------|------------------------|------------------------- Setup complexity | One decorator | Manual implementation | Requires infrastructure Eviction policy | LRU (maxsize) | Manual (TTL, size) | LRU, TTL, LFU Automatic key invalidation | Only via maxsize LRU| Must implement manually | TTL expiry, explicit delete Distributed across processes | No (per-process) | No (per-process) | Yes (shared across processes) Memory bounds | maxsize parameter | Must implement size cap | Configurable via config Cache stampede protection | No | Must implement | Some (lock patterns) Dependency invalidation | Not supported | Must implement | Must implement Serialisation overhead | None (in-memory) | None (in-memory) | Yes (pickle/json) Suitability for pure functions | Excellent | Good | Overkill if not shared ``

When to use functools.lru_cache: - The function is pure (same arguments → same result, no side effects). - The function is called repeatedly with the same arguments within a single process. - The total number of unique argument combinations fits within a reasonable memory budget (set maxsize). - You don't need time-based expiry or distributed invalidation. - The cache key is easy to derive from the function arguments (they must be hashable).

When to use a custom in-memory dict cache: - You need TTL-based expiry (e.g., cache for 5 minutes, then recompute). - You need to invalidate specific keys based on external events. - The function arguments are not hashable (e.g., lists or dicts that need custom key logic). - You need to store more than lru_cache's argument tuple can handle (very large arguments).

When to use Redis/Memcached: - The cache must be shared across multiple processes or services. - The data must survive application restarts. - You need atomic operations (increment, lock) as part of the caching pattern. - You need fine-grained TTLs on a per-key basis. - The caching logic is part of a distributed system with multiple consumers.

The most common mistake is using lru_cache when the function depends on external state (database queries, timestamps, user context) — the cache serves stale data silently. The second most common mistake is not setting maxsize and letting the cache grow unboundedly. Always start with lru_cache for pure functions with bounded key spaces, and move to custom caching only when profiling shows it's necessary.

import functools
import time


# --- Scenario 1: lru_cache for pure functions ---
@functools.lru_cache(maxsize=256)
def compute_active_score(user_tier: str, base_score: float) -> float:
    """
    Pure function: result depends only on arguments.
    No side effects, no external state.
    Perfect for lru_cache.
    """
    # Simulate a moderately expensive computation
    time.sleep(0.01)  # 10ms work
    multiplier = {"free": 1.0, "pro": 2.0, "enterprise": 3.0}
    return base_score * multiplier.get(user_tier, 1.0)


# Usage
print(compute_active_score("pro", 100.0))  # Cache miss, computes
print(compute_active_score("pro", 100.0))  # Cache hit, returns instantly
print(compute_active_score("free", 100.0)) # Cache miss (different key)
print(f"Cache info: {compute_active_score.cache_info()}")


# --- Scenario 2: Custom dict cache with TTL ---
import threading

class TTLDictCache:
    """
    Simple in-memory cache with time-based expiry.
    Use when you need TTLs that lru_cache cannot provide.
    """
    def __init__(self, default_ttl: int = 300):
        self._cache = {}
        self._default_ttl = default_ttl
        self._lock = threading.Lock()
    
    def get(self, key):
        with self._lock:
            entry = self._cache.get(key)
            if entry is None:
                return None
            value, expiry = entry
            if time.time() > expiry:
                del self._cache[key]
                return None
            return value
    
    def set(self, key, value, ttl: int = None):
        if ttl is None:
            ttl = self._default_ttl
        expiry = time.time() + ttl
        with self._lock:
            self._cache[key] = (value, expiry)


# Scenario 2: DB query caching with TTL
db_cache = TTLDictCache(default_ttl=60)  # Cache for 60 seconds

def get_user(user_id: int) -> dict:
    cached = db_cache.get(user_id)
    if cached is not None:
        return cached
    # Imagine real DB query here
    user = {"id": user_id, "name": "Alice", "role": "admin"}
    db_cache.set(user_id, user, ttl=300)
    return user

print(get_user(42))  # Cache miss
print(get_user(42))  # Cache hit (within 60s TTL)