Python Design Patterns — Singleton Connection Pool Leak

Every senior Python developer has felt it — that sinking moment when a codebase that worked perfectly at 1,000 users starts buckling at 100,000. New engineers can't follow the logic, a shared resource keeps getting instantiated twenty times, and event-driven code turns into an untraceable chain of callbacks. Design patterns are the vocabulary that lets experienced engineers name these problems and reach for proven solutions without reinventing them under pressure. They're not academic exercises — they're the difference between a codebase you're proud to hand off and one you're embarrassed to demo.

The real problem design patterns solve isn't complexity — it's communication and entropy. As a codebase grows, the same structural problems appear over and over: how do you ensure exactly one database connection pool exists? How do you add features to an object without editing its class? How do you notify dozens of components when state changes without coupling them tightly together? Without patterns, every developer on your team invents a slightly different answer, and the codebase quietly fractures. Patterns give you a shared language and a proven structure before the chaos sets in.

By the end of this article you'll understand the internal mechanics of Singleton, Factory, Observer, Strategy, and Decorator patterns in Python — not just the textbook definitions but the gotchas that bite production systems, the performance trade-offs that matter at scale, and how Python's own language features (metaclasses, descriptors, __call__, functools) interact with these patterns in ways that surprise even experienced devs. You'll be able to choose the right pattern for the right problem and explain your reasoning in a technical interview.

Why Python Design Patterns Are Not Templates

Python design patterns are reusable solutions to recurring architectural problems, but they are not templates you copy-paste. They are conceptual frameworks that exploit Python's dynamic typing, first-class functions, and duck typing to solve specific coupling, scalability, or maintainability issues. The core mechanic is separating what varies from what stays the same — encapsulating the variation behind a stable interface.

In practice, patterns in Python often collapse into simpler constructs. A Strategy pattern becomes a function passed as an argument; an Observer becomes a callback list. The key property is that patterns are not about class hierarchies — they are about communication structures between objects. Python's protocols (__enter__, __iter__, etc.) let you implement patterns with less boilerplate than in statically typed languages, but the same trade-offs apply: indirection costs readability, and over-engineering is the most common failure mode.

Use design patterns when you have a known, repeating structural problem — not because a book says so. In production systems, the Singleton pattern is the most misapplied: teams use it for connection pools, then hit thread-safety bugs and memory leaks because they ignored Python's module-level singletons (which are already singletons) or used a naive __new__ override without considering GIL interactions or garbage collection cycles.

Why Design Patterns Matter in Python — Beyond the Hype

Design patterns are not about fancy UML diagrams or memorising GOF names. In Python, many patterns are built into the language itself: decorators, generators, context managers, and even the module system. But that doesn't make the patterns obsolete. It makes them more powerful — and more dangerous when applied carelessly.

Take the Singleton pattern. Python's module system gives you a Singleton for free: a module is imported once, and its top-level names persist for the process lifetime. But many teams still write a class-based Singleton and get it wrong — they forget thread safety, or they assume the initialisation in __new__ runs only once when it actually runs on every instance creation attempt. The result? A Singleton that creates multiple instances under concurrent load.

Patterns in Python aren't about how to implement them — they're about when to use them and how to adapt them to Python's idioms. The real value comes from understanding the trade-offs: a module-level Singleton is simple but untestable; a dependency-injected Singleton is testable but requires a container. Knowing which trade-off to accept for your context is what separates senior engineers from juniors.

from io.thecodeforge.utils import get_config

# Thread-safe Singleton using module-level cache
class DatabasePool:
    _instance = None
    _lock = None

    @classmethod
    def get_instance(cls):
        if cls._instance is None:
            import threading
            if cls._lock is None:
                cls._lock = threading.Lock()
            with cls._lock:
                if cls._instance is None:
                    cls._instance = cls(get_config())
        return cls._instance

    def __init__(self, config):
        self.conn = create_connection(config)

# Better approach: use module-level singleton with lru_cache
from functools import lru_cache

@lru_cache(maxsize=1)
def get_db_pool():
    return create_connection(get_config())

Singleton Pattern — The One You Think You Know

Singleton ensures a class has only one instance and provides a global point of access. In Python, this seems trivial: just create a module-level object. But the devil is in the details. A thread-safe Singleton requires careful initialisation, and the classic double-checked locking pattern is tricky to get right because of Python's GIL and object ordering.

Let's break down the production-grade Singleton. The key is that __new__ is called every time you instantiate the class — even if the instance already exists. The common mistake is to put the instance check in __init__ instead of __new__, which means the constructor still runs on every call, potentially resetting state.

A better approach for most Python applications is to use a module-level function with @lru_cache. This gives you a thread-safe, lazily initialised singleton without locks. But it comes with a caveat: you cannot easily clear the cache for testing without accessing the wrapped function's cache clear method. Use this for objects that truly live for the entire process lifetime, like a configuration object that never changes after startup.

from functools import lru_cache
from io.thecodeforge.config import Settings

@lru_cache(maxsize=1)
def get_settings() -> Settings:
    """Production-grade singleton for application settings.
    Thread-safe, lazy-loaded, and easy to use in decorators."""
    return Settings.from_env()

# Usage anywhere in the codebase, no import-time instantiation
# The lru_cache ensures only one Settings object ever exists per process.

Factory Pattern — Object Creation Without the Pain

The Factory pattern separates object creation from business logic. In Python, this is often as simple as a function that returns different types based on input. The GoF Factory Method and Abstract Factory patterns translate naturally: Python's dynamic typing and duck typing mean you don't need complex interface hierarchies. A Factory can be a single function or a class hierarchy with a common interface.

The real power of Factory in Python comes with configuration-driven creation. You can map string keys to classes and instantiate them at runtime. This is how many frameworks (like Celery or Django's backends) work. But watch out for circular imports: if the factory function imports all possible classes, you may introduce a circular dependency. A pattern that works well is to use a lazy registry: class definitions register themselves in a dict at import time, and the factory only needs to know the dict.

Performance note: Factory dispatch via a dict is O(1) and preferable to a long if/elif chain. Use functools.singledispatch for function overloading based on argument type, but avoid using it when you need string-based dispatch for configuration.

from typing import Protocol
from io.thecodeforge.notifications import EmailSender, SMSSender, PushSender

class NotificationSender(Protocol):
    def send(self, message: str) -> bool: ...

class SenderFactory:
    _registry: dict[str, type[NotificationSender]] = {}

    @classmethod
    def register(cls, sender_type: str, sender_cls: type[NotificationSender]):
        cls._registry[sender_type] = sender_cls

    @classmethod
    def create(cls, sender_type: str, **kwargs) -> NotificationSender:
        if sender_type not in cls._registry:
            raise ValueError(f"Unknown sender type: {sender_type}")
        return cls._registry[sender_type](**kwargs)

# Registration:
SenderFactory.register('email', EmailSender)
SenderFactory.register('sms', SMSSender)

# Usage:
sender = SenderFactory.create('email', config=settings.email_config)
sender.send('Hello')

Observer Pattern — Event-Driven Without the Memory Leak

The Observer pattern defines a one-to-many dependency between objects so that when one object changes state, all its dependents are notified and updated automatically. Python has several built-in mechanisms for this: property setters can trigger callbacks, and the blinker library or PyPubSub provide publish-subscribe. But the classic GoF Observer often leads to memory leaks in Python because observers hold strong references to subjects, or vice versa.

In Python, the biggest gotcha is the __del__ method. If a subject holds a strong reference to an observer that goes out of scope normally, the observer is not garbage collected because the subject still has a reference. This can cause unbounded memory growth, especially in long-running processes like web servers.

The fix: use weakref.ref for the observer references. Python's WeakSet and WeakValueDictionary are perfect for this. But you can't use weak references for bound methods easily — a bound method holds a strong reference to its object, so even a weak reference to the method won't help. Instead, store the actual object and call the method via weak reference, or use the weakref module's finalize for cleanup.

Another approach: use callbacks as simple functions (not methods) stored in a set of weak references to functions. But functions can be closures that capture large objects — so be mindful of what the closure captures.

import weakref
from typing import Callable, Set

class Observable:
    def __init__(self):
        self._observers: Set[weakref.ref] = set()

    def register(self, callback: Callable):
        self._observers.add(weakref.ref(callback))

    def unregister(self, callback: Callable):
        # Use a temporary set to collect dead refs
        dead = set()
        for ref in self._observers:
            obj = ref()
            if obj is None:
                dead.add(ref)
            elif obj is callback:
                dead.add(ref)
        self._observers -= dead

    def notify(self, *args, **kwargs):
        dead = set()
        for ref in self._observers:
            callback = ref()
            if callback:
                callback(*args, **kwargs)
            else:
                dead.add(ref)
        self._observers -= dead

# Usage:
def on_data_change(data):
    print(f"Data changed: {data}")

obs = Observable()
obs.register(on_data_change)
# The function `on_data_change` can be garbage collected when no longer needed

Strategy Pattern — Replace if/elif Chains with Dispatch

Strategy lets you define a family of algorithms, encapsulate each one, and make them interchangeable. In Python, you don't need a formal interface; you can use a dictionary of callables. This is often simpler and more readable than a class hierarchy. The Strategy pattern shines when you have multiple ways to process the same input, like different compression algorithms or payment gateways.

Performance consideration: if the strategy selection logic is based on a configuration string that rarely changes, use a functools.lru_cache on the strategy selection function to avoid repeated parsing. If the strategies themselves have different performance characteristics, profile them independently and choose based on input characteristics (e.g., small data use compression-X, large data use compression-Y).

The biggest mistake is putting the selection logic in the caller's code, scattered across the codebase. Centralise the dispatch in a dedicated strategy registry so that adding a new strategy doesn't require changes in multiple places.

from io.thecodeforge.payment import CreditCardPay, PayPalPay, CryptoPay

# Strategy registry as a dict of callables
_payment_strategies = {
    'credit_card': CreditCardPay,
    'paypal': PayPalPay,
    'crypto': CryptoPay,
}

def get_payment_strategy(method: str):
    """Return the payment strategy class or callable."""
    strategy = _payment_strategies.get(method)
    if strategy is None:
        raise ValueError(f"Unsupported payment method: {method}")
    return strategy

# Usage:
strategy_cls = get_payment_strategy(order.payment_method)
processor = strategy_cls(config)
processor.charge(order.amount)

Decorator Pattern — Python's Superpower and Its Pitfalls

Python has a built-in syntax for the Decorator pattern: the @decorator syntax. It's one of the few languages where the GoF Decorator pattern is directly supported. But with great power comes great responsibility. Decorators can change the signature of the decorated function, break type hints, and introduce subtle performance overhead.

The three most common production issues with decorators: 1. Signature loss: The decorated function loses its original __name__, __doc__, and argument signature. Always use @functools.wraps. 2. Performance overhead: Every call to a decorated function adds the decorator's overhead. If you chain several decorators, each one adds a function call. For latency-critical paths, consider combining decorators or using a single decorator that does multiple things. 3. State leakage: If a decorator has mutable state (e.g., a cache), concurrent access can cause data races. Use thread-safe structures or limit to single-threaded contexts.

The GoF Decorator pattern (wrapping objects) is also useful in Python: e.g., wrapping a file object to add compression or encryption. Use io.IOBase as the base class and implement the wrapper, or simply use functools.partial for simple cases.

import functools
import logging
from typing import Callable

logger = logging.getLogger(__name__)

def logged(func: Callable) -> Callable:
    """Logs entry and exit of the decorated function.
    Preserves signature via wraps."""
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        logger.debug("Calling %s with args=%s kwargs=%s", func.__name__, args, kwargs)
        try:
            result = func(*args, **kwargs)
            logger.debug("%s returned %s", func.__name__, result)
            return result
        except Exception as e:
            logger.exception("%s raised %s", func.__name__, e)
            raise
    return wrapper

# Usage:
@logged
def process_order(order_id: int) -> bool:
    ...
# process_order.__name__ is still 'process_order', not 'wrapper'

The Pattern That Bites Back — Adapter Is Not a Band-Aid

You inherited that third-party API with the worst interface in existence. First instinct: slap an Adapter on it and call it a day. That works — until it doesn't. The Adapter pattern wraps an incompatible interface to make it talk to your system. But here's the trap: wrapping garbage doesn't make it clean. Every adapter adds a layer of indirection, and that layer hides the underlying complexity. When your adapter starts sprouting if/else chains to handle edge cases, you've built a leaky abstraction. The real fix is to understand why the interface is broken and fix it at the source. If you can't, then and only then, write an adapter. Keep it thin. Single responsibility. Test it like your production depends on it — because it does. I've seen adapters become the single point of failure in a critical payment pipeline. Don't let that be you.

Composite Pattern — When Trees Go to War

You've got a hierarchy: orders, line items, discounts. Each one needs the same operation: calculate total, apply tax, serialize. The Composite pattern lets you treat individual objects and groups uniformly. It's a tree structure where every node can be a leaf or a composite. In Python, this is deceptively simple. You define a common interface, then implement it in both classes. The composite class internally calls the same method on its children. Beautiful recursion. But I've seen teams turn this into a performance disaster. They add hundreds of thousands of nodes, then wonder why a simple sum takes seconds. The WHY is important: Composite shines when you have stable, shallow trees. If your hierarchy runs deep or changes frequently, you're better off with a Visitor pattern or a flat iterator. Measure before you commit. And always cache results if you're recalculating more than once.

State Pattern — When if/elif Becomes a War Crime

You've seen that function. Two hundred lines of if/elif checking a status variable. It works until someone adds a new state and forgets to update every branch. That's how payment confirmations end up in limbo. The State pattern moves each state's behavior into its own class. Instead of checking the state, you delegate to the current state object. It's like having a bouncer that only lets the right people in. In Python, this maps naturally to classes that implement a common protocol. The state machine holds a reference to the current state and swaps it when a transition happens. The catch: you need a clear, enforced transition table. I've had to rewrite messes where states jumped arbitrarily because someone added a shortcut. Document every allowed transition. Use an enum for state names. And for the love of consistent logs, log every state change with a timestamp and caller. You'll thank me during the post-mortem.