Python Design Patterns — Singleton Connection Pool Leak

Every senior Python developer has felt it — that sinking moment when a codebase that worked perfectly at 1,000 users starts buckling at 100,000. New engineers can't follow the logic, a shared resource keeps getting instantiated twenty times, and event-driven code turns into an untraceable chain of callbacks. Design patterns are the vocabulary that lets experienced engineers name these problems and reach for proven solutions without reinventing them under pressure. They're not academic exercises — they're the difference between a codebase you're proud to hand off and one you're embarrassed to demo.

The real problem design patterns solve isn't complexity — it's communication and entropy. As a codebase grows, the same structural problems appear over and over: how do you ensure exactly one database connection pool exists? How do you add features to an object without editing its class? How do you notify dozens of components when state changes without coupling them tightly together? Without patterns, every developer on your team invents a slightly different answer, and the codebase quietly fractures. Patterns give you a shared language and a proven structure before the chaos sets in.

By the end of this article you'll understand the internal mechanics of Singleton, Factory, Observer, Strategy, and Decorator patterns in Python — not just the textbook definitions but the gotchas that bite production systems, the performance trade-offs that matter at scale, and how Python's own language features (metaclasses, descriptors, __call__, functools) interact with these patterns in ways that surprise even experienced devs. You'll be able to choose the right pattern for the right problem and explain your reasoning in a technical interview.

Why Python Design Patterns Are Not Templates

Python design patterns are reusable solutions to recurring architectural problems, but they are not templates you copy-paste. They are conceptual frameworks that exploit Python's dynamic typing, first-class functions, and duck typing to solve specific coupling, scalability, or maintainability issues. The core mechanic is separating what varies from what stays the same — encapsulating the variation behind a stable interface.

In practice, patterns in Python often collapse into simpler constructs. A Strategy pattern becomes a function passed as an argument; an Observer becomes a callback list. The key property is that patterns are not about class hierarchies — they are about communication structures between objects. Python's protocols (__enter__, __iter__, etc.) let you implement patterns with less boilerplate than in statically typed languages, but the same trade-offs apply: indirection costs readability, and over-engineering is the most common failure mode.

Use design patterns when you have a known, repeating structural problem — not because a book says so. In production systems, the Singleton pattern is the most misapplied: teams use it for connection pools, then hit thread-safety bugs and memory leaks because they ignored Python's module-level singletons (which are already singletons) or used a naive __new__ override without considering GIL interactions or garbage collection cycles.

Why Design Patterns Matter in Python — Beyond the Hype

Design patterns are not about fancy UML diagrams or memorising GOF names. In Python, many patterns are built into the language itself: decorators, generators, context managers, and even the module system. But that doesn't make the patterns obsolete. It makes them more powerful — and more dangerous when applied carelessly.

Take the Singleton pattern. Python's module system gives you a Singleton for free: a module is imported once, and its top-level names persist for the process lifetime. But many teams still write a class-based Singleton and get it wrong — they forget thread safety, or they assume the initialisation in __new__ runs only once when it actually runs on every instance creation attempt. The result? A Singleton that creates multiple instances under concurrent load.

Patterns in Python aren't about how to implement them — they're about when to use them and how to adapt them to Python's idioms. The real value comes from understanding the trade-offs: a module-level Singleton is simple but untestable; a dependency-injected Singleton is testable but requires a container. Knowing which trade-off to accept for your context is what separates senior engineers from juniors.

from io.thecodeforge.utils import get_config

# Thread-safe Singleton using module-level cache
class DatabasePool:
    _instance = None
    _lock = None

    @classmethod
    def get_instance(cls):
        if cls._instance is None:
            import threading
            if cls._lock is None:
                cls._lock = threading.Lock()
            with cls._lock:
                if cls._instance is None:
                    cls._instance = cls(get_config())
        return cls._instance

    def __init__(self, config):
        self.conn = create_connection(config)

# Better approach: use module-level singleton with lru_cache
from functools import lru_cache

@lru_cache(maxsize=1)
def get_db_pool():
    return create_connection(get_config())

Singleton Pattern — The One You Think You Know

Singleton ensures a class has only one instance and provides a global point of access. In Python, this seems trivial: just create a module-level object. But the devil is in the details. A thread-safe Singleton requires careful initialisation, and the classic double-checked locking pattern is tricky to get right because of Python's GIL and object ordering.

Let's break down the production-grade Singleton. The key is that __new__ is called every time you instantiate the class — even if the instance already exists. The common mistake is to put the instance check in __init__ instead of __new__, which means the constructor still runs on every call, potentially resetting state.

A better approach for most Python applications is to use a module-level function with @lru_cache. This gives you a thread-safe, lazily initialised singleton without locks. But it comes with a caveat: you cannot easily clear the cache for testing without accessing the wrapped function's cache clear method. Use this for objects that truly live for the entire process lifetime, like a configuration object that never changes after startup.

from functools import lru_cache
from io.thecodeforge.config import Settings

@lru_cache(maxsize=1)
def get_settings() -> Settings:
    """Production-grade singleton for application settings.
    Thread-safe, lazy-loaded, and easy to use in decorators."""
    return Settings.from_env()

# Usage anywhere in the codebase, no import-time instantiation
# The lru_cache ensures only one Settings object ever exists per process.

Factory Pattern — Object Creation Without the Pain

The Factory pattern separates object creation from business logic. In Python, this is often as simple as a function that returns different types based on input. The GoF Factory Method and Abstract Factory patterns translate naturally: Python's dynamic typing and duck typing mean you don't need complex interface hierarchies. A Factory can be a single function or a class hierarchy with a common interface.

The real power of Factory in Python comes with configuration-driven creation. You can map string keys to classes and instantiate them at runtime. This is how many frameworks (like Celery or Django's backends) work. But watch out for circular imports: if the factory function imports all possible classes, you may introduce a circular dependency. A pattern that works well is to use a lazy registry: class definitions register themselves in a dict at import time, and the factory only needs to know the dict.

Performance note: Factory dispatch via a dict is O(1) and preferable to a long if/elif chain. Use functools.singledispatch for function overloading based on argument type, but avoid using it when you need string-based dispatch for configuration.

from typing import Protocol
from io.thecodeforge.notifications import EmailSender, SMSSender, PushSender

class NotificationSender(Protocol):
    def send(self, message: str) -> bool: ...

class SenderFactory:
    _registry: dict[str, type[NotificationSender]] = {}

    @classmethod
    def register(cls, sender_type: str, sender_cls: type[NotificationSender]):
        cls._registry[sender_type] = sender_cls

    @classmethod
    def create(cls, sender_type: str, **kwargs) -> NotificationSender:
        if sender_type not in cls._registry:
            raise ValueError(f"Unknown sender type: {sender_type}")
        return cls._registry[sender_type](**kwargs)

# Registration:
SenderFactory.register('email', EmailSender)
SenderFactory.register('sms', SMSSender)

# Usage:
sender = SenderFactory.create('email', config=settings.email_config)
sender.send('Hello')

Observer Pattern — Event-Driven Without the Memory Leak

The Observer pattern defines a one-to-many dependency between objects so that when one object changes state, all its dependents are notified and updated automatically. Python has several built-in mechanisms for this: property setters can trigger callbacks, and the blinker library or PyPubSub provide publish-subscribe. But the classic GoF Observer often leads to memory leaks in Python because observers hold strong references to subjects, or vice versa.

In Python, the biggest gotcha is the __del__ method. If a subject holds a strong reference to an observer that goes out of scope normally, the observer is not garbage collected because the subject still has a reference. This can cause unbounded memory growth, especially in long-running processes like web servers.

The fix: use weakref.ref for the observer references. Python's WeakSet and WeakValueDictionary are perfect for this. But you can't use weak references for bound methods easily — a bound method holds a strong reference to its object, so even a weak reference to the method won't help. Instead, store the actual object and call the method via weak reference, or use the weakref module's finalize for cleanup.

Another approach: use callbacks as simple functions (not methods) stored in a set of weak references to functions. But functions can be closures that capture large objects — so be mindful of what the closure captures.

import weakref
from typing import Callable, Set

class Observable:
    def __init__(self):
        self._observers: Set[weakref.ref] = set()

    def register(self, callback: Callable):
        self._observers.add(weakref.ref(callback))

    def unregister(self, callback: Callable):
        # Use a temporary set to collect dead refs
        dead = set()
        for ref in self._observers:
            obj = ref()
            if obj is None:
                dead.add(ref)
            elif obj is callback:
                dead.add(ref)
        self._observers -= dead

    def notify(self, *args, **kwargs):
        dead = set()
        for ref in self._observers:
            callback = ref()
            if callback:
                callback(*args, **kwargs)
            else:
                dead.add(ref)
        self._observers -= dead

# Usage:
def on_data_change(data):
    print(f"Data changed: {data}")

obs = Observable()
obs.register(on_data_change)
# The function `on_data_change` can be garbage collected when no longer needed

Strategy Pattern — Replace if/elif Chains with Dispatch

Strategy lets you define a family of algorithms, encapsulate each one, and make them interchangeable. In Python, you don't need a formal interface; you can use a dictionary of callables. This is often simpler and more readable than a class hierarchy. The Strategy pattern shines when you have multiple ways to process the same input, like different compression algorithms or payment gateways.

Performance consideration: if the strategy selection logic is based on a configuration string that rarely changes, use a functools.lru_cache on the strategy selection function to avoid repeated parsing. If the strategies themselves have different performance characteristics, profile them independently and choose based on input characteristics (e.g., small data use compression-X, large data use compression-Y).

The biggest mistake is putting the selection logic in the caller's code, scattered across the codebase. Centralise the dispatch in a dedicated strategy registry so that adding a new strategy doesn't require changes in multiple places.

from io.thecodeforge.payment import CreditCardPay, PayPalPay, CryptoPay

# Strategy registry as a dict of callables
_payment_strategies = {
    'credit_card': CreditCardPay,
    'paypal': PayPalPay,
    'crypto': CryptoPay,
}

def get_payment_strategy(method: str):
    """Return the payment strategy class or callable."""
    strategy = _payment_strategies.get(method)
    if strategy is None:
        raise ValueError(f"Unsupported payment method: {method}")
    return strategy

# Usage:
strategy_cls = get_payment_strategy(order.payment_method)
processor = strategy_cls(config)
processor.charge(order.amount)

Decorator Pattern — Python's Superpower and Its Pitfalls

Python has a built-in syntax for the Decorator pattern: the @decorator syntax. It's one of the few languages where the GoF Decorator pattern is directly supported. But with great power comes great responsibility. Decorators can change the signature of the decorated function, break type hints, and introduce subtle performance overhead.

The three most common production issues with decorators: 1. Signature loss: The decorated function loses its original __name__, __doc__, and argument signature. Always use @functools.wraps. 2. Performance overhead: Every call to a decorated function adds the decorator's overhead. If you chain several decorators, each one adds a function call. For latency-critical paths, consider combining decorators or using a single decorator that does multiple things. 3. State leakage: If a decorator has mutable state (e.g., a cache), concurrent access can cause data races. Use thread-safe structures or limit to single-threaded contexts.

The GoF Decorator pattern (wrapping objects) is also useful in Python: e.g., wrapping a file object to add compression or encryption. Use io.IOBase as the base class and implement the wrapper, or simply use functools.partial for simple cases.

import functools
import logging
from typing import Callable

logger = logging.getLogger(__name__)

def logged(func: Callable) -> Callable:
    """Logs entry and exit of the decorated function.
    Preserves signature via wraps."""
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        logger.debug("Calling %s with args=%s kwargs=%s", func.__name__, args, kwargs)
        try:
            result = func(*args, **kwargs)
            logger.debug("%s returned %s", func.__name__, result)
            return result
        except Exception as e:
            logger.exception("%s raised %s", func.__name__, e)
            raise
    return wrapper

# Usage:
@logged
def process_order(order_id: int) -> bool:
    ...
# process_order.__name__ is still 'process_order', not 'wrapper'

The Pattern That Bites Back — Adapter Is Not a Band-Aid

You inherited that third-party API with the worst interface in existence. First instinct: slap an Adapter on it and call it a day. That works — until it doesn't. The Adapter pattern wraps an incompatible interface to make it talk to your system. But here's the trap: wrapping garbage doesn't make it clean. Every adapter adds a layer of indirection, and that layer hides the underlying complexity. When your adapter starts sprouting if/else chains to handle edge cases, you've built a leaky abstraction. The real fix is to understand why the interface is broken and fix it at the source. If you can't, then and only then, write an adapter. Keep it thin. Single responsibility. Test it like your production depends on it — because it does. I've seen adapters become the single point of failure in a critical payment pipeline. Don't let that be you.

Composite Pattern — When Trees Go to War

You've got a hierarchy: orders, line items, discounts. Each one needs the same operation: calculate total, apply tax, serialize. The Composite pattern lets you treat individual objects and groups uniformly. It's a tree structure where every node can be a leaf or a composite. In Python, this is deceptively simple. You define a common interface, then implement it in both classes. The composite class internally calls the same method on its children. Beautiful recursion. But I've seen teams turn this into a performance disaster. They add hundreds of thousands of nodes, then wonder why a simple sum takes seconds. The WHY is important: Composite shines when you have stable, shallow trees. If your hierarchy runs deep or changes frequently, you're better off with a Visitor pattern or a flat iterator. Measure before you commit. And always cache results if you're recalculating more than once.

State Pattern — When if/elif Becomes a War Crime

You've seen that function. Two hundred lines of if/elif checking a status variable. It works until someone adds a new state and forgets to update every branch. That's how payment confirmations end up in limbo. The State pattern moves each state's behavior into its own class. Instead of checking the state, you delegate to the current state object. It's like having a bouncer that only lets the right people in. In Python, this maps naturally to classes that implement a common protocol. The state machine holds a reference to the current state and swaps it when a transition happens. The catch: you need a clear, enforced transition table. I've had to rewrite messes where states jumped arbitrarily because someone added a shortcut. Document every allowed transition. Use an enum for state names. And for the love of consistent logs, log every state change with a timestamp and caller. You'll thank me during the post-mortem.

Pythonic Design Patterns: First-Class Functions and Protocols

Traditional design patterns often rely on class hierarchies and inheritance, but Python's first-class functions and Protocols offer more flexible, Pythonic alternatives. Instead of implementing the Strategy pattern with abstract base classes, you can pass functions directly. Similarly, Protocols (PEP 544) allow structural subtyping, eliminating the need for explicit inheritance. This section explores how to replace classic patterns with functional and structural approaches.

Consider the Strategy pattern: instead of creating a CompressionStrategy abstract class with subclasses, define a Protocol for compressors and pass callables. This reduces boilerplate and leverages Python's dynamic nature. For example, a Compressor protocol with a compress(data: bytes) -> bytes method can be satisfied by any object with that method, including functions via __call__. This approach is more flexible and testable.

Similarly, the Template Method pattern can be replaced by passing functions for varying steps. Python's functools.partial and closures enable partial application and state encapsulation without classes. This shift reduces coupling and improves readability.

However, caution is needed: overusing Protocols can lead to implicit interfaces that are hard to trace. Use them when you need duck typing with static checking. For simple cases, plain functions suffice. The key is to recognize when a pattern's intent is better served by Python's dynamic features rather than rigid class structures.

from typing import Protocol

class Compressor(Protocol):
    def compress(self, data: bytes) -> bytes: ...

def gzip_compress(data: bytes) -> bytes:
    import gzip
    return gzip.compress(data)

def lzma_compress(data: bytes) -> bytes:
    import lzma
    return lzma.compress(data)

def compress_data(data: bytes, compressor: Compressor) -> bytes:
    return compressor.compress(data)

# Usage
compressed = compress_data(b"hello", gzip_compress)  # Works because functions have __call__

Observer Pattern with asyncio and Signal

The classic Observer pattern often suffers from memory leaks due to strong references and thread-safety issues. Python's asyncio and the built-in signal module (or third-party libraries like blinker) provide robust alternatives. This section demonstrates an async observer using asyncio.Queue and asyncio.Event to decouple publishers from subscribers without blocking.

Instead of maintaining a list of callbacks, use an asyncio.Queue to buffer events. Subscribers await events in a loop, processing them asynchronously. This naturally handles backpressure and avoids callback hell. For thread safety, asyncio runs in a single thread, eliminating race conditions. For inter-process or distributed observers, consider signal (for OS signals) or message queues.

Example: a WeatherStation emits temperature updates. Observers (like a logger or display) subscribe via an async generator. The station pushes updates to a queue, and each observer pulls from its own queue. This pattern is memory-safe because queues are bounded, and subscribers can be cancelled cleanly.

For simpler cases, Python's signal.signal can handle OS-level events, but for custom events, asyncio.Event or asyncio.Condition work well. Avoid storing strong references to observers; use weak references or let the async context manage lifetimes.

import asyncio
from typing import AsyncIterator, Callable

class AsyncObservable:
    def __init__(self):
        self._subscribers = []

    def subscribe(self, queue: asyncio.Queue):
        self._subscribers.append(queue)

    async def emit(self, event):
        for queue in self._subscribers:
            await queue.put(event)

async def observer(name: str, queue: asyncio.Queue):
    while True:
        event = await queue.get()
        print(f"{name} received: {event}")
        queue.task_done()

async def main():
    station = AsyncObservable()
    q1 = asyncio.Queue()
    q2 = asyncio.Queue()
    station.subscribe(q1)
    station.subscribe(q2)
    asyncio.create_task(observer("Logger", q1))
    asyncio.create_task(observer("Display", q2))
    await station.emit("temperature 25C")
    await asyncio.sleep(0.1)

asyncio.run(main())

Strategy Pattern with Protocols and Callables

The Strategy pattern defines a family of algorithms and makes them interchangeable. In Python, you can implement this using Protocols for static type safety or simply pass callables for maximum flexibility. This section compares both approaches and shows how to avoid common pitfalls like tight coupling and excessive boilerplate.

Using a Protocol: define an interface with a method like execute(data: dict) -> dict. Concrete strategies implement this method. This provides clear contracts and IDE support. Alternatively, use a callable: a function or lambda that takes the same arguments. This is simpler but lacks explicit interface documentation.

Example: a payment processing system with strategies for credit card, PayPal, and crypto. Using a Protocol ensures each strategy has a pay(amount: float) -> bool method. With callables, you'd pass lambda amount: .... The Protocol approach is safer for large codebases; callables are fine for small scripts.

A common mistake is storing strategies in a dictionary with string keys, leading to runtime errors. Instead, use an Enum or a registry with type hints. Also, avoid mutable state in strategies; prefer pure functions or immutable objects.

For dynamic strategy selection, use a factory function that returns the appropriate strategy based on configuration. This keeps the client code clean and adheres to the Open/Closed principle.

from typing import Protocol

class PaymentStrategy(Protocol):
    def pay(self, amount: float) -> bool: ...

class CreditCard:
    def pay(self, amount: float) -> bool:
        print(f"Paying {amount} with credit card")
        return True

def paypal_pay(amount: float) -> bool:
    print(f"Paying {amount} with PayPal")
    return True

# Using Protocol
strategy: PaymentStrategy = CreditCard()
strategy.pay(100.0)

# Using callable (function satisfies Protocol via __call__)
strategy2: PaymentStrategy = paypal_pay
strategy2.pay(200.0)