Python Magic Methods — Set Corruption When __hash__ Missing

Every time Python evaluates len(my_list), compares two objects with ==, or prints something with print(), it is secretly delegating that work to a special method buried inside the object's class. This is not magic in the stage-trick sense — it is a precisely defined protocol that makes Python's data model tick. Understanding it is the difference between writing classes that play nicely with the rest of Python's ecosystem and writing classes that feel bolted-on and awkward.

Before magic methods existed as a concept, languages forced you to register callbacks or inherit from a god-class just to make your objects behave like built-in types. Python solved this with a clean contract: implement a double-underscore method with a specific name (hence 'dunder'), and the interpreter will call it at the right moment. The result is that your custom Vector class can support +, len(), slicing, context managers, and even pickling — all without inheriting from anything.

By the end of this article you will understand not just the syntax but the CPython internals that make these calls happen, the subtle ordering rules Python follows when resolving them, the performance traps hiding in __getattr__ and __slots__, how __call__ and __enter__/__exit__ work in production, why __del__ is almost always the wrong answer to resource cleanup, and the patterns senior engineers use in production libraries. You will also have concrete answers for the interview questions that trip up even experienced Python developers.

What Are Magic Methods? The CPython Dispatch Contract

Magic methods are special method names with double underscores on both ends that Python's interpreter calls implicitly when you use certain syntax or built-in operations. The 'magic' is not runtime wizardry — it is a well-defined protocol baked into CPython's bytecode execution and the C-level type structure.

When you write len(obj), CPython calls type(obj).__len__(obj), not obj.__len__() directly. Python looks up the method on the class, not the instance. This distinction is critical: if you try to monkey-patch a dunder onto a single object — obj.__len__ = lambda: 42 — it will have no effect when you call len(obj). The built-in function goes through the type, always.

The same dispatch applies to obj + other: Python looks up __add__ on type(obj), then __radd__ on type(other), then raises TypeError if both return NotImplemented. This two-step lookup is why int + float works — int.__add__ returns NotImplemented for floats, so Python falls back to float.__radd__. The protocol enables cooperation between types that neither owns the other.

A practical guide for choosing when to implement a dunder versus a regular method: - Use a dunder when you want Python syntax (+, len(), str(), ==) to work naturally on your object. - Use a regular named method when you need a descriptive API — add_item() is clearer than __add__ for domain-specific logic. - Use __lt__ and related comparison dunders when you want your objects to work with sorted(), min(), and max(). - Use __call__ when you want an instance to behave like a function — this is more common in production than tutorials suggest.

The key mental model: each dunder is a slot in CPython's type structure (PyTypeObject). If the class defines the dunder, Python fills the slot. If not, the slot is NULL and the built-in raises TypeError. This is also why __getattr__ cannot intercept len() — __getattr__ operates at the Python level, but len() goes through a C-level slot that bypasses Python attribute lookup entirely.

class Vector:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    def __repr__(self):
        return f"Vector({self.x}, {self.y})"

    def __add__(self, other):
        if not isinstance(other, Vector):
            return NotImplemented  # signals: I cannot handle this type
        return Vector(self.x + other.x, self.y + other.y)

    def __radd__(self, other):
        # Called when other.__add__(self) returned NotImplemented
        return self.__add__(other)

    def __len__(self):
        # Euclidean magnitude, truncated to int (len() requires an int)
        return int((self.x**2 + self.y**2)**0.5)


v1 = Vector(3, 4)
v2 = Vector(1, 2)
print(v1 + v2)   # calls Vector.__add__ -> Vector(4, 6)
print(len(v1))   # calls Vector.__len__ -> 5

# Attempting to monkey-patch __len__ on the instance has no effect
v1.__len__ = lambda: 999
print(len(v1))   # still 5 — len() goes through type(v1).__len__, not the instance

__init__ vs __new__: Object Creation Under the Hood

Most developers know __init__ as the constructor, but technically __new__ is the true constructor — it allocates the object. __init__ only initializes an already-created instance. This distinction matters when you need immutable objects or when you subclass immutable types like tuple or str.

__new__ receives the class as its first argument and must return an instance, usually by calling super().__new__(cls). If __new__ returns an instance of a different class, __init__ is NOT called. That is not a bug — it is a deliberate rule: __init__ only runs if __new__ returned an instance of the class being constructed.

In production, you rarely override __new__ unless you need a singleton, a flyweight, or to subclass an immutable type. For 95% of Python classes, __init__ is all you need.

One rule worth memorising: if __new__ returns an instance of a completely different class, __init__ is skipped. This is how some serialisation libraries work — they call __new__ to allocate the shell of an object, then populate attributes directly without going through __init__.

class ImmutablePoint(tuple):
    """A point that IS a tuple — immutable by nature.

    We must use __new__ because tuple is immutable.
    By the time __init__ would run, the tuple's content is already fixed.
    Trying to set values in __init__ would raise an error.
    """

    def __new__(cls, x, y):
        # super().__new__ allocates the tuple with the given content
        instance = super().__new__(cls, (x, y))
        return instance

    # No __init__ needed — content is fixed at allocation time

    def __repr__(self):
        return f"ImmutablePoint(x={self[0]}, y={self[1]})"


pt = ImmutablePoint(3, 4)
print(pt)                          # ImmutablePoint(x=3, y=4)
print(pt[0], pt[1])                # 3 4
print(isinstance(pt, tuple))       # True

# Demonstrates singleton pattern via __new__
class Singleton:
    _instance = None

    def __new__(cls, *args, **kwargs):
        if cls._instance is None:
            cls._instance = super().__new__(cls)
        return cls._instance  # always returns the same object

    def __init__(self, value):
        self.value = value

s1 = Singleton(10)
s2 = Singleton(20)
print(s1 is s2)    # True — same object
print(s1.value)    # 20 — __init__ ran twice on the same instance

__str__ vs __repr__: Which to Use for Debugging and Logging

Both __repr__ and __str__ return string representations, but the contract is different. __repr__ should be unambiguous — ideally a string you could pass to eval() to recreate the object. __str__ should be readable to an end user. Python uses __str__ for print() and str(), and __repr__ for the interactive interpreter and the f-string debug format (f"{obj!r}").

If you define only one, define __repr__. When __str__ is missing, Python falls back to __repr__. The reverse is not true: if __str__ is defined but __repr__ is not, the interactive interpreter and logging frameworks that call repr() still show the default <__main__.MyClass object at 0x...> — completely useless in a production log.

A practical rule from years of on-call experience: if you ship a class to production without __repr__, you will eventually spend time staring at a log file trying to figure out which object was which. It takes about five minutes to implement and saves hours over the lifetime of the service.

For sensitive data — passwords, API keys, personal information — mask the value in __repr__ rather than omitting it entirely. Showing User(id=42, email='a***@example.com') is far more useful for debugging than User(id=42) and still protects the data.

class User:
    def __init__(self, user_id, name, api_key):
        self.user_id = user_id
        self.name = name
        self._api_key = api_key  # sensitive — must not appear in logs

    def __repr__(self):
        # Unambiguous, shows class name and constructor arguments
        # Masks the sensitive api_key field — shows existence but not value
        return (
            f"User(user_id={self.user_id!r}, name={self.name!r}, "
            f"api_key='***')"
        )

    def __str__(self):
        # Human-readable for display — no internal details
        return f"{self.name} (ID: {self.user_id})"


u = User(42, 'Alice', 'secret-key-abc123')
print(repr(u))    # unambiguous, safe to log
print(str(u))     # user-friendly display
print(f"{u!r}")  # same as repr(u)
print(f"{u}")    # same as str(u)

# Fallback behaviour: if __str__ is missing, Python uses __repr__
class MinimalClass:
    def __repr__(self):
        return "MinimalClass()"

obj = MinimalClass()
print(str(obj))   # MinimalClass() — falls back to __repr__

__eq__ and __hash__: The Contract That Keeps Dicts and Sets Consistent

Python's dict and set rely on hash tables. When you look up a key, Python computes its hash (via __hash__) to find the bucket, then checks equality (via __eq__) to confirm the match. The contract is simple: if two objects are equal (__eq__ returns True), their hashes MUST be equal. Break this contract and you corrupt the data structure — objects disappear, duplicates appear, and lookups return wrong results.

There are two distinct failure modes that get conflated:

Failure mode one — __eq__ without __hash__: Python implicitly sets __hash__ to None, making the object unhashable. Any attempt to add it to a set or use it as a dict key raises TypeError: unhashable type immediately. This is the loud, obvious failure.

Failure mode two — mutable hash: You implement both __eq__ and __hash__, but __hash__ is based on a mutable field. The object is inserted into a set. The field changes. Now the hash is different, the object is in the wrong bucket, and it can never be found or removed. The set appears to contain an object it cannot retrieve. This is the silent, dangerous failure — no exception, just corrupted state.

The fix for both is the same: base __hash__ only on immutable fields, and match those fields to what __eq__ uses.

For mutable objects where value-based equality is needed, the right answer is usually: do not implement __hash__ at all (leave it None explicitly), and use a list or a different data structure that does not require hashing. If you need to deduplicate mutable objects, extract an immutable key and deduplicate on that.

class Order:
    def __init__(self, order_id, symbol, quantity):
        self.order_id = order_id
        self.symbol = symbol
        self.quantity = quantity

    def __eq__(self, other):
        if not isinstance(other, Order):
            return NotImplemented
        return self.order_id == other.order_id

    def __hash__(self):
        # Hash only the immutable order_id — matches what __eq__ compares
        # Never hash mutable fields like quantity
        return hash(self.order_id)

    def __repr__(self):
        return f"Order(order_id={self.order_id!r}, qty={self.quantity})"


o1 = Order(1, 'AAPL', 100)
o2 = Order(1, 'AAPL', 200)  # same order_id, different quantity

order_set = {o1, o2}
print(order_set)         # one element — __eq__ and __hash__ agree
print(len(order_set))    # 1

# Demonstrating the mutable-hash silent corruption failure
class BrokenOrder:
    """Do NOT do this — __hash__ based on a mutable field."""
    def __init__(self, order_id):
        self.order_id = order_id

    def __eq__(self, other):
        return self.order_id == other.order_id

    def __hash__(self):
        return hash(self.order_id)  # dangerous if order_id can change

bad = BrokenOrder(1)
bad_set = {bad}
print(bad in bad_set)    # True before mutation

bad.order_id = 99        # mutate the field used in __hash__
print(bad in bad_set)    # False — object is lost in the wrong bucket
print(len(bad_set))      # 1 — the set still 'has' it but cannot find it

__getattr__, __setattr__, and __delattr__: Attribute Access Control and Pitfalls

Python gives you fine-grained control over attribute access via three hooks: __getattr__ (fallback for failed lookups), __setattr__ (every attribute assignment), and __delattr__ (attribute deletion). __getattr__ is called only when normal attribute lookup fails — it is NOT the same as __getattribute__, which intercepts every access.

The difference matters enormously in practice. If you access an attribute that exists, __getattr__ is never called — __getattribute__ handles it. __getattr__ is specifically the fallback of last resort.

The underscore guard in the proxy example below deserves an explicit explanation because engineers routinely remove it thinking it is defensive noise. Without it, accessing self._config inside __getattr__ would trigger __getattr__ again (because _config might not exist yet during the early stages of __init__ before the line self._config = config executes). The guard raises AttributeError immediately for underscore-prefixed names, which tells Python to abort the lookup rather than recurse.

Inside __setattr__, never write self.x = value — that calls __setattr__ again and creates infinite recursion. Always delegate through super().__setattr__(name, value) or object.__setattr__(self, name, value) for attributes that need to bypass the custom logic.

class ConfigProxy:
    """A proxy that delegates attribute reads to a backing dict.

    The underscore guard in __getattr__ is NOT optional.
    During __init__, before self._config is set, any attribute access
    that fails normal lookup will trigger __getattr__. Without the guard,
    accessing self._config inside __getattr__ would recurse indefinitely.
    The guard raises AttributeError immediately for private/internal names,
    which tells Python to abort the lookup cleanly.
    """

    def __init__(self, config: dict):
        # Routes through __setattr__ below, which uses super() for _config
        self._config = config

    def __getattr__(self, name):
        # This is only called when normal lookup FAILS (attribute not found)
        # The underscore guard prevents infinite recursion during __init__
        # and for any internal attribute that may not exist yet
        if name.startswith('_'):
            raise AttributeError(name)
        try:
            return self._config[name]
        except KeyError:
            raise AttributeError(f"Config key '{name}' not found")

    def __setattr__(self, name, value):
        if name == '_config':
            # Use super() for internal attributes to avoid infinite recursion
            # Writing self._config = value here would call __setattr__ again
            super().__setattr__(name, value)
        else:
            self._config[name] = value


cfg = ConfigProxy({'host': 'localhost', 'port': 5432})
print(cfg.host)          # 'localhost' — found in _config via __getattr__
print(cfg.port)          # 5432 — same path
cfg.timeout = 30         # routes through __setattr__ -> stored in _config
print(cfg.timeout)       # 30

try:
    print(cfg.missing_key)
except AttributeError as error:
    print(f"Correctly raised: {error}")

__call__, __enter__, __exit__, and __del__ — The Production Dunders Nobody Teaches

Most tutorials cover __init__, __repr__, and __eq__ and stop there. But three other dunders appear constantly in production Python and deserve explicit coverage.

__call__ makes any instance behave like a function. Any class with __call__ defined can be invoked with parentheses — instance(args). This is how decorators implemented as classes work, how stateful callables maintain configuration, and how middleware layers stay composable without inheritance. In 2026, __call__ is especially common in ML inference pipelines where a model object is callable, and in dependency injection containers that need to produce objects on demand.

__enter__ and __exit__ power the with statement. __enter__ is called at the start of a with block and its return value is bound to the as variable. __exit__ is called at the end, whether the block exits normally or through an exception. __exit__ receives the exception type, value, and traceback — if it returns True, the exception is suppressed; if it returns False or None, the exception propagates. This is the correct pattern for resource management in Python, replacing try/finally in almost every case.

__del__ is the finalizer — it is called when an object's reference count drops to zero. It looks like the right place for cleanup, but it is unreliable in practice: it may not be called if there are reference cycles (CPython's cycle garbage collector handles those separately), it is never guaranteed to run in PyPy, and it is never guaranteed to run at all during interpreter shutdown. The rule is simple: if a resource must be released, use a context manager (__enter__/__exit__) or an explicit close() method. Reserve __del__ for logging or debugging only, never for resource release that must happen.

import time

# --- __call__: making an instance callable ---
class RateLimiter:
    """A callable that enforces a minimum interval between calls.
    __call__ makes the instance usable as a function or decorator.
    """

    def __init__(self, min_interval_seconds: float):
        self._min_interval = min_interval_seconds
        self._last_called = 0.0

    def __call__(self, func):
        """Wraps a function with rate limiting."""
        def wrapper(*args, **kwargs):
            elapsed = time.monotonic() - self._last_called
            if elapsed < self._min_interval:
                time.sleep(self._min_interval - elapsed)
            self._last_called = time.monotonic()
            return func(*args, **kwargs)
        return wrapper

    def __repr__(self):
        return f"RateLimiter(min_interval={self._min_interval}s)"


# --- __enter__ and __exit__: context manager protocol ---
class DatabaseConnection:
    """Manages a database connection lifecycle.
    __enter__ opens the connection; __exit__ closes it regardless of exceptions.
    This is the correct pattern for resource management — not __del__.
    """

    def __init__(self, connection_string: str):
        self._connection_string = connection_string
        self._connection = None

    def __enter__(self):
        print(f"Opening connection to {self._connection_string}")
        self._connection = f"<Connection:{self._connection_string}>"  # simulated
        return self  # bound to the 'as' variable in the with block

    def __exit__(self, exc_type, exc_val, exc_tb):
        print(f"Closing connection (exc_type={exc_type})")
        self._connection = None
        # Return False (or None) to let exceptions propagate
        # Return True only if you intentionally want to suppress them
        return False

    def query(self, sql: str) -> str:
        if self._connection is None:
            raise RuntimeError("Not connected. Use as a context manager.")
        return f"Result of [{sql}] on {self._connection}"

    def __repr__(self):
        return f"DatabaseConnection({self._connection_string!r})"


# __call__ in action
limiter = RateLimiter(0.0)  # zero interval for demo
print(f"RateLimiter is callable: {callable(limiter)}")

@limiter
def fetch_data():
    return "data"

result = fetch_data()
print(f"fetch_data result: {result}")

# __enter__ and __exit__ in action
with DatabaseConnection("postgres://localhost/mydb") as db:
    print(db.query("SELECT 1"))

# __del__ warning — never rely on it for resource cleanup
class LeakyResource:
    def __del__(self):
        # This may never be called in PyPy, during interpreter shutdown,
        # or if there are reference cycles. Use a context manager instead.
        print("__del__ called — do not rely on this for real cleanup")

__slots__: Memory Efficiency, Inheritance Rules, and @dataclass(slots=True)

__slots__ is a declarative way to prevent the creation of a per-instance __dict__, saving memory and speeding up attribute access. Each slot reserves space for an attribute as a direct pointer in a fixed-size array, bypassing the dict lookup entirely.

The memory saving is concrete. A regular class instance in CPython 3.12 carries a __dict__ that takes roughly 200–400 bytes even when empty, plus the standard instance overhead. A slotted instance replaces that with a fixed array of pointers — typically 40–50% smaller per instance. For a service that creates millions of geolocation points or financial tick records, that difference matters.

The complexity cost is real too. Every class in the inheritance hierarchy must define its own __slots__. A subclass that does not define __slots__ will have a __dict__ anyway, defeating the memory saving. Multiple inheritance with __slots__ requires careful coordination — if two parent classes both define non-empty __slots__, the subclass must be designed to avoid layout conflicts.

For private attributes with name mangling, the slot must use the mangled name. A class Foo with self.__price needs the slot named '_Foo__price', not '__price'. This is a common mistake that produces an AttributeError that looks nothing like a slots issue.

In Python 3.10+, @dataclass(slots=True) generates both the class and its __slots__ automatically, handles the mangled names correctly, and avoids most of the manual slot management pitfalls. This is the recommended way to use slots for most teams in 2026.

import sys
from dataclasses import dataclass

# --- Manual __slots__ ---
class Point2D:
    __slots__ = ('x', 'y')

    def __init__(self, x: float, y: float):
        self.x = x
        self.y = y

    def __repr__(self):
        return f"Point2D(x={self.x}, y={self.y})"


# --- @dataclass(slots=True) — Python 3.10+: the modern way ---
@dataclass(slots=True)
class Point2DDataclass:
    x: float
    y: float


# --- Comparison ---
regular_point = Point2D(3.0, 4.0)
slot_point = Point2DDataclass(3.0, 4.0)

print(f"Manual slots instance: {regular_point}")
print(f"Dataclass slots instance: {slot_point}")

# __slots__ prevents arbitrary attribute creation — this is a safety feature
try:
    regular_point.z = 5.0
except AttributeError as error:
    print(f"Slot safety caught it: {error}")

# Without __slots__, this would silently create a new attribute
class RegularPoint:
    def __init__(self, x, y):
        self.x = x
        self.y = y

rp = RegularPoint(3.0, 4.0)
rp.z = 5.0  # silently creates a new attribute — no error
print(f"Regular class allows accidental attr: {rp.z}")

# Private attribute with mangled name in slots
class PricedItem:
    __slots__ = ('name', '_PricedItem__price')  # mangled name required

    def __init__(self, name: str, price: float):
        self.name = name
        self.__price = price  # stored in '_PricedItem__price' slot

    @property
    def price(self):
        return self.__price


item = PricedItem("Widget", 9.99)
print(f"Price via property: {item.price}")