Python Protocols: Structural Subtyping and Duck Typing Mastery

Imagine you're building a payment processing system that handles credit cards, PayPal, and cryptocurrencies. Each payment method has a pay(amount) method, but they don't share a common base class. How do you ensure type safety without forcing inheritance? Enter Python Protocols.

Introduced in Python 3.8 via PEP 544, Protocols provide a way to define structural subtyping—also known as duck typing—in static type checking. Instead of requiring explicit inheritance, a protocol specifies a set of methods and attributes that a class must have. Any class that provides those methods is considered a subtype of the protocol, even if it doesn't inherit from it.

This is a game-changer for large codebases. You can write generic functions that accept any object with the right behavior, while still catching errors at type-check time. For example, you can define a PaymentProcessor protocol with a pay method, and then use it in a function that processes payments. Any class with a pay method—whether it's CreditCard, PayPal, or CryptoWallet—will be accepted.

In this tutorial, you'll learn how to define and use protocols, the difference between nominal and structural subtyping, runtime checking, and best practices. We'll also dive into a real-world production incident where a missing protocol caused a costly outage, and provide a debugging guide for common issues.

What Are Protocols?

Protocols are a way to define structural subtyping in Python. Unlike nominal subtyping (where a class must explicitly inherit from another), structural subtyping considers two types equivalent if they have the same structure—i.e., the same methods and attributes. This is the essence of duck typing: 'If it walks like a duck and quacks like a duck, it's a duck.'

In Python, you can define a protocol by creating a class that inherits from typing.Protocol. The methods you define in the protocol are the required interface. Any class that implements those methods (with matching signatures) is considered a subtype of the protocol, even if it doesn't inherit from it.

For example, consider a simple Drawable protocol:

```python from typing import Protocol

class Drawable(Protocol): def draw(self) -> None: ... ```

Now, any class with a draw method that takes no arguments and returns None is considered a Drawable. This allows you to write functions that accept any drawable object without forcing a common base class.

Protocols are especially useful in large codebases where you want to decouple modules. Instead of importing a base class from a shared module, you can define a protocol locally and rely on structural compatibility. This reduces dependencies and makes your code more modular.

from typing import Protocol

class Drawable(Protocol):
    def draw(self) -> None: ...

class Circle:
    def draw(self) -> None:
        print("Drawing a circle")

class Square:
    def draw(self) -> None:
        print("Drawing a square")

def render(shape: Drawable) -> None:
    shape.draw()

render(Circle())  # OK
render(Square())  # OK
render(42)        # Type checker error: int is not Drawable

Defining Protocols with Method Signatures

When defining a protocol, you must specify the exact method signatures that implementing classes should have. This includes parameter names, types, and return types. The protocol body typically contains only docstrings or ellipsis (...) as placeholders.

Let's define a more complex protocol for a DataFetcher that fetches data from a URL:

```python from typing import Protocol, Any

class DataFetcher(Protocol): def fetch(self, url: str, timeout: float = 10.0) -> dict[str, Any]: ... ```

Any class that has a fetch method with the same signature (or a compatible one) is a DataFetcher. Note that the protocol method can have default arguments; implementing classes can also have additional parameters as long as they are optional.

You can also define class methods, static methods, and properties in protocols. For example:

```python from typing import Protocol

class Configurable(Protocol): @classmethod def from_config(cls, config: dict) -> 'Configurable': ... @property def name(self) -> str: ... ```

Implementing classes must provide these as class methods and properties respectively.

One important nuance: protocols are checked structurally only at type-check time (e.g., with mypy). At runtime, they are just regular classes. To enable runtime structural checks (like isinstance), you need to decorate the protocol with @runtime_checkable.

from typing import Protocol, runtime_checkable

@runtime_checkable
class DataFetcher(Protocol):
    def fetch(self, url: str, timeout: float = 10.0) -> dict: ...

class HTTPFetcher:
    def fetch(self, url: str, timeout: float = 5.0) -> dict:
        # Simulate fetching
        return {"data": "example"}

class MockFetcher:
    def fetch(self, url: str) -> dict:
        return {"data": "mock"}

print(isinstance(HTTPFetcher(), DataFetcher))  # True
print(isinstance(MockFetcher(), DataFetcher))  # True (timeout is optional)
print(isinstance(42, DataFetcher))            # False

Runtime Checkable Protocols

By default, protocols are not runtime checkable. That means isinstance(obj, MyProtocol) will always return False unless you decorate the protocol with @runtime_checkable. This decorator enables structural checks at runtime, but it has limitations: it only checks for the existence of methods, not their signatures.

For example:

```python from typing import Protocol, runtime_checkable

@runtime_checkable class Flyable(Protocol): def fly(self) -> None: ...

class Bird: def fly(self) -> None: print("Flying")

class Airplane: def fly(self, speed: int) -> None: print(f"Flying at {speed}")

print(isinstance(Bird(), Flyable)) # True print(isinstance(Airplane(), Flyable)) # True (signature not checked) ```

As you can see, Airplane has a fly method with a different signature, but isinstance still returns True. This is because @runtime_checkable only checks that the method exists, not its parameter list. For strict signature checking, you must rely on static type checkers like mypy.

When should you use @runtime_checkable? Use it when you need runtime type checking, such as in serialization or dependency injection frameworks. However, be aware of the performance cost and the lack of signature verification.

Another caveat: @runtime_checkable works by checking if the object's class has all the protocol's methods. It does not check for attributes defined in the protocol (e.g., x: int). For attribute checking, you need to implement a custom __instancecheck__ or use a library like typing_extensions.

from typing import Protocol, runtime_checkable

@runtime_checkable
class HasName(Protocol):
    name: str

class Person:
    def __init__(self, name: str):
        self.name = name

print(isinstance(Person("Alice"), HasName))  # False! Attribute not checked

# To check attributes, you need a custom approach
class StrictHasName(Protocol):
    @property
    def name(self) -> str: ...

@runtime_checkable
class RuntimeHasName(StrictHasName):
    pass

print(isinstance(Person("Alice"), RuntimeHasName))  # True (property exists)

Protocols vs Abstract Base Classes

Both protocols and ABCs define interfaces, but they differ in philosophy and usage. ABCs use nominal subtyping: a class must explicitly inherit from the ABC to be considered a subtype. Protocols use structural subtyping: any class with the right methods is a subtype, regardless of inheritance.

ABCs are useful when you want to enforce a class hierarchy and provide default implementations. For example, the collections.abc module defines ABCs like Iterable and Sequence. These ABCs also provide mixin methods (e.g., __contains__ from Iterable).

Protocols, on the other hand, are lightweight and don't provide any implementation. They are ideal for duck typing scenarios where you want to accept any object that behaves a certain way.

Here's a comparison:

In practice, you can use both together. For example, you can define an ABC that also implements a protocol. But generally, prefer protocols for interfaces that are meant to be implemented by unrelated classes.

When should you choose ABC over protocol? If you need to provide shared behavior (like a template method) or if you want to enforce a class hierarchy (e.g., for serialization). Otherwise, protocols are more flexible.

from abc import ABC, abstractmethod
from typing import Protocol

# ABC approach
class ShapeABC(ABC):
    @abstractmethod
    def area(self) -> float: ...

class Circle(ShapeABC):
    def __init__(self, radius):
        self.radius = radius
    def area(self) -> float:
        return 3.14 * self.radius ** 2

# Protocol approach
class ShapeProtocol(Protocol):
    def area(self) -> float: ...

class Square:
    def __init__(self, side):
        self.side = side
    def area(self) -> float:
        return self.side ** 2

def print_area(shape: ShapeProtocol) -> None:
    print(f"Area: {shape.area()}")

print_area(Circle(5))  # OK
print_area(Square(4))  # OK

Generic Protocols

Protocols can be generic, meaning they can work with multiple types. This is useful when you want to define a protocol that operates on a type parameter, like a Comparable protocol that works with any type that supports comparison.

To define a generic protocol, use typing.Generic and a type variable:

```python from typing import Protocol, TypeVar

T = TypeVar('T')

class Comparable(Protocol[T]): def __lt__(self, other: T) -> bool: ... ```

Now, any class that has a __lt__ method that takes an argument of the same type is a Comparable. For example, int and str are Comparable because they have __lt__.

You can also use generic protocols with multiple type variables:

```python from typing import Protocol, TypeVar

K = TypeVar('K') V = TypeVar('V')

class KeyValueStore(Protocol[K, V]): def get(self, key: K) -> V: ... def set(self, key: K, value: V) -> None: ... ```

Generic protocols are powerful for building type-safe abstractions. For instance, you can write a function that sorts any list of Comparable items:

``python def sort_items(items: list[Comparable]) -> list[Comparable]: return sorted(items) ``

This function will work with lists of integers, strings, or any custom class that implements __lt__.

from typing import Protocol, TypeVar

T = TypeVar('T')

class Comparable(Protocol[T]):
    def __lt__(self, other: T) -> bool: ...

def sort_items(items: list[Comparable]) -> list[Comparable]:
    return sorted(items)

print(sort_items([3, 1, 2]))  # [1, 2, 3]
print(sort_items(['c', 'a', 'b']))  # ['a', 'b', 'c']

# Custom class
class Person:
    def __init__(self, name: str, age: int):
        self.name = name
        self.age = age
    def __lt__(self, other: 'Person') -> bool:
        return self.age < other.age
    def __repr__(self):
        return f"Person({self.name}, {self.age})"

people = [Person("Alice", 30), Person("Bob", 25), Person("Charlie", 35)]
print(sort_items(people))  # [Person(Bob,25), Person(Alice,30), Person(Charlie,35)]

Best Practices and Common Pitfalls

When using protocols, follow these best practices to avoid common mistakes:

1. Keep protocols focused: Each protocol should represent a single responsibility. Avoid bloated protocols with many methods.
2. Use @runtime_checkable sparingly: Only use it when you need runtime type checking. Overuse can lead to performance issues and false positives.
3. Prefer static type checking: Use mypy in strict mode to catch protocol violations at development time.
4. Document protocol expectations: Clearly document what methods and attributes are expected, including signatures.
5. Avoid mutable class variables in protocols: Protocols can define class variables, but implementing classes may accidentally share state.

Common pitfalls:

• Missing methods: The most common error is forgetting to implement a required method. Mypy will catch this.
• Incorrect signatures: Even if the method exists, a different signature can cause runtime errors. Mypy checks signatures.
• Assuming runtime checking works without decorator: Without @runtime_checkable, isinstance returns False.
• Overusing protocols: Not every interface needs a protocol. Sometimes a simple callable type hint is enough.

Let's look at an example of a common mistake: using a protocol with a mutable class variable.

```python from typing import Protocol

class Counter(Protocol): count: int = 0 # Mutable class variable

class MyCounter: count: int = 0

c1 = MyCounter() c2 = MyCounter() c1.count += 1 print(c2.count) # 0 (instance variable shadows class variable) ```

This can lead to confusion. Instead, define count as a property or method.

from typing import Protocol, runtime_checkable

# Good: focused protocol
@runtime_checkable
class Logger(Protocol):
    def log(self, message: str) -> None: ...

# Bad: too many methods
class Everything(Protocol):
    def log(self, msg: str) -> None: ...
    def save(self) -> None: ...
    def load(self) -> None: ...
    def delete(self) -> None: ...

# Use static checking
# mypy --strict will catch missing methods

class ConsoleLogger:
    def log(self, message: str) -> None:
        print(f"LOG: {message}")

def process(logger: Logger) -> None:
    logger.log("Processing...")

process(ConsoleLogger())  # OK
process(42)  # mypy error