Python Pickle — __reduce__() Remote Code Execution

Every serious Python application eventually hits the same wall: you spend time computing something valuable — a trained machine learning model, a complex graph structure, a parsed configuration tree — and then your program ends and all of it vanishes. The next run starts from scratch, wasting time and resources. This is one of the most quietly expensive problems in Python development, and most beginners don't realise there's a clean, built-in solution sitting right in the standard library.

The pickle module solves this by giving you object serialization: the ability to convert any Python object into a byte stream that can be written to disk, stored in a database, or sent across a network — and then deserialized back into an identical live object. Unlike writing to a CSV or JSON file, pickle doesn't care what shape your data is in. It handles nested objects, custom class instances, lambda functions, and even circular references without you lifting a finger.

By the end of this article you'll understand exactly how pickle works under the hood, when it's the right tool versus when you should reach for JSON or shelve, how to safely serialize and deserialize complex Python objects including class instances, and — critically — the security trap that catches even experienced developers off guard. You'll walk away with patterns you can drop into real projects immediately.

What is pickle Module in Python?

pickle is Python's built-in serialization module. It converts any Python object into a byte stream that can be saved to disk or sent over a network, then reconstructed later. The key advantage over formats like JSON or CSV is that pickle can handle arbitrary Python objects — including custom class instances, nested structures, and even circular references — without you writing any conversion code.

Here's the simplest possible example: serialize a dictionary to a file, then read it back.

import pickle

# Serialize a dict to a file
data = {'name': 'model_v1', 'accuracy': 0.95, 'layers': [128, 64, 10]}
with open('model.pkl', 'wb') as f:
    pickle.dump(data, f)

# Deserialize it back
with open('model.pkl', 'rb') as f:
    restored = pickle.load(f)

print(restored == data)  # True
print(restored['name'])  # model_v1

How to Serialize and Deserialize Objects with pickle

The two core functions are pickle.dump() to write an object to a file, and pickle.load() to read it back. You can also use pickle.dumps() to get bytes and pickle.loads() from bytes. Always open your files in binary mode: 'wb' for writing, 'rb' for reading. The protocol argument controls the binary format version.

Protocol versions range from 0 (text-based, readable) to 5 (binary, with out-of-band data support). Specifying protocol=pickle.HIGHEST_PROTOCOL gives you the best speed and smallest size, but may not be compatible with older Python versions.

import pickle

# Pickle a complex object with different protocols
data = {"name": "model", "params": [1.2, 3.4], "hyperparams": {"lr": 0.001}}

with open('data_proto4.pkl', 'wb') as f:
    pickle.dump(data, f, protocol=4)

with open('data_proto5.pkl', 'wb') as f:
    pickle.dump(data, f, protocol=5)

# Read back (protocol auto-detected)
with open('data_proto4.pkl', 'rb') as f:
    restored = pickle.load(f)
print(restored)

Customizing Serialization with __getstate__ and __setstate__

Not every object attribute is serializable by default. File handles, network connections, and database sessions need special handling. Override __getstate__ to return a dict of picklable state, and __setstate__ to restore resources after deserialization.

A common pattern: your class holds a database connection that cannot be pickled. You exclude it in __getstate__ and recreate it in __setstate__.

class DatabaseConnection:
    def __init__(self, dsn):
        self.dsn = dsn
        self.conn = self._create_connection()

    def _create_connection(self):
        return f"Connected to {self.dsn}"

    def __getstate__(self):
        return {'dsn': self.dsn}

    def __setstate__(self, state):
        self.__dict__.update(state)
        self.conn = self._create_connection()

import pickle
conn = DatabaseConnection("db://host:port/db")
with open('conn.pkl', 'wb') as f:
    pickle.dump(conn, f)

with open('conn.pkl', 'rb') as f:
    loaded = pickle.load(f)
print(loaded.conn)

Security Risks and Safe Usage

The biggest danger with pickle is that pickle.load() can execute arbitrary code. This happens through the __reduce__ protocol, which allows objects to specify any callable and arguments to reconstruct themselves. A malicious pickle can run os.system, open network sockets, or delete files.

The only safe rule: never unpickle data from an untrusted source. If you must accept external input, use a restricted Unpickler that whitelists allowed classes. Even then, consider using a separate process or container for isolation.

Performance Comparison: pickle vs JSON vs dill

pickle is generally faster than JSON for Python-native objects because it uses a binary format and can encode arbitrary types without additional conversion. However, for simple dicts and lists, JSON can be faster due to optimized C implementations. dill extends pickle and adds support for lambdas and closures, but comes with a size and speed overhead.

Here's a rough benchmark: pickling a 10MB list of strings is about 3x faster than JSON with protocols 4 or 5. The file size is also ~20% smaller. But if you need interoperability, JSON wins.

Frequently Asked Questions