Amazon DocumentDB: When MongoDB Compatibility Bites Back in Production

Here's the truth nobody at AWS wants to shout: Amazon DocumentDB is not MongoDB. It's a separate engine that speaks MongoDB's wire protocol — mostly. I've seen teams migrate their MongoDB 3.6 workloads thinking it's a drop-in replacement, only to have their aggregation pipelines fail silently at 2 AM. The docs say 'compatible.' The reality is a compatibility matrix you need to study like a pilot's pre-flight checklist.

The problem DocumentDB solves is real: managing MongoDB yourself on EC2 is a nightmare. Backups, failover, scaling — it's a full-time job. AWS offers to handle all that, plus replication across three AZs. For many workloads, that trade-off is worth it. But you need to know exactly what you're giving up.

By the end of this, you'll know exactly when to use DocumentDB, how to configure it for production, and — more importantly — the three failure modes that will burn you if you treat it like real MongoDB. You'll walk away with a decision framework, not just syntax.

Why DocumentDB Exists: The Managed MongoDB Nightmare

Running MongoDB on EC2 is like adopting a pet elephant. It's powerful, but you're cleaning up after it constantly. Backups? You script them. Failover? You build a replica set. Patching? You schedule downtime. Amazon DocumentDB takes that elephant and puts it in a zoo — AWS feeds it, cleans its enclosure, and handles the vet visits. You just throw queries at it.

The real win is operational simplicity. DocumentDB replicates your data across three Availability Zones automatically. Storage scales from 10 GB to 64 TB without you lifting a finger. Backups are continuous to S3 with point-in-time recovery. For teams that don't have a dedicated DBA, that's gold.

But here's the catch: you're locked into AWS's implementation. If MongoDB releases a killer feature in 4.2 or 5.0, you won't get it until AWS decides to support it — if ever. DocumentDB currently targets MongoDB 3.6 and 4.0 API compatibility. That's a 5-year-old version as of 2024.

// io.thecodeforge — DevOps tutorial

// Provision a DocumentDB cluster via AWS CLI
aws docdb create-db-cluster \
    --db-cluster-identifier production-cluster \
    --engine docdb \
    --master-username admin \
    --master-user-password 'SuperS3cret!' \
    --vpc-security-group-ids sg-0123456789abcdef0 \
    --db-subnet-group-name my-subnet-group \
    --availability-zones us-east-1a us-east-1b us-east-1c \
    --backup-retention-period 7 \
    --preferred-backup-window 03:00-04:00 \
    --preferred-maintenance-window sun:04:00-sun:05:00

// Create a writer instance (minimum 1)
aws docdb create-db-instance \
    --db-instance-identifier production-writer \
    --db-cluster-identifier production-cluster \
    --db-instance-class db.r5.large \
    --engine docdb

// Add a reader for read scaling
aws docdb create-db-instance \
    --db-instance-identifier production-reader-1 \
    --db-cluster-identifier production-cluster \
    --db-instance-class db.r5.large \
    --engine docdb

echo "Cluster endpoint: production-cluster.cluster-xxxxx.us-east-1.docdb.amazonaws.com:27017"
echo "Reader endpoint: production-cluster.cluster-ro-xxxxx.us-east-1.docdb.amazonaws.com:27017"

Connection Pooling: The Silent Throttle

Every DocumentDB instance has a max connection limit. For db.r5.large, it's 1000. Sounds like a lot until your microservices each open 50 connections. I've seen a single service exhaust the pool at 3 AM during a traffic spike, causing cascading failures across the entire stack.

The fix isn't just increasing the instance size — that's expensive. You need to tune your connection pool. MongoDB drivers default to 100 connections per pool. For DocumentDB, that's aggressive. Drop it to 10-20 per service instance and use connection pooling with a queue.

Also, DocumentDB doesn't support the maxPoolSize option in the same way as MongoDB. The driver will respect it, but DocumentDB's server-side limits are hard. Monitor DatabaseConnections CloudWatch metric and set alarms at 80% of the limit.

// io.thecodeforge — DevOps tutorial

// Node.js MongoDB driver connection string with tuned pool
const uri = 'mongodb://admin:SuperS3cret!@production-cluster.cluster-xxxxx.us-east-1.docdb.amazonaws.com:27017/?ssl=true&ssl_ca_certs=rds-combined-ca-bundle.pem&replicaSet=rs0&readPreference=secondaryPreferred&maxPoolSize=20&minPoolSize=5&waitQueueTimeoutMS=5000';

// Python pymongo example
from pymongo import MongoClient
client = MongoClient(
    'mongodb://admin:SuperS3cret!@production-cluster.cluster-xxxxx.us-east-1.docdb.amazonaws.com:27017/',
    ssl=True,
    ssl_ca_certs='rds-combined-ca-bundle.pem',
    replicaSet='rs0',
    readPreference='secondaryPreferred',
    maxPoolSize=20,
    minPoolSize=5,
    waitQueueTimeoutMS=5000
)

# Java driver (MongoClientSettings)
MongoClientSettings settings = MongoClientSettings.builder()
    .applyConnectionString(new ConnectionString(
        "mongodb://admin:SuperS3cret!@production-cluster.cluster-xxxxx.us-east-1.docdb.amazonaws.com:27017/?ssl=true&ssl_ca_certs=rds-combined-ca-bundle.pem&replicaSet=rs0&readPreference=secondaryPreferred"))
    .applyToConnectionPoolSettings(builder ->
        builder.maxSize(20)
               .minSize(5)
               .maxWaitTime(5000, TimeUnit.MILLISECONDS))
    .build();

Aggregation Pipeline: What Works and What Doesn't

DocumentDB supports a subset of MongoDB's aggregation pipeline. The basics work: $match, $group, $sort, $project, $limit, $skip. But the moment you reach for $lookup with a pipeline, $graphLookup, $facet, or $bucket, you're in unsupported territory. The engine won't error — it'll silently return empty or incomplete results.

I've debugged a production issue where a $lookup with let and pipeline variables returned zero documents. No error, no warning. The team spent two days thinking it was a data problem. The fix was to split the aggregation into two queries and join in application code.

Here's the rule: if your aggregation uses anything beyond the basic stages, test it on DocumentDB before deploying. The official compatibility docs list supported stages, but they miss edge cases. For example, $addFields works, but $set (alias) doesn't.

// io.thecodeforge — DevOps tutorial

// MongoDB aggregation that works on DocumentDB
db.orders.aggregate([
    { $match: { status: "shipped" } },
    { $group: { _id: "$customerId", total: { $sum: "$amount" } } },
    { $sort: { total: -1 } },
    { $limit: 10 }
])

// MongoDB aggregation that FAILS silently on DocumentDB
db.orders.aggregate([
    { $lookup: {
        from: "customers",
        let: { custId: "$customerId" },
        pipeline: [
            { $match: { $expr: { $eq: ["$_id", "$$custId"] } } },
            { $project: { name: 1 } }
        ],
        as: "customer"
    }}
])
// Returns empty array for 'customer' field — no error!

// Workaround: two queries in application code
const orders = await db.collection('orders').find({ status: 'shipped' }).toArray();
const customerIds = orders.map(o => o.customerId);
const customers = await db.collection('customers').find({ _id: { $in: customerIds } }).toArray();
const customerMap = new Map(customers.map(c => [c._id, c]));
const result = orders.map(o => ({ ...o, customer: customerMap.get(o.customerId) }));

Indexing Strategy: The Write Performance Trap

DocumentDB uses a different storage engine than MongoDB. It's built on AWS's own distributed storage, not WiredTiger. That means index behavior differs. Specifically, DocumentDB's indexes are B-tree based and don't support partial indexes, sparse indexes, or TTL indexes. If you rely on TTL to auto-delete expired documents, you'll need a different approach.

More critically, every index you add slows down writes. On MongoDB, the impact is linear. On DocumentDB, it's worse because the storage layer adds latency. I've seen a 4-index collection on a db.r5.large instance handle 500 writes/sec. Adding a fifth index dropped it to 200 writes/sec. The fix: keep indexes minimal. Only index fields you query on, and use compound indexes instead of multiple single-field indexes.

Another gotcha: DocumentDB doesn't support background index creation. All index builds block writes. Schedule them during maintenance windows.

// io.thecodeforge — DevOps tutorial

// Check existing indexes
db.collection('orders').getIndexes()

// Create a compound index (preferred over multiple single-field indexes)
db.collection('orders').createIndex(
    { customerId: 1, createdAt: -1 },
    { name: "customerId_createdAt" }
)

// This will FAIL on DocumentDB (partial index not supported)
db.collection('orders').createIndex(
    { status: 1 },
    { partialFilterExpression: { status: { $exists: true } } }
)
// Error: "The option 'partialFilterExpression' is not supported"

// Workaround: index the field anyway, accept extra index entries

// Monitor index usage
db.collection('orders').aggregate([
    { $indexStats: {} }
])
// Note: $indexStats may not return all data on DocumentDB; use CloudWatch metrics instead

Backup and Restore: The Hidden Costs

DocumentDB automatically backs up your cluster to S3 with point-in-time recovery within your backup retention window (1-35 days). That's great. But restoring a cluster takes time — up to several hours for multi-TB databases. And you can't restore to an existing cluster; you get a new one. That means DNS changes, connection string updates, and potential downtime.

Another hidden cost: backups consume storage I/O. During backup windows, you might see increased latency. Schedule backups during off-peak hours and monitor BackupRetentionPeriod and BackupStorageUsed metrics.

For cross-region disaster recovery, DocumentDB doesn't support cross-region snapshots natively. You have to manually copy snapshots to another region using AWS CLI or console. That's an operational burden.

// io.thecodeforge — DevOps tutorial

// Create a manual snapshot (faster than point-in-time restore)
aws docdb create-db-cluster-snapshot \
    --db-cluster-snapshot-identifier pre-deploy-snapshot \
    --db-cluster-identifier production-cluster

// Restore from snapshot (creates a new cluster)
aws docdb restore-db-cluster-from-snapshot \
    --db-cluster-identifier production-cluster-restored \
    --snapshot-identifier pre-deploy-snapshot \
    --vpc-security-group-ids sg-0123456789abcdef0 \
    --db-subnet-group-name my-subnet-group

// Copy snapshot to another region for DR
aws docdb copy-db-cluster-snapshot \
    --source-db-cluster-snapshot-identifier arn:aws:rds:us-east-1:123456789012:cluster-snapshot:pre-deploy-snapshot \
    --target-db-cluster-snapshot-identifier pre-deploy-snapshot-dr \
    --source-region us-east-1 \
    --region us-west-2

echo "Restore complete. New cluster endpoint: production-cluster-restored.cluster-xxxxx.us-west-2.docdb.amazonaws.com:27017"

When Not to Use DocumentDB: The Honest Assessment

DocumentDB is not a universal MongoDB replacement. Avoid it if:

1. You need MongoDB 4.0+ features like transactions, change streams, or $graphLookup. DocumentDB's transaction support is limited to single-document atomicity — no multi-document transactions.
2. Your workload is write-heavy with high throughput. DocumentDB's write performance degrades under sustained load due to its journaling mechanism. MongoDB on provisioned IOPS SSDs can outperform it.
3. You rely on MongoDB-specific tools like Compass, Atlas Search, or Ops Manager. They won't work with DocumentDB.
4. You need cross-region active-active replication. DocumentDB only supports single-region with read replicas in the same region.

For read-heavy, moderate-write workloads that don't need cutting-edge MongoDB features, DocumentDB is a solid choice. The operational savings often outweigh the feature gaps. But if you're building a new project and expect to need MongoDB's full feature set, just use MongoDB Atlas on AWS. It's more expensive but you get the real thing.

// io.thecodeforge — DevOps tutorial

// Decision matrix for DocumentDB vs MongoDB Atlas
const workload = {
    writeThroughput: 2000, // writes/sec
    readThroughput: 5000,  // reads/sec
    needTransactions: true,
    needChangeStreams: false,
    budget: 'moderate'
};

if (workload.needTransactions || workload.needChangeStreams) {
    console.log('Use MongoDB Atlas — DocumentDB lacks these features');
} else if (workload.writeThroughput > 1000) {
    console.log('Consider MongoDB Atlas or self-managed MongoDB on EC2 with provisioned IOPS');
} else if (workload.budget === 'low') {
    console.log('DocumentDB is cost-effective for moderate workloads');
} else {
    console.log('MongoDB Atlas for full compatibility and less operational headache');
}