Siddhant Deval
Siddhant DevalAuthor
Senior Full-Stack Engineer·Jul 12, 2026·17 min read

Document & NoSQL Databases: MongoDB & DynamoDB in Production

Document databases trade join-ability for write flexibility. This article covers the embedding vs. referencing decision, DynamoDB single-table design, event sourcing with Streams, schema versioning discipline, cost modeling at scale, and GDPR-compliant erasure strategies for denormalized document stores.

Document & NoSQL Databases: MongoDB & DynamoDB in Production

Selecting the right database is a foundational architectural decision; data gravity ultimately dictates the scalability and resilience of an application — choose the paradigm first, the product second. The promise of document databases is seductive: no rigid schema, rapid iteration, flexible structure. The broken pattern is believing that "schemaless" means "schema-free." Every document store develops an implicit schema — it is just encoded in the application layer where no database constraint can enforce it, and every migration must be written by hand. This article shows you how to use document databases correctly: when the flexibility earns its cost, and when you are simply moving the rigor problem from the database to the application.

1. The Document Model Mental Model

A document database stores each entity as a self-contained JSON document. The design decision that dominates everything else is: embed or reference?
javascript
// ❌ Broken pattern: always embedding — leads to unbounded document growth
// A blog post that embeds all comments directly
{
  "_id": "post_abc",
  "title": "Understanding MVCC",
  "comments": [          // unbounded array — grows without limit
    { "author": "alice", "text": "Great post!" },
    { "author": "bob", "text": "Very helpful!" },
    // ... 50,000 comments later — document hits 16MB limit and all writes fail
  ]
}

// ✅ Correct: reference for high-cardinality, frequently-updated child entities
// Post document — stays small, never bloats
{ "_id": "post_abc", "title": "Understanding MVCC", "commentCount": 50000 }

// Comment document — separate collection, independently queryable
{ "_id": "comment_xyz", "postId": "post_abc", "author": "alice", "text": "Great post!" }

1.1 Embedding vs. Referencing Decision Matrix

DimensionEmbedReference
CardinalityOne-to-few (< 100 items)One-to-many, one-to-millions
Update frequencyChild rarely changes independentlyChild updated independently of parent
Query patternParent and child always fetched togetherChild queried independently or in batches
Atomicity needSingle-document atomicity sufficientMulti-document transactions needed
Document size growthBounded, predictableParent stays small
Mental Model Check
Embed when the child entity has no life outside the parent — an order's line items, a profile's address list. Reference when the child entity is independently queryable, independently updated, or unbounded in count.

2. Data Modeling

2.1 MongoDB Schema Versioning

The "schemaless" label is the most dangerous myth in document databases. Without explicit versioning, schema drift turns a document collection into an archaeology site:
javascript
// ❌ Broken: no schema version — reading old documents requires defensive checks in every query
const user = await db.users.findOne({ _id: userId })
// Is user.address a string? An object? An array? It changed 3 times over 2 years.
const city = user.address?.city ?? user.city ?? user.location?.city ?? 'Unknown'

// ✅ Correct: explicit _schemaVersion field — readers branch on version, migrations are explicit
// v1 document (2024)
{ "_id": "user_1", "_schemaVersion": 1, "address": "123 Main St, NYC" }

// v2 document (2025 — structured address)
{ "_id": "user_2", "_schemaVersion": 2, "address": { "street": "123 Main St", "city": "NYC" } }

// v3 document (2026 — multi-address support)
{ "_id": "user_3", "_schemaVersion": 3, "addresses": [{ "type": "home", "city": "NYC" }] }

// Migration script: run as background job, not a blocking migration
await db.users.updateMany(
  { _schemaVersion: { $lt: 3 } },
  [{ $set: { /* transform to v3 shape */ } }]
)

2.2 DynamoDB Single-Table Design

DynamoDB requires all access patterns to be defined before the first write. The key insight is that a single table, with carefully chosen partition and sort keys, can serve multiple entity types:
typescript
// Table: AppTable
// Access patterns to support:
//   1. Get user by ID
//   2. Get all orders for a user
//   3. Get order by order ID
//   4. Get all order items for an order

// Item shapes in a single table:
type UserItem = {
  PK: `USER#${string}`      // PK: USER#userId
  SK: `USER#${string}`      // SK: USER#userId (same — point lookup)
  entityType: 'USER'
  email: string
  name: string
}

type OrderItem = {
  PK: `USER#${string}`      // PK: USER#userId — enables "get all orders for user"
  SK: `ORDER#${string}`     // SK: ORDER#orderId — enables "get order by ID"
  entityType: 'ORDER'
  total: number
  status: 'pending' | 'shipped' | 'delivered'
}

type OrderLineItem = {
  PK: `ORDER#${string}`     // PK: ORDER#orderId — enables "get all items for order"
  SK: `ITEM#${string}`      // SK: ITEM#productId
  entityType: 'ORDER_ITEM'
  quantity: number
  unitPrice: number
}
Performance / Safety Warning
Retrofitting access patterns onto a DynamoDB table with an existing key design is expensive — it often requires a full table scan and re-write via DynamoDB Streams or a batch migration. Design the key schema for ALL access patterns before the first write hits production.

2.3 The 16MB Document Limit

javascript
// ❌ Anti-pattern: audit log embedded in a document — hits 16MB silently in production
{
  "_id": "user_42",
  "name": "Alice",
  "auditLog": [    // grows indefinitely — each login, each update appends here
    { "action": "login", "timestamp": "2026-01-01T00:00:00Z" },
    { "action": "update_email", "timestamp": "2026-01-02T00:00:00Z" },
    // ... 8,000 events later — write fails with BSONObjectTooLarge
  ]
}

// ✅ Correct: separate audit_events collection — unbounded data never embedded
db.audit_events.insertOne({
  userId: "user_42",
  action: "login",
  timestamp: new Date(),
  metadata: { ip: "1.2.3.4", userAgent: "..." }
})

3. MongoDB Aggregation Pipeline

The aggregation pipeline is MongoDB's equivalent of SQL's GROUP BY, JOIN, and window functions — but sequential, document-stream-based:
javascript
// Find top 5 product categories by total revenue in the last 30 days
db.orders.aggregate([
  // Stage 1: filter to recent orders
  { $match: {
    status: 'delivered',
    createdAt: { $gte: new Date(Date.now() - 30 * 24 * 60 * 60 * 1000) }
  }},

  // Stage 2: unwind the items array into individual documents
  { $unwind: '$items' },

  // Stage 3: join to products collection to get the category
  { $lookup: {
    from: 'products',
    localField: 'items.productId',
    foreignField: '_id',
    as: 'product'
  }},
  { $unwind: '$product' },

  // Stage 4: group by category, sum revenue
  { $group: {
    _id: '$product.category',
    totalRevenue: { $sum: { $multiply: ['$items.quantity', '$items.unitPrice'] } },
    orderCount: { $sum: 1 }
  }},

  // Stage 5: sort and limit
  { $sort: { totalRevenue: -1 } },
  { $limit: 5 }
])
Architectural Note
MongoDB's $lookup (join) is expensive relative to a Postgres JOIN — it does not use indexes from the foreign collection efficiently. For join-heavy reporting workloads run at high frequency, Postgres is faster. Reserve aggregation pipelines for analytical queries run on a schedule or on demand, not for every page render.

4. DynamoDB Streams — Native Event Sourcing

typescript
// DynamoDB Stream record — triggered on every INSERT, MODIFY, or REMOVE
type DynamoDBStreamRecord = {
  eventName: 'INSERT' | 'MODIFY' | 'REMOVE'
  dynamodb: {
    Keys: { PK: { S: string }; SK: { S: string } }
    NewImage?: Record<string, AttributeValue>
    OldImage?: Record<string, AttributeValue>
  }
}

// Lambda function that processes order status changes
export const handler = async (event: DynamoDBStreamEvent) => {
  for (const record of event.Records) {
    if (record.eventName !== 'MODIFY') continue

    const newStatus = record.dynamodb.NewImage?.status?.S
    const oldStatus = record.dynamodb.OldImage?.status?.S

    if (oldStatus !== 'pending' || newStatus !== 'shipped') continue

    // Trigger fulfillment notification without a separate event bus
    await ses.sendEmail({ /* ... */ })
    await sqs.sendMessage({ /* ... fulfillment queue */ })
  }
}
Pro Tip & Optimization
DynamoDB Streams remove the need for a separate Kafka or SQS bus for document-mutation triggers. The stream is ordered per partition key, retained for 24 hours, and can fan out to up to 2 Lambda consumers. For event volumes exceeding Lambda concurrency limits, route through Kinesis Data Streams.

5. Atlas Vector Search — MongoDB as a Combined Store

For teams already running MongoDB who need RAG search at moderate scale (under 5M vectors):
javascript
// Create a vector search index via Atlas Search UI or API
{
  "name": "article_embeddings_index",
  "type": "vectorSearch",
  "definition": {
    "fields": [{ "type": "vector", "path": "embedding", "numDimensions": 1536, "similarity": "cosine" }]
  }
}

// Query: semantic similarity search combined with a metadata filter
db.articles.aggregate([
  { $vectorSearch: {
    index: "article_embeddings_index",
    path: "embedding",
    queryVector: [/* 1536-dimension query vector */],
    numCandidates: 150,
    limit: 5,
    filter: { category: "backend", publishedAt: { $gte: new Date("2025-01-01") } }
  }},
  { $project: { title: 1, summary: 1, score: { $meta: "vectorSearchScore" } }}
])

6. Cost Model at Scale

ScenarioDynamoDB On-DemandDynamoDB ProvisionedMongoDB Atlas (M30)
1M reads/day + 100K writes/day~$2.10/day~$0.40/day (pre-provisioned)~$6/day (flat)
10M reads/day + 1M writes/day~$21/day~$4/day~$6/day (flat — same tier)
100M reads/day + 10M writes/day~$210/day~$40/day~$40/day (M50 tier)
Crucial Requirement
DynamoDB's on-demand pricing is convenient but expensive at sustained high throughput. Switch to provisioned capacity with auto-scaling once your traffic is predictable — it is typically 3–8× cheaper. MongoDB Atlas pricing is compute-based (instance tier), so cost is flat until you upgrade the tier.

7. Security

7.1 DynamoDB Fine-Grained Access Control

json
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": ["dynamodb:GetItem", "dynamodb:PutItem", "dynamodb:UpdateItem"],
    "Resource": "arn:aws:dynamodb:us-east-1:123456789:table/AppTable",
    "Condition": {
      "ForAllValues:StringEquals": {
        "dynamodb:LeadingKeys": ["${cognito-identity.amazonaws.com:sub}"]
      }
    }
  }]
}
This IAM condition restricts each Cognito-authenticated user to only read/write DynamoDB items where the partition key matches their own Cognito identity — enforced at the AWS API layer, not the application layer.

7.2 MongoDB Atlas Field-Level Encryption

typescript
// Client-side field-level encryption — PII never reaches MongoDB in plaintext
const encryptedClient = new MongoClient(uri, {
  autoEncryption: {
    keyVaultNamespace: 'encryption.__keyVault',
    kmsProviders: { aws: { accessKeyId: '...', secretAccessKey: '...' } },
    schemaMap: {
      'mydb.users': {
        properties: {
          ssn: { encrypt: { bsonType: 'string', algorithm: 'AEAD_AES_256_CBC_HMAC_SHA_512_Deterministic' }},
          creditCard: { encrypt: { bsonType: 'string', algorithm: 'AEAD_AES_256_CBC_HMAC_SHA_512_Random' }}
        }
      }
    }
  }
})
Crucial Requirement
GDPR Compliance: Document databases store denormalized copies of user PII across multiple documents. Implement a userId compound index on every collection containing PII, and a scheduled erasure job that bulk-deletes all documents by userId. Do not rely on TTL for compliance erasure — TTL is for expiry, not for right-to-erasure. For DynamoDB, create a GSI on userId as the erasure scan key. Document your data map (which collections contain which PII fields) before your first compliance audit.

8. When NOT to Use Document / NoSQL

ScenarioWhy Document FailsBetter Alternative
Complex multi-entity joins at query time$lookup is expensive; no query planner optimization across collectionsPostgreSQL — B-tree indexes, JOIN optimizer
Strong ACID across multiple documentsMulti-document transactions in MongoDB exist but carry significant write overhead and limit horizontal scalingPostgreSQL — native multi-statement transactions
Arbitrary reporting / ad-hoc analyticsAggregation pipeline requires knowing the query shape upfront; slow for exploratory queriesPostgreSQL + analytics view, or a dedicated OLAP store
Strictly typed, constraint-enforced dataSchema validation exists in MongoDB but is opt-in and not enforced at the storage levelPostgreSQL — typed columns, CHECK constraints, FK enforcement

Summary

ConceptRule
Embedding vs. referencingEmbedding optimizes for read performance; referencing optimizes for write flexibility — choose based on the dominant operation.
DynamoDB key designDynamoDB single-table design requires modeling all access patterns upfront — retrofitting is expensive and often requires a full migration.
Schema disciplineSchema evolution in MongoDB requires explicit versioning discipline — 'schemaless' enables chaos, not speed.
DynamoDB StreamsDynamoDB Streams eliminate a separate event bus for document-mutation triggers in serverless architectures.
When to switchDocument databases are wrong for complex reporting, multi-entity ACID transactions, and workloads requiring ad-hoc joins.

What's Next

In Part 4, we cover distributed SQL — how CockroachDB solves PostgreSQL's horizontal scaling ceiling with Raft consensus, and the cross-region write latency, serializable isolation overhead, and clock skew failure modes you must understand before committing.
Research & Synthesis Note

This article was developed with AI-assisted deep search, specification cross-referencing, and technical research synthesis.

#MongoDB#DynamoDB#NoSQL#Document Database#Single-Table Design#Schema Design#GDPR
Siddhant Deval

Written by Siddhant Deval

Senior Full-Stack Engineer building high-scale architectures, browser performance engineering systems, and SaaS platforms.