DynamoDB at Scale: Partition Internals, GSI Backpressure & ACID Transactions
DynamoDB's performance ceiling is determined at schema design time — a partition key with insufficient cardinality creates a hot partition that table-level throughput scaling cannot fix, because the limit is enforced per-partition at the storage node. This article covers partition key hashing, per-partition hard limits, hot partition detection and salt-key mitigation, GSI backpressure propagation, ACID transactions, and Global Tables conflict resolution.
AWS Serverless Engineering: Lambda to Production
DynamoDB at Scale: Partition Internals, GSI Backpressure & ACID Transactions
Every AWS primitive is a tradeoff surface, not a feature toggle. DynamoDB's "scale to any workload" promise has a hard asterisk: per-partition hard limits. A table provisioned at 10,000 WCU cannot write faster than 1,000 WCU to a single partition key, because that limit is enforced at the storage node — not at the table control plane. This is why provisioning more capacity does not fix hot-partition throttling. This article covers the physical storage model that produces this constraint, the GSI backpressure failure mode that causes base table write throttling from an under-provisioned index, and the ACID transaction semantics that enable financial-grade consistency in a non-relational system.
Prerequisite chain: Document & NoSQL Databases: MongoDB & DynamoDB → Part 7: Access-Pattern-First Modeling → this article.
1. Partition Internals — The Storage Model
DynamoDB stores data in partitions — independently managed storage and compute units. Understanding the partition model is the prerequisite for understanding every performance and cost constraint in DynamoDB.
1.1 Partition Key Hashing
DynamoDB uses an internal hashing algorithm to map partition key values to physical partitions. The hash distributes keys across partitions to balance load. The key insight: all items with the same partition key value land on the same partition, served by the same storage node.
If 100,000 orders all share STATUS#PENDING as their partition key, all 100,000 writes route to partition_node_A. Even if the table has 10,000 WCU provisioned, partition_node_A is hard-capped at 1,000 WCU.
1.2 Per-Partition Hard Limits
| Limit | Value | Nature |
|---|---|---|
| Storage per partition | 10 GB | Hard limit — DynamoDB automatically splits at 10GB |
| Read Capacity Units per partition | 3,000 RCU | Per storage node — cannot be increased |
| Write Capacity Units per partition | 1,000 WCU | Per storage node — cannot be increased |
| Maximum item size | 400 KB | Per item — binary blobs belong in S3 |
The per-partition WCU and RCU limits are hard constraints enforced at the storage node level. Increasing table-level provisioned capacity or switching to On-Demand mode does NOT increase these limits. The only mitigation for a hot partition is to distribute writes across more partition key values.
1.3 Hot Partition Detection
Before mitigating, detect. A hot partition signature in CloudWatch:
The diagnostic that confirms hot partition (vs general over-provisioning): ConsumedWriteCapacityUnits is low (e.g., 200 WCU average) but ThrottledRequests is high. The table has capacity — the problem is concentration on one partition.
1.4 Write Sharding — Synthetic Salt Keys
The mitigation for hot write partitions: add synthetic cardinality to the partition key with a salt suffix, then scatter-gather reads across shards.
Write sharding trades write scalability for read complexity. A scatter-gather read across 10 shards issues 10 parallel queries and merges results in the application. This is acceptable for infrequent bulk reads (fulfillment queue drain, admin dashboard) but adds latency and cost for high-frequency individual reads. Model both the write improvement and the read cost before applying sharding.
2. GSI Backpressure — The Upstream Write Throttle
This is the most dangerous DynamoDB failure mode because it is invisible: a GSI under-provisioned relative to the base table write rate throttles base table writes, not GSI reads.
2.1 The Failure Chain
2.2 Monitoring and Mitigation
On-Demand mode eliminates GSI backpressure — DynamoDB scales GSI capacity automatically with table write throughput. For Provisioned mode, the GSI WCU must be sized to match the write rate of items that carry the GSI key. A GSI on a low-cardinality attribute (status field with 3 values) concentrates writes and is a hot-partition risk even in On-Demand mode.
3. ACID Transactions
DynamoDB supports ACID transactions via TransactWriteItems (up to 25 write operations) and TransactGetItems (up to 25 read operations) across any number of tables in the same account and region.
3.1 TransactWriteItems — Cross-Item Consistency
TransactWriteItems consumes 2× WCU per item — once for the write operation and once for the transactional overhead. A 3-item transaction costs 6 WCU regardless of item size. At high write rates, this doubles the effective write cost. Use transactions only when cross-item consistency is a genuine business requirement — not as a convenience wrapper for multi-item writes that do not require atomicity.
3.2 Conditional Writes — Idempotent Alternatives to Atomic Counters
4. DynamoDB Streams and Lambda Integration
| Stream view type | Content | Use case | Cost |
|---|---|---|---|
KEYS_ONLY |
PK + SK only | Change detection | Lowest |
NEW_IMAGE |
New item state | Event sourcing | Medium |
OLD_IMAGE |
Previous state | Soft deletes, undo log | Medium |
NEW_AND_OLD_IMAGES |
Both states | Full audit trail, diff detection | Highest |
5. On-Demand vs Provisioned — Cost Crossover
| Attribute | On-Demand | Provisioned + Auto Scaling |
|---|---|---|
| Billing model | Per request ($1.25/million WRU, $0.25/million RRU) | Per hour ($0.00065/WCU-hr, $0.00013/RCU-hr) |
| Zero-traffic cost | $0 | Minimum provisioned hours × rate |
| Traffic spike handling | Automatic | Delayed by Auto Scaling (3–15 min lag) |
| Cost crossover | Wins for spiky/unpredictable traffic | Wins for steady, predictable traffic |
| Typical break-even | ~200 WCU equivalent sustained | ~200 WCU sustained 24/7 |


Summary
| Concept | Rule |
|---|---|
| Per-partition limits | 1,000 WCU / 3,000 RCU per partition — table capacity cannot override these |
| Hot partition detection | Consumed WCU << Provisioned WCU + ThrottledRequests > 0 = hot partition, not under-provisioning |
| Write sharding | Synthetic salt key distributes writes; scatter-gather adds read complexity |
| GSI backpressure | Under-provisioned GSI throttles base table writes — alarm on GSI metrics, not only table metrics |
| TransactWriteItems | 2× WCU per item, 25 item max — use for genuine financial-grade consistency only |
| Conditional writes | Idempotent optimistic concurrency — correct alternative to atomic counters on retry paths |
What's Next
In Part 9: Messaging Foundations — SQS, SNS & Fan-Out Patterns, we shift from the data layer to the messaging layer: why SNS and SQS solve completely different problems, what happens when you use SNS as a work queue, and how the visibility timeout is the mechanism behind SQS's at-least-once delivery guarantee.
This article was developed with AI-assisted deep search, specification cross-referencing, and technical research synthesis.