Siddhant Deval
Siddhant DevalAuthor
Senior Full-Stack Engineer·Jun 28, 2026·14 min read

The Database Decision Framework: Choose the Paradigm Before the Product

Defaulting to a familiar database is an architectural mistake. This article introduces a structured decision process — workload classification, access pattern analysis, and consistency model selection — that must precede any product evaluation, plus a diagnostic checklist for ruling out application-layer bottlenecks before adding a new database.

Technical Series

Modern Database Paradigms

Part 1 of 8

The Database Decision Framework: Choose the Paradigm Before the Product

Selecting the right database is a foundational architectural decision; data gravity ultimately dictates the scalability and resilience of an application — choose the paradigm first, the product second. The most expensive database mistake is not choosing the wrong engine — it is choosing an engine before understanding the workload. Teams that default to a familiar tool ("we use Postgres for everything") eventually reach a wall: a sharding problem Postgres cannot solve, a graph traversal that recursive CTEs cannot serve, or a write throughput ceiling that a single primary cannot absorb. This article gives you the structured decision process that prevents that wall.

1. Diagnostic Checklist — Is My Database Actually the Bottleneck?

Before adding a new database to your stack, rule out that your existing database is the problem. Most "we need a new database" decisions are actually application-layer problems in disguise.
Work through this checklist against your primary store before evaluating alternatives:
sql
-- Step 1: Identify the slowest queries
SELECT query, calls, mean_exec_time, total_exec_time
FROM pg_stat_statements
ORDER BY mean_exec_time DESC
LIMIT 20;

-- Step 2: Check for missing indexes (sequential scans on large tables)
SELECT relname, seq_scan, idx_scan,
       round(seq_scan::numeric / (seq_scan + idx_scan + 1) * 100, 1) AS seq_pct
FROM pg_stat_user_tables
WHERE seq_scan > 0
ORDER BY seq_scan DESC;
Performance / Safety Warning
If seq_pct exceeds 10% on a table with over 100K rows, you have a missing index — not a database paradigm problem.
SymptomRoot CauseFix Before Switching
Slow queries on large tablesMissing indexes, bad query planAdd targeted index; run ANALYZE
App threads blocking under loadN+1 query patternFix query at the ORM/query layer
Connection refused / timeoutConnection pool exhaustionDeploy PgBouncer in transaction mode
High memory usageMissing query result limitsAdd LIMIT clauses; paginate
Write latency spike at high volumeNo connection pooling on writesPgBouncer + async write queue
Crucial Requirement
Only proceed to a new database paradigm when the bottleneck cannot be eliminated by indexing, query rewriting, connection pooling, or vertical scaling. Adding a second database before fixing the application layer compounds the problem — you now have two stores to debug.

2. Workload Taxonomy

Every database paradigm is optimized for a workload class. The first step in any database selection is classifying your own.
Workload ClassDominant OperationLatency RequirementConsistency Requirement
OLTP (Online Transactional Processing)Point reads + writes, short transactions< 10ms P99Strong (ACID)
OLAP (Online Analytical Processing)Full-table scans, aggregations, joinsSeconds acceptableEventual or snapshot
HTAP (Hybrid)Mixed OLTP + light analytics< 50ms for OLTP pathStrong for writes, eventual for reads
SearchFull-text, faceted, fuzzy matching< 50ms P99Eventual
GraphMulti-hop relationship traversal< 100ms for 3–5 hopsCausal or strong
Cache / SessionSingle key lookup, TTL expiry< 1ms P99Eventual (loss-tolerant)
Vector / SimilarityANN search on high-dimensional embeddings< 100ms P99Eventual
Time-Series / Append-OnlySequential writes, range-time reads< 5ms write P99Eventual
Mental Model Check
If you can describe your workload with "mostly point reads and writes on structured entities," you have an OLTP workload. If you say "we need to find all users similar to this user" or "run this report across the last 90 days," you have a different workload class entirely.
5-minute classification: Count your top 10 queries by frequency. What fraction are point lookups (fetch by primary key or unique index)? What fraction are scans or aggregations? What fraction follow relationships across 2+ entities? The largest bucket defines your primary workload.

3. Access Pattern Analysis

Workload class narrows the field. Access patterns determine the specific paradigm.

3.1 Point Lookup

sql
-- ✅ Relational: index seek on primary key — O(log n)
SELECT * FROM orders WHERE order_id = 'ord_8f3k2';

-- ✅ Redis: O(1) hash get — sub-millisecond
GET session:user:42

-- ✅ DynamoDB: partition key hash — single-digit millisecond at any scale
GetItem({ TableName: 'orders', Key: { orderId: 'ord_8f3k2' } })
Point lookups by a known key are served equally well by relational, key-value, and document stores. The differentiator is what else you need — if the answer is "complex queries," relational wins; if "extreme write throughput," key-value or document wins.

3.2 Range Scan

sql
-- ✅ Relational: B-tree range scan — efficient with index
SELECT * FROM events
WHERE user_id = 42 AND created_at BETWEEN '2026-01-01' AND '2026-03-31';

-- ❌ DynamoDB: requires a GSI on (user_id, created_at) upfront — retrofitting is costly
-- ❌ MongoDB: works but requires a compound index; aggregation pipeline for reporting adds overhead
Range scans favor relational databases (B-tree indexes) and Cassandra (clustering columns). DynamoDB can handle ranges but requires the full access pattern to be modeled into the table key design before the first write.

3.3 Graph Traversal

cypher
-- ✅ Neo4j Cypher: 3-hop traversal in milliseconds — native graph storage
MATCH (u:User)-[:PURCHASED]->(p:Product)<-[:PURCHASED]-(similar:User)
-[:PURCHASED]->(rec:Product)
WHERE u.id = 42 AND NOT (u)-[:PURCHASED]->(rec)
RETURN rec, COUNT(similar) AS strength
ORDER BY strength DESC LIMIT 10;
sql
-- ❌ SQL recursive CTE: same 3-hop traversal — seconds at scale
WITH RECURSIVE recs AS (
  SELECT p2.product_id, 1 AS depth FROM orders o1
  JOIN orders o2 ON o1.product_id = o2.product_id AND o1.user_id != o2.user_id
  JOIN orders p2 ON o2.user_id = p2.user_id AND p2.product_id != o1.product_id
  WHERE o1.user_id = 42
)
SELECT product_id, COUNT(*) FROM recs GROUP BY product_id ORDER BY 2 DESC LIMIT 10;
Pro Tip & Optimization
The SQL version is not just slower — it scales with O(n³) join complexity. At 10M orders, the SQL approach is not just slow; it is impractical. Graph traversal at 3+ hops is the clearest signal to use a graph database.

3.4 Aggregation

sql
-- ✅ Relational: window functions and GROUP BY — optimized for ad-hoc
SELECT user_id,
       SUM(amount) AS total_spend,
       RANK() OVER (ORDER BY SUM(amount) DESC) AS spend_rank
FROM orders
WHERE created_at >= NOW() - INTERVAL '30 days'
GROUP BY user_id;
Complex, ad-hoc aggregations favor relational databases. If you can define the aggregation shape upfront, Cassandra counter tables or DynamoDB GSI projections can work — but you lose flexibility. Arbitrary aggregations require SQL.

4. Consistency Model Spectrum

Mental Model Check
Consistency is a contract between the database and the reader: what version of the data are you guaranteed to see after a write completes?
ModelGuaranteeLatency ImpactUse When
Strong / LinearizableEvery read sees the most recent committed write+RTT for quorumFinancial transactions, inventory, seat reservation
CausalReads see writes that causally preceded them+small overheadSocial feeds, collaborative editing
Read-Your-WritesYou always see your own writesMinimalUser profile updates, session data
EventualReads eventually converge to the latest writeLowestCounters, analytics aggregates, search indexes
typescript
// ❌ Broken pattern: assuming eventual consistency is safe for inventory
async function reserveSeat(seatId: string, userId: string) {
  const seat = await dynamoDB.get({ Key: { seatId } }) // eventual read!
  if (seat.available) {
    await dynamoDB.put({ Item: { seatId, userId, available: false } })
    // Two concurrent users can both read available=true → double booking
  }
}

// ✅ Correct: use conditional write (optimistic concurrency) for strong consistency
async function reserveSeat(seatId: string, userId: string) {
  await dynamoDB.put({
    Item: { seatId, userId, available: false },
    ConditionExpression: 'attribute_not_exists(userId)',
    // Fails with ConditionalCheckFailedException if already booked
  })
}

5. The CAP Theorem in Practice

CAP states that a distributed system can guarantee at most two of: Consistency, Availability, Partition Tolerance. In cloud deployments, network partitions are not theoretical — they happen. Partition tolerance is non-negotiable.
This makes CAP a real-world choice between CP (consistent under partition, may reject writes) and AP (available under partition, may serve stale reads).
DatabaseCAP PositionWhat It Means in Practice
PostgreSQLCPUnder partition: primary continues; replicas may serve stale reads
CockroachDBCP (serializable)Under partition: Raft majority required for writes; minority nodes pause
MongoDB (majority write concern)CPUnder partition: primary election; secondary reads may be stale
DynamoDBAP (default)Under partition: serves eventual reads; conditional writes enforce consistency
CassandraAP (tunable)QUORUM consistency trades availability for consistency at write time
RedisAP (no persistence)Under partition: stale reads possible; data loss if primary crashes without AOF

6. Polyglot Persistence Cost Model

Adding a second database to your stack is an operational multiplier, not just a technical decision.
Per-database ongoing cost:
Operational overhead per database:
  + Backup configuration and monitoring
  + Separate connection pool management
  + Dedicated dashboards (latency, error rate, disk usage)
  + Failover procedures and runbooks
  + On-call expertise (team must be able to debug it at 3 AM)
  + Security patching and upgrade cycles
  + Schema migration tooling
Crucial Requirement
Before adding a second database, answer: does the performance delta of the specialized store at my current scale exceed the fully-loaded cost of operating it? If your dataset is under 10M rows/documents/vectors, the answer is almost always no — a single Postgres instance with the right extensions outperforms the operational cost of a second store.
When polyglot is justified:
  • Write throughput on the primary store is saturated and cannot be relieved by PgBouncer or read replicas
  • A search tier (Elasticsearch) is required for full-text relevance that Postgres tsvector cannot serve
  • Session/cache reads must be sub-millisecond and the primary DB cannot provide that latency
  • Vector search at > 10M vectors requires recall performance that pgvector cannot achieve

7. The Decision Tree

What is your workload class?
│
├── OLTP (transactions, point reads/writes)
│     ├── Need complex queries / reporting → PostgreSQL
│     ├── Need extreme write scale (millions/s) → DynamoDB or Cassandra
│     └── Need horizontal SQL + multi-region HA → CockroachDB
│
├── Search (full-text, faceted, fuzzy)
│     └── Elasticsearch / OpenSearch
│
├── Graph (3+ hop relationship traversal)
│     └── Neo4j
│
├── Cache / Session (sub-millisecond, loss-tolerant)
│     └── Redis
│
├── Vector / Similarity (ANN search on embeddings)
│     ├── < 10M vectors → pgvector (inside Postgres)
│     └── > 10M vectors → Pinecone / Qdrant / Milvus
│
└── Time-series / Append-only (sensor, metrics, logs)
      ├── Moderate scale → PostgreSQL + BRIN index or TimescaleDB
      └── High write volume + distributed → Cassandra (TWCS compaction)
A decision tree flowing from workload classification (OLTP, Search, Graph, Cache, Vector, Time-Series) through paradigm selection to a product shortlist — each branch labeled with the access pattern and scale threshold that drives it.
A decision tree flowing from workload classification (OLTP, Search, Graph, Cache, Vector, Time-Series) through paradigm selection to a product shortlist — ea…

Summary

ConceptRule
Paradigm before productClassify the workload before naming a product; the paradigm is the unit of decision.
Diagnostic checklist firstRun the diagnostic checklist first — most 'database problems' are application-layer problems in disguise.
Access pattern → paradigmAccess patterns (point lookup, scan, aggregation, graph) determine index strategy and therefore paradigm fit.
Consistency costStrong consistency is not free — it trades latency for correctness; choose it deliberately, not by default.
Polyglot costPolyglot persistence multiplies operational burden; a single store with extensions is superior at moderate scale.
Reading-order hierarchy for the Modern Database Paradigms series — Part 1 at root, branches showing which articles build on which prerequisites, and which can stand alone after Part 1.
Reading-order hierarchy for the Modern Database Paradigms series — Part 1 at root, branches showing which articles build on which prerequisites, and which ca…

What's Next

In Part 2, we cover the relational paradigm in depth — PostgreSQL's extension model, MVCC under load, data modeling with JSONB, Row-Level Security for multi-tenancy, and the exact signals that tell you you've hit Postgres's ceiling.
Research & Synthesis Note

This article was developed with AI-assisted deep search, specification cross-referencing, and technical research synthesis.

#Database Architecture#System Design#OLTP#CAP Theorem#Polyglot Persistence
Siddhant Deval

Written by Siddhant Deval

Senior Full-Stack Engineer building high-scale architectures, browser performance engineering systems, and SaaS platforms.