AWS Managed Messaging Primitives: SQS, SNS, and EventBridge
SQS, SNS, and EventBridge are AWS's production-ready implementations of the queue and pub/sub patterns — but their managed constraints (visibility timeout, message group IDs, filter policies, content-based routing rules) require deliberate design, not default configuration. This article maps the broker-agnostic concepts from Parts 1–5 to concrete AWS primitives using AWS SDK v3 and LocalStack.
Messaging at Cloud Scale
AWS Managed Messaging Primitives: SQS, SNS, and EventBridge
The payment service deploys to production. The first load test reveals that at 3,000 requests per second, the downstream fulfillment Lambda starts timing out — not from a code bug, but because the SQS visibility timeout is set to its default 30 seconds, and each message takes up to 45 seconds to process. After 30 seconds, SQS assumes the consumer failed and makes the message visible again. A second Lambda invocation picks it up. Now two invocations are processing the same payment simultaneously. The fulfillment database records a double-charge.
The visibility timeout is not a tuning parameter. It is the at-least-once delivery contract — the mechanism by which SQS implements the retry semantics that Kafka implements through committed offsets. Getting it wrong has the same consequences: duplicate processing.
Series positioning: This is Part 1 of Messaging at Cloud Scale (Series 2). It assumes full familiarity with delivery guarantees (Series 1, Part 5), consumer patterns (Part 6), and dead-letter queues (Part 10). This article maps every broker-agnostic concept from Series 1 to concrete AWS managed primitives — SQS, SNS, and EventBridge — using @aws-sdk/client-sqs@3.x, @aws-sdk/client-sns@3.x, and LocalStack for local development.
1. SQS: The Managed Queue
1.1 Visibility Timeout — The At-Least-Once Contract
SQS does not use committed offsets. Instead, when a consumer receives a message, SQS marks it invisible for VisibilityTimeout seconds. The consumer must explicitly delete the message before the timeout expires. If it does not, the message reappears:
Size VisibilityTimeout to p99 processing latency × 1.5, not the average. The average will result in timeouts for the slowest 50% of messages. If processing time varies widely (e.g., 5ms–120s), use the heartbeat extension pattern rather than a fixed timeout — it keeps the message invisible throughout the actual processing duration.
1.2 SQS Standard vs FIFO
| Feature | SQS Standard | SQS FIFO |
|---|---|---|
| Ordering | Best-effort (not guaranteed) | Strict within MessageGroupId |
| Throughput | Unlimited | 300 msg/s per queue (3,000 with batching) |
| Deduplication | None (application must be idempotent) | Content-based or MessageDeduplicationId (5-min window) |
| Price | Lower | Higher |
| Use case | Decoupled async work, idempotent processing | Ordered entity events, payment sequences |
1.3 SQS Dead-Letter Queue Configuration
2. SNS: Fan-Out and Filter Policies
2.1 SNS + SQS Fan-Out Pattern
SNS delivers a single published event to multiple SQS subscribers simultaneously — the managed equivalent of a Kafka topic with multiple consumer groups:
2.2 SNS Filter Policies
Enable RawMessageDelivery: true on SNS→SQS subscriptions. Without it, SQS receives the SNS envelope (a JSON wrapper containing the original message, topic ARN, signature, etc.) rather than your raw message body. Consumers must then unwrap the envelope before parsing. Raw delivery simplifies consumer code and removes the SNS-specific parsing dependency.
3. EventBridge: Content-Based Routing
3.1 EventBridge vs SNS Filter Policies
EventBridge is the managed equivalent of a RabbitMQ topic exchange — it routes events based on content-based rules applied to the event payload, not just message attributes:
| Feature | SNS Filter Policies | EventBridge Rules |
|---|---|---|
| Routing basis | Message attributes only | Full event payload (any JSON field) |
| Rule expressiveness | Simple match / prefix | prefix, suffix, equals, numeric range, exists, anything-but |
| Schema registry | No | Yes (Schema Registry built-in) |
| Cross-account routing | No | Yes (cross-account event buses) |
| Max targets per rule | 1 subscription = 1 target | 5 targets per rule |
| Archive and replay | No | Yes (EventBridge Archive) |
| Pricing | Per message delivery | Per event published ($1/million) |
3.2 EventBridge Rule Implementation
3.3 EventBridge Archive and Replay
4. LocalStack: Local AWS Development
Summary
| Concept | Rule |
|---|---|
| Visibility timeout contract | SQS visibility timeout IS the at-least-once contract — if your consumer takes longer than the timeout to process, the message reappears and is processed twice; size the timeout to p99 processing latency × 1.5. |
| FIFO MessageGroupId | SQS FIFO's MessageGroupId is the partition key equivalent: all messages in the same group are strictly ordered and processed sequentially by a single consumer. |
| EventBridge routing | EventBridge content-based routing is the managed equivalent of RabbitMQ topic exchange bindings — prefer it over SNS filter policies when routing rules depend on event payload fields, not just attributes. |
What's Next
Part 2: AWS Streaming Primitives — Kinesis Data Streams and Amazon MSK covers the managed streaming alternatives to self-hosted Kafka: Kinesis Data Streams for serverless, per-shard streaming at cloud scale, and Amazon MSK for teams that need full Kafka compatibility without operating the brokers. The shard-vs-partition model, enhanced fan-out consumers, and the MSK cost model are the primary decision points.
This article was developed with AI-assisted deep search, specification cross-referencing, and technical research synthesis.