Queues vs. Streams: Handling Replay and Retention

September 2, 2026

Queues are primarily designed for distributing work and smoothing load, not for durable history, replay, or retention of events. While queues can buffer bursts and handle backpressure, they typically do not retain event history for long-term replay or auditability. For these capabilities, event streams are the more appropriate choice.

Understanding Queues and Their Limitations

Queues serve as a mechanism for work orchestration and load leveling in event-driven systems. They are effective at absorbing bursts of messages, allowing consumers to process at their own pace and preventing system overload. This buffering capability helps manage backpressure, where producers generate work faster than consumers can process it.

However, queues generally lack the inherent ability to retain a durable history of events. Their primary function is to deliver messages for processing, often with at-least-once delivery semantics, meaning consumers must be idempotent to handle potential duplicates. Once a message is consumed, it is typically removed from the queue, making long-term retention and replay challenging or impossible without additional mechanisms.

Backpressure and Buffering in Queues

Queues implement backpressure by buffering messages, which helps absorb bursts and allows consumers to catch up. This prevents a "pile-up" that could stop the entire system. Operational signals like "lag" (queue growth) are crucial for identifying bottlenecks and adjusting producer rates, consumer concurrency, or batch sizes.

Dead Letter Queues (DLQs) and Limited Replay

While standard queues don't offer general replay capabilities, Dead Letter Queues (DLQs) provide a limited form of message replay for failed messages. DLQs are used to quarantine messages that couldn't be processed successfully due to transient or permanent faults. Operators can then inspect these messages, fix underlying issues, and potentially replay them back into the system. This mechanism is for managing failures, not for replaying an entire historical log of events.

Event Streams for Replay and Retention

In contrast to queues, event streams are specifically designed for durable history, replay, and fan-out to multiple independent consumers. A stream acts as an append-only log of events, retaining the entire event history. This capability is fundamental to event sourcing, where the event log becomes the system of record.

Key Capabilities of Event Streams

Durable History: Streams store events durably, ensuring that the history is preserved.
Replay: The retained event history allows for replaying events to rebuild state, fix bugs, or create new projections. This is crucial for auditability and "time travel".
Fan-out: Multiple independent consumers can read from the event log at their own pace, subscribing to events as needed. New services can subscribe later and process historical events.
Auditability: By retaining every event, streams provide a complete timeline of "what happened," enabling detailed audits and answering "when and why" questions.

Streams vs. Queues: A Practical Rule of Thumb

The choice between a queue and a stream depends on the primary need.

Primitive	Strengths	Best for
Queue	Distributes work, smooths load, buffers bursts	Work orchestration, command-like jobs, transient tasks
Stream	Durable history, replay, fan-out, auditability	Event history, state rebuild, facts over time

Most real-world systems often utilize both: streams for events (representing immutable facts) and queues for command-like jobs (representing work distribution). For example, Kafka topics, a modern stream implementation, retain events for days or weeks, enabling new consumers to replay history.

Event Sourcing and the Role of Streams

Event sourcing makes the event log the system of record, storing state changes as an append-only sequence of immutable events. The current state is derived by replaying these events. This pattern inherently relies on the durable history and replay capabilities provided by event streams. Without a stream or similar durable storage, achieving event sourcing semantics with queues alone is not possible.

Frequently Asked Questions

Can queues store events for long-term retention?

No, queues are generally not designed for long-term retention of events. Their primary purpose is to distribute work and buffer messages for immediate processing, not to maintain a durable historical log.

How do streams enable replay?

Streams enable replay by storing events durably in an append-only log, retaining the entire event history. This allows consumers to re-read past events to rebuild state, fix issues, or create new projections.

When should I choose a queue over a stream?

You should choose a queue when your primary need is work orchestration, distributing tasks to a single consumer, or smoothing out load bursts. Queues are ideal for command-like jobs.

Can Dead Letter Queues (DLQs) be used for general event replay?

DLQs offer a limited form of replay for failed messages, allowing operators to inspect, fix, and reprocess them. However, they are not a substitute for the comprehensive historical replay capabilities of event streams, which store all events for auditability and state reconstruction.

What is the main difference between an event in a stream and a task in a queue?

An event in a stream represents "this happened" – an immutable fact over time. A task in a queue represents "do this" – a unit of work to be distributed and processed.

Conclusion

While queues are excellent for work distribution, load leveling, and buffering bursts, they are not designed to handle durable event retention or comprehensive replay capabilities. For these critical functions, especially in event-sourced architectures, event streams are the appropriate primitive. Streams provide the necessary durable history, auditability, and the ability to replay events, making them indispensable for systems that require a complete and reconstructible record of "what happened".

Sources & References

Want to actually learn Queues vs. Streams: Handling Replay and Retention?

Curo turns topics like this into a personalized, guided learning board - built around what you already know. Free to start.

Try Curo