Queue vs Pub/Sub: When to Use Each Pattern

An order is accepted. One component must create a shipping label, while analytics, fraud, email, and inventory systems may all need to react. Calling every service inline makes checkout slow and fragile. Calling them asynchronously is better, but the correct messaging pattern depends on one question: is this a task that needs one owner, or a fact that several independent systems should receive?

This queue vs pub/sub guide makes that choice concrete. It follows what a message queue is and connects to idempotency in REST APIs, because retry safety matters whichever pattern you choose. Use the Backend Interview Mastery series for the learning path and the Backend tag for related concepts.

Table of Contents

The Decision in One Sentence

Use a queue when one worker should own and complete a unit of work. Use pub/sub when an event should be independently delivered to multiple subscribers. That distinction is about the consumption contract, not the product name: Kafka, RabbitMQ, cloud services, and managed brokers can support more than one pattern.

A queue is a work distribution mechanism. The producer says, “generate this invoice,” and one eligible worker should do it. Pub/sub is an event distribution mechanism. The producer says, “invoice was generated,” and billing, analytics, auditing, and notifications can each decide what to do. Treating a command as an event obscures who owns completion; treating an event as a single queue hides useful subscribers behind a central worker.

How a Queue Works

In a queue, a producer submits a task and competing consumers receive work. After one consumer successfully acknowledges a message, other consumers do not process that same task. Adding workers increases throughput, but it does not create more business recipients.

flowchart TD
    A[Checkout service] --> B[Shipping-label queue]
    B --> C[Worker one]
    B --> D[Worker two]
    C --> E[Carrier API]
    D --> E
    E --> F[Acknowledge completed task]

This model is a strong fit for thumbnail generation, sending one transactional email, calling a rate-limited partner API, and processing a file. The queue absorbs a burst and gives the work an explicit owner. Its important trade-off is that a queue is not proof of successful business completion. A worker can crash after calling the carrier but before acknowledging the task, so the message may be redelivered. The downstream operation must tolerate that duplicate.

How Pub/Sub Works

In pub/sub, a publisher writes an event to a topic. Each subscription has its own delivery state, so one slow subscriber does not prevent another from receiving and processing the event. A new subscription may receive only future events or may be able to replay retained history, depending on the broker and subscription configuration.

flowchart TD
    A[Order service] --> B[order.created topic]
    B --> C[Inventory subscription]
    B --> D[Analytics subscription]
    B --> E[Notification subscription]
    C --> F[Reserve stock]
    D --> G[Record business metric]
    E --> H[Send order update]

The publisher knows the event contract, not the subscriber list. That loose coupling makes pub/sub useful for domain events, audit feeds, search-index updates, data pipelines, and integrations that may be added later. It also increases governance needs: an event schema becomes a shared interface, access to a topic must be controlled, and a broken subscriber needs its own retry and dead-letter path.

Queue vs Pub/Sub Comparison

There are hybrids. A pub/sub topic can feed a subscription that behaves like a work queue, and a broker can route a message to several queues. The decision still starts with semantics. Ask whether every recipient needs a copy, then ask what must happen when a recipient is unavailable.

Command or Event: The Real Modeling Decision

A command asks a known owner to do something. charge_payment has one payment service owner and a clear success or failure outcome. The sender often needs a status record, a retry policy, and a deadline. A queue maps naturally to this work because handing the same command to several workers would risk several charges.

An event records that something happened. payment.authorized should not tell analytics how to aggregate revenue or tell fulfillment how to pick an item. It gives each consumer a stable fact and lets that consumer own its reaction. Pub/sub prevents the payment service from becoming a coordinator for every future side effect.

Do not publish a state-changing command and call it an event just to obtain fanout. If several systems need to react, complete the authoritative command first, persist its result, and then publish an event from that committed state. The database transaction guide explains why a database write and broker publication are not automatically one atomic action; an outbox and idempotent consumers are common ways to bridge that boundary.

Delivery, Retries, and Idempotency

Both patterns commonly provide at-least-once delivery. That is a deliberate trade-off: after a network failure, redelivery is safer than silently losing important work. It also means a consumer can see the same message twice.

Make the consumer’s business effect idempotent. For a payment event, store a durable event ID with a unique constraint before applying the effect. For an email, use a delivery record keyed by the event and template. Do not deduplicate only in memory; a restart would erase the protection.

CREATE TABLE processed_events (
  consumer_name text NOT NULL,
  event_id uuid NOT NULL,
  processed_at timestamptz NOT NULL DEFAULT now(),
  PRIMARY KEY (consumer_name, event_id)
);

The consumer inserts the (consumer_name, event_id) record in the same local transaction as its database effect. A duplicate violates the primary key and becomes a safe no-op. For external side effects, use an idempotency key the provider understands or persist a delivery state before and after the call. Then retry only transient failures with exponential backoff and move exhausted messages to a dead-letter queue where an operator can inspect, correct, and replay them.

Ordering, Backpressure, and Retention

Neither word “queue” nor “pub/sub” automatically means global ordering. Many systems preserve order only within a partition, key, or single subscription stream. If events for one order must be processed in order, route them with the order ID as the partition key and do not parallelize that key across workers. Global ordering often sacrifices most parallelism, so require it only for a real invariant.

Backpressure is equally important. A broker can buffer a spike, but it cannot create unlimited storage or downstream capacity. Measure backlog age, not only depth; a queue with one million messages may be healthy if it drains quickly, while a small queue with an oldest message age of thirty minutes is a user-visible failure. Cap retries, apply producer limits, and define what to shed or delay when the consumer cannot catch up.

Retention changes recovery. A task queue may delete a message after a successful acknowledgment because its purpose is completed work. An event stream may retain records so a new analytics service can replay history. Retention costs storage and requires schema compatibility, access controls, and a plan for personally identifiable data.

A Checkout Design That Uses Both

Checkout often needs both patterns, but at different boundaries:

flowchart TD
    A[Checkout request] --> B[Order transaction]
    B --> C[Outbox record]
    C --> D[order.created topic]
    D --> E[Fulfillment subscription]
    D --> F[Analytics subscription]
    D --> G[Notification subscription]
    E --> H[Shipping-label queue]
    H --> I[Carrier worker]

The checkout service commits the order and an outbox record together. A relay publishes order.created after that commit, so a database success is not silently separated from an event failure. Analytics and notifications each consume the event independently. Fulfillment then creates a shipping label task for one carrier worker because there must be exactly one business owner for that attempted label. The message may still be delivered twice, so the worker calls the carrier with a stable idempotency key.

This design is more complex than a single synchronous call, but every component has a reason: pub/sub lets independent reactions evolve; the queue controls a bounded, rate-limited task; the outbox closes a local consistency gap. Do not adopt all three for a tiny application unless the failure modes justify their operational cost.

How to Choose in Production

Use these questions in order:

1. Is the payload a command or an immutable fact? Commands usually need a known owner; facts can have many consumers.
2. How many independent recipients need it now or later? One owner favors a queue. Independent fanout favors pub/sub.
3. What does success mean? Define the durable business outcome, acknowledgement point, timeout, retry budget, and DLQ owner.
4. What ordering is truly required? Preserve order by an entity key when needed; avoid a global-order promise by default.
5. Can consumers replay history? If yes, define retention, schema evolution, and privacy deletion behavior before publishing broadly.
6. How will overload be seen and contained? Alert on oldest-message age, consumer errors, retry rate, DLQ growth, and producer rejection.

For an interview, state the simplest design first. “I need one worker to generate a PDF, so I choose a queue with idempotent processing and a DLQ.” Then add pub/sub only if a stated requirement says several services need the resulting event.

Interview Questions

1. Can a pub/sub system act like a queue?

Yes. A single subscription with several competing consumers can distribute each delivered message to one consumer, which resembles queue semantics. The crucial detail is the subscription: other subscriptions can still receive their own copy. I would name the consumption contract, acknowledgement behavior, retry policy, and retention setting rather than assuming the product’s label decides the pattern.

2. Why are idempotent consumers required for both patterns?

A consumer can finish a side effect and fail before the broker observes its acknowledgement. The broker then redelivers, which is correct for reliability but dangerous for a non-idempotent payment, email, or inventory update. A durable event ID or provider idempotency key makes repeated delivery converge on one business outcome.

3. When should a command become an event?

After the command’s authoritative owner has completed and durably recorded the state transition. CreateOrder is a command to the order service; order.created is the event that other systems may observe. Publishing a future-tense command to many subscribers makes ownership and failure handling ambiguous.

4. Does pub/sub guarantee event ordering?

Not globally. Ordering is usually scoped to a partition, ordering key, or one subscription, and failures or retries can complicate processing order. I would choose an entity key where order matters, build handlers that reject impossible state transitions, and avoid global serialization unless the workload truly requires it.

5. How do you prevent a slow subscriber from taking down publishers?

The broker should buffer delivery up to a declared retention and quota boundary, while the subscriber scales or applies backpressure independently. I monitor its backlog age, errors, retry count, and DLQ volume. When the boundary is reached, I make an explicit product decision to throttle producers, drop noncritical events, or degrade the subscriber rather than allowing unlimited storage growth.

Conclusion

A queue assigns work; pub/sub distributes facts. The most reliable choice comes from modeling who owns the action, who must observe the result, and how duplicate delivery and overload are handled. Start with the message contract, then choose the broker configuration that enforces it.

References

1. Google Cloud: Event-Driven Architecture with Pub/Sub
2. Google Cloud: Choosing Pub/Sub or Cloud Tasks
3. RabbitMQ: Work Queues
4. Apache Kafka: Introduction

YouTube Videos

1. Message Queues vs Pub/Sub | System Design