Message queues — decoupling producers from consumers
What actually breaks when a service calls another service directly, and how a queue in between fixes it — plus the delivery-guarantee trade-off that comes with it.
Intermediate
2 min read
What breaks without a queue
Order Service publishes a message ("send confirmation for order #123") to the queue and moves on immediately — it doesn't wait for the email to actually send, and it doesn't know or care whether Email Service is fast, slow, or temporarily down. Email Service consumes messages from the queue at its own pace. Three direct consequences:
Decoupling — the two services no longer need to be up at the same time. If Email Service is down for five minutes, messages simply queue up and get processed once it recovers, instead of the order flow failing.
Buffering traffic spikes — 10,000 orders placed in one second become 10,000 messages sitting in a queue, consumed at whatever steady rate Email Service can actually handle, instead of 10,000 simultaneous demands on it.
Independent scaling — Email Service can run more or fewer worker processes based on queue depth, entirely independent of how many Order Service instances exist.
This is the exact pattern behind Celery-with-Redis or Celery-with-RabbitMQ in a Django app: .delay() publishes a task message and returns immediately; a separate worker process consumes it whenever it gets to it.
The trade-off queues introduce: delivery guarantees
Once processing is asynchronous, "did this message actually get handled" becomes a real question with three possible answers, not an assumption:
Guarantee
What it means
Risk
Typical use
At-most-once
Message delivered 0 or 1 times
Silent data loss if consumer crashes
Metrics / analytics — losing one event is ok
At-least-once
Message guaranteed to be processed, possibly >1 time
Duplicate processing — consumer must be idempotent
Most real-world queues (SQS default)
Exactly-once
Delivered precisely once
Very hard to truly guarantee end-to-end
Usually at-least-once + idempotent consumer
The practical takeaway: default to designing consumers as if delivery is at-least-once, because in most real message queue systems, it is.