Stopping the stampeding herd problem with lazy queues

A sudden rush of incoming messages can inundate your RabbitMQ cluster and grind processing to a halt. Here lazy queues can come to the rescue.

What are lazy queues, anyway? Lazy queues are queues where the messages are automatically stored to disk. Only when they are needed are messages loaded into memory.

Messages in memory and on disk in Rabbit

This differs from how default queues work, where messages are only paged out to disk when the memory is starting to get full. This paging process can take some time and will block the queue from processing messages. If there is a lot to page out, it can take considerable time and will effect the performance of the broker. With lazy queues the messages go straight to disk and thereby the RAM usage is minimized. A potential drawback with lazy queues is that the I/O to the disk will increase.

CloudAMQP has found the lazy queue feature so good that our new clusters (with RabbitMQ versions 3.6.0 and higher) comes with this policy enabled from the start.

If you are running a 3.6.x cluster without lazy queues you can enable it by adding a policy for it.

Policy with queue-mode = lazy

CloudAMQP - industry leading RabbitMQ as a service

Start your managed cluster today. CloudAMQP is 100% free to try.

13,000+ users including these smart companies