CloudAMQP is happy to announce that RabbitMQ version 3.10 is now available for our customers. In this blog post, we will go through the most significant changes and features.
Classic Queue Version 2 with new queue index and message store implementations
The RabbitMQ team has made improvements to the index and message store implementation for classic queues. These improvements are called Classic Queue Version 2 or CQv2 for short. For replicated queues across multiple nodes, we still recommend Quorum Queues.
The benefits of using Classic Queue v2 (CQv2) compared to v1 (CQv1) are:
- Increased throughput for workloads where consumers keep up with producers
- Less memory usage
- Less variable memory usage
- More efficient consumer delivery
How do I enable Classic Queue version 2?
CQv2 can be enabled via a policy, or as a queue argument:
- Queue argument:
How do I migrate a classic queue version 1 to version 2?
In order to take advantage of the new features of Classic Queue v2, you need to change the queue version to 2 via a policy as described above. After only a few seconds, queues with up to a million messages will be migrated. The migration requires some server resources so here are a few tips:
- If your queues have transient messages, and/or a lot of messages, converting a queue from v1 to v2 can be memory intensive. To avoid running out of memory we do recommend that you apply a lazy policy to the queue before converting queues from v1 to v2.
- Deleting the policy will downgrade the queue back to v1. We do not recommend downgrading v2 queues to v1 as this showed to be more memory intensive and time-consuming.
- Make sure your clients are setting the x-queue-version queue argument to 2 when declaring queues.
Performance test Classic Queue Version 1 versus Version 2
The following test scenarios are used to discover how a CloudAMQP instance performs on Classic Queues version 1 versus version 2. We have performed these tests on a single node Awesome Ape plan in AWS using:
- m6g.2xlarge with 8 CPUs and 32GB memory
- 100GB gp3 disk with 3000 IOPS and 125MB/s
- RabbitMQ Perftest tool
- RabbitMQ flow control disabled
Scenario 1: High throughput
Let's see how Classic Queue version 1 and version 2 perform in an optimal scenario to get the maximum throughput. Messages are just 1 byte and 100 bytes in size and the number of queues and producers are the same as the number of cores, which is 8.
Classic queues version 1
bin/runjava com.rabbitmq.perf.PerfTest --queue-pattern 'cqv1-1-%d' --queue-pattern-from 1 --queue-pattern-to 8 --size 1 --producers 8 --consumers 16 -z 300 -f persistent -c 50000 -H $URL
Classic queues version 2
bin/runjava com.rabbitmq.perf.PerfTest --queue-pattern 'cqv2-1-%d' --queue-pattern-from 1 --queue-pattern-to 8 --size 1 --producers 8 --consumers 16 -z 300 -f persistent -c 50000 --queue-args x-queue-version=2 -H $URL
Summary scenario 1
With very small messages and a limited number of queues and connections, Classic Queue version 2 performs 20% faster than version 1 with the same amount of memory usage. In general, the larger the message size, the less throughput (msgs/s).
The first peak is the number of messages delivered in a v1 queue, and peak 2 for a version 2 queue with a 1-byte message size:
Scenario 2: 100 Queues, 100 producers, 200 consumers
In this scenario, we can see how Classic Queues version 1 performs with an increased number of queues and connections, compared to version 2.
Classic queues version 1
bin/runjava com.rabbitmq.perf.PerfTest --queue-pattern 'cqv1-2-%d' --queue-pattern-from 1 --queue-pattern-to 100 --size 1 --producers 100 --consumers 200 -z 300 -f persistent -c 50000 -H $URL
1-byte messages: 20600 msg/s
Classic queues version 2
bin/runjava com.rabbitmq.perf.PerfTest --queue-pattern 'cqv2-2-%d' --queue-pattern-from 1 --queue-pattern-to 100 --size 1 --producers 100 --consumers 200 -z 300 -f persistent -c 50000 --queue-args x-queue-version=2 -H $URL
1-byte messages: 53800 msg/s
Summary scenario 2
Consumer delivery is much faster with Classic Queues version 2 than version 1 in a scenario where there are many queues and connections. Publishing and consuming rates vary a lot more when using more queues and connections. Even though it is using the same amount of memory, version 2 is around 160% faster than version 1. In general, having many queues and connections takes up more memory if we compare scenario 1 with scenario 2.
The above test scenarios show that Classic Queue version 2 is outperforming version 1 in both cases.
Quorum Queues messages move to disk
Previously you had to set x-max-in-memory-length or x-max-in-memory-bytes to control the memory usage of a quorum queue, but with RabbitMQ 3.10 messages are automatically moved to disk as quickly as possible, ignoring those two settings. This will make throughput and latency less variable.
Message TTL for Quorum Queues
Message TTL has long been supported for classic queues, and with RabbitMQ 3.10 it’s also supported in quorum queues. Specify the message TTL as before, by setting the expiration field when sending a basic.publish.
Dead lettering strategies for Quorum Queues
You can now use two different strategies when using dead lettering in Quorum Queues, ‘at-most-once’ or ‘at-least-once’:
is the same behavior as classic and quorum queues in RabbitMQ 3.9, meaning
messages can get lost during transit between queues. This is the default
is introduced in RabbitMQ 3.10 and guarantees that a message has been
transferred between the queues. Dead lettered messages will be consumed from
the original queue and published to the target queue using publisher confirms
to make sure they are delivered. To use this strategy you need to opt in.
How to enable ‘at-least-once’ Dead Lettering for Quorum Queues?
You need to opt-in to use at-least-once Dead Lettering for quorum queues, as at-most-once is the default. Here are the policies that need to be applied:
- Set ‘dead-letter-strategy’ = ‘at-least-once’
- Set ‘overflow’ = ‘reject-publish’ (default is ‘drop-head’).
- Configure a dead letter exchange.
It can also be set via queue arguments, just start them with ‘x-’
How to upgrade to RabbitMQ 3.10
If you already have a CloudAMQP cluster you can easily upgrade to 3.10 from the Versions view in the CloudAMQP Console or via the API. Upgrading minor versions of RabbitMQ will result in some downtime, as the whole cluster has to be offline. The minimum allowed Erlang version is 23.2, so you might need to upgrade Erlang first. You can read more about the details of upgrading RabbitMQ and Erlang.