LavinMQ performance testing: throughput and latency

This article discusses the process of conducting performance testing for LavinMQ, including practical steps to perform it on your own, as well as some recent benchmarks for your reference.

The objective of this performance testing is to:

  • Measure the volume of messages LavinMQ could process in a given time (Throughput) and what this looks like across different message sizes (16 bytes and 1024 bytes).
  • Measure the total time it takes a message to move from a producer to a consumer in LavinMQ (Latency). Again we will test this with different message sizes (16 bytes and 1024 bytes).

Before we proceed, note that the producer, consumer, and broker configuration utilized for the testing may not align with your needs. Our aim is not to replicate a realistic setup, as each person's configuration is unique. Rather, we seek to present an example of a trivial one.

LavinMQ throughput and latency

This section will present the throughput and latency values of our benchmarking exercise

Throughput

As metioned earlier, we did the throughput test with messages of different sizes: 16 bytes and 1024 bytes

For each message size, we tested with 3 producers and 5 consumers. The result of our experiment is summarized in the chart below.

Figure 1 - LavinMQ Throughput

In the chart above, we see LavinMQ performing extremely well, at around 400k+ messages/sec, when tested with lightweight messages of about 16 bytes. We also see LavinMQ performing well at around 200k+ messages/sec, when tested with medium sized messages of about 1024 bytes.

Latency

The table below shows the latency distribution of LavinMQ for different message sizes– because generally, latency data does not have a normal distribution, reporting it in terms of percentiles gives a more holistic picture.

Figure 2 - LavinMQ Latency Table

Even though the table above reports the latency data from the minimum all the way up to the 99th percentile, we will use the 50th percentile/median to do a quick analysis of LavinMQ’s latency performance.

The median is a very fair metric to look at because it represents the central tendency of a distribution – what this means is, if we have a distribution where the median is, let’s say 7.5, then a decent percentage of the values in that distribution would be slightly above or below 7.5

The chart below shows the median latency of LavinMQ across messages of different sizes.

Figure 2 - LavinMQ Median Latency

LavinMQ benchmark methodology and tooling

This section will cover the server configuration we used to run the benchmarks, and generally how we arrived at the throughput and latency results we got.

Throughput: test bed

For the LavinMQ throughput test we used two instances of AWS EC2 servers with EBS GP3 drive – precisely m6g.medium(a single vCPU, 4GiB RAM). Both instances ran in a VPC.

To execute the benchmarks, we ran the load generator (lavinmqperf) on one of the EC2 instances and the LavinMQ message broker on the other instance – simply to avoid having the load generator and the broker running on the same machine and by extension reduce the effect of resource sharing.

The primary function of the load generator, lavinmqperf is to spin up makeshift producers and consumers. It then uses the producers and consumers to generate tons of messages, publish them to the broker, and then consume the messages eventually.

What broker the load generator publishes its messages to, the number of producers and consumers it spins up, and the size of messages it creates, always depends on the command it is being passed.

For example, in our case, the command below tells lavinmqperf to spin up 3 and 5 producers and consumers respectively, create 16 bytes messages, and forward the messages to the broker whose uri was specified in the command.

lavinmqperf throughput -z 120 -x 3 -y 5 -s 16 --uri=amqp://dummy-lavinm-server-uri

In the command above, -z 120 tells lavinmqperf to run the throughput test for 2 minutes = 120 seconds.

Latency: test bed

The major change we made here, as per our configuration, was the switch from lavinmqperf to rabbitmq-perf-test as our load generator. This is mostly because, rabbitmq-perf-test reports latency out of the box and lavinmqperf doesn’t.

Other than that, we used the same machines as the ones used for the throughput test – only that now, one of the instances runs rabbitmq-perf-test instead of lavinmqperf.

Again, the rabbitmq-perf-test will spin up the number of producers and consumers, create the message size and connect to the broker we pass to it in a command that’s similar to that of lavinmqperf.

The limitations of our approach

At the outset of this article, we explicitly stated that our producer, broker, and consumer configuration is not representative of real-world scenarios.

To elaborate, both lavinmqperf and rabbitmq-perf-test create consumers and producers on the same machine, which is atypical in reality. In practical settings, it is more common to have hundreds of producers and consumers distributed across multiple machines.

Conclusion

This article has demonstrated LavinMQ's performance across messages of varying sizes. We have also published a similar article that details RabbitMQ's performance benchmarks.

Ready to spin up a LavinMQ instance? Create a free LavinMQ instance at CloudAMQP.

Leave your suggestions, questions, or feedback in the comment section below or email contact@cloudamqp.com

CloudAMQP - industry leading RabbitMQ as a service

Start your managed cluster today. CloudAMQP is 100% free to try.

13,000+ users including these smart companies