CloudAMQP and RabbitMQ Use cases

RabbitMQ is a simple/traditional publish-subscribe message broker. It is usually used as middleman between microservices, where a system simply needs to notify another part of the system to start to work on a task, like ordering handling in a webshop (order placed, update order status, send order, payment, etc.). Another main use case is for long-running tasks or when you want to perfome reliable background jobs.

RabbitMQ use case example

Imagine that you have a web service that receives many requests every second, where no request is afford to get lost and all requests needs to be processed by a process that is time consuming.

Imagine that your web service always has to be highly available and ready to receive new request instead of being locked by the processing of previous received requests.

In this case it is ideal to put a queue between the web service and the processing service. The two processes will be decoupled from each other and does not need to wait for each other. If you have a lot of requests coming in a short amount of time, the processing system will be able to process them all anyway. The queue will persist requests if their number becomes really huge.

You can then imagine that your business and your workload is growing and you need to scale up your system. All that is needed to be done now is to add more workers to work off the queues faster.

In addition to providing a buffer between a web service and another processing service, message queues can be used for more advanced scenarios. RabbitMQ can be configured to route and distribute messages according to different rules and different processes. Messages can be added to different queues depending on how the messages should be handled. The processing systems can then process them at their leisure.

CloudAMQP Customer stories

Long-running tasks at Softonic

Read up about how RabbitMQ can be used in an event-based microservices architecture, to support 100 million users a month.

The software and app discovery portal Softonic is reached by over 100 million users per month, delivers more than 2 million downloads per day and has a constant flow of events and commands between their services. RabbitMQ is used as a message queue between microservices, contributes to a reliable, fast and effective architecture perfect for their purpose.

Users can upload files to Softonic. All uploaded files are scanned for virus and information about the file is collected, before the file is distributed to other users. The new binary data is, first of all, persisted within a dedicated service, and a notification about the upload is sent to the message queue. Other services collect this information which in the end will be added to the website. In this case, the user gets notified straight after the upload has succeeded and a scanning event is simply placed on the message queue for other services to handle. The message queue allows web servers to respond to requests quickly instead of being forced to perform a resource-heavy process on the spot, and instead of keeping the user waiting.

Breaking down a monolithic system into microservices

Parkster (a digital parking service) are breaking down their system into multiple microservices by using RabbitMQ.

Like many other companies, Netflix just to mention one, Parkster started out with a monolithic architecture. They wanted to have their business model proven before they went further. A monolithic application is where the whole application is built as a single unit. All code for a system is in a single codebase that is compiled together and produces a single system.

Having one codebase seemed as the easiest and fastest solution at the time. It solved their core business problems, which included connecting devices with people, parking zones, billing, and payments. A few years later, they decided to break up the monolith into multiple small codebases. Which they did through multiple microservices communicating via message queues.

Long-running tasks - image scaling at Hemnet

The property listing platform, Hemnet, moved from a full on-premise solution to a cloud-based solution in less than a year. This is the story about how they changed from in-house to the cloud, and how RabbitMQ became an important player during the migration.

Hemnet uses RabbitMQ, among other things, as a pipeline for image scaling. Once a real estate broker adds a new image of the property, an image scaling task is given to RabbitMQ. The image scaling task stays in the message queue until Hemnet’s image scaling service grabs the task from the queue, scale the selected image, and in the end, the image is ready to be shown on the website or in the app with the new size.

Machine-to-machine chat application at FarmBot

FarmBot is an open-source robotic hardware kit designed for gardeners, researchers, and educators to interact with agricultural projects in a more efficient way. RabbitMQ communicates to the devices in the field by AMQP and acts as a message queue to the backend services. MQTT protocol is used for real-time events to the frontend user interface.

RabbitMQ is now an important component of the FarmBot web API, where it handles various tasks including:

  • Passing push notifications between users and devices
  • Passing background messages between server background workers
  • Uses a set of custom authorization plugins (to prevent unauthorized use)

Because RabbitMQ is a real-time message broker, there is no need to check for new messages. Messages, such as a user clicking the "move" button on the user interface, are sent back and forth between client, device, and server without initiating requests.

In many ways, the message broker acts as a machine-to-machine chat application. Any software package, whether it be the REST API, FarmBot OS or third-party firmware, can send a message to any other entity that is currently connected to the message broker, provided it has the correct authorization.

The complete user story can be found here.

A Business Intelligence solution based on SpringXD and RabbitMQ

A while ago Cortex Intelligence decided to redesign their Business Intelligence (BI) solution. The usage of their platform was growing and their SQL server was not acting as quickly as they wished for. Changes had to be made and RabbitMQ became part of their product. The complete use case can be found here: A Business Intelligence solution based on SpringXD and RabbitMQ

Why should we use message queues and CloudAMQP?

A message queue will keep the processes in your application separated and independent of each other. The first process will never need to invoke another process, or post notifications to another process, or follow the process flow of the other processes. It can just put the message on the queue and then continue processing. The other processes can also handle their work independently. They can take the messages from the queue when they are able to process them.

This way of handling messages creates a system that is easy to maintain and easy to scale.

CloudAMQP Features and Benefits

  • Application decoupling

    Instead of building a massive application, is it beneficial to decouple different concerns in your application and only communicate between them asynchronously with messages. That way different parts of your application can evolve independently, be written in different languages and/or maintained by separated developer teams.

  • Reliable

    Messages can be stored-and-forwarded. A message can be redelivered until the message is processed.

  • Offload your data store

    Instead of polling your data store, publish a message when new data is inserted. Interested parties will be notified immediately and your data store will be ready to answer qualified queries instead.

  • Scalability

    Message queueing is the key to make your application scale. As the workload goes up you only have to add more workers to work off the queues faster.

  • Resilience

    Message queues decouples your processes. If a process that is processing messages from the queue fails, other messages can still be added to the queue and be processed when the system has recoved.

  • Asynchronous Communication

    Message queues enable asynchronous processing, meaning that it allows you to put a message on a queue without processing it immediately.

  • Realtime applications

    CloudAMQP is an excellent backend for realtime applications. Notifications and message streaming is handled very effectively by RabbitMQ, you will be able to push thousands of messages per second.

Benefits of cloud-based messaging

Cloud-based messaging will increase the speed of getting the application to the market. Developers can focus on the core part of their applications, instead of managing and maintaining servers and queueing infrastructure, like updates and mirroring of clusters.