RabbitMQ is a simple, traditional publish-subscribe message broker. It is usually used as middleman between microservices, where a system simply needs to notify another part of the system to start to work on a task, like order handling in a webshop (order placed, update order status, send order, payment, etc.). Long-running tasks or performing reliable background jobs are also significant use cases for RabbitMQ.
Imagine a web service that receives many requests every second. Nothing can afford to get lost, and all requests need to run through a time-consuming process.
Further, imagine that your web service always has to be highly available and ready to receive new requests instead of being locked by processing previous requests.
In this case, placing a queue between the web service and the processing service is ideal. The two processes will be decoupled and do not need to wait for each other. If you have a lot of requests coming in a short amount of time, the processing system will be able to take care of them all. The queue will persist requests if their number becomes huge.
Imagine that your business and workload are growing, and you need to scale up the system. All that is needed is to add more workers to work off the queues faster.
In addition to providing a buffer between a web service and another processing service, message queues are also suitable for more advanced scenarios, like routing and distributing messages according to different rules and different processes. Messages can be added to different queues depending on how they should be handled. The processing systems can then process them at their leisure.
Read about how RabbitMQ can be used in an event-based microservices architecture, to support 100 million users a month as it does for Softonic.
The software and app discovery portal Softonic reaches over 100 million users per month, delivers more than 2 million downloads per day, and has a constant flow of events and commands between its services. RabbitMQ is used as a message queue between microservices, contributing to a reliable, fast, and effective architecture perfect for Softonic’s purpose.
Users upload files to Softonic, and all uploaded files are scanned for viruses while information about the file is collected before being distributed to other users. The new binary data is first persisted within a dedicated service, and a notification about the upload is sent to the message queue. Other services collect this information which, in the end, will be added to the website. In this case, the user gets notified immediately after the upload has succeeded, and a scanning event is simply placed on the message queue for other services to handle. The message queue allows web servers to respond to requests quickly instead of performing a resource-heavy process on the spot and keeping the user waiting.
Parkster, a digital parking service, is breaking down its system into multiple microservices by using RabbitMQ.
Like many other companies, including Netflix, Parkster started with a monolithic architecture. They wanted to prove the business model before they developed further. In monolithic applications, the whole application is built as a single unit. All code for a system is in a single codebase compiled together and produced as a single system.
Having one codebase seemed like the easiest and fastest solution at the time. It solved their core business problems, including connecting devices with people, parking zones, billing, and payments. A few years later, Parkster decided to break up the monolith into multiple small codebases, which they did through numerous microservices communicating via message queues.
The property listing platform Hemnet moved from a full-on premise solution to a cloud-based solution in less than a year. Read their story about how they changed from in-house to the cloud, and how RabbitMQ became an important player during the migration.
Hemnet uses RabbitMQ, among other things, as a pipeline for image scaling. Once a real estate broker adds a new property image, it sends an image scaling task to RabbitMQ.
The task stays in the message queue until Hemnet’s image scaling service grabs it from the queue, scales the selected image, making it ready to be shown on the website or in the app with the new size.
FarmBot is an open-source robotic hardware kit designed for gardeners, researchers, and educators to interact with agricultural projects in a more efficient way. RabbitMQ communicates to the devices in the field by AMQP and acts as a message queue to the backend services. MQTT protocol is used for real-time events to the frontend user interface.
RabbitMQ is now an essential component of the FarmBot web API, where it handles various tasks, including:
Because RabbitMQ is a real-time message broker, there is no need to check for new messages. When users click the "move" button on the user interface, they send back and forth between client, device, and server without initiating requests.
In many ways, the message broker acts as a machine-to-machine chat application. Any software package, whether it be the REST API, FarmBot OS, or third-party firmware, can send a message to any other entity currently connected to the message broker, given it has the correct authorization.
Read the whole user story here.
When using a message broker , the different parts of your application works independently, detached from each other. This means that one process won't need to consult another or even post notifications to it.
This way of addressing messages creates a system that is easy to maintain and easy to scale.
Instead of building a massive application, it is beneficial to decouple different concerns in your application and only communicate between them asynchronously with messages. That way, other parts of your application can evolve independently, be written in various languages and/or be maintained by separated developer teams.
Messages can be stored and forwarded. A message can be delivered multiple times until the message is processed.
Instead of polling your data store, publish a message when new data is inserted. Interested parties will be notified immediately, and your data store will be ready to answer qualified queries instead.
Message queueing is the key to making your application scale. As the workload increases you only have to add more workers to work off the queues faster.
Message queues decouple your application. If a process handling messages from the queue fails, other messages can still be added to the queue and be dealt with when the system has recovered.
Message queues enable asynchronous processing, meaning that a message can be put on a queue without dealing with it immediately.
CloudAMQP is an excellent backend for real-time applications. Notifications and message streaming is handled very effectively by RabbitMQ, thanks to its ability to push thousands of messages per second.
Cloud-based messaging increases the speed of getting the application to the market. Developers can focus on the core part of their applications instead of managing and maintaining servers and queueing infrastructure like updates and mirroring of clusters.