Imagine that you have a web service that receives many requests every second, where no request is afford to get lost and all requests needs to be processed by a process that is time consuming.
Imagine that your web service always has to be highly available and ready to receive new request instead of being locked by the processing of previous received requests.
In this case it is ideal to put a queue between the web service and the processing service. The two processes will be decoupled from each other and does not need to wait for each other. If you have a lot of requests coming in a short amount of time, the processing system will be able to process them all anyway. The queue will persist requests if their number becomes really huge.
You can then imagine that your business and your workload is growing and you need to scale up your system. All that is needed to be done now is to add more workers to work off the queues faster.
In addition to providing a buffer between a web service and another processing service, message queues can be used for more advanced scenarios. RabbitMQ can be configured to route and distribute messages according to different rules and different processes. Messages can be added to different queues depending on how the messages should be handled. The processing systems can then process them at their leisure.
A while ago the Brazilian based company Cortex Intelligence decided to redesign their Business Intelligence (BI) solution. The usage of their platform was growing and their SQL server was not acting as quickly as they wished for. Changes had to be made and RabbitMQ became part of their product. The complete use case can be found here: A Business Intelligence solution based on SpringXD and RabbitMQ
A message queue will keep the processes in your application separated and independent of each other. The first process will never need to invoke another process, or post notifications to another process, or follow the process flow of the other processes. It can just put the message on the queue and then continue processing. The other processes can also handle their work independently. They can take the messages from the queue when they are able to process them.
This way of handling messages creates a system that is easy to maintain and easy to scale.
Instead of building a massive application, is it beneficial to decouple different concerns in your application and only communicate between them asynchronously with messages. That way different parts of your application can evolve independently, be written in different languages and/or maintained by separated developer teams.
Messages can be stored-and-forwarded. A message can be redelivered until the message is processed.
Instead of polling your data store, publish a message when new data is inserted. Interested parties will be notified immediately and your data store will be ready to answer qualified queries instead.
Message queueing is the key to make your application scale. As the workload goes up you only have to add more workers to work off the queues faster.
Message queues decouples your processes. If a process that is processing messages from the queue fails, other messages can still be added to the queue and be processed when the system has recoved.
Message queues enable asynchronous processing, meaning that it allows you to put a message on a queue without processing it immediately.
CloudAMQP is an excellent backend for realtime applications. Notifications and message streaming is handled very effectively by RabbitMQ, you will be able to push thousands of messages per second.
Cloud-based messaging will increase the speed of getting the application to the market. Developers can focus on the core part of their applications, instead of managing and maintaining servers and queueing infrastructure, like updates and mirroring of clusters.