![RabbitMQ](/img/hero/rabbitmq-logomark.svg) # RabbitMQ monitoring and observability What to monitor in RabbitMQ, and how we do it across thousands of production clusters. ## From early warning to root cause RabbitMQ sits in the middle of your system. When a consumer crashes, a queue grows. When a client leaks connections, memory follows. Monitoring shows you the signs before they turn into an outage, and when something does break, the right metrics and logs point you to the cause instead of leaving you to guess. ## What to watch in RabbitMQ Start with these. Some come from RabbitMQ itself, others from the server it runs on. ### Resource headroom Memory and disk compared with their limits. When RabbitMQ hits a resource alarm, it blocks publishers. ### Queue trends Watch whether ready and unacked messages are growing, not only how many there are. A queue that has grown for 30 minutes needs attention. ### Consumers A queue with no consumers usually means something crashed. Consumer capacity shows whether consumers keep up. ### Connection churn Lots of connections opening and closing wastes resources and often points to a client bug. Keep an eye on file descriptors too. ### Message flow Published, delivered, acknowledged, redelivered, unroutable and dead-lettered messages. Changes in the ratios tell you more than the totals. ### Cluster health A node missing from the cluster needs attention right away. For quorum queues and streams, watch Raft commit latency and consumer offset lag. [See the exact metric names and ready-made alert rules in our monitoring guide](/blog/monitoring-rabbitmq-in-production/) ## How we monitor RabbitMQ at CloudAMQP We run one of the largest fleets of RabbitMQ clusters in the world. This is the setup we use on every dedicated instance. - **Prometheus metrics from every node.** We collect from the RabbitMQ Prometheus plugin, not the management API, which competes with message routing for resources. - **Server metrics next to RabbitMQ metrics.** RabbitMQ metrics stop at the Erlang VM, so we also collect CPU, memory, disk, IO and network from every server. - **Logs from every node,** labeled by cluster and node so you can tell them apart. - **Alarms with a duration.** Thresholds sit well below the limits and must hold for a while before they fire. That keeps alerts actionable. ### Our default alarms | Alarm | Fires when | | --- | --- | | Memory | Above 80% of server memory for 10 minutes | | CPU | Above 90% for 10 minutes | | Disk | Less than 5 GiB free for 10 minutes | On CloudAMQP, RabbitMQ blocks publishers at 81% of RAM (the memory watermark) and when free disk drops below 10% of the data disk. The memory alarm at 80% tells you the server is nearly full, so add your own warning at around 70% to get time to act. ## A real case: one campaign, one restart A customer running RabbitMQ on CloudAMQP sent a large outbound messaging campaign one evening: about 425,000 messages over five hours, almost all into one queue. This is what their metrics showed: - Queue depth climbed as consumers fell behind. - Disk use, flat at 1.3 GiB for weeks, rose to 3.8 GiB of 4 GiB. - Memory reached its limit and the server started swapping. - RabbitMQ blocked publishers. Publishes that normally took 0.1 seconds took 30 to 140 seconds. - Diagnostics in the console flagged the failing alarm check, and the server ran out of memory and restarted. The campaign was the trigger, not the whole cause. Queues without consumers had been growing slowly for weeks, adding to memory and disk use, without anyone being notified. Our support team moved the instance to a larger plan so it could recover, and the customer drained the backlog. Afterwards they set up memory, disk, queue-length and consumer alarms with notifications to their team, and cleaned up the forgotten queues. ## Learn how to monitor RabbitMQ Our guides, based on lessons from thousands of production clusters. [Monitoring RabbitMQ in production Which metrics, logs and alerts matter, and how to set thresholds that don't wake you up for nothing. Read the guide](/blog/monitoring-rabbitmq-in-production/) [RabbitMQ production checklist What to check before you go live, including alarms and metrics export. Read the checklist](/blog/rabbitmq-checklist-for-production-environments-a-complete-guide/) ## Built into CloudAMQP Running RabbitMQ on CloudAMQP? All of the above comes ready on dedicated instances. ### Alarms Memory, CPU and disk alarms are on from day one. Add your own for queues, consumers and connections, and send notifications to email, Slack, PagerDuty or a webhook. [Learn more about alarms](/docs/cloudamqp-alarms/) ### Diagnostics Automatic diagnostics runs 24 checks on your cluster, from RabbitMQ version and unrouted messages to connection leaks and outdated clients. Each failed check explains why it matters and how to fix it. [Learn more about diagnostics](/docs/cloudamqp-diagnostics/) ### Metrics and logs Follow every node in the console, from RabbitMQ metrics down to the server. [Learn more about metrics](/docs/cloudamqp-metrics/) ### Integrations An OpenTelemetry Collector on every node sends metrics and logs to your own stack. No agent to install. [See all integrations](/docs/cloudamqp-integrations/) ## Works with your stack Send metrics and logs to Datadog, Grafana Cloud, Prometheus, AWS CloudWatch and more. If your platform accepts Prometheus remote write or OTLP, it works with CloudAMQP. - **Native Prometheus format.** Your existing dashboards and alerts keep working. - **Structured logs.** Severity, cluster and node arrive as attributes, so you don't need parsing rules. - **Metrics filter.** Send only what you use and keep costs down. [See the full list of integrations](/docs/cloudamqp-integrations/) ## Integrations in detail What each integration does and how it works. For setup instructions, see the [docs](/docs/cloudamqp-integrations/). [Metrics Grafana Cloud Send RabbitMQ metrics to Grafana Cloud in Prometheus format and query them with PromQL. Read the post](/blog/send-broker-metrics-to-grafana-cloud-mimir/) [Metrics Prometheus remote write Push metrics to Mimir, Thanos, VictoriaMetrics or any other remote write endpoint. Read the post](/blog/prometheus-remote-write-integration/) [Logs Grafana Cloud Ship RabbitMQ logs to Grafana Cloud over OpenTelemetry. Read the post](/blog/cloudamqp-otlp-log-integrations/) [Logs Datadog Structured logs with severity tagging and one log stream per node. Read the post](/blog/datadog-v2-log-integration/) [Logs Coralogix Structured logs with region selection, sent through the OpenTelemetry Collector. Read the post](/blog/coralogix-v2-log-integration/) [Logs Any OTLP backend Send logs to SigNoz, Honeycomb or your own OTLP gateway. Read the post](/blog/otlp-log-integration/) ## Monitoring included from day one Every dedicated RabbitMQ instance on CloudAMQP comes with alarms, diagnostics, metrics, logs and integrations. [View dedicated plans](/plans/) [Read the monitoring guide](/blog/monitoring-rabbitmq-in-production/) Running a large setup or migrating? [Talk to our team](/contact/) #### Monitoring RabbitMQ yourself? Metric names, alert rules and Grafana dashboards, from thousands of production clusters. [Read the monitoring guide](/blog/monitoring-rabbitmq-in-production/) * * * #### Running on CloudAMQP? Alarms, metrics, logs and integrations are built into every dedicated instance. [View dedicated plans](/plans/)