![LavinMQ](/img/hero/lavinmq-logomark.svg) # LavinMQ monitoring and observability What to monitor in LavinMQ, where the metrics come from, and what you get on CloudAMQP. ## Disk first, then everything else LavinMQ is built to be lean: it writes messages to disk and keeps memory use low. That shifts what you watch. Free disk space is the limit that matters most, followed by queues that grow, consumers that fall behind and connections that come and go. Monitoring shows you the signs before they turn into an outage, and when something does break, the right metrics and logs point you to the cause instead of leaving you to guess. ## Where LavinMQ metrics come from LavinMQ has everything you need built in. No plugins to enable. - **Prometheus endpoint.** Aggregated metrics on `/metrics`, port 15692, bound to 127.0.0.1 by default. Point your scraper there. - **Detailed metrics.** Per-queue, per-connection and per-channel series on `/metrics/detailed`. Ask for specific families and vhosts to keep the number of series down. - **Management interface and API.** Statistics every five seconds, with about ten minutes of history. - **Logs.** Stream them live from the API, download recent logs, or route them to the internal `amq.lavinmq.log` exchange with severity as the routing key. ## What to watch in LavinMQ Start with these. Each card lists the metrics to look for on the Prometheus endpoint. ### Disk space LavinMQ blocks publishing when free disk drops below the `free_disk_min` setting. Alert well before that. `disk_space_available_bytes` ### Memory and file descriptors Compare resident memory with its limit, and open file descriptors with the maximum. `process_resident_memory_bytes``resident_memory_limit_bytes``process_open_fds``process_max_fds` ### Queue depth Watch the trend, not only the size. A rising unacked count usually means slow or stuck consumers. `queue_messages_ready``queue_messages_unacked` ### Connection churn Compare how fast connections and channels open and close. A spike points to a connection storm or a client bug. `connections_opened_total``connections_closed_total` ### Dead-lettering Any growth deserves a look. The `dead_letter_strategy` label tells you whether messages went to a dead-letter exchange or were discarded. `global_messages_dead_lettered_*_total` ### Replication In a cluster, check that every follower is connected and keeping up with the leader. `total_connected_followers``follower_lag_in_bytes` [See every metric in the LavinMQ monitoring docs](https://lavinmq.com/documentation/monitoring) ### Tips from the LavinMQ docs - Alert on queue depth, unacked growth, dead-letter growth, memory and file descriptor headroom, low disk space and follower lag. - Query the replication counters with `rate()` or `increase()`, since they reset when a follower reconnects. - Watch `socket_buffer_pool_dropped_total`. Steady growth means more busy connections at once than the buffer cache can hold. ## A real case: slow, but not overloaded A customer running LavinMQ on CloudAMQP saw message latency climb above 30 seconds and asked to scale up. CPU was at 60 percent, so the server had headroom. The metrics showed that something was slow, but not why. The logs did. For every websocket session, the customer's app opened a new connection and never closed it. Each one stayed open until LavinMQ's heartbeat timeout ended it, about 30 seconds later, and any unacknowledged messages on it were held until then. Many connections had piled up hundreds of channels that were never closed, some up to 1,000. A bigger server would only have bought time. The leak would have kept growing until it caught up again. The real fix was in the client: close connections when the session ends, and reuse channels instead of opening new ones. Watch connection and channel counts, not just CPU and memory. A steady climb with no matching rise in traffic usually means a leak. ## Learn more [LavinMQ monitoring documentation Every metric, endpoint and setting, from the LavinMQ team. Read the docs](https://lavinmq.com/documentation/monitoring) [Live broker logs in the CloudAMQP console How the Broker logstream works, and how it runs on LavinMQ streams. Read the post](/blog/live-broker-logs-in-the-cloudamqp-console-for-lavinmq-and-rabbitmq/) ## Built into CloudAMQP Running LavinMQ on CloudAMQP? This comes ready on dedicated instances. ### Alarms Set alarms on CPU, memory, disk, queues, consumers and connections, and send notifications to email, Slack, PagerDuty or a webhook. [Learn more about alarms](/docs/cloudamqp-alarms/) ### Live broker logs The Broker logstream follows your LavinMQ log live, with three hours of history for every node. [Learn more about logs](/docs/cloudamqp-log/) ### Metrics Follow every node in the console, from LavinMQ metrics down to the server. [Learn more about metrics](/docs/cloudamqp-metrics/) ### Integrations An OpenTelemetry Collector on every node sends metrics and logs to your own stack. No agent to install. [See all integrations](/docs/cloudamqp-integrations/) ## Works with your stack Send metrics and logs to Datadog, Grafana Cloud, Prometheus, AWS CloudWatch and more. If your platform accepts Prometheus remote write or OTLP, it works with CloudAMQP. - **Native Prometheus format.** Your existing dashboards and alerts keep working. - **Structured logs.** Severity, cluster and node arrive as attributes, so you don't need parsing rules. - **Metrics filter.** Send only what you use and keep costs down. [See the full list of integrations](/docs/cloudamqp-integrations/) ## Integrations in detail What each integration does and how it works. For setup instructions, see the [docs](/docs/cloudamqp-integrations/). [Metrics Grafana Cloud Send LavinMQ metrics to Grafana Cloud in Prometheus format and query them with PromQL. Read the post](/blog/send-broker-metrics-to-grafana-cloud-mimir/) [Metrics Prometheus remote write Push metrics to Mimir, Thanos, VictoriaMetrics or any other remote write endpoint. Read the post](/blog/prometheus-remote-write-integration/) [Logs Grafana Cloud Ship LavinMQ logs to Grafana Cloud over OpenTelemetry. Read the post](/blog/cloudamqp-otlp-log-integrations/) [Logs Datadog Structured logs with severity tagging and one log stream per node. Read the post](/blog/datadog-v2-log-integration/) [Logs Coralogix Structured logs with region selection, sent through the OpenTelemetry Collector. Read the post](/blog/coralogix-v2-log-integration/) [Logs Any OTLP backend Send logs to SigNoz, Honeycomb or your own OTLP gateway. Read the post](/blog/otlp-log-integration/) [Logs Uptrace Send LavinMQ logs to Uptrace over OTLP. Read the post](/blog/uptrace-log-integration/) ## Monitoring included from day one Every dedicated LavinMQ instance on CloudAMQP comes with alarms, metrics, live logs and integrations. [View dedicated plans](/plans/) [Read the monitoring docs](https://lavinmq.com/documentation/monitoring) Running a large setup or migrating? [Talk to our team](/contact/) #### Monitoring LavinMQ yourself? Which metrics, logs and health checks matter, and which alerts to set up, from thousands of production brokers. [Read the monitoring guide](/blog/monitoring-lavinmq-in-production/) * * * #### Running on CloudAMQP? Alarms, metrics, live logs and integrations are built into every dedicated instance. [View dedicated plans](/plans/)