Monitoring Activemq Queue Depths On Openshift With Prometheus And Grafana
In this post I will explain how to add the Prometheus agent to the Red Hat ActiveMQ image, so that Prometheus is able to scrape queue depths from the ActiveMQ running in your Openshift environment.
Much of this blog references https://brunonetid.github.io/2017/11/27/camel-prometheus-openshift.html which explains how to add the Prometheus agent for Camel.
For this blog I have used minishift.
Building the AMQ Image Extension
We need to add the Prometheus jar into the ActiveMQ image. This code is all available at my github amq63-prom-sti repository.
Override the assemble script to move the prometheus agent lib into the
A default Prometheus configuration file is also added into the image which configures the agent to expose all of the queue size JMX statistics to Prometheus.
The Openshift template then configures and extra service for prometheus and ensures that the port is exposed on the container. The template in my repository is based on the upstream template from jboss-openshift / application-templates.
In addition, the ACTIVEMQ_OPTS are in the template specifying the location of the promethus configuration file:
In order to build the AMQ63 image with the agent built in and get it running on Openshift, use the following commands:
At this point you should have a broker pod up and running, let’s get its name, and rsh into the pod and see the statistics that the prometheus agent is exposing. It should be seen that the ActivMQ destinations are exposed.
Now we need to deploy the prometheus application into the project.
At this point there should be a prometheus route available on your Openshift cluster. Use
oc get routes and navigate in your browser. You should be able to query for queue_size, as per the image below:
Note that the broker-amq-prom service is hardcoded into the
Now we need to deploy Grafana in order to see the queue depths graphed.
At this point there should be a Grafana route available on your Openshift cluster. Use
oc get routes and navigate in your browser. You should be able to use Grafana and start charting your queue depths: