Toggle Menu

Monitoring Cassandra Clusters in Kubernetes with Prometheus and Grafana

As your Kubernetes system grows, it becomes more taxing to manage your cluster including:

  • Configuring all the cluster’s components
  • Detecting anomalous conditions to prevent failures
  • When failures occur, diagnosing the root cause quickly
  • Optimizing performance and resource utilization
  • Planning for future capacity requirements

Performing this level of management at Kubernetes-scale is no simple manual process - you need powerful monitoring tools so you know how your cluster is behaving. This is where Prometheus and Grafana come in.

Prometheus is a useful tool for capturing metrics. It is a mainstay for monitoring components of Kubernetes clusters. Grafana is a customizable metrics visualization tool that integrates well with Prometheus. Together, these two tools let you monitor and successfully manage complex Kubernetes clusters.

Get Started
How Does Prometheus and Grafana fit into a Kubernetes Architecture?

How Does Prometheus and Grafana fit into a Kubernetes Architecture?

Within a Kubernetes cluster, the job of Prometheus is to capture metrics from components of interest. Prometheus captures these metrics by “scraping” endpoints provided by the components.

Since many Kubernetes components include a service, you can use a Prometheus Custom Resource Definition (CRD), called a ServiceMonitor, to create an endpoint interface for Kubernetes services.

Grafana provides a set of browser-based visualizations for the metrics Prometheus collects. Grafana does this by querying Prometheus using PromQL, the Prometheus query language.

You can deploy both Prometheus and Grafana by installing the Prometheus Operator. This operator also adds the ServiceMonitor CRD to the Kubernetes environment.

Next: How Does Prometheus Work?

How Does Prometheus Work?

The Prometheus Server consists of three modules:

  • The metrics capture component
  • A time-series database
  • The query server module

The metrics capture component “scrapes” endpoints to retrieve metrics. Prometheus allows you to specify and configure the list of endpoints it accesses. These endpoints present themselves as HTTP servers and usually have the name format of hostname/metrics. The ServiceMonitor, provided by the Prometheus Operator, connects to a Kubernetes service and presents the necessary HTTP server.

The time-series database stores the metrics retrieved by the metrics capture module.

The query server module provides access to the time-series database using PromQL as a query language. In addition to the query server, Prometheus also provides a web-based interface.

Prometheus also provides a browser-based UI for inspecting endpoints.

Note that Prometheus has a “pull-based” architecture (as opposed to a push-based approach). This architecture prevents Prometheus from being swamped with metrics being pushed by many endpoints.

Next: How do I view metrics in Grafana?
How Does Prometheus Work?
How do I view metrics in Grafana?

How do I view metrics in Grafana?

Grafana retrieves metrics from Prometheus (using PromQL) and presents these metrics in Dashboards. Grafana comes with some preconfigured Kubernetes dashboards, and you can also build your own. You configure dashboards by using a ConfigMap manifest file, which defines the dashboards, and then applying the manifest file to create the Kubernetes ConfigMap.

Next: Grafana Dashboards

Grafana Dashboards

Grafana Dashboards consist of sets of panels for viewing specific metrics. The panels include counters, gauges and histograms.

If you look at some of the dashboard configuration files, you will see that configuring Grafana dashboards can be tedious and challenging. Fortunately, this is just one of the many things that K8ssandra does for you.

Next: Skill Building
Grafana Dashboards

Skill Building

Are you ready to try it out? See how to deploy the Spring Pet Clinic Example on Kubernetes.

Monitoring with Prometheus and Grafana

Learn to monitor Cassandra with Prometheus

CONTRIBUTORS

As your Kubernetes cluster (and your Cassandra cluster) grows, you need to know how it is behaving. This is where Proemtheus and Grafana come in!

This scenario picks up where the scenario named Managing Cassandra Clusters in Kubernetes Using the Cass-Operator leaves off. If you have not yet worked through that scenario, you may want to check it out here.

In this scenario, we'll:

  • Set up a Cassandra cluster running on Kubernetes and install the example Pet Clinic app
  • Install and configure Prometheus to monitor Cassandra and Kubernetes
  • Install and configure Grafana to view metrics for Cassandra and Kubernetes
  • Configure Prometheus and Grafana to monitor our Pet Clinic app

Note: When using Kubernetes and Cassandra, it is important to recognize that we are working with two types of clusters:

  • Kubernetes clusters - a set of machines called Kubernetes nodes
  • Cassandra clusters - a set of those Kubernetes nodes that host and run the Cassandra software

 

You're going to love Prometheus and Grafana!

Time to Complete

20 minutes

Difficulty

Intermediate

Share

Do you have a suggestion or issue you’ve discovered?

Share Feedback

More Resources

Items related to Kubernetes

Prometheus Architecture Explained

Prometheus Architecture Explained

Fully understand how Prometheus Monitoring works

Watch Now
Monitoring Data

Monitoring Data

Follow these simple steps to access the Prometheus and Grafana monitoring interfaces.

Learn More
Metrics Collector

Metrics Collector

How do you know if your cluster is healthy?

Learn More