Manikandan Srinivasan

<p>Our mission at DataStax is to deliver strong value to our customers by providing advanced tools and solutions to manage and monitor&nbsp;<u><a href="https://www.datastax.com/products/datastax-enterprise">DataStax Enterprise</a></u>&nbsp;(DSE) with ease. With complex applications deployed across different environments, sending DSE diagnostic metrics to a centralized, unified, scalable, and flexible monitoring solution that provides a fast and accurate problem resolution is critical to helping administrators troubleshoot problems.</p>

<p>Furthermore, to troubleshoot production slowdowns, it is vital to be able to correlate application and network activity with the impact on the database from a single lens. When it comes to monitoring a distributed platform, there are many ways to go about it. At DataStax we understand this, and with DSE 6.7 we are proud to announce the&nbsp;<u><a href="https://docs.datastax.com/en/dse/6.7/dse-dev/datastax_enterprise/tools/metricsCollector/mcIntroduction.html">DSE Metrics Collector</a></u>, which lets you integrate DSE metrics with any&nbsp;<u><a href="https://collectd.org/documentation/manpages/collectd.conf.5.shtml">standard</a></u>&nbsp;centralized monitoring solution of your choice.</p>

<p>In this post, we’ll explain the high-level architecture of the DSE Metrics Collector and show you how to set up simple end-end monitoring with Prometheus exporter visualized via a Grafana Dashboard.</p>

<h2>Architecture</h2>

<p>To provide a flexible and familiar experience, the DSE Metrics Collector is based on an industry standard collector. This implementation allows users to integrate the DSE and OS metrics into any standard enterprise monitoring system such as Prometheus, Nagios, Graphite, and many more. Furthermore, to improve operator efficiency, the DSE server automatically manages this service for you; i.e., no separate agent software is required.</p>

<p><img alt="" height="552" src="https://www.datastax.com/sites/default/files/content/blog/image-1-1-1.jpg" width="700" /></p>

<h3>Customize the DSE Metrics Collector</h3>

<p>The DSE Metrics Collector is enabled by default and comes with a comprehensive set of knobs and configuration options that can be customized based on your environment. For instance, to view the default configuration, run the following command:</p>

<p><img alt="" height="276" src="https://www.datastax.com/sites/default/files/content/blog/image-2-1-1.jpg" width="700" /></p>

<p>By default, the DSE Metrics Collector ships with a minimal set of filters with the goal of running the Collector transparently alongside your DSE installation. Based on your requirements, you can always override these or define your own set of rules. To view the default filters, run the following command:</p>

<p><img alt="" height="304" src="https://www.datastax.com/sites/default/files/content/blog/image-3-2.jpg" width="700" /></p>

<p>Refer to&nbsp;<u><a href="https://docs.datastax.com/en/dse/6.7/dse-admin/datastax_enterprise/tools/dsetool/dsetoolInsights_filters.html">documentation</a></u>&nbsp;to view the exhaustive list of options.</p>

<h3>Export and Visualize metrics</h3>

<p>After configuring the DSE Metrics Collector, you can export the DSE metrics to any centralized monitoring environment.</p>

<h4>Enable the Prometheus plugin on DSE Cluster</h4>

<p>Before configuring Prometheus, each node must be configured to expose metrics to the Prometheus server. First, let’s add a Prometheus exporter on all DSE nodes. Depending on the install type, create a prometheus.conf file in one of the indicated&nbsp;<u><a href="https://docs.datastax.com/en/dse/6.7/dse-dev/datastax_enterprise/tools/metricsCollector/mcConfigureMetricsCollection.html">directories</a></u>, and add the following snippet to the configuration file.</p>

<p><img alt="" height="186" src="https://www.datastax.com/sites/default/files/content/blog/image-4-2.jpg" width="504" /></p>

<h4>Configure the Prometheus server</h4>

<p>To use Prometheus with DSE Metrics Collector, download the&nbsp;<u><a href="https://github.com/datastax/dse-metric-reporter-dashboards/blob/master/prometheus/prometheus.yml">prometheus.yaml</a></u>&nbsp;file from GitHub and save it in the directory appropriate for your installation type. The prometheus.yaml file is preconfigured to scrape metrics reported by the DSE Metrics Collector. Additionally, download the&nbsp;<u><a href="https://github.com/datastax/dse-metric-reporter-dashboards/blob/master/prometheus/tg_dse.json">tg_dse.json</a></u>&nbsp;file, which the prometheus.yaml file references to obtain the name of your cluster and IP addresses for each node.</p>

<p><img alt="" height="319" src="https://www.datastax.com/sites/default/files/content/blog/image-5-1.jpg" width="598" /></p>

<p>Finally, start your Prometheus server</p>

<p>./prometheus –config.file=/path/prometheus.yaml</p>

<h4>Create dashboard visualizations</h4>

<p>Once Prometheus is scraping the endpoints where the DSE Metrics Collector is running, add a data source pointing to the Prometheus server configured in the previous step.</p>

<p><img alt="" height="703" src="https://www.datastax.com/sites/default/files/content/blog/image-6.jpg" width="511" /></p>

<p>Now, you can create a Grafana dashboard using the configured Prometheus data source.</p>

<p><img alt="" height="614" src="https://www.datastax.com/sites/default/files/content/blog/image-7.jpg" width="1112" /></p>

<p>To make this simpler, we have created a getting-started Github&nbsp;<u><a href="https://github.com/datastax/dse-metric-reporter-dashboards">project</a></u>&nbsp;that automates all the steps outlined in this post. Simply clone the project, follow the README, and explore the sample dashboards that we have shipped with this project!</p>

<h2>Conclusion</h2>

<p>The DSE Metrics Collector simplifies your monitoring experience and provides you with the capability to integrate DSE metrics into any standard centralized monitoring solution of your choice. We are confident that you will enjoy this new feature to monitor your DSE clusters with confidence!&nbsp;<u><a href="https://academy.datastax.com/quick-downloads">Download DSE 6.7</a></u>&nbsp;for free today and let us know what you think.</p>


Improved Performance Diagnostics With DataStax Metrics Collector

Manikandan Srinivasan

Discover more

Share

More Company

DataStax Astra DB Nabs Three Prestigious 2023 TrustRadius “Best of” Awards, Dominates the Vector Databases Category

DataStax Has Achieved ISO 27001 Certification

Keeping the Momentum Rolling with Google Cloud

My Internship Journey: A Dive into AI

NoSQL and Vector DB
for Generative AI,
Instantly, At Scale

Manikandan Srinivasan

Discover more

Share

More Company

DataStax Astra DB Nabs Three Prestigious 2023 TrustRadius “Best of” Awards, Dominates the Vector Databases Category

DataStax Has Achieved ISO 27001 Certification

Keeping the Momentum Rolling with Google Cloud

My Internship Journey: A Dive into AI

NoSQL and Vector DBfor Generative AI,Instantly, At Scale

NoSQL and Vector DB
for Generative AI,
Instantly, At Scale