David Jones-Gilardi

<p>This is Part 3 of a six-part series on Apache Cassandra&reg;. In the last post, we discussed <a href="https://www.datastax.com/blog/advanced-data-modeling-on-apache-cassandra">advanced data modeling</a>. Now, we&rsquo;ll use our open-source benchmarking tool,&nbsp;<a href="https://www.datastax.com/blog/nosqlbench">NoSQLBench</a>, to benchmark and stress test NoSQL databases.</p>
<p>Operators and developers benchmark and stress test their data models regularly. With mission-critical infrastructure like&nbsp;<a href="https://www.datastax.com/cassandra">Apache Cassandra&reg;</a>, it&rsquo;s standard practice, especially before going to production.</p>
<p>There are several benchmarking tools in the market but most of them require esoteric coding knowledge. NoSQLBench is simple to use while providing sophisticated benchmarking for Cassandra and other NoSQL databases. It provides results within minutes.</p>
<p>In this post, you&rsquo;ll get hands-on experience with benchmarking and stress testing Cassandra using NoSQLBench. Rather than going in-depth, our&nbsp;<a href="https://dtsx.io/38fowtF">tutorial</a>&nbsp;will scratch the surface and cover:</p>
<ol>
<li>Understanding parameters and key metrics for benchmarking</li>
<li>How cycles, bindings and statements work together</li>
<li>Experimenting with stdout</li>
<li>Scaling up a test and customizing your own scenarios</li>
<li>Packaging a performance test with named scenarios</li>
</ol>
<h2>What is NoSQLBench?</h2>
<p>NoSQLBench is an open-source, pluggable testing tool for the NoSQL ecosystem. It&rsquo;s primarily designed to test Cassandra, but you can also use it for other NoSQL technology like&nbsp;<a href="https://kafka.apache.org/documentation/">Apache Kafka</a>,&nbsp;<a href="https://docs.mongodb.com/drivers/">MongoDB</a>, and&nbsp;<a href="https://astra.datastax.com/">DataStax Astra DB</a>.</p>
<p>NoSQLBench evolved from the internal testing tool at DataStax, dsbench. It made a quantum leap in testing abilities for the Cassandra community. Engineers and customers were using it for performance testing, data model design, sizing, and deployment of new clusters.</p>
<p>After we completed dsbench, we turned it into an open-source project, so the NoSQL community could improve it further. NoSQLBench now integrates with different kinds of workloads and protocols, including CQL support.</p>
<p>If you&rsquo;d like to contribute to the project in any way, including mentoring and code reviews, let us know on&nbsp;<a href="https://dtsx.io/3MTuCiK">Twitter</a>.</p>
<h2>Why should you benchmark your data models?</h2>
<p>Companies handle massive traffic everyday on their sites as consumers spend more time online. Benchmarking ensures your data models can handle unexpected spikes without breaking down.</p>
<p>It&rsquo;s critical to test your data model before going into production. It lets you understand how your system behaves and performs as it scales, before you release it into the world.</p>
<p>Highly experienced data operators learn to gauge signals and metrics without performing a benchmark. For everyone else however, testing your data models is imperative &ndash; especially if you like to play it safe.</p>
<h2>NoSQLBench with Cassandra exercises</h2>
<p>The benchmarking exercises take roughly two hours to complete. They contain preconfigured scenarios on&nbsp;<a href="https://katacoda.com/datastax/courses/nosqlbench-intro">Katacoda</a>, resources on&nbsp;<a href="https://dtsx.io/3uSfDiK">GitHub</a>, and step-by-step instructions in this&nbsp;<a href="https://dtsx.io/38fowtF">YouTube video</a>. We handled the Cassandra backend so you can focus entirely on NoSQLbench.</p>
<p>In the hands-on exercises, you will:</p>
<ol>
<li>Run your first NoSQLBench benchmark against Cassandra with pre-packaged data models</li>
<li>Investigate NoSQLBench metrics from Cassandra using Grafana and Prometheus</li>
<li>Create your custom workloads to benchmark with NoSQLBench</li>
</ol>
<h3>Exercise 1: Executing NoSQLBench commands against Cassandra</h3>
<p>The first exercise shows you how to execute NoSQLBench commands against Cassandra with pre-packaged data models. You can follow this exercise manually using&nbsp;<a href="https://dtsx.io/3rBK9LC">GitHub</a>&nbsp;or to run this<a href="https://katacoda.com/datastax/courses/nosqlbench-intro/nosqlbench">&nbsp;Katacoda scenario</a>.</p>
<p>If you&rsquo;re doing this manually, set up Docker through this&nbsp;<a href="https://dtsx.io/3vqFrlb">0-setup readme</a>&nbsp;first. Docker is an open-source virtualization software that automates deployment of software applications inside containers. Once your Docker is running, start a Cassandra database and&nbsp;<a href="https://dtsx.io/3vqFrlb">download NoSQLBench as a jar file</a>.</p>
<p>We strongly recommend that you create a directory to use with NoSQLBench and download it there. This will make it easier for you to organize the many files that you&rsquo;ll generate during these exercises. Follow along our&nbsp;<a href="https://dtsx.io/3MsjE3r">video</a>&nbsp;tutorial or this&nbsp;<a href="https://dtsx.io/3rBK9LC">GitHub link</a>&nbsp;to complete the exercises below:</p>
<ul>
<li><a href="https://dtsx.io/3rBK9LC">Execute an initial run of your NoSQLBench commands</a></li>
<li><a href="https://dtsx.io/3JVDpie">Create a test schema</a></li>
<li><a href="https://dtsx.io/3xEoNBi">Write initial ramp-up data</a></li>
<li><a href="https://dtsx.io/36xL1tC">Perform a benchmark test</a></li>
<li><a href="https://dtsx.io/3KZJJ9s">Analyze the results</a></li>
</ul>
<h3>Exercise 2: Investigate NoSQLBench metrics from Cassandra with Grafana</h3>
<p>In this exercise, you&rsquo;ll view and analyze the metrics in charts and graphs on Grafana. Grafana is an open-source analytics and interactive visualization application. You can also run this&nbsp;<a href="https://katacoda.com/datastax/courses/nosqlbench-intro/nosqlbench-grafana">Katacoda scenario</a>&nbsp;or get the code for this section on&nbsp;<a href="https://dtsx.io/38420Ut">GitHub</a>. Follow the instructions in this&nbsp;<a href="https://dtsx.io/3L880uh">video</a>.</p>
<p>Here&rsquo;s what you need to do for this exercise:</p>
<ul>
<li><a href="https://dtsx.io/3Op6N3C">Export metrics to Grafana</a></li>
<li><a href="https://dtsx.io/3uWjqLU">Launch Grafana</a></li>
<li><a href="https://dtsx.io/3rBz8Ks">View various metrics</a></li>
<li><a href="https://dtsx.io/3OrlVOe">Launch Prometheus and view metrics in more detail</a></li>
</ul>
<p>When you&rsquo;ve completed these steps, we can look at four types of metrics on Grafana and what they represent:</p>
<ol>
<li><strong>Ops and successful ops</strong>: this graph indicates the operation rate (reads/writes per second) of the benchmark. There should be no discrepancies between ops and success metrics</li>
<li><strong>Error Counts</strong>: if there are discrepancies between ops and success metrics, you will see errors here. Ideally, there shouldn&rsquo;t be any</li>
<li><strong>Service Time distribution</strong>: service time measures how quickly your NoSQL database responds to requests coming in from the client&rsquo;s application server</li>
<li><strong>Op tries distribution</strong>: this graph shows the tries or retries it took to execute your operation. If there are too many retries, your database is overloaded, which warrants an investigation immediately</li>
</ol>
<p><em><img src="https://cdn.sanity.io/images/bbnkhnhl/production/49fc47994728d1b9c391c5cac0542dc076ae7a22-512x486.png" alt="Screenshot of Grafana Metrics" width="512" height="486" /></em></p>
<p><em>Figure 1. Four types of metrics on Grafana.</em></p>
<h3>Exercise 3: Customizing workloads in NoSQLBench</h3>
<p>In the previous exercises, you used pre-packaged workloads to run the NoSQLBench commands. Now, we&rsquo;ll guide you to create custom workloads for your own application or database. This section is optional but recommended.</p>
<p>To get started, run this&nbsp;<a href="https://katacoda.com/datastax/courses/nosqlbench-intro/nosqlbench-modify-workload">Katacoda scenario</a>&nbsp;or use the code on&nbsp;<a href="https://dtsx.io/38RsLvQ">GitHub</a>. Find detailed instructions in this&nbsp;<a href="https://dtsx.io/3xPMuGK">video</a>&nbsp;and complete these steps:</p>
<ul>
<li><a href="https://dtsx.io/3OjK2hA">List workloads and named scenarios</a></li>
<li><a href="https://dtsx.io/3vqFRrL">Copy workloads</a></li>
<li><a href="https://dtsx.io/394dmZl">Build your own workload</a></li>
<li><a href="https://dtsx.io/3KZJ0Fw">Combine everything into a single workload file</a></li>
</ul>
<h2>Conclusion</h2>
<p>You&rsquo;re already halfway through our Cassandra series. If you want to keep learning, be on the lookout for our upcoming Part 4 where we'll dive into Storage-Attached Indexes. Then to finish off strong, in Part 5 and Part 6 we'll&nbsp; show you how to migrate your SQL applications to NoSQL and give you a hands-on exercise.</p>
<p>Our&nbsp;<a href="https://www.datastax.com/blog/get-your-definitive-guide-apache-cassandra-40">Definitive Cassandra Guide</a>&nbsp;will also be incredibly useful to learn everything that you need to know about Cassandra.&nbsp;</p>
<p>Check out our&nbsp;<a href="https://dtsx.io/3rITFNk">YouTube channel</a>&nbsp;for tutorials and&nbsp;<a href="https://dtsx.io/3xMdk2H">DataStax Developers on Twitter</a>&nbsp;for the latest news about our developer community.</p>
<h2>Resources</h2>
<ol>
<li><a href="https://www.datastax.com/blog/nosqlbench">DataStax NoSQLBench</a></li>
<li><a href="https://dtsx.io/38fowtF">YouTube Tutorial: Benchmark your NoSQL Database</a></li>
<li><a href="https://www.datastax.com/products/datastax-astra">Astra DB: Multi-cloud DBaaS built on Apache Cassandra</a></li>
<li><a href="https://katacoda.com/datastax/courses/nosqlbench-intro">NoSQLBench Learning Series for Apache Cassandra by DataStax</a></li>
<li><a href="https://dtsx.io/3uSfDiK">NoSQL Bench Workshop Online GitHub</a></li>
<li><a href="https://auth.cloud.datastax.com/auth/realms/CloudUsers/protocol/saml/clients/absorb">DataStax Academy</a></li>
<li><a href="https://community.datastax.com">DataStax Community</a></li>
<li><a href="https://www.datastax.com/blog/get-your-definitive-guide-apache-cassandra-40">Definitive Cassandra Guide by DataStax</a></li>
<li><a href="https://dtsx.io/3L9K5KL">Apache Pulsar Performance Testing with NoSQLBench</a></li>
</ol>

A Beginner's Guide to Benchmarking with NoSQLBench

David Jones-Gilardi

Discover more

Share

Share

What is NoSQLBench?

Why should you benchmark your data models?

NoSQLBench with Cassandra exercises

Exercise 1: Executing NoSQLBench commands against Cassandra

Exercise 2: Investigate NoSQLBench metrics from Cassandra with Grafana

Exercise 3: Customizing workloads in NoSQLBench

Conclusion

Resources

More Technology

How Winweb Built its AI Assistant with DataStax Astra DB and LangChain

Vercel + Astra DB: Get Data into Your GenAI Apps Fast

Simplifying Agent Development with Astra DB Connector for Vertex AI Search

Making Astra DB easier for MongoDB developers

One-stop Data API for Production GenAI