Raghavan Srinivas

<p>The daily operations of a cloud-native platform can be challenging and time-consuming. Especially when it involves persistent data. One minute you&rsquo;re scaling up nodes, because your application grows. The next you&rsquo;re restoring backups. The good news is, you can automate many operations with K8ssandra &ndash; the operational data platform for Kubernetes.</p>
<p>A typical cloud-native platform has one or more of the following features:</p>
<ul>
<li>Infrastructure-as-a-service (IaaS) offering computation power and storage</li>
<li>A Kubernetes engine to help manage and scale containerized nodes and applications</li>
<li>A database-as-a-service (DBaaS) to help with scaling and operations (like&nbsp;<a href="https://www.datastax.com/products/datastax-astra">DataStax Astra DB</a>)</li>
<li>Microservices that help orchestrate the complete solution</li>
</ul>
<p>You can spend a lot of your time on day-to-day challenges, if you&rsquo;re manually managing a cloud-native platform. Thankfully there are automated solutions to help you create backups, access data, and serve simple monitoring.</p>
<h2>Using K8ssandra to manage containers</h2>
<p>If you want to automate processes like deploying, managing, and scaling containerized applications &ndash; and still get the benefits of a distributed NoSQL database &ndash; then&nbsp;<a href="https://dtsx.io/3rGkS2O">K8ssandra</a>&nbsp;is your solution.</p>
<p>K8ssandra is a cloud-native distribution of&nbsp;<a href="https://www.datastax.com/cassandra">Apache Cassandra</a>&reg; that runs on&nbsp;<a href="https://kubernetes.io/">Kubernetes</a>. You get Cassandra&rsquo;s power of scale and high availability combined with Kubernetes&rsquo; capability to manage containers. Whether you&rsquo;re installing locally or in the cloud,&nbsp;<a href="https://dtsx.io/3vrEOaM">K8ssandra</a>&nbsp;is&nbsp;<a href="https://dtsx.io/3EwboN3">easy to install</a>&nbsp;with a Helm command and ability to provide custom values in k8ssandra-values.yaml.&nbsp;</p>
<h2>Day-to-day challenges in cloud-native platforms</h2>
<p>K8ssandra comes drop-loaded with different components, each with the goal to help automate your daily operations.</p>
<h3>Monitoring Kubernetes with K8ssandra</h3>
<p>Monitoring is an essential aspect of all systems and applications. The sooner you know something is wrong, the faster you can start to fix it. K8ssandra deploys&nbsp;<a href="https://dtsx.io/3rJ8hMB">metrics collectors for Apache Cassandra</a>&nbsp;(MCAC) to the Kubernetes environment in order to provide useful metrics.</p>
<p><a title="Dashboard images showing Prometheus monitoring screens" href="https://cdn.sanity.io/images/bbnkhnhl/production/985b8c3196c8f42a1685b521c83ed1cb79242f2b-512x274.png"><img src="https://cdn.sanity.io/images/bbnkhnhl/production/985b8c3196c8f42a1685b521c83ed1cb79242f2b-512x274.png" alt="Graphic showing API extensions available through Stargate.io" width="512" height="274" /></a></p>
<p><em>Figure 1. Visuals of the dashboard that can be created by MCAC.</em></p>
<p>MCAC is built on the open-source solutions&nbsp;<a href="https://prometheus.io/">Prometheus</a>&nbsp;and&nbsp;<a href="https://grafana.com/">Grafana</a>&nbsp;for monitoring and dashboards. MCAC tracks these metrics via a Java agent and can give you over 100,000 unique metric series per node, such as operations per second or heap usage.</p>
<p>Prometheus gathers all the data and stores it as a multi-dimensional data model.&nbsp;<a href="https://prometheus.io/docs/prometheus/latest/querying/basics/">PromQL</a>, the flexible query language for Prometheus, is used to access this data model. You&rsquo;ll find that PromQL is much more powerful and rich for queries than its competitors.</p>
<p>Having lots of metrics doesn&rsquo;t do you any good if you can&rsquo;t read them easily. And that&rsquo;s where Grafana can help. Grafana turns collected data into highly customizable dashboards.</p>
<h3>Management API for Cassandra</h3>
<p>Managing Cassandra operations is mainly command-line driven. It&rsquo;s usually outsourced to a team of experts who understand management of the operational tools. This approach produces an uneven set of best practices and risks corners being cut in the process.</p>
<p>With K8ssandra, you get the power of a management API and the ability to:</p>
<ul>
<li>Start nodes</li>
<li>Stop nodes</li>
<li>Customize configuration</li>
<li>Perform Kubernetes liveness/readiness checks</li>
</ul>
<h3>Scaling up and down Cassandra</h3>
<p>If your web application is flourishing, and more people are using it, your database will likely need to grow quickly. You can add nodes and remove them in Cassandra simply by altering the k8ssandra-values.yaml because of the distributed database structure.</p>
<p><img src="https://cdn.sanity.io/images/bbnkhnhl/production/884930dfe6522f3e9353c0f431fff9150b8fe321-512x270.png" alt="Graph showing number of operations per second" width="512" height="270" /></p>
<p><em>Figure 2. A graph showing the linear progression in the number of operations per second</em></p>
<p>These are the properties within the k8ssandra-values.yaml:</p>
<ul>
<li>Cluster name for your Cassandra installation</li>
<li>Name of data center used with your Stargate installation</li>
<li>The size of the data center</li>
</ul>
<p>A simple YAML used in the Helm install could look something like this:</p>
<p><code>cassandra:</code></p>
<p><code>clusterName: my-k8ssandra</code></p>
<p><code>datacenters:</code></p>
<p><code>- name: dc1</code></p>
<p><code>size: 3</code></p>
<p>Altering the size of the installation is as simple as changing the size property and applying the changes with helm upgrade:</p>
<p><code>helm upgrade my-k8ssandra k8ssandra/k8ssandra -f k8ssandra-values.yaml</code></p>
<h3>API access with K8ssandra</h3>
<p>A database isn&rsquo;t much use if it&rsquo;s just sitting there storing data. And how useful will your application be if a user can&rsquo;t interact with it? You need to be able to complete the four basic operations:</p>
<ul>
<li>Create</li>
<li>Read</li>
<li>Update</li>
<li>Delete</li>
</ul>
<p>To help you with this K8ssandra deploys&nbsp;<a href="https://dtsx.io/3Moiyp5">Stargate</a>. Stargate is a unified API gateway that sits between your app and your database. It gives you a range of different APIs to do the above basic operations. Some of the APIs are:</p>
<ul>
<li>REST</li>
<li>GraphQL</li>
<li>Document API</li>
<li>gRPC</li>
</ul>
<p><img src="https://cdn.sanity.io/images/bbnkhnhl/production/1f1a50e1c8e697aa833236f3fa1dfdf686fb7359-512x200.png" alt="Graphic showing various API extensions available through Stargate.io" width="512" height="200" /></p>
<p><em>Figure 3. A diagram showing the various API extensions available through Stargate.io.</em></p>
<p>You can either use an API you&rsquo;re familiar with or pick the one that&rsquo;s just right for the job from the mix of different APIs.</p>
<h3>Backup and restore with K8ssandra</h3>
<p>No matter how careful you may be, sometimes the worst happens. Fortunately, K8ssandra allows you to backup and restore your Cassandra database with the help of&nbsp;<a href="https://dtsx.io/3OrTfEM">Medusa</a>.</p>
<p>You can backup and restore single nodes with Medusa or do a full database backup. It supports Amazon S3, Google Cloud Storage, MinIO and Azure. You can even create a snapshot of your database. So if disaster strikes, you can roll back to your previous backup and continue as if nothing happened.</p>
<h3>Repairs with K8ssandra</h3>
<p>Data can go out of sync within Cassandra because of entropy. Entropy is technically a natural event caused by downed nodes and data deletions among other things. If left alone, data across nodes may become inconsistent and eventually unstable.</p>
<p>That&rsquo;s why running repairs is an important task. Running repairs will help keep your nodes healthy and battle entropy. You can run incremental repairs daily and full repairs weekly or monthly.</p>
<p>K8ssandra deploys&nbsp;<a href="https://dtsx.io/3vxwxlJ">Reaper</a>&nbsp;for anti-entropy repair operations on Cassandra. Reaper helps automate repairs and schedule repairs. It also has a simple UI where you can quickly see the health of your clusters.</p>
<p>Technical issues no longer have to be a major headache for you, even though they are a fact of life when running cloud-native apps. Certain tools and technology can greatly simplify many of the operational tasks you come across.&nbsp;<a href="https://dtsx.io/3rGkS2O">Learn more about K8ssandra</a>&nbsp;to see what challenges it can solve for you.</p>
<p>Check out our&nbsp;<a href="https://dtsx.io/3vAAWoa">YouTube</a>&nbsp;channel for tutorials and follow DataStax Developers on&nbsp;<a href="https://dtsx.io/3vAiY51">Twitter</a>&nbsp;for the latest news about our developer community.</p>
<h2>Resources</h2>
<ol>
<li><a href="http://astra.datastax.com">DataStax Astra DB</a>&nbsp;</li>
<li><a href="https://dtsx.io/3rJ8hMB">Metrics Collectors for Apache Cassandra</a></li>
<li><a href="https://dtsx.io/3Moiyp5">Stargate</a></li>
<li><a href="https://dtsx.io/3OrTfEM">Medusa</a></li>
<li><a href="https://dtsx.io/3vxwxlJ">Reaper</a></li>
<li><a href="https://community.datastax.com/index.html">DataStax Community Platform</a></li>
<li><a href="https://auth.cloud.datastax.com/auth/realms/CloudUsers/protocol/saml/clients/absorb">DataStax Academy</a></li>
</ol>

How to Manage the Data Challenges of a Cloud-Native Platform

Raghavan SrinivasDeveloper Advocate

Discover more

Share

More Technology

Why You Can't Miss Cassandra Summit

Mitigate the OpenAI Debacle with Open Source and Cloud

Supercharge AI Assistants with Superagent and DataStax

DataStax and AWS: Accelerating GenAI Applications into Production

NoSQL and Vector DB
for Generative AI,
Instantly, At Scale