Patrick McFadin

Apache Cassandra&reg; 4.1 will be released sometime in July, so now&rsquo;s a good time to look back over the last year and reflect on where we are as a project. We committed to a yearly cadence, and guess what? We kept the promise!
If you haven&rsquo;t been watching the Cassandra community for a while, it&rsquo;s been a busy place. I know many of you have Cassandra clusters you installed years ago and completely forgot about.
By nature, a good database should be boring and forgettable&mdash;something that&rsquo;s always there and ready. So we invite you to come back and see what&rsquo;s new. Tell us about your absolutely boring database. Maybe an upgrade is in your future? Let me tell you what&rsquo;s coming.
<h2>Stability as a feature</h2>
<a href="https://thenewstack.io/the-end-of-the-beginning-for-apache-cassandra/">Cassandra 4.0</a>&nbsp;marked a significant milestone for a 10-year-old database. The project put enormous resources into building tools that define and validate a stable database and integrate those into the build and release pipeline to ensure thorough testing of future changes and prevent regression.
The result was one of the most stable databases ever shipped. Teams have been deploying 4.0 clusters over the past year at a furious pace, and the reports so far are overwhelmingly positive. Stability is a killer feature for a database, and if the world depends on Cassandra for storing critical data, we&rsquo;re delighted that it&rsquo;s living up to this promise.
Do you know another killer feature of a database? Shipping new features. There was a regrettable length of time between 3.11 and 4.0, but with a solid foundation built, we&rsquo;re now moving at a regular cadence. Cassandra 4.1 is currently in pre-release, on track with our goal for major yearly releases. In addition, all the validation work done in 4.0 is paying off with a build pipeline that gives contributors confidence when building new functionality in Cassandra.
<h2>4.1&rsquo;s shiny new thing: pluggability</h2>
So, what do we get out of this database and building on a stable core? The theme for Cassandra 4.1 is enabling feature plug-ins. Why are plug-ins a valuable new thing, you ask? It&rsquo;s the structured way to add features to an existing product without changing the core code. For Cassandra, that means adding important new features without actually changing Cassandra.
One of the early drivers of this idea was Instagram. They had built a version of Cassandra that used RocksDB as the underlying storage engine, called&nbsp;<a href="https://thenewstack.io/instagram-supercharges-cassandra-pluggable-rocksdb-storage-engine/">Rocksandra</a>. It radically changed how we thought of storage with Cassandra without changing the networking and node coordination. However, it also surfaced two distinct problems that needed addressing. First is the need for a clear interface to Cassandra internals, so there&rsquo;s an understood contract between them when using outside code. The Rocksandra team had to rely on deep knowledge of Cassandra internals to make it work.
The second was having a reliable testing framework. Instagram understood its use case and had its own acceptance testing. For a general-purpose database, however, there needs to be a much wider scope of functional testing. Based in part on those lessons, both of those problems have been tackled in the project.
Some of the new plug-in features that are available in 4.1 include:
<ul>
<li><a href="https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-11%3A+Pluggable+memtable+implementations">Storage</a>.&nbsp;The feature talks about memtables, but those translate into the underlying storage because they&rsquo;re mapped to be written and read by Cassandra. Expect to see some interesting implementations focusing on specific use cases, including fast memory storage and columnar formats.</li>
<li><a href="https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-9%3A+Make+SSLContext+creation+pluggable">Network Encryption</a>.&nbsp;Previous to this change, any SSL certificates had to be in the local file system. This change allows external key providers such as HashiCorp Vault to make key management easier in large deployments.</li>
<li><a href="https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-16%3A+Auth+Plugin+Support+for+CQLSH">Authentication</a>.&nbsp;External and centralized authentication is a desirable feature for most organizations that manage many infrastructures. This change allows the Cassandra command-line tool, CQLSH, to use LDAP, Kerberos, and others.</li>
<li><a href="https://issues.apache.org/jira/browse/CASSANDRA-17044">Schema</a>.&nbsp;The only option to store cluster schema is in system tables. For global coordination, especially in Kubernetes, external schema storage such as&nbsp;<a href="https://etcd.io/">etcd</a>&nbsp;is now an option.</li>
<li><a href="https://cassandra.apache.org/_/blog/Apache-Cassandra-4.1-Features-Guardrails-Framework.html">Guardrails</a>.&nbsp;Operators around the world rejoice! You now can restrict anti-patterns in your production environment. An example would be limiting the number of indexes you can add to a table. Guardrails is already in use with&nbsp;<a href="https://auth.cloud.datastax.com/auth/realms/CloudUsers/protocol/openid-connect/auth?client_id=auth-proxy&amp;redirect_uri=https%3A%2F%2Fgatekeeper.auth.cloud.datastax.com">DataStax Astra DB</a>, our managed Cassandra service, which donated to the project.</li>
</ul>
And now, the future! ACID transactions and more.
The future of Cassandra after 4.1 will be 5.0, and those discussions are already in full swing. Cassandra is the database designed for growth and for cloud-native applications in years to come. The following 10 years of Apache Cassandra are about rising to that challenge by building on the solid foundation of the previous 10 years.
The most transformative new 5.0 Astra DB feature may shock you, but here we go: Cassandra will be adding&nbsp;<a href="https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-15%3A+General+Purpose+Transactions">full ACID transactions</a>.
If you&rsquo;re in the camp of people saying you could never use Cassandra because it &ldquo;doesn&rsquo;t support transactions,&rdquo; prepare to make room for more Cassandra in your life. Cassandra 2.0 added what we called &ldquo;lightweight transactions&rdquo; based on&nbsp;<a href="https://en.wikipedia.org/wiki/Paxos_(computer_science)">Paxos</a>.&nbsp;
To keep the guarantees that Cassandra delivers while not destroying performance, Paxos was the right choice at the time. However, since 2013, new consensus protocols have emerged, such as&nbsp;<a href="https://cloud.google.com/spanner">Spanner</a>&nbsp;and&nbsp;<a href="https://raft.github.io/">Raft</a>, which are very popular in the database world &mdash; but they require tradeoffs that aren&rsquo;t aligned with the linear scale and uptime guarantees that Cassandra users expect.
A protocol known as Accord describes the next generation of a distributed consensus protocol, which does consensus in one round trip and doesn&rsquo;t require complicated leader failover mechanisms. In short, it enables Cassandra to be Cassandra while delivering full ACID transactions. We&rsquo;re building it now, so if you want to join in the conversation, we&rsquo;d appreciate your input.
A more nonspecific but important direction for Cassandra is the move to being more cloud-native. The ability to add plugins and shape the deployment of your Cassandra cluster is a step in the right direction. Serverless databases in Kubernetes are the future, and Cassandra is well-positioned to be that database for years to come.&nbsp;<a href="http://www.datastax.com">DataStax</a>&nbsp;released a&nbsp;<a href="https://www.datastax.com/resources/whitepaper/astra-serverless">white paper</a>&nbsp;on a cloud-native database based on Cassandra that could be a blueprint for the future.
For end-users, it means scaling without manual effort and true multi-tenancy where multiple applications share the same infrastructure. For operators, especially site reliability engineers, the native Kubernetes deployments separate compute from storage to scale independently. Most important, you only provision what you need asynchronously to save on those ever-growing infrastructure bills.
<h2>Time to celebrate</h2>
Last year, we had a party for the release of 4.0, and this year will be no different! The&nbsp;<a href="https://www.cassandraworldparty.org/">Cassandra World Party</a>&nbsp;is a one-hour, online-only event that we&rsquo;ll hold at three different times on July 20 to cover as many time zones as possible. The highlight will be the five-minute lightning talk format where the user community can get on and tell their story&hellip; quickly!&nbsp;Make sure you&nbsp;<a href="https://www.cassandraworldparty.org/">register</a>&nbsp;to save your spot; don&rsquo;t miss out!
We hope you&rsquo;ll join us and celebrate with the rest of the community. We&rsquo;d love to hear your story&mdash;if you&rsquo;re up to the challenge.&nbsp;<a href="https://www.cassandraworldparty.org/">Just tell us your topic</a>&nbsp;and which time zone works for you. See you there!
<h2>Resources</h2>
<ol>
<li><a href="https://cassandra.apache.org/_/index.html">Apache Cassandra</a></li>
<li><a href="https://www.cassandraworldparty.org/">Cassandra World Party</a></li>
<li><a href="https://thenewstack.io/the-end-of-the-beginning-for-apache-cassandra/">The End of the Beginning for Apache Cassandra</a></li>
<li><a href="https://thenewstack.io/instagram-supercharges-cassandra-pluggable-rocksdb-storage-engine/">Instagram Supercharges Cassandra with a Pluggable RocksDB Storage Engine</a></li>
<li><a href="https://auth.cloud.datastax.com/auth/realms/CloudUsers/protocol/openid-connect/auth?client_id=auth-proxy&amp;redirect_uri=https%3A%2F%2Fgatekeeper.auth.cloud.datastax.com">Astra DB</a></li>
<li><a href="https://en.wikipedia.org/wiki/Paxos_(computer_science)">Paxos</a></li>
<li><a href="https://cloud.google.com/spanner">Spanner</a></li>
<li><a href="https://raft.github.io/">Raft</a></li>
<li><a href="https://www.datastax.com/resources/whitepaper/astra-serverless">DataStax Astra DB: Designing a Serverless Cloud-Native Database-as-a-Service</a></li>
</ol>

Apache Cassandra 4.1: Building the Database Your Kids Will Use

Patrick McFadinDeveloper Relations

Discover more

Share

Share

Stability as a feature

4.1's shiny new thing: pluggability

Time to celebrate

Resources

More Company

DataStax Acquires Langflow to Accelerate Generative AI Development

The Top 5 DataStax Stories from 2023

2023 Recap: Data = AI

DataStax Astra DB Nabs Three Prestigious 2023 TrustRadius “Best of” Awards, Dominates the Vector Databases Category

One-stop Data API for Production GenAI