Jonathan Ellis

Moving data structures off of the Java heap to native memory is important to keep up with datasets that continue to grow, while the JVM stays stuck at heap sizes of about 8GB. As of&nbsp;<a href="https://www.datastax.com/dev/blog/whats-under-the-hood-in-cassandra-2-0">Cassandra 2.0</a>, there are two major pieces of the storage engine that still depend on the JVM heap:&nbsp;<a href="https://www.datastax.com/documentation/cassandra/2.0/cassandra/dml/dml_write_path_c.html">memtables</a>&nbsp;and the&nbsp;<a href="https://www.datastax.com/documentation/cassandra/2.0/cassandra/operations/ops_configuring_caches_c.html">key cache</a>.

Cassandra 2.1 addresses the larger of these, memtables. Memtables are not moved entirely off heap, but two options are provided to reduce their footprint in the new&nbsp;<tt>memtable_allocation_type</tt>&nbsp;configuration option:

<ul>
	<li><tt>offheap_buffers</tt>&nbsp;moves the&nbsp;<a href="https://www.datastax.com/dev/blog/does-cql-support-dynamic-columns-wide-rows">cell</a>&nbsp;name and value to&nbsp;<tt>DirectBuffer</tt>&nbsp;objects. This has the lowest impact on reads -- the values are still "live" Java buffers -- but only reduces heap significantly when you are storing large strings or blobs.</li>
	<li><tt>offheap_objects</tt>&nbsp;moves the entire cell off heap, leaving only&nbsp;<a href="https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/db/AbstractNativeCell.java">the NativeCell reference</a>&nbsp;containing a pointer to the native (off-heap) data. This makes it effective for small values like ints or uuids as well, at the cost of having to copy it back on-heap temporarily when reading from it.</li>
</ul>

The default behavior is unchanged for 2.1 (<tt>heap_buffers</tt>). We anticipate changing the default to offheap_objects for 3.0.

Let's look at how this plays out in the lab:<img alt="offheap vs onheap memtables" data-entity-type="file" data-entity-uuid="f3acf4e7-e11b-46e8-bd74-954eec526a5c" src="https://www.datastax.com/sites/default/files/inline-images/Screen-Shot-2014-07-15-at-10.34.02-PM-700x468.png" />

&nbsp;

Writes are about 5% faster with offheap_objects enabled, primarily because Cassandra doesn't need to flush as frequently. Bigger sstables means less compaction is needed.

<img alt="off heap vs on heap memtables" data-entity-type="file" data-entity-uuid="6813c8d5-d4c8-4a25-abc5-2fbb25e0859e" src="https://www.datastax.com/sites/default/files/inline-images/Screen-Shot-2014-07-15-at-10.34.28-PM-700x470.png" />

Reads are dead even within the accuracy of the benchmark, but this is a best-case scenario for offheap_objects: most of the reads are coming from sstables on disk, not unflushed data in memtables. We have&nbsp;<a href="https://issues.apache.org/jira/browse/CASSANDRA-7519">a ticket open</a>&nbsp;to add profiles to cassandra-stress that will work the memtables much harder. Stay tuned!

Off-heap memtables in Cassandra 2.1

Jonathan EllisTechnology

Share

More Technology

Why DataStax Loves LangChain

Getting Started with DataStax Astra DB and Amazon Bedrock

Why You Can't Miss Cassandra Summit

Mitigate the OpenAI Debacle with Open Source and Cloud

NoSQL and Vector DB
for Generative AI,
Instantly, At Scale