email iconemail phone iconcall

Off-heap memtables in Cassandra 2.1

By Jonathan Ellis, CTO and Co-founder -  July 15, 2014 | 3 Comments

Moving data structures off of the Java heap to native memory is important to keep up with datasets that continue to grow, while the JVM stays stuck at heap sizes of about 8GB. As of Cassandra 2.0, there are two major pieces of the storage engine that still depend on the JVM heap: memtables and the key cache.

Cassandra 2.1 addresses the larger of these, memtables. Memtables are not moved entirely off heap, but two options are provided to reduce their footprint in the new memtable_allocation_type configuration option:

  • offheap_buffers moves the cell name and value to DirectBuffer objects. This has the lowest impact on reads -- the values are still "live" Java buffers -- but only reduces heap significantly when you are storing large strings or blobs.
  • offheap_objects moves the entire cell off heap, leaving only the NativeCell reference containing a pointer to the native (off-heap) data. This makes it effective for small values like ints or uuids as well, at the cost of having to copy it back on-heap temporarily when reading from it.

The default behavior is unchanged for 2.1 (heap_buffers). We anticipate changing the default to offheap_objects for 3.0.

Let's look at how this plays out in the lab:

Screen Shot 2014-07-15 at 10.34.02 PM

Writes are about 5% faster with offheap_objects enabled, primarily because Cassandra doesn't need to flush as frequently. Bigger sstables means less compaction is needed.

Screen Shot 2014-07-15 at 10.34.28 PM

Reads are dead even within the accuracy of the benchmark, but this is a best-case scenario for offheap_objects: most of the reads are coming from sstables on disk, not unflushed data in memtables. We have a ticket open to add profiles to cassandra-stress that will work the memtables much harder. Stay tuned!

DataStax has many ways for you to advance in your career and knowledge.

You can take free classes, get certified, or read one of our many white papers.

register for classes

get certified

DBA's Guide to NoSQL


  1. DuyHai DOAN says:

    Interesting results. Going offheap also mean GCing offheap. Did you see any performance dip during collections on a long run over hours ?

    1. Carl Mueller says:

      going offheap implies that JVM GC no longer controls those memory allocations. It is up to Cassandra to deallocate those buffers when not used anymore.

      Just like C pointers and malloc…

  2. Liang Xie says:

    Could you share your cassadra server config/setting in test report? I am running offheap test these days, it would be helpful to have a reference:)


Your email address will not be published. Required fields are marked *

Subscribe for newsletter: