Apache Cassandra™ 2.0

Recommended production settings

Recommendations for production environments; adjust them accordingly for your implementation.

Disable zone_reclaim_mode on NUMA systems

The Linux kernel can be inconsistent in enabling/disabling zone_reclaim_mode. This can result in odd performance problems.
To ensure that zone_reclaim_mode is disabled:
echo 0 > /proc/sys/vm/zone_reclaim_mode

For more information, see Peculiar Linux kernel performance problem on NUMA systems.

User resource limits

You can view the current limits using the ulimit -a command. Although limits can also be temporarily set using this command, DataStax recommends making the changes permanent:

Packaged installs: Ensure that the following settings are included in the /etc/security/limits.d/cassandra.conf file:
cassandra - memlock unlimited
cassandra - nofile 100000
cassandra - nproc 32768
cassandra - as unlimited
Tarball installs: Ensure that the following settings are included in the /etc/security/limits.conf file:
* - memlock unlimited
* - nofile 100000
* - nproc 32768
* - as unlimited
If you run Cassandra as root, some Linux distributions such as Ubuntu, require setting the limits for root explicitly instead of using *:
root - memlock unlimited
root - nofile 100000
root - nproc 32768
root - as unlimited
For CentOS, RHEL, OEL systems, also set the nproc limits in /etc/security/limits.d/90-nproc.conf :
* - nproc 32768
For all installations, add the following line to /etc/sysctl.conf :
vm.max_map_count = 131072
To make the changes take effect, reboot the server or run the following command:
$ sudo sysctl -p
To confirm the limits are applied to the Cassandra process, run the following command where pid is the process ID of the currently running Cassandra process:
$ cat /proc/<pid>/limits

For more information, see Insufficient user resource limits errors.

Disable swap

You must disable swap entirely. Failure to do so can severely lower performance. Because Cassandra has multiple replicas and transparent failover, it is preferable for a replica to be killed immediately when memory is low rather than go into swap. This allows traffic to be immediately redirected to a functioning replica instead of continuing to hit the replica that has high latency due to swapping. If your system has a lot of DRAM, swapping still lowers performance significantly because the OS swaps out executable code so that more DRAM is available for caching disks.

If you insist on using swap, you can set vm.swappiness=1. This allows the kernel swap out the absolute least used parts.

$ sudo swapoff --all

To make this change permanent, remove all swap file entries from /etc/fstab.

For more information, see Nodes seem to freeze after some period of time.

Synchronize clocks

The clocks on all nodes should be synchronized. You can use NTP (Network Time Protocol) or other methods.

This is required because columns are only overwritten if the timestamp in the new version of the column is more recent than the existing column.

Optimum blockdev --setra settings for RAID

Typically, a readahead of 128 is recommended, especially on Amazon EC2 RAID0 devices.

Check to ensure setra is not set to 65536:

sudo blockdev --report /dev/<device>

To set setra:

sudo blockdev --setra 128 /dev/<device>

Java Virtual Machine

The latest 64-bit version of Java 7 is recommended, not the OpenJDK.

Java Native Access

Java Native Access (JNA) is required for production installations.

Show/hide