Configuring Replication¶

Partitioner Settings¶

When you deploy a Cassandra cluster, make sure that each node is responsible for roughly an equal amount of data. To accomplish this load balancing, configure the partitioner for each node, and assign the node an initial-token value.

DataStax strongly recommends using the RandomPartitioner (the default) for all cluster deployments. Assuming use of this partitioner, each node in the cluster is assigned a token that represents a hash value within the range of 0 to 2¹²⁷ -1.

You can calculate tokens for a cluster having nodes in a single data center by dividing the range by the total number of nodes in the cluster. In multiple data center deployments, tokens should be calculated such that each data center is individually load balanced as well. Partition each data center as if it were its own distinct ring. See Generating Tokens for the different approaches to generating tokens for nodes in single and multiple data center clusters.

Snitch Settings¶

The snitch is responsible for knowing the location of nodes within your network topology. The location of nodes affects where replicas are placed and how requests are routed between replicas. All nodes must have exactly the same snitch configuration. The endpoint_snitch property configures the snitch for a node.

In cassandra.yaml, the snitch is set to the DSE Delegated Snitch (endpoint_snitch: com.datastax.bdp.snitch.DseDelegateSnitch). The Delegated Snitch is used to implement Elastic Workload Re-provisioning. The following sections describe a few commonly-used snitches. All snitches are described in the Apache Cassandra documentation.

# Data Center One

175.56.12.105=DC1:RAC1
175.50.13.200=DC1:RAC1
175.54.35.197=DC1:RAC1

120.53.24.101=DC1:RAC2
120.55.16.200=DC1:RAC2
120.57.102.103=DC1:RAC2

# Data Center Two

110.56.12.120=DC2:RAC1
110.50.13.201=DC2:RAC1
110.54.35.184=DC2:RAC1

50.33.23.120=DC2:RAC2
50.45.14.220=DC2:RAC2
50.17.10.203=DC2:RAC2

# Analytics Replication Group

172.106.12.120=DC3:RAC1
172.106.12.121=DC3:RAC1
172.106.12.122=DC3:RAC1

# default for unknown nodes
default=DC3:RAC1

Choosing Keyspace Replication Options¶

When you create a keyspace, you must define the replica placement strategy and the number of replicas you want. DataStax recommends choosing NetworkTopologyStrategy for single and multiple data center clusters. It is as easy to use as SimpleStrategy and allows for expansion to multiple data centers in the future, should that become useful. It is much easier to configure the most flexible replication strategy up front, than to reconfigure replication after you have already loaded data into your cluster.

NetworkTopologyStrategy takes as options the number of replicas you want per data center. Even for single data center (or single node) clusters, you can use this replica placement strategy and just define the number of replicas for one data center. For example (using cassandra-cli):

CREATE KEYSPACE test
  WITH strategy_class = 'NetworkTopologyStrategy'
  AND strategy_options:us-east = 6;

Or for a multiple data center cluster:

CREATE KEYSPACE test2 WITH strategy_class = 'NetworkTopologyStrategy’
  AND strategy_options:DC1 = 3 AND strategy_options:DC2 = 3;

When declaring the keyspace strategy-options, what you name your data centers depends on the snitch you have chosen for your cluster. The data center names must correlate to the snitch you are using in order for replicas to be placed in the correct location.

As a general rule, the number of replicas should not exceed the number of nodes in a replication group. However, it is possible to increase the number of replicas, and then add the desired number of nodes afterwards. When the replication factor exceeds the number of nodes, writes will be rejected, but reads will still be served as long as the desired consistency level can be met.

Changing Replication Settings¶

The default replication for system keyspaces, such as the HiveMetaStore keyspace, is 1. The default replication factor is suitable for development and testing of a single node. If you use Hive in a production environment, you should increase the cfs and HiveMetaStore keyspace replication factors to at least 2 to be resilient to single-node failures.

The procedure for changing the replication of data in the Cassandra File System (CFS) involves these tasks:

• Update the replication factor of the default keyspaces on the node: cfs and, if you use cfs_archive, also update it.
• Update the default HiveMetaStore keyspace if the node is an Analytics node that uses Hive.

To change the replication of data in the CFS

1. Check the name of the data center of the node.

   bin/nodetool -h localhost ring
   

   The output tells you that the name of the data center for your node is, for example, datacenter1.

2. Change the replication factor of the cfs and cfs_archive keyspaces:

   [default@unknown] UPDATE KEYSPACE cfs
      WITH strategy_options = {datacenter1:3};
   
   [default@unknown] UPDATE KEYSPACE cfs_archive
      WITH strategy_options = {datacenter1:3};
   

3. If you use Hive, update the HiveMetaStore keyspace accordingly:

   [default@unknown] UPDATE KEYSPACE HiveMetaStore
      WITH strategy_options = {datacenter1:3};
   

If you change the replication factor of a keyspace that contains data, run nodetool repair to avoid having missing data problems or data unavailable exceptions.

Choosing Node Configuration Options¶

A major part of planning your Cassandra cluster deployment is understanding and setting the various node configuration properties. This section explains the various configuration decisions that need to be made before deploying a Cassandra cluster, be it a single-node, multi-node, or multiple data center cluster.

The properties mentioned in this section are set in the cassandra.yaml configuration file. Each node should be correctly configured before starting it for the first time.

-Dcassandra.load_ring_state=false