Configuring Multiple Data Centers Quick Start¶

Configuration Example (Tarball Installation)¶

The procedures for configuring multiple data centers on Tarball and Debian/RPM installations are the same except the configuration files are located in different places.

To configure multiple data centers:

1. Install DSE on the nodes. Suppose, for example, you install DSE on these nodes:

   10.168.66.41
   10.176.43.66
   10.168.247.41
   10.176.170.59
   10.169.61.170
   10.169.30.138

2. Decide on names for each data center, for example DC1, DC2 or 100, 200.

3. Calculate tokens for nodes using one of the procedures that ensure even distribution of data. For example:

   token 0: 0
   token 1: 56713727820156410577229101238628035242
   token 2: 113427455640312821154458202477256070485
   token 3: 10
   token 4: 56713727820156410577229101238628035252
   token 5: 113427455640312821154458202477256070495

4. For each node, change the cassandra.yaml file in <install_location>/resources/cassandra/conf/:

   • Assign an endpoint-snitch.
   • Specify an initial_token.
   • Specify a listen IP address.

   For example:

   endpoint_snitch: com.datastax.bdp.snitch.DseDelegateSnitch
   initial_token: <one of the tokens from step 3>
   seeds: 10.168.66.41,10.176.170.59
   listen_address: <localhost IP address>

   Note

   It is a best practice to have at least one seed node per data center.

   If you are configuring a mixed workload cluster, you need at least one seed node for each type of Cassandra and Hadoop workload. Solr nodes don’t require a seed node. All nodes need to have the same snitch configuration and the same list of seeds. The seed list needs to include a node from each data center.

5. For each node, change the dse.yaml file in <install_location>/resources/dse/conf to specify the snitch to be delegated by the DseDelegateSnitch. You can use the default, DseSimpleSnitch, or another snitch. For more information about snitches, see the snitches section.

6. In the cassandra-topology.properties file, assign a data center name and rack name to the IP addresses of each node. For example, use DC1 and DC2 for data center names and RAC1 for the rack name.

   # Cassandra Node IP=Data Center:Rack
   
   10.168.66.41=DC1:RAC1
   10.176.43.66=DC2:RAC1
   10.168.247.41=DC1:RAC1
   10.176.170.59=DC2:RAC1
   10.169.61.170=DC1:RAC1
   10.169.30.138=DC2:RAC1

7. Also, in the cassandra-topologies.properties file, assign a default data center name and rack name for unknown nodes.

   # default for unknown nodes
   default=DC1:RAC1

8. Start the nodes.

9. Run the nodetool utility on one of the nodes to check that your multiple data centers are up and running:

   ./nodetool -h localhost ring

For more information about how to configure multiple data centers, see Initializing a Multi-Node or Multi-Data Center Cluster.

Balancing the Data Center Nodes¶

When you deploy a Cassandra cluster, you need to use the partitioner to distribute roughly an equal amount of data to nodes. You also use identifiers for each data center (see step 6) in a formula to calculate tokens that balance nodes within a data center (DC). For example, assign each DC a numerical name that is a multiple of 100. Then for each DC, determine the tokens as follows: token = (2^127 / num_nodes_in_dc * n + DC_ID) where n is the node for which the token is being calculated and DC_ID is the numerical name.

Frequently Asked Questions¶

1. Can all the application data be 100% owned by Cassandra nodes?

There is no ownership. You set a replication factor (RF) for each data center, including the virtual analytics one. So, you might have one copy of the data in each of C1, C2, C3, and Analytics (AN), for example. Regardless of what data center or nodes you write to, the data is replicated to all four data centers. Replication is configured per keyspace.

For example, one keyspace with RF = {C1:1, C2:1, C3:1, AN:0}, and a different keyspace with RF = {C1:0, C2:0, C3:0, AN:1}. In such a configuration, if you write into the first keyspace, the analytics nodes do not have any copies of the data. If you write into the second keyspace only the analytics nodes have copies. If you write data (e.g. flat files) directly into CFS, by default only the AN nodes have copies of the data. The assumption is only the AN nodes need access to the flat files because their only use is for analytics. This is actually accomplished by having a Cassandra File System (CFS) keyspace, where AN has a RF > 0 and the others have RF=0.

2. Through replication, can you give the analytics nodes all of the application data?

Yes, as exemplified in the previous answer.

3. If all the analytics data is written to column families (CF), how can application nodes get a copy of the data?

The destination CFs used for the output of the analytics jobs are in a keyspace where only the Cassandra data centers have a RF > 0 (that is, the output of the analytics jobs do not need to be stored on the analytics nodes. There are common exceptions to this. If the output does not really belong in Cassandra, for some reason, such as the output is for analysis and not part of the operational data set, you can write the output into a keyspace where only the analytics DC had RF > 0. If you want both Cassandra and analytic nodes to have copies of the data, it is just a matter of setting the RF correctly on the keyspace you write to.

4. If you add or remove a node from a ring, do all nodes in the data centers need to be rebalanced?

You need to rebalance a data center after adding or removing a node. Nodes in other data centers do not have to be rebalanced. You need to stagger tokens between data centers for maximum data availability.

More Information About Configuring Data Centers¶

Links to more information about configuring a data center:

• Configuring nodes
• Choosing keyspace replication options
• Replication in a physical or virtual data center