Support Forums : DataStax

DataStax Support Forums » DataStax Enterprise

ec2snitch/propertyfile snitches + Brisk Datacenter Name

(10 posts) (2 voices)

Started 1 year ago by
Latest reply from joaquin
This topic is not resolved

Tags:

EC2
snitches

Anonymous
Unregistered

Hi all,

I am setting up Brisk on EC2 in a multi-az setup. I had used the ec2snitch with Apache Cassandra but we need the capabilities Brisk provides. From what I can tell, Brisk uses very custom datacenter names to ensure replication factors. I did try setting up the ec2snitch in the cassandra.yaml file and the system immediately halted and I had to revert to the BriskSimpleSnitch.

So here is my problem.

I need to ensure the data is in two datacenters. I have puppet running so I could derive a custom fact in ERB and put that into the property file (but I don't know if that is even used in the default yaml file). Ideally we would like to have the ec2snitch work, but that snitch then messes with the datacenter name and causes Brisk to halt due to replicas being under the allowed limit. My quesiton is this: is there a custom ec2snitch for Brisk which will fix this or can I fix this some other way (propertyfile snitch if it works and custom facts)? I would like to avoid the propertyfile hack at all costs, but I need to ensure our data is in multiple AZ's.

Thanks,
Tom

Posted 1 year ago #
joaquin
Administrator

Hello,

If you were to start up nodes in availability zone 1 with the task tracker enabled, and others in availability zone 2 as just vanilla Cassandra, then Brisk would be able to find itself without issue, split into two datacenters, and you ensure that each part of the ring is in different availability zones.

You could also setup the propertyfile snitch, and keep them the same on all machines, which would be the most robust way to manage such a cluster.

Thanks,
Joaquin

Posted 1 year ago #
Anonymous
Unregistered

We are deploying all nodes as Brisk nodes (with hadoop enabled). How would I go about using propertyfile snitch without making the system blow up?

Does the datacenter name have to be "Brisk"?

Posted 1 year ago #
joaquin
Administrator

With regarding Brisk, the BriskSimpleSnitch uses DataCenter name "Brisk" and "Cassandra", and Rack name as "rack1". Basically if a Cassandra node is running with a task tracker, then the DataCenter that this node belongs to would be noted as "Brisk", otherwise it would be "Cassandra".

What you can do is change the strategy_options to use the EC2Snitch names. To do so, access the cassandra-cli, and update the keyspace strategy_options to not use Brisk and Cassandra and instead use the EC2Snitch names.

Due keep https://issues.apache.org/jira/browse/CASSANDRA-3114 in mind since you may hit this bug during your changes.

Thanks,
Joaquin

Posted 1 year ago #
Anonymous
Unregistered

Does that mean that Brisk natively looks at the "rack" name for replication placement?

Posted 1 year ago #
joaquin
Administrator

The way that racks normally work is like this:
http://www.datastax.com/docs/0.8/operations/datacenter

What I was suggesting was leaving everything as is for the EC2Snitch, but when you see a Strategy option of ['Brisk':1, 'Cassandra': 0], change this to ['az1': 1, 'az2':1, Cassandra:'0'], for the names that show up naturally with the EC2Snitch, since this has currently slipped my mind. Using this form of replication, will ensure that the data is available in each of the availability zones.

Thanks,
Joaquin

Posted 1 year ago #
joaquin
Administrator

I'm sorry, Jackson has brought the internal workings of the EC2Snitch to my attention. I thought availability zones mapped to datacenters, but it turns out regions:azs map to dcs:racks.

Sorry for the confusion.

So what you would need to do is use the NetworkTopologyProperties to set the availability zones to map to Datacenters. Once that happens, then you would be able to follow the rest of what I had posted.

Posted 1 year ago #
Anonymous
Unregistered

Joaquin,

I'm confused here. How could I use the NetworkTopologyProperties without using the Property File snitch or am I missing something. Could you possibly provide a sample keyspace update statement? I would imaging this is a question many people would have who are changing to Brisk.

Posted 1 year ago #
joaquin
Administrator

Hello,

You are correct. As of now you would have to use the Property File to make this work.

It is in no way a hack and is actually the recommended way to bring up servers in production. This guarantees a more stable layout, one that you have provided. The EC2Snitch and BriskSnitch are convenience snitches, of sorts.

http://www.datastax.com/docs/0.8/operations/datacenter

The link above shows DC1 and DC2 being created. (Ignore the creation of racks, since you won't know this information from EC2.)

Then for the strategy options you would change both brisk_system and cfs to from [Brisk:1, Cassandra:0] to [DC1: 1, DC2: 1] using these lines:

update keyspace brisk_system with strategy_options = [{DC1:1, DC2:1}];
update keyspace cfs with strategy_options = [{DC1:1, DC2:1}];

Let me know if you still have issues or concerns.

Thanks,
Joaquin

Posted 1 year ago #
joaquin
Administrator

Also, just to make sure we are on the same page.

The reason Brisk starts up with 2 DataCenters is to avoid your real-time datacenter getting bogged down by your analytics. But by using Cassandra's innate datacenter knowledge, you are also able to run analytics on real-time data without having to do a full move of data to a separate cluster.

I'm assuming this is not the case with your cluster, but I wanted to make sure though.

Thanks,
Joaquin

Posted 1 year ago #

RSS feed for this topic

Reply

You must log in to post.