Support Forums : DataStax

DataStax Support Forums » DataStax Enterprise

Help to understand configuration for DSE (based on DataStax AMI)

(4 posts) (2 voices)

Started 1 year ago by
Latest reply from
This topic is not a support question

Tags:

No tags yet.

Anonymous
Unregistered

I have a DSE(1.x) cluster on Amazon which I setup using the DataStax ami.

I've loaded data and experimented with both Hive and Pig, but I am trying to understand the configuration -- trying to find out what options are used, etc -- and I'm really stumped trying to figure a few things out.

Here's the output of nodetool ring

x.x.x.x Cassandra rack1 Up Normal 22.29 GB 16.67% 0
x.x.x.x Analytics rack1 Up Normal 2.05 GB 16.67% 28356863910078205288614550619314017621
x.x.x.x Cassandra rack1 Up Normal 16.81 GB 16.67% 56713727820156410577229101238628035242
x.x.x.x Analytics rack1 Up Normal 2.16 GB 16.67% 85070591730234615865843651857942052863
x.x.x.x Cassandra rack1 Up Normal 15.61 GB 16.67% 113427455640312821154458202477256070485
x.x.x.x Analytics rack1 Up Normal 2.16 GB 16.67% 141784319550391026443072753096570088106

1) where do the datacenter and rack names come from? from doc, it seems like it should be cassandra-topology.properties, but mine seem to be the default, sample entries, certainly nothing that matches Cassandra or Analytics

2) and what configuration parameters indicate that the Analytics nodes won't have much data.. From reading the docs, I would have thought the initial tokens would be different (e.g. minimal gap for analytics)

3) According to docs, I added the following strategy_options for any CF:
strategy_options={Brisk:1, Cassandra:1};
Why Brisk? Shouldn't this be Analytics? --- Hmmm, just had a thought.. .is that the reason for lack of data on Analytics nodes? will have to explore that. But I didn't get any errors when using the schema...

4) How do the analytics nodes find each other for hadoop jobs? I assume that the dse hadoop runtime could figure it out via cassandra API, but just curious..

I'm guessing I'll have a few head slaps when I get answers... but that's ok..

Thanks,
JohnLiberty

Posted 1 year ago #
Anonymous
Unregistered

Hi JohnLiberty,

The answer to you first question is here http://www.datastax.com/docs/1.0/cluster_architecture/replication, take a note on DseSimpleSnitch.

2). Analytics nodes should have the same amount of data as C* nodes to be able to run jobs efficiently.

3). It should be Analytics, "Brisk" comes from the older project name which is not used anymore...

4). They are organized into the pseudo-DC called Analytics, JobTracker, inside of that DC, knows locations of the TaskTracker nodes and it's is responsible of submitting jobs to the them. Here is info on the JT https://wiki.apache.org/hadoop/JobTracker (except NameNode part).

Posted 1 year ago #
Anonymous
Unregistered

Ah, I thanks, I read that info, but upon re-reading the DseSimpleSnitch section, this line finally sunk in:

If using this snitch, use Analytics or Cassandra as your data center names when defining your keyspace strategy_options.

Which implies that the names are pre-defined (head slap) ... which means, I need to re-do my strategy options to refer to Analytics instead of Brisk, which should resolve my ring confusion.

Thanks!

Posted 1 year ago #
Anonymous
Unregistered

Just a follow-up... the unequal node layout was indeed a result of using the incorrect data center name for the strategy_options.

After updating column family, the nodes started to stream and eventually the load was distributed

x.x.x.x Cassandra rack1 Up Normal 23.37 GB 16.67% 0
x.x.x.x Analytics rack1 Up Normal 16.32 GB 16.67% 28356863910078205288614550619314017621
x.x.x.x Cassandra rack1 Up Normal 17.9 GB 16.67% 56713727820156410577229101238628035242
x.x.x.x Analytics rack1 Up Normal 18.4 GB 16.67% 85070591730234615865843651857942052863
x.x.x.x Cassandra rack1 Up Normal 16.61 GB 16.67% 113427455640312821154458202477256070485
x.x.x.x Analytics rack1 Up Normal 23.31 GB 16.67% 141784319550391026443072753096570088106

Posted 1 year ago #

RSS feed for this topic

Reply

You must log in to post.