No jobtracker found

(6 posts) (2 voices)

• Started 2 years ago by vwonthecup
• Latest reply from joaquin
• This topic is resolved

Tags:

• ami
• brisk
• jobtracker

1. 

   vwonthecup
   Member

   Hi,

   I have been wanting to try out the hive, hadoop, and cassandra combination for a long time and nothing comes closer to it than brisk. So for the record I have initiated a demo cluster on aws using the brisk ami's and have followed the tutorial (http://www.datastax.com/docs/0.8/brisk/install_brisk_ami) very closely (including launching 6 instances). However, things havent gone to plan.

   my nodetool dump:
   Address Status State Load Owns Token
   141784319550391026443072753096570088105
   10.170.175.47 Up Normal 11.13 KB 16.67% 0
   10.171.53.68 Up Normal 8.92 KB 16.67% 28356863910078205288614550619314017621
   10.170.227.11 Up Normal 8.92 KB 16.67% 56713727820156410577229101238628035242
   10.171.51.195 Up Normal 8.92 KB 16.67% 85070591730234615865843651857942052863
   10.171.38.197 Up Normal 8.92 KB 16.67% 113427455640312821154458202477256070484
   10.170.123.20 Up Normal 8.92 KB 16.67% 141784319550391026443072753096570088105

   cassandra.yaml:
   - seeds: "10.170.175.47,10.171.51.195"
   #note it only shows two IP's whereas there are 6 showing in the nodetool dump

   mapred-site.xml:
   <value>${brisk.job.tracker}</value>
   # note there is not fixed IP in here, I have tried replacing this with the line below but yet no result
   <value>10.170.175.47</value>

   finally $ brisk/bin/brisktool jobtracker
   SLF4J: Class path contains multiple SLF4J bindings.
   SLF4J: Found binding in [jar:file:/home/ubuntu/brisk/resources/cassandra/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
   SLF4J: Found binding in [jar:file:/home/ubuntu/brisk/resources/hadoop/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
   SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
   No jobtracker found

   I have the same error popping out when I start Hive, nothing seems to work !

   Thanks,
   Harsh

   Posted 2 years ago #
2. 

   joaquin
   Administrator

   Hello,

   So far everything seems alright.

   1. Your nodetool shows 6 nodes, but not all 6 nodes have to be seeds. Seeds just help with the coordination of tasks within Cassandra, primarily for the nodes to latch onto each other at boot, but after that, the cluster as a whole survives. So there is no real concept of Master/Slave nodes, nor to all seeds have to take such a high role. So this is fine. The primary thing is that all 6 nodes are now Up and running.

   2. The mapred-site.xml setting was a nice addition by Jake. Brisk does not need this tracker to be set, and actually doesn't listen to it at all since Brisk chooses the IP address with the lowest sorted value in the Brisk DataCenter to be the job tracker. mapred-site.xml is provided so that non-Brisk commands can be sent over to this cluster. When that happens, what is returned is ${brisk.job.tracker} which translates back as the jobtracker that we were talking of.

   3. The SLF4J errors are harmless and cosmetic. We tried to get them off before release, but ended up sticking with them until we filter out the combination of jars between all three programs (cassandra, hadoop, and hive).

   My best guess is that you're not running a Hadoop enabled node. What are your parameter's that you sent in? They are viewable at the top of your bootscreen.

   Thanks,
   Joaquin

   Posted 2 years ago #
3. 

   joaquin
   Administrator

   Oh and for further clarification on point 2: We choose one by sorting because we eliminate any single point of failure for the jobtracker. All Hadoop-enabled nodes have to fail before we lose the jobtracker.

   Posted 2 years ago #
4. 

   vwonthecup
   Member

   Dear Joaquin,

   Thanks a million for explaining this in detail and un_clouding my understanding.

   Alright I started the cluster using the same options in the howto file, (see i changed nothing)

   Cluster started with these options:
   -s 6 -v 3 -c 2 -o james:bond

   Let me know if I made a mistake somewhere.

   Harsh

   Posted 2 years ago #
5. 

   joaquin
   Administrator

   Yes, -v 3 means that the first 3 nodes are Vanilla Cassandra nodes. This means that the first tracker node is that with AMI Index = 3 (since it starts at 0).

   Login to the third machine and you should be able to run the command.

   Posted 2 years ago #
6. 

   joaquin
   Administrator

   I just wanted to check in an confirm that it now works for you?

   Posted 2 years ago #

RSS feed for this topic

Reply

You must log in to post.