http://www.datastax.com/support-forums/topic/is-it-ok-to-manually-specify-the-jobtracker-in-mapred-sitexml Software, Support, and Training for Apache Cassandra en-US Thu, 20 Jun 2013 04:45:38 +0000 http://bbpress.org/?v=1.0.3 q http://www.datastax.com/support-forums/search.php http://www.datastax.com/support-forums/topic/is-it-ok-to-manually-specify-the-jobtracker-in-mapred-sitexml#post-122 Thu, 26 May 2011 16:53:10 +0000 joaquin 122@http://www.datastax.com/support-forums/ <p>The reason that the jobtracker is set that way is to allow Brisk to elect a job tracker in the event that the currently elected one fails, thereby eliminating job trackers as a single point of failure.</p> <p>You could either change your map-reduce job to evaluate the statement, which I would recommend, or you can specify the jobtracker yourself. </p> http://www.datastax.com/support-forums/topic/is-it-ok-to-manually-specify-the-jobtracker-in-mapred-sitexml#post-120 Thu, 26 May 2011 01:23:53 +0000 matt 120@http://www.datastax.com/support-forums/ <p>All,</p> <p>This is something I've tried, and it seems to work, but I don't know if it's due to coincidence. I have a map-reduce job that we have been running for ages on a regular hadoop cluster. For some reason, it appears to be manually checking mapred-site.xml for the jobtracker address, and when I leave it set to the default, the job fails with:</p> <p>java.lang.RuntimeException: Not a host:port pair: ${brisk.job.tracker}<br /> at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:138)<br /> at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:125)<br /> at org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:2549)<br /> at org.apache.hadoop.mapred.JobClient.init(JobClient.java:454)<br /> at org.apache.hadoop.mapred.JobClient.&lt;init&gt;(JobClient.java:437)<br /> at org.apache.hadoop.mapreduce.Job$1.run(Job.java:477)<br /> at java.security.AccessController.doPrivileged(Native Method)<br /> at javax.security.auth.Subject.doAs(Subject.java:396)<br /> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)<br /> at org.apache.hadoop.mapreduce.Job.connect(Job.java:475)<br /> at org.apache.hadoop.mapreduce.Job.submit(Job.java:464)<br /> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:494)<br /> ....</p> <p>However, if I explicitly set the job tracker address in mapred-site.xml everything seems to work fine. But I'm not sure if the cluster will ever elect a job tracker that isn't the one I specify, thereby screwing things up. I realize that by explicitly setting it, I lose a great deal of flexibility, but will it actively break anything?</p> <p>Thanks,<br /> Matt </p>