DataStax Support Forums » Topic: getting "Not a host:port pair: ${brisk.job.tracker}" while running hadoop example jar

DataStax Support Forums » Topic: getting "Not a host:port pair: ${brisk.job.tracker}" while running hadoop example jar http://www.datastax.com/support-forums/topic/getting-not-a-hostport-pair-briskjobtracker-while-running-hadoop-example-jar Software, Support, and Training for Apache Cassandra en-US Fri, 24 May 2013 05:22:36 +0000 http://bbpress.org/?v=1.0.3 <![CDATA[Search]]> q http://www.datastax.com/support-forums/search.php tjake on "getting "Not a host:port pair: ${brisk.job.tracker}" while running hadoop example jar" http://www.datastax.com/support-forums/topic/getting-not-a-hostport-pair-briskjobtracker-while-running-hadoop-example-jar#post-264 Tue, 05 Jul 2011 16:52:11 +0000 tjake 264@http://www.datastax.com/support-forums/ It looks like this example doesn't use hadoop's ParseGenericOptions method as the other examples do, this is how brisk sets the job tracker address. The workaround is to run "brisktool jobtracker" to get the current jobtrackers location and replace ${brisk.job.tracker} with this in mapred-site.xml billau on "getting "Not a host:port pair: ${brisk.job.tracker}" while running hadoop example jar" http://www.datastax.com/support-forums/topic/getting-not-a-hostport-pair-briskjobtracker-while-running-hadoop-example-jar#post-248 Thu, 30 Jun 2011 18:54:08 +0000 billau 248@http://www.datastax.com/support-forums/ I just tried the wordcount program in the hadoop example jar and that works fine. Not sure why that works but the grep program does not with both using the same configuration. joaquin on "getting "Not a host:port pair: ${brisk.job.tracker}" while running hadoop example jar" http://www.datastax.com/support-forums/topic/getting-not-a-hostport-pair-briskjobtracker-while-running-hadoop-example-jar#post-247 Thu, 30 Jun 2011 18:11:18 +0000 joaquin 247@http://www.datastax.com/support-forums/ Hello, I haven't seen this error before, but as far as the job tracker goes, that is correct. Brisk auto-detects the next available jobtracker so that there is never a single point of failure. I'll ask a member of the hive team to take a look at this. Thanks, Joaquin billau on "getting "Not a host:port pair: ${brisk.job.tracker}" while running hadoop example jar" http://www.datastax.com/support-forums/topic/getting-not-a-hostport-pair-briskjobtracker-while-running-hadoop-example-jar#post-246 Thu, 30 Jun 2011 17:50:09 +0000 billau 246@http://www.datastax.com/support-forums/ I am new to brisk. I am trying out beta2. I am just trying to run a simple example from the hadoop example jar: bin/brisk hadoop jar resources/hadoop/hadoop-examples-0.20.203.1-brisk1-beta2.jar grep /user/billa/apachelog output 'Java' I am getting an RuntimeException after the job complete and there is no output: 11/06/30 10:43:20 INFO mapred.JobClient: Running job: job_201106300609_0013 11/06/30 10:43:21 INFO mapred.JobClient: map 0% reduce 0% 11/06/30 10:43:29 INFO mapred.JobClient: map 100% reduce 0% 11/06/30 10:43:41 INFO mapred.JobClient: map 100% reduce 100% 11/06/30 10:43:43 INFO mapred.JobClient: Job complete: job_201106300609_0013 11/06/30 10:43:43 INFO mapred.JobClient: Counters: 27 11/06/30 10:43:43 INFO mapred.JobClient: Job Counters 11/06/30 10:43:43 INFO mapred.JobClient: Launched reduce tasks=1 11/06/30 10:43:43 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=8547 11/06/30 10:43:43 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 11/06/30 10:43:43 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 11/06/30 10:43:43 INFO mapred.JobClient: Rack-local map tasks=1 11/06/30 10:43:43 INFO mapred.JobClient: Launched map tasks=2 11/06/30 10:43:43 INFO mapred.JobClient: Data-local map tasks=1 11/06/30 10:43:43 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=9317 11/06/30 10:43:43 INFO mapred.JobClient: File Input Format Counters 11/06/30 10:43:43 INFO mapred.JobClient: Bytes Read=113747 11/06/30 10:43:43 INFO mapred.JobClient: File Output Format Counters 11/06/30 10:43:43 INFO mapred.JobClient: Bytes Written=86 11/06/30 10:43:43 INFO mapred.JobClient: FileSystemCounters 11/06/30 10:43:43 INFO mapred.JobClient: FILE_BYTES_READ=20 11/06/30 10:43:43 INFO mapred.JobClient: FILE_BYTES_WRITTEN=53400 11/06/30 10:43:43 INFO mapred.JobClient: CFS_BYTES_READ=114001 11/06/30 10:43:43 INFO mapred.JobClient: CFS_BYTES_WRITTEN=86 11/06/30 10:43:43 INFO mapred.JobClient: Map-Reduce Framework 11/06/30 10:43:43 INFO mapred.JobClient: Map output materialized bytes=32 11/06/30 10:43:43 INFO mapred.JobClient: Map input records=653 11/06/30 10:43:43 INFO mapred.JobClient: Reduce shuffle bytes=32 11/06/30 10:43:43 INFO mapred.JobClient: Spilled Records=0 11/06/30 10:43:43 INFO mapred.JobClient: Map output bytes=0 11/06/30 10:43:43 INFO mapred.JobClient: Map input bytes=112803 11/06/30 10:43:43 INFO mapred.JobClient: Combine input records=0 11/06/30 10:43:43 INFO mapred.JobClient: SPLIT_RAW_BYTES=254 11/06/30 10:43:43 INFO mapred.JobClient: Reduce input records=0 11/06/30 10:43:43 INFO mapred.JobClient: Reduce input groups=0 11/06/30 10:43:43 INFO mapred.JobClient: Combine output records=0 11/06/30 10:43:43 INFO mapred.JobClient: Reduce output records=0 11/06/30 10:43:43 INFO mapred.JobClient: Map output records=0 java.lang.RuntimeException: Not a host:port pair: ${brisk.job.tracker} at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:138) at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:125) at org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:2549) at org.apache.hadoop.mapred.JobClient.init(JobClient.java:454) at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:437) at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1199) at org.apache.hadoop.examples.Grep.run(Grep.java:84) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.hadoop.examples.Grep.main(Grep.java:93) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68) at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139) at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) I do see the brisk.job.tracker property set in my job conf but it comes after where mapred.job.tracker is set: <property><name>mapred.job.tracker</name><value>${brisk.job.tracker}</value></property> So does that mean I need to do something specific for brisk for the auto-detect of jobtracker in my map/reduce job?