Failure running Hive examples on DSE

(7 posts) (2 voices)

• Started 11 months ago by
• Latest reply from
• This topic is not a support question

Tags:

• hive
• setup

1. 

   Anonymous
   Unregistered

   It seems to be similar to this problem
   http://www.datastax.com/support-forums/topic/there-was-a-problem-with-the-cassandra-hive-metastore

   But the error is slightly different.

   hive> CREATE TABLE invites (foo INT, bar STRING)
   > PARTITIONED BY (ds STRING);
   FAILED: Error in metadata: MetaException(message:Got exception: java.io.IOException UnavailableException())
   FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

   We are trying to run the lastest DSE up on amazon ec2. Any thoughts on our configuration error?

   Thanks,

   Todd

   Posted 11 months ago #
2. 

   Anonymous
   Unregistered

   What is your cluster configuration (dsetool ring)?
   Can you post a full exception stacktrace from hive logs?

   Posted 11 months ago #
3. 

   Anonymous
   Unregistered

   dsetool ring >>>

   Address DC Rack Workload Status State Load Owns Token
   113427455640312821154458202477256070485
   10.10.21.85 us-east 1a Analytics(tt) Up Normal 5.49 MB 33.33% 0
   10.120.206.93 us-east 1b Cassandra Up Normal 5.22 MB 33.33% 56713727820156410577229101238628035242
   10.68.78.182 us-east 1d Cassandra Up Normal 5.59 MB 33.33% 113427455640312821154458202477256070485

   hive.log>>

   2012-06-06 01:56:14,558 ERROR hive.log (MetaStoreUtils.java:logAndThrowMetaException(780)) - Got exception: java.io.IOException UnavailableException()
   2012-06-06 01:56:14,562 ERROR hive.log (MetaStoreUtils.java:logAndThrowMetaException(783)) - java.io.IOException: UnavailableException()
   at com.datastax.bdp.hadoop.cfs.CassandraFileSystemThriftStore.readSlice(CassandraFileSystemThriftStore.java:686)
   at com.datastax.bdp.hadoop.cfs.CassandraFileSystemThriftStore.readINodeColumns(CassandraFileSystemThriftStore.java:617)
   at com.datastax.bdp.hadoop.cfs.CassandraFileSystemThriftStore.retrieveINode(CassandraFileSystemThriftStore.java:569)
   at com.datastax.bdp.hadoop.cfs.CassandraFileSystemThriftStore.retrieveINode(CassandraFileSystemThriftStore.java:561)
   at com.datastax.bdp.hadoop.cfs.CassandraFileSystem.getFileStatus(CassandraFileSystem.java:508)
   at org.apache.hadoop.hive.metastore.Warehouse.isDir(Warehouse.java:377)
   at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1040)
   at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.access$700(HiveMetaStore.java:143)
   at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$15.run(HiveMetaStore.java:1081)
   at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$15.run(HiveMetaStore.java:1078)
   at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.executeWithRetry(HiveMetaStore.java:362)
   at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table(HiveMetaStore.java:1078)
   at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:400)
   at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:540)
   at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3479)
   at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:225)
   at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:133)
   at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
   at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332)
   at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123)
   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
   at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
   at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
   at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
   Caused by: UnavailableException()
   at org.apache.cassandra.thrift.Cassandra$get_slice_result.read(Cassandra.java:7764)
   at org.apache.cassandra.thrift.Cassandra$Client.recv_get_slice(Cassandra.java:570)
   at org.apache.cassandra.thrift.Cassandra$Client.get_slice(Cassandra.java:542)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at com.datastax.bdp.util.CassandraProxyClient.invoke(CassandraProxyClient.java:407)
   at $Proxy5.get_slice(Unknown Source)
   at com.datastax.bdp.hadoop.cfs.CassandraFileSystemThriftStore.readSlice(CassandraFileSystemThriftStore.java:668)
   ... 30 more

   2012-06-06 01:56:14,564 ERROR exec.Task (SessionState.java:printError(380)) - FAILED: Error in metadata: MetaException(message:Got exception: java.io.IOException UnavailableException())
   org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Got exception: java.io.IOException UnavailableException())
   at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:546)
   at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3479)
   at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:225)
   at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:133)
   at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
   at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332)
   at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123)
   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
   at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
   at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
   at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
   Caused by: MetaException(message:Got exception: java.io.IOException UnavailableException())
   at org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:784)
   at org.apache.hadoop.hive.metastore.Warehouse.isDir(Warehouse.java:385)
   at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1040)
   at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.access$700(HiveMetaStore.java:143)
   at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$15.run(HiveMetaStore.java:1081)
   at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler$15.run(HiveMetaStore.java:1078)
   at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.executeWithRetry(HiveMetaStore.java:362)
   at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table(HiveMetaStore.java:1078)
   at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:400)
   at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:540)
   ... 17 more

   2012-06-06 01:56:14,565 ERROR ql.Driver (SessionState.java:printError(380)) - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

   Posted 11 months ago #
4. 

   Anonymous
   Unregistered

   What happened to your jobtracker? From dsetool ring you don't seem to be running one.
   Try starting the analytics node with -j option.

   Posted 11 months ago #
5. 

   Anonymous
   Unregistered

   How can you tell there is no jobtracker? If I run

   #dsetool jobtracker
   10.10.21.85:8012

   I see there is a (TT) in the ring i was wondering if that should be a (JT). If so how do you change that with service des start method.

   the contents of /etc/default/dse

   # enable this to start Hadoop's JobTracker and/or TaskTrackers on this machine.
   # If left disabled, this will act as a regular Cassandra node.
   HADOOP_ENABLED=1

   # enable this to set the replication factor for CFS. Note that this will only
   # have an effect the first time a cluster is started with HADOOP_ENABLED=1 and
   # after that will be a no-op. Defaults to 1.
   #CFS_REPLICATION_FACTOR=1

   # enable this to start Solr search indexing on this machine. If left disabled,
   # this will act as a regular Cassandra node.
   SOLR_ENABLED=0
   ~

   Posted 11 months ago #
6. 

   Anonymous
   Unregistered

   We created a cluster across an ec2 availability zones within a region. We are using EC2Snitch and we are having to bring up the nodes one at a time. What would the proper procedure be fore bringing up a 3 cassandra node, one in each availability zone, then bringing up an analytical node, then possibly adding more nodes as needed.

   Posted 11 months ago #
7. 

   Anonymous
   Unregistered

   When you start up all your nodes, use dsetool movejt to set the jobtracker at the specified datacenter / location. Probably the analytics node is not a seed, so it doesn't start up the jobtracker automatically. It should remember the setting.

   Posted 11 months ago #

RSS feed for this topic

Reply

You must log in to post.