http://www.datastax.com/support-forums/topic/storing-counters-in-hive Software, Support, and Training for Apache Cassandra en-US Tue, 21 May 2013 10:27:51 +0000 http://bbpress.org/?v=1.0.3 q http://www.datastax.com/support-forums/search.php http://www.datastax.com/support-forums/topic/storing-counters-in-hive#post-1393 Tue, 20 Mar 2012 00:06:57 +0000 Anonymous 1393@http://www.datastax.com/support-forums/ <p>I am trying to store Counters CF from cassandra to Hive. Below is the CREATE TABLE syntax in Hive:</p> <p>DROP TABLE IF EXISTS Counters;<br /> create external table Counters(row_key string, column_name string, value string)<br /> STORED BY 'org.apache.hadoop.hive.cassandra.CassandraStorageHandler'<br /> WITH SERDEPROPERTIES ("cassandra.columns.mapping" = ":key,:column,:value",<br /> "cassandra.ks.name" = "BAMSchema",<br /> "cassandra.ks.repfactor" = "1",<br /> "cassandra.ks.strategy" = "org.apache.cassandra.locator.SimpleStrategy",<br /> "cassandra.cf.name" = "Counters" ,<br /> "cassandra.host" = "127.0.0.1" ,<br /> "cassandra.port" = "9160",<br /> "cassandra.partitioner" = "org.apache.cassandra.dht.RandomPartitioner")<br /> TBLPROPERTIES (<br /> "cassandra.input.split.size" = "64000",<br /> "cassandra.range.size" = "1000",<br /> "cassandra.slice.predicate.size" = "1000"); </p> <p>and Counter CF is defined as :</p> <p>create column family Counters<br /> with comparator = UTF8Type<br /> and default_validation_class=CounterColumnType<br /> and replicate_on_write=true;</p> <p>I am not able to import the Counter value in Hive. I am getting other row_key and column_name properly.</p> <p>Below is the output from Hive:</p> <p>hive&gt; select * from Counters;<br /> OK<br /> 213_debit_1326690000 1-sess_count d<br /> 213_debit_1326690000 1-total_db_time<br /> 213_debit_1326690000 1-total_exec_time<br /> 213_debit_1326690000 1-txn_count<br /> 213_debit_1326690000 2-sess_count<br /> 213_debit_1326690000 2-total_db_time<br /> 213_debit_1326690000 2-total_exec_time<br /> 213_debit_1326690000 2-txn_count<br /> Time taken: 0.263 seconds</p> <p>Below is output from Cassandra:</p> <p>[default@BAMSchema] list Counters;<br /> Using default limit of 100<br /> -------------------<br /> RowKey: 213_debit_1326690000<br /> =&gt; (counter=1-sess_count, value=100)<br /> =&gt; (counter=1-total_db_time, value=20)<br /> =&gt; (counter=1-total_exec_time, value=30)<br /> =&gt; (counter=1-txn_count, value=1)<br /> =&gt; (counter=2-sess_count, value=30)<br /> =&gt; (counter=2-total_db_time, value=30)<br /> =&gt; (counter=2-total_exec_time, value=30)<br /> =&gt; (counter=2-txn_count, value=1)</p> <p>As you can see that "d" junk letter is getting added in Hive when import happens from Cassandra to Hive. Wondering what am I missing.</p> <p>Thanks for your help! </p>