http://www.datastax.com/support-forums/topic/mapping-composite-column-names-on-hive Software, Support, and Training for Apache Cassandra en-US Fri, 24 May 2013 20:51:08 +0000 http://bbpress.org/?v=1.0.3 q http://www.datastax.com/support-forums/search.php http://www.datastax.com/support-forums/topic/mapping-composite-column-names-on-hive#post-9018 Thu, 14 Feb 2013 11:45:49 +0000 dipesh 9018@http://www.datastax.com/support-forums/ <p>If possible, can you please share a sample code for serializer and deserializer?<br /> And/or provide necessary pointers that could help in that regard.</p> <p>Here is what I have, but not working.</p> <pre><code>package com.cisco.iep.hive.plugins; import java.io.ByteArrayOutputStream; import java.nio.ByteBuffer; import java.util.ArrayList; import java.util.List; import org.apache.hadoop.hive.ql.exec.UDF; import org.apache.hadoop.io.BytesWritable; import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; public class WriteCompositeLongString extends UDF{ public BytesWritable evaluate(final LongWritable l, final Text s) { List&lt;byte[]&gt; b = new ArrayList&lt;byte[]&gt;(2); b.add(longWritableToByteArray(l)); b.add(textToByteArray(s)); return new BytesWritable(makeComposite(b)); } private byte[] textToByteArray(Text s){ if(s==null){ return null; } return s.getBytes(); } private byte[] longWritableToByteArray(final LongWritable l){ if(l==null){ return null; } return ByteBuffer.allocate(Long.SIZE).putLong(l.get()).array(); } private byte[] makeComposite(final List&lt;byte[]&gt; b) { ByteArrayOutputStream bos = new ByteArrayOutputStream(); for (int i = 0; i &lt; b.size(); i++) { bos.write((byte) ((b.get(i).length &gt;&gt; (7 + 1)) &amp; 0xFF)); bos.write((byte) (b.get(i).length &amp; 0xFF)); for (int j = 0; j &lt; b.get(i).length; j++) { bos.write(b.get(i)[j] &amp; 0xFF); } bos.write((byte) 0); } return bos.toByteArray(); } }</code></pre> <p>Thanks in advance!! </p> http://www.datastax.com/support-forums/topic/mapping-composite-column-names-on-hive#post-7442 Thu, 08 Nov 2012 14:07:24 +0000 pierantonio.merlino 7442@http://www.datastax.com/support-forums/ <p>Thank you for the answer.<br /> I solved the problem mapping the composite columns in to a struct in Hive and writing a custom serializer/deserializer. </p> http://www.datastax.com/support-forums/topic/mapping-composite-column-names-on-hive#post-7159 Mon, 29 Oct 2012 09:37:26 +0000 Piotr Kołaczkowski 7159@http://www.datastax.com/support-forums/ <p>There is no dedicated support in Hive for composite columns. In wide-row mode, it scans each row sequentially and fetches each column to produce a triple (row-key, column-name, value). Outputting as triples is hardcoded. Anyway, this could make a nice feature-request. </p> http://www.datastax.com/support-forums/topic/mapping-composite-column-names-on-hive#post-6763 Thu, 04 Oct 2012 10:00:30 +0000 pierantonio.merlino 6763@http://www.datastax.com/support-forums/ <p>Hi,<br /> I've to map Cassandra columns families on Hive tables.<br /> Since I've dynamic fields in Cassandra, I use the following string</p> <pre><code>CREATE EXTERNAL TABLE Table1 (keyName string, columnName string, value string) STORED BY &#039;org.apache.hadoop.hive.cassandra.CassandraStorageHandler&#039; WITH SERDEPROPERTIES (&quot;cassandra.columns.mapping&quot; = &quot;:key,:column,:value&quot; );</code></pre> <p>in order to create the external table on Hive. The result is a table where each row is as follows:</p> <p><code>rowName1 columnName1 value1</code><br /> <code>rowName2 columnName2 value2</code><br /> ...</p> <p>Since I use composite column names in Cassandra (Composite(columnNameA, columnNameB)), is it possible to map them into two different columns on Hive?<br /> What I want is as follows:</p> <p><code>rowName1 columnNameA1 columnNameB1 value1</code>.</p> <p>How can I map these columns on Hive?</p> <p>Thanks in advance. </p>