cartera_alex Software, Support, and Training for Apache Cassandra en-US Mon, 20 May 2013 19:22:31 +0000 http://bbpress.org/?v=1.0.3 q http://www.datastax.com/support-forums/search.php http://www.datastax.com/support-forums/topic/using-pigunit-with-pig-scripts-that-write-to-cassandrastorage#post-9758 Mon, 08 Apr 2013 22:09:06 +0000 cartera_alex 9758@http://www.datastax.com/support-forums/ <p>Hi,</p> <p>Does anyone know if it's possible to use PigUnit to test scripts that use "store... using CassandraStorage()"? I'd like to know so I don't waste time writing tests if this is not going to work.</p> <p>Thanks. </p> http://www.datastax.com/support-forums/topic/error-defining-pig-schema-in-dse-30#post-9757 Mon, 08 Apr 2013 22:05:26 +0000 cartera_alex 9757@http://www.datastax.com/support-forums/ <p>I was actually able to make it work like this:</p> <p>compo = load 'cassandra://keyspace/column_fam' using CassandraStorage()<br /> as (key:chararray, columns: bag{c:(name:(),value:int)});</p> <p>I can then do something like this:</p> <p>compo_cols = foreach compo generate key, flatten(columns.name) as compo_col; </p> http://www.datastax.com/support-forums/topic/error-defining-pig-schema-in-dse-30#post-9701 Fri, 05 Apr 2013 18:50:29 +0000 alexliu 9701@http://www.datastax.com/support-forums/ <p>The column family created by cql with "COMPACT STORAGE" and composite key doesn't have composite column defined. You can check it by cassandra-cli using show schema command.</p> <p>A workaround is </p> <p>e.g.<br /> compo = load 'cassandra://keyspace/column_fam' using CassandraStorage();</p> <p>compo = foreach compo generate key as method, flatten(columns);</p> <p>dump compo;</p> <p>It shows composite key and values for each line, then you can use dump data a resource for more processing. </p> http://www.datastax.com/support-forums/topic/error-defining-pig-schema-in-dse-30#post-9641 Mon, 01 Apr 2013 17:43:41 +0000 cartera_alex 9641@http://www.datastax.com/support-forums/ <p>Hi,</p> <p>I'm trying to load data from a Cassandra column family defined with "COMPACT STORAGE" and composite key.<br /> When I try the following load command in Pig</p> <p>grunt&gt; data = load 'cassandra://keyspace/column_fam'<br /> &gt;&gt; using CassandraStorage()<br /> &gt;&gt; AS (mid: int, segs: {seg: (key: (b, d, s, m), o: int)} ); </p> <p>I get this error:</p> <p>2013-04-01 13:27:45,693 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1200: &lt;line 3, column 54&gt; mismatched input ';' expecting RIGHT_PAREN</p> <p>As far as I can tell, all parens match. The same statement works fine with DSE 2.2.</p> <p>Any ideas? </p> http://www.datastax.com/support-forums/topic/using-sqoop-with-query-option-to-import-data-into-cassandra-column-family-doesnt-seem-to-work#post-8154 Thu, 27 Dec 2012 12:05:44 +0000 ernipur 8154@http://www.datastax.com/support-forums/ <p>Hi,</p> <p>I would rather suggest to use outdir switch. As per my experience looks like RBDMS column name and Cassandra column name should be same for input to be successful. Some thing like this should work.</p> <p>dse sqoop import --connect jdbc:mysql://db.server.address/schema_name --username user --p --outdir /some/output/dir/ --query "select x, y, z from table where cond1 and cond2 and \$CONDITIONS" --target-dir /some/dir --split-by x --cassandra-keyspace cks1 --cassandra-column-family ccf1 --cassandra-row-key x --cassandra-thrift-host 127.0.0.1 --append</p> <p>Thanks ,</p> <p>_Nipur </p> http://www.datastax.com/support-forums/topic/using-composite-columns-in-mapreduce#post-8051 Fri, 21 Dec 2012 06:19:06 +0000 cartera_alex 8051@http://www.datastax.com/support-forums/ <p>This is, in fact, a manifestation of <a href="https://issues.apache.org/jira/browse/CASSANDRA-4377" rel="nofollow">https://issues.apache.org/jira/browse/CASSANDRA-4377</a> </p> http://www.datastax.com/support-forums/topic/using-composite-columns-in-mapreduce#post-8050 Fri, 21 Dec 2012 05:32:48 +0000 cartera_alex 8050@http://www.datastax.com/support-forums/ <p>I'm starting to think that this is a bug in Thrift.<br /> Since posting the original post, I have implemented a whole new output format that uses the Astyanax client. I have also tried this from a stand-alone program using Astyanax.</p> <p>What I noticed is that I am able to insert columns that have UTF8Type (string) values while any other insertions fail. Surprisingly, if I attempt to insert a string value into a column that was defined as a non-"varchar" type in CQL3 from a Thrift-based client it succeeds! I also noticed that cassandra-cli lists this CF as having default_validation_class = 'UTF8Type'.</p> <p>Inserting non-string values into this CF from cqlsh using CQL3 works fine. </p> http://www.datastax.com/support-forums/topic/using-composite-columns-in-mapreduce#post-8008 Tue, 18 Dec 2012 15:47:40 +0000 cartera_alex 8008@http://www.datastax.com/support-forums/ <p>Hello,<br /> I'm trying to use ColumnFamilyOutputFormat to write to a Cassandra column family with composite columns from my reducer. Since Thrift is the only available API in this case (I wish it wasn't so), I followed an example outlined here: <a href="http://stackoverflow.com/questions/11801288/inserting-and-getting-from-cassandra-with-composite-key-invalidrequestexception" rel="nofollow">http://stackoverflow.com/questions/11801288/inserting-and-getting-from-cassandra-with-composite-key-invalidrequestexception</a></p> <p>Unfortunately, I'm getting the following exception when I run the job:</p> <p>12/12/17 21:37:32 INFO mapred.JobClient: Task Id : attempt_201212120944_0011_r_000003_2, Status : FAILED<br /> java.io.IOException: InvalidRequestException(why:(String didn't validate.) [c8][txnspermember][12:345:ac0a2000-effb-11e1-c000-000000000000:1234:56789:somememberid:txn_date] failed validation)<br /> at org.apache.cassandra.hadoop.ColumnFamilyRecordWriter$RangeClient.run(ColumnFamilyRecordWriter.java:316)<br /> Caused by: InvalidRequestException(why:(String didn't validate.) [c8][txnspermember][12:345:ac0a2000-effb-11e1-c000-000000000000:1234:56789:somememberid:txn_date] failed validation)<br /> at org.apache.cassandra.thrift.Cassandra$batch_mutate_result.read(Cassandra.java:20253)<br /> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)<br /> at org.apache.cassandra.thrift.Cassandra$Client.recv_batch_mutate(Cassandra.java:922)<br /> at org.apache.cassandra.thrift.Cassandra$Client.batch_mutate(Cassandra.java:908)<br /> at org.apache.cassandra.hadoop.ColumnFamilyRecordWriter$RangeClient.run(ColumnFamilyRecordWriter.java:308)</p> <p>Note that "somememberid" is what I would expect my row key to be, yet it appears in the middle of the composite string for some reason.</p> <p>My column family looks like this:<br /> <pre><code>CREATE TABLE txnspermember ( member_id text, org_id int, brand_id int, post_date &#039;TimeUUIDType&#039;, merchant_id bigint, offer_id bigint, txn_id text, agreed_rebate text, catalog_id text, cb_elig boolean, cli_shr_pct double, conv_rate double, description text, first_four int, first_name text, g_merchant_id bigint, last_four int, mbr_shr_pct double, mcc bigint, member_zip text, mid text, mn_shr_pct double, rebate_amt double, rebate_val double, spend_amt double, txn_date &#039;TimeUUIDType&#039;, PRIMARY KEY (member_id, org_id, brand_id, post_date, merchant_id, offer_id, txn_id) ) WITH comment=&#039;&#039; AND caching=&#039;KEYS_ONLY&#039; AND read_repair_chance=0.100000 AND gc_grace_seconds=864000 AND replicate_on_write=&#039;true&#039; AND compaction_strategy_class=&#039;SizeTieredCompactionStrategy&#039; AND compression_parameters:sstable_compression=&#039;SnappyCompressor&#039;;</code></pre> http://www.datastax.com/support-forums/topic/using-sqoop-with-query-option-to-import-data-into-cassandra-column-family-doesnt-seem-to-work#post-7098 Wed, 24 Oct 2012 15:18:31 +0000 cartera_alex 7098@http://www.datastax.com/support-forums/ <p>Hi,</p> <p>I'm trying to use sqoop to import some data from a MySQL database into a Cassandra column family. I've been able to do this successfully when I used the --table option (like shown in the sqoop demo in the docs), but using --query option doesn't seem to work.</p> <p>If I run a command like this:</p> <p><code>dse sqoop import --connect jdbc:mysql://db.server.address/schema_name --username user -P --query &quot;select x, y, z from table where cond1 and cond2 and \$CONDITIONS&quot; --split-by x --cassandra-keyspace cks1 --cassandra-column-family ccf1 --cassandra-row-key x --cassandra-thrift-host 127.0.0.1</code></p> <p>it fails and tells me that --target-dir option must be specified.</p> <p>If I then add the --target-dir option like this:</p> <p><code>dse sqoop import --connect jdbc:mysql://db.server.address/schema_name --username user -P --query &quot;select x, y, z from table where cond1 and cond2 and \$CONDITIONS&quot; --split-by x --target-dir /some/dir --cassandra-keyspace cks1 --cassandra-column-family ccf1 --cassandra-row-key x --cassandra-thrift-host 127.0.0.1</code></p> <p>the job runs, but does not import any records.</p> <p>Running this import without the cassandra options works and dumps the data into CFS.</p> <p>Am I doing something wrong or is it just not possible to import directly into a Cassandra column family using the --query option?</p> <p>I am using DSE 2.2. </p>