http://www.datastax.com/support-forums/topic/etl-tools-to-transfer-data-from-cassandra-into-other-relational-databases Software, Support, and Training for Apache Cassandra en-US Wed, 19 Jun 2013 04:56:05 +0000 http://bbpress.org/?v=1.0.3 q http://www.datastax.com/support-forums/search.php http://www.datastax.com/support-forums/topic/etl-tools-to-transfer-data-from-cassandra-into-other-relational-databases#post-8153 Thu, 27 Dec 2012 06:46:11 +0000 cko 8153@http://www.datastax.com/support-forums/ <p>Hi Srini,</p> <p>Thanks for your response.</p> <p>A few months ago, we did try Pentaho briefly. The job caused an OutOfMemoryError in Cassandra. The Cassandra Pentaho plugin does not handle retrieving rows in batches. It was mentioned that the next version of the plugin will address that issue. We will look at it again when it is available.</p> <p>Thanks again.<br /> Chin </p> http://www.datastax.com/support-forums/topic/etl-tools-to-transfer-data-from-cassandra-into-other-relational-databases#post-8015 Tue, 18 Dec 2012 18:09:37 +0000 Srini 8015@http://www.datastax.com/support-forums/ <p>This should help you. It's just an overview. Drill down more as per your requirement.</p> <p><a href="http://ovum.com/2012/03/08/pentaho-expands-big-data-coverage/" rel="nofollow">http://ovum.com/2012/03/08/pentaho-expands-big-data-coverage/</a></p> <p><a href="http://wiki.pentaho.com/display/DATAMINING/Cassandra+Source+and+Sink+in+Weka" rel="nofollow">http://wiki.pentaho.com/display/DATAMINING/Cassandra+Source+and+Sink+in+Weka</a> </p> http://www.datastax.com/support-forums/topic/etl-tools-to-transfer-data-from-cassandra-into-other-relational-databases#post-7972 Fri, 14 Dec 2012 03:13:03 +0000 cko 7972@http://www.datastax.com/support-forums/ <p>We will use Cassandra as logging storage in one of our web application. The application only insert rows into Cassandra but never update or delete any rows. The CF is expected to grow by about 0.5 million rows per day.</p> <p>We need to transfer the data in Cassandra to another relational database daily. Due to the large size of the CF, instead of truncating the relational table and reloading all rows into it each time, we plan to run a job to select the "delta" rows since the last run and insert them into the relational database.</p> <p>We know we can use Java, Pig or Hive to extract the delta rows to a flat file and load the data into the target relational table. We are particularly interested in a process that can extract delta rows without scanning the entire CF.</p> <p>Has anyone used any other ETL tools to do this kind of delta extraction from Cassandra? We appreciate any comments and experience.</p> <p>Thanks,<br /> Chin </p>