http://www.datastax.com/support-forums/topic/bulkloading-sstables-including-solr-content Software, Support, and Training for Apache Cassandra en-US Wed, 22 May 2013 10:24:39 +0000 http://bbpress.org/?v=1.0.3 q http://www.datastax.com/support-forums/search.php http://www.datastax.com/support-forums/topic/bulkloading-sstables-including-solr-content#post-1746 Mon, 23 Apr 2012 20:06:35 +0000 jas 1746@http://www.datastax.com/support-forums/ <p>Given this read-only content of mine, is it possible to issue a Solr query with a consistency level of ONE? I see in the DSE 2 docs that <code>cl=ONE</code> can be specified when using the Solr HTTP API. However, the example and discussion pertain to issuing and update, not a query. Given I have RF&gt;1, it seems that perhaps response time could be reduced slightly with cl=ONE vs. requiring waiting for two nodes to respond?</p> <p>Can the cl parameter be applied to a query? What's the default? I'm guessing LOCAL_QUORUM perhaps?</p> <p>Thanks,</p> <p>Jeff </p> http://www.datastax.com/support-forums/topic/bulkloading-sstables-including-solr-content#post-1745 Mon, 23 Apr 2012 16:21:34 +0000 jas 1745@http://www.datastax.com/support-forums/ <p>Thanks Jake.</p> <p>The more I think about it, the more I like the notion of defining a new column family per quarter. When that's done and verified, signal the app in some manner to refer to that new core. I could even make the name of the core tenant specific and allow a tenant individual some flexibility as to when to migrate to the new content.</p> <p>There are other non-search related column families as well that are associated with the quarterly update, but the same mechanism can be applied to them as well.</p> <p>Look forward to see what appears in 2.1. :)</p> <p>Jeff </p> http://www.datastax.com/support-forums/topic/bulkloading-sstables-including-solr-content#post-1744 Mon, 23 Apr 2012 15:39:11 +0000 tjake 1744@http://www.datastax.com/support-forums/ <p>Hi Jeff,</p> <p>I'll work backwards. Correct, solrjson:["Human","Mouse","Rat","Cow"] is how you define a multivalued field.</p> <p>Next, If you currently drop your solr index and rebuild it each quarter why not use a new column family per quarter and instruct your app to use the new one? You can also take one node out of the cluster at a time (assuming RF&gt;1) and rebuild the data on it then bring it back in.</p> <p>We are working on a workflow for migrating from one schema to another using the same data but not in 2.0.</p> <p>-Jake </p> http://www.datastax.com/support-forums/topic/bulkloading-sstables-including-solr-content#post-1743 Mon, 23 Apr 2012 03:09:08 +0000 jas 1743@http://www.datastax.com/support-forums/ <p>Hello:</p> <p>A major aspect of my application is that in addition to tenant provided content, there is a quantity of 'canonical content' that gets updated once a quarter. I am making it available for both search and retrieval. I'm looking for a more production worthy way to accomplish this.</p> <p>Prior to the release of DSE 2.0, this content existed in both Cassandra (non-search) and Solr. For the former, the content build process generates SSTables using org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter, and the latter by generating native Solr XML formatted files. Being pre-production at the time, and only using a single Cassandra node, it was pretty straightforward to shutdown Cassandra, and replace the SSTables with the new ones. Likewise for Solr, I could remove the old index and ingest the new Solr XML files.</p> <p>Regardless of DSE 2, moving to multiple Cassandra nodes requires more finesse than my procedure above. It seems what I need to do in that case is described here:</p> <p><a href="http://www.datastax.com/dev/blog/bulk-loading" rel="nofollow">http://www.datastax.com/dev/blog/bulk-loading</a></p> <p>However, Sylvain's blog post is dated August 1, 2011 and speaks of Cassandra 0.8.1. Is using sstableloader still the way to go in the Cassandra 1.0.x, and soon to be 1.1, world? I'm already generating the SSTables, but they need to be uploaded to the cluster rather than just a single node.</p> <p>With DSE 2, it sounds like I can continue to index the existing Solr XML files I generate, or more directly upload the new content directly into Cassandra, also using sstableloader. The search related keyspace and column family (Solr core) definition may or may not change as well for a given quarterly update.</p> <p>With Solr today, I can issue a deletion query and remove all documents, then process a series of files to define the new content, and at the end of that, issue a commit. Once that has finally completed, then new queries will use that data. Until then, current searches will continue to refer to the old content. Is there a way to accomplish something similar going through Cassandra via sstableloader? I'm guessing the content will be updated as it goes, becoming eventually consistent across the cluster.</p> <p>I suppose another approach would be to create a new version of the core via the HTTP API, then bulk load the content, and then instruct my app to then refer to the new core. The name of the current content core could be set in Cassandra itself, which the app will read and start using. At some point, the old core can be deleted. I kind of like this approach since I'll only have to deal with SSTables and Cassandra, and not Solr XML etc.</p> <p>Finally, the canonical content is comprised of numerous multi-valued fields. If I pull up such a document field in the CLI, I see:</p> <pre><code>(column=n_macromolecule_species, value=solrjson:[&quot;Human&quot;,&quot;Mouse&quot;,&quot;Rat&quot;,&quot;Cow&quot;], timestamp=1333552307242000)</code></pre> <p>Is that column value a standard I can make use of to define multi-valued fields, or is it some secret DSE Solr integration thing?</p> <p>Thanks!</p> <p>Jeff </p>