Re-indexing partial fields in DSE 2.0

(4 posts) (3 voices)

• Started 11 months ago by
• Latest reply from
• This topic is not a support question

Tags:

• m n v cv v .

1. 

   Anonymous
   Unregistered

   We are evaluating DSE 2.0 for it's ability to re-index fields individually. Our application has lot of update-able fields. We currently store all these fields in the index itself.

   I am curious as to how DSE 2.0 handles partial updates for fields? Even though it looks like the request updates a single field, under-the-hood will the entire document be read from cassandra and then delete/add to lucene performed?

   Is my understanding correct or is there some finer details I am missing

   Regards,
   Ravi

   Posted 11 months ago #
2. 

   tjake
   Moderator

   That is correct. Since we hold all the fields in Cassandra we can rebuild the document internally. Please note, to get this behavior using the solr apis you must pass a extra http param. see ,

   http://www.datastax.com/docs/datastax_enterprise2.0/search/dse_search_cluster#updating-individual-fields-in-a-solr-document

   Posted 11 months ago #
3. 

   Anonymous
   Unregistered

   Thanks Jake.
   Right now, we have built a proprietary distributed search over hadoop using native lucene and are looking for alternatives.

   I have a few more questions on this architecture

   a) How do we keep lucene indexes and CFs in sync? Will index write be attempted first and after that CFs get updated? If CF writes fail, do we rely on commitlog to replay the mutations?

   b) Will read-repair work even for native solr queries also?. If we read/write at QUORUM, are we guaranteed to see the correct results returning from nodes?

   c) Also we faced a power-cut scenario some time back and corrupted many of our indexes in hadoop. I guess enterprise search is also vulnerable to such issues. In those cases, will a simple re-build of indexes suffice to restore the data?

   Regards,
   Ravi

   Posted 11 months ago #
4. 

   Anonymous
   Unregistered

   > a) How do we keep lucene indexes and CFs in sync? Will index write be attempted first and after that CFs get updated? If CF writes fail, do we rely on commitlog to replay the mutations?

   The index is in RAM until it is flushed, however the commit log guarantees that data is not lost.

   > b) Will read-repair work even for native solr queries also?. If we read/write at QUORUM, are we guaranteed to see the correct results returning from nodes?

   DSE supports writes at quorum.

   > c) Also we faced a power-cut scenario some time back and corrupted many of our indexes in hadoop. I guess enterprise search is also vulnerable to such issues. In those cases, will a simple re-build of indexes suffice to restore the data?

   Lucene is generally pretty good in regards to handling things like power outages. With DSE the advantage over regular Solr is the index can be completely rebuilt from the data stored in Cassandra (which as a side notes is also compressed, something Solr / Lucene do not offer).

   Posted 11 months ago #

RSS feed for this topic

Reply

You must log in to post.