Support Forums : DataStax

DataStax Support Forums » DataStax Enterprise

Do not trigger Solr commit when updating a stored only field

(5 posts) (2 voices)

Started 11 months ago by
Latest reply from
This topic is not a support question

Tags:

Anonymous
Unregistered

I am trying DSE and Solr.
I started from the wikipedia example and i changed the Solr schema setting a field as stored only. I am amazed from the fact that stored fields (with indexed=false) are not persisted inside the Solr index, great feature!

I tried to execute a partial update on one document, changing the value of the stored-only field. I saw that the update triggers a Solr commit with a subsequent cache refresh.
I am just wondering if it make sense to avoid the Solr commit when stored-only fields are updated. Ideally it could improve query performance since caches are not regenerated.

What do you think? Is it something doable?

Posted 11 months ago #
Anonymous
Unregistered

Hi, great question. Are you using SolrJ or CQL to update the document / row?

Posted 11 months ago #
Anonymous
Unregistered

Hi rockbrain,
i have tried with both SolrJ and CQL.

Just to keep you in the loop, in case someone else is interested, i tried to make few changes to the Solr configuration, but without luck :(
- removed the autocommit option which is set in solrconfig.xml (updateHandler). Without it the commit is not fired but documents are not updated
- removed Solr documentCache

After a partial update if you inspect the document using CQL changes are visible (as expected). However with a Solr query the document remains unchanged until a commit is called (or you restart DSE).

Posted 11 months ago #
Anonymous
Unregistered

Edo, Ok sounds like it could be a bug. Are you aware of this?

http://www.datastax.com/docs/datastax_enterprise2.0/search/dse_search_cluster#updating-individual-fields-in-a-solr-document

Also this is a thread on a similar topic:

http://www.datastax.com/support-forums/topic/re-indexing-partial-fields-in-dse-20

Internally DSE is getting the key from Solr then loading the data from Cassandra. Maybe there is a missing step that requires a commit. Is there a large problem with calling even a soft commit for you?

Posted 11 months ago #
Anonymous
Unregistered

Hi rockbrain,
i actually make it work!
I was aware of the links you posted. I don't understand why it wasn't behaving as expected but it's sorted now :)

I still have a curiosity, as it's mentioned in the other topic (http://www.datastax.com/support-forums/topic/re-indexing-partial-fields-in-dse-20), whenever you update a field, under the hood the entire document is deleted and added from the Solr index. Does it means that it performs a soft-commit?

I guess these information are stored in memory. Is there a way to limit the amount of memory used (i imagine a case where you have a lot of updates which means a lot of memory used to keep these new documents before being flushed to disk).
thank you!

Posted 11 months ago #

RSS feed for this topic

Reply

You must log in to post.