tristanls Software, Support, and Training for Apache Cassandra en-US Thu, 23 May 2013 00:00:03 +0000 http://bbpress.org/?v=1.0.3 q http://www.datastax.com/support-forums/search.php http://www.datastax.com/support-forums/topic/dse-30-reindexing-is-not-working-for-solr-copyfield#post-11030 Wed, 15 May 2013 00:59:46 +0000 jwlee 11030@http://www.datastax.com/support-forums/ <p>Sergio,</p> <p>All right. Good to hear it is in the plan.<br /> We'll do the workaround in the meantime.</p> <p>Thanks!<br /> Regards,<br /> JW </p> http://www.datastax.com/support-forums/topic/dse-30-reindexing-is-not-working-for-solr-copyfield#post-11007 Tue, 14 May 2013 07:24:59 +0000 sergio 11007@http://www.datastax.com/support-forums/ <p>Hi,</p> <p>the reason we don't currently support reindexing of *new* copy fields is that we always store them in Cassandra (that is, we always assume stored=true), hence we need the copy field value to be known upfront at ingestion time.</p> <p>We are planning to support non-stored mode too (that is, stored=false): this will not store any copy fields in Cassandra, but will allow for them to be reindexed post-ingestion, even new ones.</p> <p>In the meantime, you can use the workaround provided above.</p> <p>Hope that helps.<br /> Cheers,</p> <p>Sergio B. </p> http://www.datastax.com/support-forums/topic/dse-30-reindexing-is-not-working-for-solr-copyfield#post-10999 Tue, 14 May 2013 03:35:59 +0000 jwlee 10999@http://www.datastax.com/support-forums/ <p>Thanks to all for your replies.<br /> Just want to add that the copyfield is an important feature for us.<br /> We are frequently changing the Solr schema to suit the search behaviors required by our users.<br /> Really do hope that the reindexing for the copyfields are automatically applied.<br /> After all, isn't it suppose to be ETL free? </p> http://www.datastax.com/support-forums/topic/dse-30-reindexing-is-not-working-for-solr-copyfield#post-10976 Mon, 13 May 2013 16:28:44 +0000 rockbrain 10976@http://www.datastax.com/support-forums/ <p>Tristan,</p> <p>The following contains the examples necessary to use the CopyFieldInputTransformer.</p> <p><a href="http://pastebin.com/bi7cLQud" rel="nofollow">http://pastebin.com/bi7cLQud</a></p> <p><a href="http://pastebin.com/MpbnCNsH" rel="nofollow">http://pastebin.com/MpbnCNsH</a> </p> http://www.datastax.com/support-forums/topic/dse-30-reindexing-is-not-working-for-solr-copyfield#post-10975 Mon, 13 May 2013 15:39:10 +0000 tristanls 10975@http://www.datastax.com/support-forums/ <p>Jack, that's good to hear. My main concern was that this might get dropped, but that's been alleviated.</p> <p>Also, I only get to complain on these forums, but overall it's been a pretty easy and good experience so far. So keep up the good work. </p> http://www.datastax.com/support-forums/topic/dse-30-reindexing-is-not-working-for-solr-copyfield#post-10974 Mon, 13 May 2013 15:19:33 +0000 Jack Krupansky 10974@http://www.datastax.com/support-forums/ <p>Tristan, thanks for the detailed justification - it will help fuel our internal discussions.</p> <p>Why not do it? Timeframe, priorities, compatibility, perceived importance, architectural complexity - the usual bane of software engineering. It sometimes just takes some time to work through it all. But, rest assured, it is on our radar. copyField is a very important Solr feature.</p> <p>Workarounds are, by definition, never good architecture. They're simply temporary, stopgap measures. </p> http://www.datastax.com/support-forums/topic/dse-30-reindexing-is-not-working-for-solr-copyfield#post-10970 Mon, 13 May 2013 15:09:53 +0000 tristanls 10970@http://www.datastax.com/support-forums/ <p>rockbrain, yes, it would be great to know even if it's brittle right now and might change </p> http://www.datastax.com/support-forums/topic/dse-30-reindexing-is-not-working-for-solr-copyfield#post-10968 Mon, 13 May 2013 14:42:45 +0000 rockbrain 10968@http://www.datastax.com/support-forums/ <p>Tristan, 3.0.1 has a way to do copy fields, however at this point we are still changing the implementation so it is undocumented, are you interested in how to do this? </p> http://www.datastax.com/support-forums/topic/dse-30-reindexing-is-not-working-for-solr-copyfield#post-10967 Mon, 13 May 2013 14:38:28 +0000 tristanls 10967@http://www.datastax.com/support-forums/ <p>Hey Jack,</p> <p>Why would you not change this? Consider a workflow for improving an autosuggest feature (since you've indicated fixing this is only barely under consideration):</p> <p>1. create a copyField to copy someField to someField_autosuggest with different analyzers<br /> 2. index stuff</p> <p>Or another:</p> <p>1. create dynamic field "*_autosuggest" which does autosuggest analyzing<br /> 2. deploy<br /> 3. business guy: "hey I need this other thing to autosuggest also"<br /> 4. I add a copyField to copy with "_autosuggest" extension<br /> 5. reindex (broken!)</p> <p>I expect to upload a schema, reload, and reindex. Copy fields not being applied breaks this workflow and is unexpected.</p> <p>Let's be honest, "simply read each Cassandra row (or Solr document)" isn't simple nor is it good architecture, especially if I'm doing it over a network.</p> <p>Cheers,</p> <p>Tristan </p> http://www.datastax.com/support-forums/topic/dse-30-reindexing-is-not-working-for-solr-copyfield#post-10966 Mon, 13 May 2013 13:40:46 +0000 Jack Krupansky 10966@http://www.datastax.com/support-forums/ <p>Sorry, but in DSE the Solr copy fields are only applied when the data is inserted into Cassandra ("insert time"), and NOT when data is reindexed ("index time"). We are considering whether to change this, but at the present time, this is the way it is.</p> <p>One workaround is to simply read each Cassandra row (or Solr document) and update it with the same values. This should then invoke the "insert time" copy field processing, with the new copy fields. </p> http://www.datastax.com/support-forums/topic/dse-30-reindexing-is-not-working-for-solr-copyfield#post-10963 Mon, 13 May 2013 08:36:48 +0000 cybc16 10963@http://www.datastax.com/support-forums/ <p>I am facing the same issue. Does anyone have any idea?</p> <p>Please help me if you have solve this.</p> <p>Thanks. </p> http://www.datastax.com/support-forums/topic/dse-30-reindexing-is-not-working-for-solr-copyfield#post-10962 Mon, 13 May 2013 08:27:29 +0000 jwlee 10962@http://www.datastax.com/support-forums/ <p>Hi,</p> <p>I have an existing single instance of DSE Search/Solr.<br /> After adding some copyfields to the schema (and posted), I did a Re-index/Full-index via the DSE Search/Solr Admin UI.<br /> The new copyfields are still empty after the re-indexing.</p> <p>In addition, I have also tried the follow to rebuild index from command line as mentioned in the docs,<br /> Follow the below steps to rebuild your solr indexes.<br /> 1) Stop DSE node<br /> 2) Delete all sub-directories under /var/lib/cassandra/data/solr.data/directory.<br /> 3) Start DSE in search mode.<br /> 4) Run nodetool rebuild_index for each column family.</p> <p>Similarly, the copyfields are empty.</p> <p>Any help is much appreciated!</p> <p>Thanks!<br /> Regards,<br /> JW </p> http://www.datastax.com/support-forums/topic/dse-searchsolr-index-consistency-bug-status#post-10528 Tue, 30 Apr 2013 18:23:08 +0000 tristanls 10528@http://www.datastax.com/support-forums/ <p>Thanks Sergio,</p> <p>I'm looking forward to the fix.</p> <p>Cheers,</p> <p>Tristan </p> http://www.datastax.com/support-forums/topic/dse-searchsolr-index-consistency-bug-status#post-10527 Tue, 30 Apr 2013 17:22:49 +0000 sergio 10527@http://www.datastax.com/support-forums/ <p>Hi Tristan,</p> <p>sorry for this late reply.</p> <p>That actually looks like a bug affecting distributed field/column deletes (rather than generally soft commits): we're looking into it.</p> <p>Cheers,</p> <p>Sergio B. </p> http://www.datastax.com/support-forums/topic/dse-searchsolr-index-consistency-bug-status#post-10366 Thu, 25 Apr 2013 18:26:37 +0000 sergio 10366@http://www.datastax.com/support-forums/ <p>Hi Tristan,</p> <p>thanks for the additional information and test case: I'll have a look into it and get back. </p>