DataStax Support Forums » Tag: EC2

DataStax Support Forums » Tag: EC2 - Recent Posts http://www.datastax.com/support-forums/tags/ec2 Software, Support, and Training for Apache Cassandra en-US Fri, 24 May 2013 17:13:11 +0000 http://bbpress.org/?v=1.0.3 <![CDATA[Search]]> q http://www.datastax.com/support-forums/search.php joaquin on "Expanding cluster with AMI: version issue" http://www.datastax.com/support-forums/topic/expanding-cluster-with-ami-version-issue#post-7303 Wed, 31 Oct 2012 18:40:09 +0000 joaquin 7303@http://www.datastax.com/support-forums/ Thanks again for the report! Alain on "Expanding cluster with AMI: version issue" http://www.datastax.com/support-forums/topic/expanding-cluster-with-ami-version-issue#post-7302 Wed, 31 Oct 2012 18:28:56 +0000 Alain 7302@http://www.datastax.com/support-forums/ Hi Joaquin, It works like a charm now. Thanks. joaquin on "Expanding cluster with AMI: version issue" http://www.datastax.com/support-forums/topic/expanding-cluster-with-ami-version-issue#post-7297 Wed, 31 Oct 2012 17:50:40 +0000 joaquin 7297@http://www.datastax.com/support-forums/ Hello, This is now patched in the AMI codebase. --release 1.1.2 should work now. Thanks for the bug report! Joaquin Alain on "Expanding cluster with AMI: version issue" http://www.datastax.com/support-forums/topic/expanding-cluster-with-ami-version-issue#post-7213 Tue, 30 Oct 2012 15:45:43 +0000 Alain 7213@http://www.datastax.com/support-forums/ I just tried with the release 1.1.2-1 instead of 1.1.2: Result: Cluster started with these options: --clustername expand --totalnodes 1 --version community --release 1.1.2-1 --opscenter no ------------------------------------ DataStax AMI for DataStax Enterprise and DataStax Community AMI version 2.2 DataStax Community version 1.1.6-1 ------------------------------------ Alain on "Expanding cluster with AMI: version issue" http://www.datastax.com/support-forums/topic/expanding-cluster-with-ami-version-issue#post-7210 Tue, 30 Oct 2012 14:49:23 +0000 Alain 7210@http://www.datastax.com/support-forums/ Hi joaquin, 3 month after my last message I need to extend my cluster again and finally can try your solution. in my console after adding an instance I have : Cluster started with these options: --clustername expand --totalnodes 1 --version community --opscenter no --release 1.1.2 But "nodetool -h localhost" version returns ReleaseVersion: 1.1.6 Did I make any mistake ? joaquin on "Expanding cluster with AMI: version issue" http://www.datastax.com/support-forums/topic/expanding-cluster-with-ami-version-issue#post-3145 Tue, 17 Jul 2012 00:12:21 +0000 joaquin 3145@http://www.datastax.com/support-forums/ Hello Alain, Use the --release option to set your DSE/DSC version. <a href="https://github.com/riptano/ComboAMI/blob/2.2/SWITCHES.md" rel="nofollow">https://github.com/riptano/ComboAMI/blob/2.2/SWITCHES.md</a> I've messaged our docs team to ensure this option is visible through our docs as well. Thanks, Joaquin Alain on "Expanding cluster with AMI: version issue" http://www.datastax.com/support-forums/topic/expanding-cluster-with-ami-version-issue#post-3119 Mon, 16 Jul 2012 09:01:45 +0000 Alain 3119@http://www.datastax.com/support-forums/ Hi, I'm trying to expand my cluster on Amazon EC2. The point is my cluster is on version 1.0.9 and the new nodes generated by the AMI will be on 1.1.2. I would like to stay on 1.0.x version for now. How can I add capacity ? Alain jas on "Levers for increasing DSE 2 Solr performance?" http://www.datastax.com/support-forums/topic/levers-for-increasing-dse-2-solr-performance/page/2#post-1830 Thu, 03 May 2012 15:44:20 +0000 jas 1830@http://www.datastax.com/support-forums/ Hi Rock: I had some things to take care of as well as I pursued the FVH problem using the more formal support ticket process. It turns out there was an FVH implementation bug in DSE 2.0 which has been reproduced and fixed and should be included in the DSE 2.1 release, whenever that is. I cannot speak for DataStax, of course, but I'm told the fix is in. :) That means I feel confident I can continue to develop around DSE 2.x and know that fix is coming. I look forward to when I can abandon plain old Solr completely, but until then, for high performance demos/testing, I'll continue to use Solr with FVH, and also work to incorporate DSE and Solr 4 features into my application. Cheers, Jeff Anonymous on "Levers for increasing DSE 2 Solr performance?" http://www.datastax.com/support-forums/topic/levers-for-increasing-dse-2-solr-performance/page/2#post-1805 Mon, 30 Apr 2012 20:47:00 +0000 Anonymous 1805@http://www.datastax.com/support-forums/ Jeff, To see the term vectors, this Solr component should be installed: <a href="http://wiki.apache.org/solr/TermVectorComponent" rel="nofollow">http://wiki.apache.org/solr/TermVectorComponent</a> Can you post your data and queries etc so we can reproduce the problem? It is likely a FVH bug. jas on "Levers for increasing DSE 2 Solr performance?" http://www.datastax.com/support-forums/topic/levers-for-increasing-dse-2-solr-performance/page/2#post-1794 Sat, 28 Apr 2012 20:14:45 +0000 jas 1794@http://www.datastax.com/support-forums/ Is it possible there is a bug in DSE 2 with regards to term offsets and positions when using FVH? I don't have a Solr 4.0 setup handy, but I did update my previous Solr 3.5.0 configuration to match that with DSE and I reindexed my content. Why the document ID is present in the DSE FVH snippet is a mystery. I do not see those in my Solr 3.5.0 output using FVH. It looks like the DSE non-FVH version posted previously, except there is a trailing space after each of the two closed <code></code> tag. So, why do I get document IDs appearing in the snippets using FVH with DSE 2? Looking at that particular document (ING:6xwoe) in Cassandra: <pre><code>[default@IngenuityContent] get SearchMain['ING:6xwoe']; ... => (column=n_name, value=egfr, timestamp=1335497830220000) ... => (column=n_synonym, value=egfr, timestamp=1335497830220000) ... => (column=type, value=node, timestamp=1335497830220000) Returned 24 results. Elapsed time: 22 msec(s). [default@IngenuityContent]</code></pre> Those field values do not contain the document ID. :) Is it possible to see the term offsets and positions somewhere? Thanks, Jeff jas on "Levers for increasing DSE 2 Solr performance?" http://www.datastax.com/support-forums/topic/levers-for-increasing-dse-2-solr-performance/page/2#post-1788 Fri, 27 Apr 2012 17:45:32 +0000 jas 1788@http://www.datastax.com/support-forums/ Anybody got any opinions regarding my posting from 4 days ago regarding how to best make use of 34GB of RAM using the heap, Solr caches, Cassandra row and key caches? I realize there is not specific magic answer, but a push in the right direction would be appreciated. Thanks, Jeff jas on "Levers for increasing DSE 2 Solr performance?" http://www.datastax.com/support-forums/topic/levers-for-increasing-dse-2-solr-performance/page/2#post-1785 Fri, 27 Apr 2012 16:30:40 +0000 jas 1785@http://www.datastax.com/support-forums/ I've not yet updated my EC2 instances, but I've been trying to resolve the FVH issue using my 2010 Core i7 iMac, with 16GB of RAM, using the DSE tarball install. A typical query generated by our application is used below, using the standard highlighter and then the fast vector highlighter. I'm using curl to speak directly to Solr/DSE, eliminating any additional application processing delays. The curl request is issued on the same system running DSE (localhost). <code>curl "http://localhost:8983/solr/IngenuityContent.SearchMain/select?hl.requireFieldMatch=true&facet=true&facet.offset=0&facet.mincount=1&facet.limit=8&hl=true&rows=10&fl=id,n_type,n_typeCategory,n_is_group,n_nameExact,n_synonymExact,n_macromolecule_name,n_macromolecule_id,n_macromolecule_species,n_macromolecule_summary,n_c_pubchem_cid,n_c_formula,n_c_cas_number,n_m_acc,n_m_descr&facet.sort=index&start=0&q=egfr&facet.field=n_diseaseFacet&facet.field=n_typeFacet&facet.field=n_tissue_typeFacet&facet.field=n_pathway_nameExact&facet.field=n_macromolecule_species&facet.field=n_locationFacet&facet.field=n_functionFacet&hl.usePhraseHighlighter=true&qt=partner-tmo&fq=type:node&debug=timing" > dse-search-egfr.timing.xml</code> The DSE log file shows: <pre><code>INFO [http-8983-3] 2012-04-27 10:09:39,469 SolrCore.java (line 1470) [IngenuityContent.SearchMain] webapp=/solr path=/select params={hl.requireFieldMatch=true&facet=true&facet.mincount=1&facet.offset=0&facet.limit=8&debug=timing&hl=true&rows=10&fl=id,n_type,n_typeCategory,n_is_group,n_nameExact,n_synonymExact,n_macromolecule_name,n_macromolecule_id,n_macromolecule_species,n_macromolecule_summary,n_c_pubchem_cid,n_c_formula,n_c_cas_number,n_m_acc,n_m_descr&facet.sort=index&start=0&q=egfr&facet.field=n_diseaseFacet&facet.field=n_typeFacet&facet.field=n_tissue_typeFacet&facet.field=n_pathway_nameExact&facet.field=n_macromolecule_species&facet.field=n_locationFacet&facet.field=n_functionFacet&qt=partner-tmo&hl.usePhraseHighlighter=true&fq=type:node} hits=2354 status=0 QTime=285</code></pre> Running this several times, the QTime varied from 279-292 ms. Well, the first time was 704 ms, but let's say that filled the caches... The Solr timing information for that request above, attributes 281 ms to <code>HighlightComponent</code>. Now, I added <code>hl.useFastVectorHighlighter=true</code> to the request: <code>curl "http://localhost:8983/solr/IngenuityContent.SearchMain/select?hl.requireFieldMatch=true&facet=true&facet.offset=0&facet.mincount=1&facet.limit=8&hl=true&hl.useFastVectorHighlighter=true&rows=10&fl=id,n_type,n_typeCategory,n_is_group,n_nameExact,n_synonymExact,n_macromolecule_name,n_macromolecule_id,n_macromolecule_species,n_macromolecule_summary,n_c_pubchem_cid,n_c_formula,n_c_cas_number,n_m_acc,n_m_descr&facet.sort=index&start=0&q=egfr&facet.field=n_diseaseFacet&facet.field=n_typeFacet&facet.field=n_tissue_typeFacet&facet.field=n_pathway_nameExact&facet.field=n_macromolecule_species&facet.field=n_locationFacet&facet.field=n_functionFacet&hl.usePhraseHighlighter=true&qt=partner-tmo&fq=type:node&debug=timing" > dse-search-egfr.timing.fast.xml</code> Log indicates: <pre><code>INFO [http-8983-3] 2012-04-27 10:15:48,687 SolrCore.java (line 1470) [IngenuityContent.SearchMain] webapp=/solr path=/select params={hl.requireFieldMatch=true&facet=true&facet.mincount=1&facet.offset=0&facet.limit=8&debug=timing&hl=true&rows=10&fl=id,n_type,n_typeCategory,n_is_group,n_nameExact,n_synonymExact,n_macromolecule_name,n_macromolecule_id,n_macromolecule_species,n_macromolecule_summary,n_c_pubchem_cid,n_c_formula,n_c_cas_number,n_m_acc,n_m_descr&hl.useFastVectorHighlighter=true&facet.sort=index&start=0&q=egfr&facet.field=n_diseaseFacet&facet.field=n_typeFacet&facet.field=n_tissue_typeFacet&facet.field=n_pathway_nameExact&facet.field=n_macromolecule_species&facet.field=n_locationFacet&facet.field=n_functionFacet&qt=partner-tmo&hl.usePhraseHighlighter=true&fq=type:node} hits=2354 status=0 QTime=182</code></pre> This was pretty consistent, varying from 181-192 ms. I will take a gain of 100 ms, thank you very much! :) The timing info reports 177 ms for <code>HighlightComponent</code>. This is admittedly a pretty narrow test case. But it shows I can get it to work and enjoy some benefits. Now, my problem is that the highlights produced by FVH are not yet ready for prime time. For example, for a given document, the old highlighter provides me: <pre><code><lst name="ING:6xwoe"> <arr name="n_name"> <str>egfr</str> </arr> <arr name="n_synonym"> <str>egfr</str> </arr> </lst></code></pre> For that same document, FVH gives: <pre><code><lst name="ING:6xwoe"> <arr name="n_name"> <str>ING:6xwoe egfr </str> </arr> <arr name="n_synonym"> <str>ING:6xwoe egfr </str> </arr> </lst></code></pre> For whatever reason, the document ID is included in the snippet, and FVH is highlighting a portion of it rather than egfr. Go figure... Oh well, that's what I have to do now. :) Cheers, Jeff jas on "Levers for increasing DSE 2 Solr performance?" http://www.datastax.com/support-forums/topic/levers-for-increasing-dse-2-solr-performance/page/2#post-1768 Wed, 25 Apr 2012 21:01:58 +0000 jas 1768@http://www.datastax.com/support-forums/ Rock: Hey, this is great! I will check out those links. Getting FVH to work is only one of several things i have to get done soon. :) Increasing the index size is not a problem. It's only a few gigs anyway, but highlighting is extremely important for the application. When issuing queries (to Solr 3.5.0 or DSE 2) with <code>debug=timing</code> it's apparent that highlighting is taking 90+% of the time typically. I've not yet gotten any responses from solr-users, but a good description of my issue can be found here: <a href="http://lucene.472066.n3.nabble.com/Re-FastVectorHighlighter-gt-no-highlights-td3933592.html" rel="nofollow">http://lucene.472066.n3.nabble.com/Re-FastVectorHighlighter-gt-no-highlights-td3933592.html</a> Cheers, Jeff Anonymous on "Levers for increasing DSE 2 Solr performance?" http://www.datastax.com/support-forums/topic/levers-for-increasing-dse-2-solr-performance/page/2#post-1767 Wed, 25 Apr 2012 18:53:09 +0000 Anonymous 1767@http://www.datastax.com/support-forums/ Also according to this link, multi term queries are not supported: <a href="http://grokbase.com/t/lucene/solr-user/1168fc837m/does-multiterm-highlighting-work-with-the-fastvectorhighlighter" rel="nofollow">http://grokbase.com/t/lucene/solr-user/1168fc837m/does-multiterm-highlighting-work-with-the-fastvectorhighlighter</a> Anonymous on "Levers for increasing DSE 2 Solr performance?" http://www.datastax.com/support-forums/topic/levers-for-increasing-dse-2-solr-performance/page/2#post-1766 Wed, 25 Apr 2012 18:44:06 +0000 Anonymous 1766@http://www.datastax.com/support-forums/ Jeff, Solr / Lucene is very fast for searching. The DSE / Cassandra interaction is minimal and so is not a factor in the highlighting speed. Highlighting is expensive because the entire document must be re-analyzed. Fast vector highlighter avoids this by utilizing term vectors. However, turning on (and storing) term vectors will increase the index size a lot. Sounds like FVH may be the best option for your situation, lets try to get it working for you. Can you post the solrconfig.xml you are using? This thread seems to illustrate the performance issues you are seeing with highlighting: <a href="http://lucene.472066.n3.nabble.com/Difference-between-regular-Highlighter-and-Fast-Vector-Highlighter-td2763162.html" rel="nofollow">http://lucene.472066.n3.nabble.com/Difference-between-regular-Highlighter-and-Fast-Vector-Highlighter-td2763162.html</a>