http://www.datastax.com/support-forums/topic/major-compaction-yesno Software, Support, and Training for Apache Cassandra en-US Sat, 18 May 2013 10:21:34 +0000 http://bbpress.org/?v=1.0.3 q http://www.datastax.com/support-forums/search.php http://www.datastax.com/support-forums/topic/major-compaction-yesno#post-1276 Fri, 02 Mar 2012 16:58:16 +0000 jbellis 1276@http://www.datastax.com/support-forums/ <p>Reduce the compaction throughput setting. </p> http://www.datastax.com/support-forums/topic/major-compaction-yesno#post-1274 Fri, 02 Mar 2012 08:03:35 +0000 Anonymous 1274@http://www.datastax.com/support-forums/ <p>Thank you both for your answers, much appreciated!</p> <p>I don't think leveled compaction is an improvement for us. we do mostly writes and are not concerned with duplicate data. Our biggest worry at the moment are minor compactions on CF's that are quite big (100GB). they make the node response very slow to our reads.</p> <p>Any tips on solving slow reads if a node is doing (minor) compaction on a big CF ?</p> <p>thanks a bunch.<br /> Fransen </p> http://www.datastax.com/support-forums/topic/major-compaction-yesno#post-1256 Wed, 29 Feb 2012 04:05:09 +0000 jbellis 1256@http://www.datastax.com/support-forums/ <p>It isn't that minor compactions are run less frequently because of the major compaction per se, but because major compaction creates One Big Data File by definition, it will be some time before new data files get large enough to be eligible to be compacted with it. Newly flushed data files will thus continue to be compacted with each other normally.</p> <p>If you're finding that normal minor compactions aren't doing a good enough job of merging together overwritten row versions, you should give the leveled compaction strategy a try. I wrote more about compaction in general and leveled in particular here: <a href="http://www.datastax.com/dev/blog/leveled-compaction-in-apache-cassandra" rel="nofollow">http://www.datastax.com/dev/blog/leveled-compaction-in-apache-cassandra</a> </p> http://www.datastax.com/support-forums/topic/major-compaction-yesno#post-1251 Wed, 29 Feb 2012 01:56:25 +0000 joaquin 1251@http://www.datastax.com/support-forums/ <p>Not a problem Fransen.</p> <p>More info on compaction can be found here:<br /> <a href="http://www.datastax.com/docs/1.0/dml/about_writes#about-compaction" rel="nofollow">http://www.datastax.com/docs/1.0/dml/about_writes#about-compaction</a></p> <p>Without major compactions, that big sstable won't go through any of those changes.</p> <p>If you won't be reading much from that keyspace and are noticing minor compaction still occurring then you should relatively not have to worry.</p> <p>If you wish to empty out the entire data directory and nuke all the previous information, this is fine too as long as you understand you lose that data. If that's the case, then the record of major compactions will be ultimately deleted.</p> <p>Thanks,<br /> Joaquin </p> http://www.datastax.com/support-forums/topic/major-compaction-yesno#post-1250 Tue, 28 Feb 2012 23:20:35 +0000 Anonymous 1250@http://www.datastax.com/support-forums/ <p>thanks again Joaquin !</p> <p>hmm thats disappointing... so even a complete emptying of the data directory will not work?</p> <p>What will happen if we don't do those major compactions?<br /> In our setup, i can see that the node that had one major compaction in the past it is doing normal compactions sometimes... </p> http://www.datastax.com/support-forums/topic/major-compaction-yesno#post-1249 Tue, 28 Feb 2012 21:49:52 +0000 joaquin 1249@http://www.datastax.com/support-forums/ <p>No, I'm sorry, once a major compaction is run, you have to stick with performing them. Typically once per gc_grace_period.</p> <p>You should probably look into leveled compaction if you are on 1.0, this should help relieve some of the stress and process of having to run major compactions.</p> <p><a href="http://www.datastax.com/dev/blog/leveled-compaction-in-apache-cassandra" rel="nofollow">http://www.datastax.com/dev/blog/leveled-compaction-in-apache-cassandra</a></p> <p>Thanks,<br /> Joaquin </p> http://www.datastax.com/support-forums/topic/major-compaction-yesno#post-1248 Tue, 28 Feb 2012 20:22:41 +0000 Anonymous 1248@http://www.datastax.com/support-forums/ <p>"Also, once you run a major compaction, automatic minor compactions are no longer triggered frequently forcing you to manually run major compactions on a routine basis."</p> <p>does anybody know how to undo this ? </p> http://www.datastax.com/support-forums/topic/major-compaction-yesno#post-1179 Mon, 20 Feb 2012 00:04:55 +0000 Anonymous 1179@http://www.datastax.com/support-forums/ <p>Thanks Joaquin,</p> <p>just a follow up:<br /> We did do one major compaction on one of our nodes in the past, would we be able to so to speak 'reset' it? perhaps by doing a full data reload on that node? </p> http://www.datastax.com/support-forums/topic/major-compaction-yesno#post-1174 Fri, 17 Feb 2012 17:51:54 +0000 joaquin 1174@http://www.datastax.com/support-forums/ <p>Hello Walt,</p> <p>Most times those who run major compactions are doing so since they had already run the first major compaction and need to continue doing so.</p> <p>If you have something like a queue application, which generates many tombstones, with SSDs, then running major compaction isn't much of an issue and thus recommended.</p> <p>If your workload generates many tombstones, then major compaction may be an option, provided you can handle the IO. But in most cases you're better off without major compactions and instead properly tuning your compaction:<br /> <a href="http://www.datastax.com/docs/1.0/operations/tuning#tuning-compaction" rel="nofollow">http://www.datastax.com/docs/1.0/operations/tuning#tuning-compaction</a></p> <p>Thanks,<br /> Joaquin </p> http://www.datastax.com/support-forums/topic/major-compaction-yesno#post-1173 Fri, 17 Feb 2012 09:10:26 +0000 Anonymous 1173@http://www.datastax.com/support-forums/ <p>Hi,</p> <p>On this documentation page from Datastax (<a href="http://www.datastax.com/docs/1.0/operations/tuning" rel="nofollow">http://www.datastax.com/docs/1.0/operations/tuning</a>) they suggests to NOT run major compaction (all the way at the bottom), however elsewhere on the net i read that people still do major compaction.</p> <p>Perhaps somebody can explain when to not run it and when you should?</p> <p>thanks! </p>