http://www.datastax.com/support-forums/topic/exception-when-compacting-running-compaction-task Software, Support, and Training for Apache Cassandra en-US Thu, 20 Jun 2013 06:49:46 +0000 http://bbpress.org/?v=1.0.3 q http://www.datastax.com/support-forums/search.php http://www.datastax.com/support-forums/topic/exception-when-compacting-running-compaction-task#post-8900 Mon, 04 Feb 2013 13:33:44 +0000 jamesr 8900@http://www.datastax.com/support-forums/ <p>Hi,</p> <p>we have (currently) a system where we are running Cassandra on two machines on a live setup. Every night the system takes automatic back-ups of the databases (Cassandra files) and then copies them to another set of ("development system") machines, which are running on identical set-up. Then the system stops the Cassandras running on dev system machines, copies the backups to said machines and starts the Cassandras again. </p> <p>This has worked very well for a quite long time, but now we consistently see EOFException during the Cassandra start-up on the dev system machine. This happens every night when Cassandras are started. I'll paste the full exception log below. Has anyone encountered similar behavior / can we do something to fix this?</p> <p>-J</p> <p>ERROR [CompactionExecutor:4] 2013-02-04 01:11:20,442 AbstractCassandraDaemon.java (line 135) Exception in thread Thread[CompactionExecutor:4,1,main]java.io.IOError: java.io.EOFException<br /> at org.apache.cassandra.db.compaction.PrecompactedRow.merge(PrecompactedRow.java:116) at org.apache.cassandra.db.compaction.PrecompactedRow.&lt;init&gt;(PrecompactedRow.java:99)<br /> at org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:176) at org.apache.cassandra.db.compaction.CompactionIterable$Reducer.getReduced(CompactionIterable.java:83)<br /> at org.apache.cassandra.db.compaction.CompactionIterable$Reducer.getReduced(CompactionIterable.java:68)<br /> at org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:118)<br /> at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:101)<br /> at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)<br /> at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135) at com.google.common.collect.Iterators$7.computeNext(Iterators.java:614)<br /> at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)<br /> at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)<br /> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:137)<br /> at org.apache.cassandra.db.compaction.CompactionManager$2.runMayThrow(CompactionManager.java:164) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)<br /> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)<br /> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)<br /> at java.util.concurrent.FutureTask.run(FutureTask.java:138)<br /> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)<br /> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662)<br /> Caused by: java.io.EOFException at java.io.RandomAccessFile.readFully(RandomAccessFile.java:399) at java.io.RandomAccessFile.readFully(RandomAccessFile.java:377)<br /> at org.apache.cassandra.utils.BytesReadTracker.readFully(BytesReadTracker.java:95) at org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:401)<br /> at org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:363)<br /> at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:120)<br /> at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:37)<br /> at org.apache.cassandra.db.ColumnFamilySerializer.deserializeColumns(ColumnFamilySerializer.java:144) at org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:234)<br /> at org.apache.cassandra.db.compaction.PrecompactedRow.merge(PrecompactedRow.java:112)<br /> ... 20 more </p>