paulingalls Software, Support, and Training for Apache Cassandra en-US Wed, 19 Jun 2013 10:04:43 +0000 http://bbpress.org/?v=1.0.3 q http://www.datastax.com/support-forums/search.php http://www.datastax.com/support-forums/topic/cql3-select-fails-with-request-did-not-complete-within-rpc_timeout#post-8256 Thu, 03 Jan 2013 01:44:10 +0000 paulingalls 8256@http://www.datastax.com/support-forums/ <p>running<br /> nodetool invalidatekeycache<br /> and<br /> nodetool invalidaterowcache</p> <p>made the problem go away. I'm guessing there is a bug somewhere in the caching code... </p> http://www.datastax.com/support-forums/topic/cql3-select-fails-with-request-did-not-complete-within-rpc_timeout#post-8255 Thu, 03 Jan 2013 00:35:28 +0000 paulingalls 8255@http://www.datastax.com/support-forums/ <p>I have a pretty simple CQL3 table:</p> <p>CREATE TABLE user_team_scores (<br /> username text,<br /> team_slug text,<br /> score double,<br /> PRIMARY KEY (username, team_slug)<br /> ) WITH<br /> comment='' AND<br /> caching='KEYS_ONLY' AND<br /> read_repair_chance=0.100000 AND<br /> gc_grace_seconds=864000 AND<br /> replicate_on_write='true' AND<br /> compaction_strategy_class='SizeTieredCompactionStrategy' AND<br /> compression_parameters:sstable_compression='SnappyCompressor';</p> <p>I can run queries like:</p> <p>select * from user_team_scores where username='paulingalls';</p> <p>and it succeeds for some usernames, but fails for others with a timeout. </p> <p>I'm running a 3 node cluster, and on the node I assume has the token range for the username that fails I see the following stack trace in the logs:</p> <p>ERROR [ReadStage:66211] 2013-01-03 00:11:26,169 AbstractCassandraDaemon.java (line 135) Exception in thread Thread[ReadStage:66211,5,main]<br /> java.lang.AssertionError: DecoratedKey(82585475460624048733030438888619591812, 001373616e2d6672616e636973636f2d343965727300000573636f726500) != DecoratedKey(45868142482903708675972202481337602533, 7061756c696e67616c6c73) in /mnt/datadrive/lib/cassandra/fanzo/user_team_scores/fanzo-user_team_scores-hf-1-Data.db<br /> at org.apache.cassandra.db.columniterator.SSTableSliceIterator.&lt;init&gt;(SSTableSliceIterator.java:60)<br /> at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:67)<br /> at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:79)<br /> at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:256)<br /> at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:64)<br /> at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1345)<br /> at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1207)<br /> at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1142)<br /> at org.apache.cassandra.db.Table.getRow(Table.java:378)<br /> at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:69)<br /> at org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:51)<br /> at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:59)<br /> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)<br /> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)<br /> at java.lang.Thread.run(Thread.java:662)</p> <p>I'm wondering if there is a way to fix this, or if I'm hosed....</p> <p>Thanks!</p> <p>Paul </p>