Fail to Insert 1 Million Records

(2 posts) (2 voices)

• Started 10 months ago by Khan
• Latest reply from nickmbailey
• This topic is not a support question

Tags:

No tags yet.

1. 

   Khan
   Member

   I am getting an exception when I try to insert only 1 million records on a local machine. Here is the exception I am getting

   INFO [FlushWriter:14] 2012-07-07 12:04:18,536 Memtable.java (line 283) Completed flushing \var\lib\cassandra\data\lmeselect_hs\orders-hc-59-Data.db (8342500 bytes)
   INFO [CompactionExecutor:102] 2012-07-07 12:07:37,925 AutoSavingCache.java (line 269) Saved lmeselect_hs-orders-KeyCache (200000 items) in 687 ms
   INFO [CompactionExecutor:104] 2012-07-07 12:17:18,641 AutoSavingCache.java (line 269) Saved lmeselect_hs-orders-KeyCache (200000 items) in 344 ms
   INFO [CompactionExecutor:105] 2012-07-07 12:31:40,373 AutoSavingCache.java (line 269) Saved lmeselect_hs-orders-KeyCache (200000 items) in 407 ms
   INFO [ScheduledTasks:1] 2012-07-07 12:34:35,484 GCInspector.java (line 122) GC for ParNew: 9072 ms for 1 collections, 573919664 used; max is 1065025536
   INFO [ScheduledTasks:1] 2012-07-07 12:34:36,624 StatusLogger.java (line 50) Pool Name Active Pending Blocked
   INFO [ScheduledTasks:1] 2012-07-07 12:34:37,390 StatusLogger.java (line 65) ReadStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 12:34:37,390 StatusLogger.java (line 65) RequestResponseStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 12:34:37,421 StatusLogger.java (line 65) ReadRepairStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 12:34:37,452 StatusLogger.java (line 65) MutationStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 12:34:37,452 StatusLogger.java (line 65) ReplicateOnWriteStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 12:34:37,484 StatusLogger.java (line 65) GossipStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 12:34:37,484 StatusLogger.java (line 65) AntiEntropyStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 12:34:37,499 StatusLogger.java (line 65) MigrationStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 12:34:37,531 StatusLogger.java (line 65) StreamStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 12:34:37,531 StatusLogger.java (line 65) MemtablePostFlusher 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 12:34:37,546 StatusLogger.java (line 65) FlushWriter 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 12:34:37,562 StatusLogger.java (line 65) MiscStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 12:34:37,593 StatusLogger.java (line 65) InternalResponseStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 12:34:37,593 StatusLogger.java (line 65) HintedHandoff 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 12:34:37,702 StatusLogger.java (line 70) CompactionManager 0 0
   INFO [ScheduledTasks:1] 2012-07-07 12:34:37,734 StatusLogger.java (line 82) MessagingService n/a 0,0
   INFO [ScheduledTasks:1] 2012-07-07 12:34:37,749 StatusLogger.java (line 86) ColumnFamily Memtable ops,data Row cache size/cap Key cache size/cap
   INFO [ScheduledTasks:1] 2012-07-07 12:34:39,124 StatusLogger.java (line 89) system.NodeIdInfo 0,0 0/0 0/1
   INFO [ScheduledTasks:1] 2012-07-07 12:34:39,187 StatusLogger.java (line 89) system.IndexInfo 0,0 0/0 2/2
   INFO [ScheduledTasks:1] 2012-07-07 12:34:39,233 StatusLogger.java (line 89) system.LocationInfo 0,0 0/0 2/2
   INFO [ScheduledTasks:1] 2012-07-07 12:34:39,280 StatusLogger.java (line 89) system.Versions 3,103 0/0 0/1
   INFO [ScheduledTasks:1] 2012-07-07 12:34:39,358 StatusLogger.java (line 89) system.Migrations 0,0 0/0 0/2
   INFO [ScheduledTasks:1] 2012-07-07 12:34:39,390 StatusLogger.java (line 89) system.HintsColumnFamily 0,0 0/0 0/1
   INFO [ScheduledTasks:1] 2012-07-07 12:34:39,421 StatusLogger.java (line 89) system.Schema 0,0 0/0 0/2
   INFO [ScheduledTasks:1] 2012-07-07 12:34:39,437 StatusLogger.java (line 89) lmeselect_hs.orders 4500,1668525 0/0 2/200000
   INFO [ScheduledTasks:1] 2012-07-07 13:13:52,389 GCInspector.java (line 122) GC for ParNew: 11912 ms for 1 collections, 580257176 used; max is 1065025536
   INFO [ScheduledTasks:1] 2012-07-07 13:13:53,357 StatusLogger.java (line 50) Pool Name Active Pending Blocked
   INFO [ScheduledTasks:1] 2012-07-07 13:13:53,607 StatusLogger.java (line 65) ReadStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:13:53,967 StatusLogger.java (line 65) RequestResponseStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:13:53,982 StatusLogger.java (line 65) ReadRepairStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:13:53,998 StatusLogger.java (line 65) MutationStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:13:54,013 StatusLogger.java (line 65) ReplicateOnWriteStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:13:54,045 StatusLogger.java (line 65) GossipStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:13:54,045 StatusLogger.java (line 65) AntiEntropyStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:13:54,060 StatusLogger.java (line 65) MigrationStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:13:54,310 StatusLogger.java (line 65) StreamStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:13:54,310 StatusLogger.java (line 65) MemtablePostFlusher 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:13:54,326 StatusLogger.java (line 65) FlushWriter 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:13:54,326 StatusLogger.java (line 65) MiscStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:13:54,326 StatusLogger.java (line 65) InternalResponseStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:13:54,326 StatusLogger.java (line 65) HintedHandoff 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:13:54,388 StatusLogger.java (line 70) CompactionManager 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:13:54,404 StatusLogger.java (line 82) MessagingService n/a 0,0
   INFO [ScheduledTasks:1] 2012-07-07 13:13:54,420 StatusLogger.java (line 86) ColumnFamily Memtable ops,data Row cache size/cap Key cache size/cap
   INFO [ScheduledTasks:1] 2012-07-07 13:13:54,482 StatusLogger.java (line 89) system.NodeIdInfo 0,0 0/0 0/1
   INFO [ScheduledTasks:1] 2012-07-07 13:13:54,498 StatusLogger.java (line 89) system.IndexInfo 0,0 0/0 2/2
   INFO [ScheduledTasks:1] 2012-07-07 13:13:54,513 StatusLogger.java (line 89) system.LocationInfo 0,0 0/0 2/2
   INFO [ScheduledTasks:1] 2012-07-07 13:13:54,529 StatusLogger.java (line 89) system.Versions 3,103 0/0 0/1
   INFO [ScheduledTasks:1] 2012-07-07 13:13:54,560 StatusLogger.java (line 89) system.Migrations 0,0 0/0 0/2
   INFO [ScheduledTasks:1] 2012-07-07 13:13:54,576 StatusLogger.java (line 89) system.HintsColumnFamily 0,0 0/0 0/1
   INFO [ScheduledTasks:1] 2012-07-07 13:13:54,591 StatusLogger.java (line 89) system.Schema 0,0 0/0 0/2
   INFO [ScheduledTasks:1] 2012-07-07 13:13:54,591 StatusLogger.java (line 89) lmeselect_hs.orders 4500,1668525 0/0 2/200000
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,159 GCInspector.java (line 122) GC for ParNew: 1471 ms for 1 collections, 581185064 used; max is 1065025536
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,159 StatusLogger.java (line 50) Pool Name Active Pending Blocked
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 65) ReadStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 65) RequestResponseStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 65) ReadRepairStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 65) MutationStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 65) ReplicateOnWriteStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 65) GossipStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 65) AntiEntropyStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 65) MigrationStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 65) StreamStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 65) MemtablePostFlusher 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 65) FlushWriter 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 65) MiscStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 65) InternalResponseStage 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 65) HintedHandoff 0 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 70) CompactionManager 0 0
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 82) MessagingService n/a 0,0
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 86) ColumnFamily Memtable ops,data Row cache size/cap Key cache size/cap
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 89) system.NodeIdInfo 0,0 0/0 0/1
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 89) system.IndexInfo 0,0 0/0 2/2
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 89) system.LocationInfo 0,0 0/0 2/2
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 89) system.Versions 3,103 0/0 0/1
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 89) system.Migrations 0,0 0/0 0/2
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 89) system.HintsColumnFamily 0,0 0/0 0/1
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 89) system.Schema 0,0 0/0 0/2
   INFO [ScheduledTasks:1] 2012-07-07 13:54:05,175 StatusLogger.java (line 89) lmeselect_hs.orders 4500,1668525 0/0 2/200000

   INFO [CompactionExecutor:106] 2012-07-07 15:01:13,030 AutoSavingCache.java (line 269) Saved lmeselect_hs-orders-KeyCache (21837 items) in 250 ms
   INFO [CompactionExecutor:107] 2012-07-07 15:24:36,594 AutoSavingCache.java (line 269) Saved lmeselect_hs-orders-KeyCache (2 items) in 0 ms

   I am still not able to figure it out what is the problem. Any help would be appreciated.

   Posted 10 months ago #
2. 

   nickmbailey
   Key Master

   There are no exceptions in that log. That log is just giving stats about memory usage due to gc/heap pressure. It sounds like you are are overloading your node.

   You could try increasing the heap size. Have you changed it from the default setting? It is set in cassandra-env.sh.

   Posted 10 months ago #

RSS feed for this topic

Reply

You must log in to post.