DataStax Support Forums » Topic: Nodetool Effective-Ownership

DataStax Support Forums » Topic: Nodetool Effective-Ownership > 100% http://www.datastax.com/support-forums/topic/nodetool-effective-ownership-100 Software, Support, and Training for Apache Cassandra en-US Thu, 23 May 2013 00:48:20 +0000 http://bbpress.org/?v=1.0.3 <![CDATA[Search]]> q http://www.datastax.com/support-forums/search.php AndyBallingall on "Nodetool Effective-Ownership > 100%" http://www.datastax.com/support-forums/topic/nodetool-effective-ownership-100#post-5329 Mon, 06 Aug 2012 09:36:30 +0000 AndyBallingall 5329@http://www.datastax.com/support-forums/ Hi Throbs, I understand what happened now. Yes, I had pointed OpsCenter at the cluster, so that would explain the 50%. So everything was operating as intended all along, and it's nice to see it recovering from that hard reset. The 'nodetool ring' output appears to show 'Owns' as the percentage heading when you haven't put any data in. It's now reading 'Effective-Ownership' again. Thanks for your help. thobbs on "Nodetool Effective-Ownership > 100%" http://www.datastax.com/support-forums/topic/nodetool-effective-ownership-100#post-5046 Fri, 03 Aug 2012 22:13:01 +0000 thobbs 5046@http://www.datastax.com/support-forums/ "Effective Ownership" is referring to what percentage of the ring the node is a replica for for a given keyspace, because it depends on the replication strategy and strategy options. So, if your keyspace has a replication factor of 2 (which the OpsCenter keyspace does), each node of the four nodes is a replica for 50% of the ring. "Owns" just covers the "primary range" for a node, which is the range counter-clockwise from that node until the next node. I'm not sure why it switched back to showing this. Perhaps the OpsCenter keyspace is gone now? AndyBallingall on "Nodetool Effective-Ownership > 100%" http://www.datastax.com/support-forums/topic/nodetool-effective-ownership-100#post-4992 Fri, 03 Aug 2012 11:01:51 +0000 AndyBallingall 4992@http://www.datastax.com/support-forums/ In response to my own post, I decided to stop the cluster, delete all the data, and restart it. Now, when I run 'nodetool ring', it looks ok: Address DC Rack Status State Load Owns Token 127605887595351923798765477786913079296 192.9.200.157 datacenter1 rack1 Up Normal 11.13 KB 25.00% 0 192.9.200.6 datacenter1 rack1 Up Normal 15.55 KB 25.00% 42535295865117307932921825928971026432 192.9.200.115 datacenter1 rack1 Up Normal 15.52 KB 25.00% 85070591730234615865843651857942052864 192.9.200.130 datacenter1 rack1 Up Normal 15.55 KB 25.00% 127605887595351923798765477786913079296 The column with percentages has changed to saying 'Owns' again (like it did yesterday), and the total adds up to 100%. What does 'Effective-Ownership' mean, and what does it mean if it adds up to 200%? AndyBallingall on "Nodetool Effective-Ownership > 100%" http://www.datastax.com/support-forums/topic/nodetool-effective-ownership-100#post-4981 Fri, 03 Aug 2012 10:08:17 +0000 AndyBallingall 4981@http://www.datastax.com/support-forums/ I've been experimenting with a 4 node cluster running on my Windows PC. Yesterday, I'm pretty sure 'nodetool ring' reported that the ownership was split equally among the 4 nodes, 25% each. Last night, my machine rebooted after a system update and I had to restart the 4 node VMs from a hard reset. Once I got cassandra up and running again, the 'effective-ownership' reports that each node shares 50% - that's 200% in total! What's going on? I'm not sure nodetool said 'Effective-Ownership' yesterday... is this a clue that something's gone wrong? Here's the output of nodetool ring: [root@sweeney-delta ~]# nodetool ring Address DC Rack Status State Load Effective-Ownership Token 127605887595351923798765477786913079296 192.9.200.157 datacenter1 rack1 Up Normal 80.63 KB 50.00% 0 192.9.200.6 datacenter1 rack1 Up Normal 68.74 KB 50.00% 42535295865117307932921825928971026432 192.9.200.115 datacenter1 rack1 Up Normal 99.43 KB 50.00% 85070591730234615865843651857942052864 192.9.200.130 datacenter1 rack1 Up Normal 55.73 KB 50.00% 127605887595351923798765477786913079296 The initialisation log from /var/log/cassandra/cassandra.log for one of the nodes is below. (I can't see anything erroring there) Thanks in advance, Andy ---- INFO 09:56:37,170 Logging initialized INFO 09:56:37,178 JVM vendor/version: Java HotSpot(TM) 64-Bit Server VM/1.6.0_33 INFO 09:56:37,178 Heap size: 101711872/101711872 INFO 09:56:37,182 Classpath: /etc/cassandra/conf:/usr/share/java/jna.jar:/usr/share/cassandra/lib/antlr-3.2.jar:/usr/share/cassandra/lib/apache-cassandra-1.1.2.jar:/usr/share/cassandra/lib/apache-cassandra-clientutil-1.1.2.jar:/usr/share/cassandra/lib/apache-cassandra-thrift-1.1.2.jar:/usr/share/cassandra/lib/avro-1.4.0-fixes.jar:/usr/share/cassandra/lib/avro-1.4.0-sources-fixes.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang-2.4.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.3.jar:/usr/share/cassandra/lib/guava-r08.jar:/usr/share/cassandra/lib/high-scale-lib-1.1.2.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.2.5.jar:/usr/share/cassandra/lib/jline-0.9.94.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.7.0.jar:/usr/share/cassandra/lib/log4j-1.2.16.jar:/usr/share/cassandra/lib/metrics-core-2.0.3.jar:/usr/share/cassandra/lib/servlet-api-2.5-20081211.jar:/usr/share/cassandra/lib/slf4j-api-1.6.1.jar:/usr/share/cassandra/lib/slf4j-log4j12-1.6.1.jar:/usr/share/cassandra/lib/snakeyaml-1.6.jar:/usr/share/cassandra/lib/snappy-java-1.0.4.1.jar:/usr/share/cassandra/lib/snaptree-0.1.jar:/usr/share/cassandra/lib/stress.jar:/usr/share/cassandra//lib/jamm-0.2.5.jar INFO 09:56:37,709 JNA mlockall successful INFO 09:56:37,718 Loading settings from file:/etc/cassandra/default.conf/cassandra.yaml INFO 09:56:37,887 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap INFO 09:56:38,192 Global memtable threshold is enabled at 32MB INFO 09:56:38,604 Initializing key cache with capacity of 4 MBs. INFO 09:56:38,621 Scheduling key cache save to each 14400 seconds (going to save all keys). INFO 09:56:38,622 Initializing row cache with capacity of 0 MBs and provider org.apache.cassandra.cache.SerializingCacheProvider INFO 09:56:38,626 Scheduling row cache save to each 0 seconds (going to save all keys). INFO 09:56:38,784 Opening /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hd-15 (1409 bytes) INFO 09:56:38,801 Opening /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hd-14 (1325 bytes) INFO 09:56:38,802 Opening /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hd-13 (8140 bytes) INFO 09:56:38,814 reading saved cache /var/lib/cassandra/saved_caches/system-schema_columns-KeyCache INFO 09:56:38,822 Opening /var/lib/cassandra/data/system/schema_columns/system-schema_columns-hd-1 (1232 bytes) INFO 09:56:38,830 Opening /var/lib/cassandra/data/system/Versions/system-Versions-hd-2 (247 bytes) INFO 09:56:38,832 Opening /var/lib/cassandra/data/system/Versions/system-Versions-hd-1 (247 bytes) INFO 09:56:38,833 Opening /var/lib/cassandra/data/system/Versions/system-Versions-hd-3 (247 bytes) INFO 09:56:38,841 reading saved cache /var/lib/cassandra/saved_caches/system-schema_keyspaces-KeyCache INFO 09:56:38,843 Opening /var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-hd-10 (246 bytes) INFO 09:56:38,845 Opening /var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-hd-9 (246 bytes) INFO 09:56:38,855 reading saved cache /var/lib/cassandra/saved_caches/system-LocationInfo-KeyCache INFO 09:56:38,857 Opening /var/lib/cassandra/data/system/LocationInfo/system-LocationInfo-hd-17 (500 bytes) INFO 09:56:39,261 Opening /var/lib/cassandra/data/OpsCenter/events/OpsCenter-events-hd-1 (179 bytes) INFO 09:56:39,268 Opening /var/lib/cassandra/data/OpsCenter/events_timeline/OpsCenter-events_timeline-hd-1 (183 bytes) INFO 09:56:39,273 completed pre-loading (10 keys) key cache. INFO 09:56:39,389 Replaying /var/lib/cassandra/commitlog/CommitLog-179731308142.log, /var/lib/cassandra/commitlog/CommitLog-179887522269.log, /var/lib/cassandra/commitlog/CommitLog-179893848845.log, /var/lib/cassandra/commitlog/CommitLog-179890119025.log, /var/lib/cassandra/commitlog/CommitLog-179902583184.log, /var/lib/cassandra/commitlog/CommitLog-179892431840.log INFO 09:56:39,394 Replaying /var/lib/cassandra/commitlog/CommitLog-179731308142.log INFO 09:56:39,425 Finished reading /var/lib/cassandra/commitlog/CommitLog-179731308142.log INFO 09:56:39,425 Replaying /var/lib/cassandra/commitlog/CommitLog-179887522269.log INFO 09:56:39,426 Finished reading /var/lib/cassandra/commitlog/CommitLog-179887522269.log INFO 09:56:39,426 Replaying /var/lib/cassandra/commitlog/CommitLog-179893848845.log INFO 09:56:39,446 Finished reading /var/lib/cassandra/commitlog/CommitLog-179893848845.log INFO 09:56:39,446 Replaying /var/lib/cassandra/commitlog/CommitLog-179890119025.log INFO 09:56:39,447 Finished reading /var/lib/cassandra/commitlog/CommitLog-179890119025.log INFO 09:56:39,447 Replaying /var/lib/cassandra/commitlog/CommitLog-179902583184.log INFO 09:56:39,447 Finished reading /var/lib/cassandra/commitlog/CommitLog-179902583184.log INFO 09:56:39,448 Replaying /var/lib/cassandra/commitlog/CommitLog-179892431840.log INFO 09:56:39,448 Finished reading /var/lib/cassandra/commitlog/CommitLog-179892431840.log INFO 09:56:39,449 Log replay complete, 0 replayed mutations INFO 09:56:39,472 Cassandra version: 1.1.2 INFO 09:56:39,472 Thrift API version: 19.32.0 INFO 09:56:39,478 CQL supported versions: 2.0.0,3.0.0-beta1 (default: 2.0.0) INFO 09:56:39,529 Loading persisted ring state INFO 09:56:39,540 Starting up server gossip INFO 09:56:39,544 Enqueuing flush of Memtable-LocationInfo@1956297961(41/51 serialized/live bytes, 1 ops) INFO 09:56:39,544 Writing Memtable-LocationInfo@1956297961(41/51 serialized/live bytes, 1 ops) INFO 09:56:39,729 Completed flushing /var/lib/cassandra/data/system/LocationInfo/system-LocationInfo-hd-18-Data.db (80 bytes) for commitlog position ReplayPosition(segmentId=929465670550, position=363) INFO 09:56:39,778 Starting Messaging Service on port 7000 INFO 09:56:39,803 Using saved token 0 INFO 09:56:39,805 Enqueuing flush of Memtable-LocationInfo@1190265908(62/77 serialized/live bytes, 2 ops) INFO 09:56:39,805 Writing Memtable-LocationInfo@1190265908(62/77 serialized/live bytes, 2 ops) INFO 09:56:39,865 Completed flushing /var/lib/cassandra/data/system/LocationInfo/system-LocationInfo-hd-19-Data.db (148 bytes) for commitlog position ReplayPosition(segmentId=929465670550, position=529) INFO 09:56:39,868 Node sweeney-delta.london.thefoundry.co.uk/192.9.200.157 state jump to normal INFO 09:56:39,876 Bootstrap/Replace/Move completed! Now serving reads. INFO 09:56:39,877 Will not load MX4J, mx4j-tools.jar is not in the classpath