http://www.datastax.com/support-forums/tags/opscenter-agent Software, Support, and Training for Apache Cassandra en-US Thu, 23 May 2013 12:50:54 +0000 http://bbpress.org/?v=1.0.3 q http://www.datastax.com/support-forums/search.php http://www.datastax.com/support-forums/topic/trouble-running-opscenter-agent-manually-please-help-1#post-1808 Tue, 01 May 2012 11:11:34 +0000 Anonymous 1808@http://www.datastax.com/support-forums/ <p>Problem Solved.........i was using an old version of cassandra..........there are some additions in JMX paramters in new version like AllDataLocations etc.........those were missing in my version.</p> <p>Thanks thobbs for your answer...now i have upgraded to latest version cassandra.........and it is working fine.........i read your answer today..........but i had figured it out day before yesterday...........i had to decompile opscenter-agent code...:).....but it was really hard after decompilation......can't we have open source code avaiable on the website for the community edition of opscenter......that will help. </p> http://www.datastax.com/support-forums/topic/trouble-running-opscenter-agent-manually-please-help-1#post-1801 Mon, 30 Apr 2012 15:06:02 +0000 thobbs 1801@http://www.datastax.com/support-forums/ <p>What version of OpsCenter and Cassandra are you running? If you connect to Cassandra with jconsole and go to MBeans -&gt; org.apache.cassandra.db -&gt; StorageService -&gt; Attributes, do you see fields named "AllDataFileLocations", "CommitLogLocation", and "SavedCachesLocation"? If so, what are their values? (You may need to double click the AllDataFileLocations value.) </p> http://www.datastax.com/support-forums/topic/trouble-running-opscenter-agent-manually-please-help-1#post-1793 Sat, 28 Apr 2012 16:48:16 +0000 Anonymous 1793@http://www.datastax.com/support-forums/ <p>I have manually installed agent through Tarball Installations instructions but i am having trouble running agent in foreground as well as background....i am not able to get Disk IO Metrics........everything else is working fine.</p> <p>ERROR [pool-4-thread-3] 2012-04-28 12:15:01,692 Error getting devices and partitions<br /> java.io.IOException: Process failed: df<br /> Exit val: 1<br /> Output:<br /> df: `': No such file or directory<br /> at opsagent.proc$handle_proc_results.invoke(proc.clj:68)<br /> at opsagent.proc$run_proc.doInvoke(proc.clj:92)<br /> at clojure.lang.RestFn.invoke(RestFn.java:410)<br /> at opsagent.os.linux_disks$get_partition.invoke(linux_disks.clj:124)<br /> at clojure.lang.Var.invoke(Var.java:365)<br /> at clojure.lang.AFn.applyToHelper(AFn.java:161)<br /> at clojure.lang.Var.applyTo(Var.java:482)<br /> at clojure.core$apply.invoke(core.clj:540)<br /> at opsagent.os.disks$os_specific.doInvoke(disks.clj:12)<br /> at clojure.lang.RestFn.invoke(RestFn.java:423)<br /> at opsagent.os.disks$get_partition.invoke(disks.clj:24)<br /> at opsagent.nodedetails$get_disk_info.invoke(nodedetails.clj:77)<br /> at opsagent.nodedetails$run_longtime_collection$fn__2803.invoke(nodedetails.clj:167)<br /> at opsagent.jmx$jmx_wrap.doInvoke(jmx.clj:100)<br /> at clojure.lang.RestFn.invoke(RestFn.java:410)<br /> at opsagent.nodedetails$run_longtime_collection.invoke(nodedetails.clj:159)<br /> at opsagent.nodedetails$start_pool$fn__2829$fn__2830.invoke(nodedetails.clj:210)<br /> at opsagent.util$error_wrap.doInvoke(util.clj:114)<br /> at clojure.lang.RestFn.invoke(RestFn.java:423)<br /> at opsagent.nodedetails$start_pool$fn__2829.invoke(nodedetails.clj:209)<br /> at clojure.lang.AFn.run(AFn.java:24)<br /> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)<br /> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)<br /> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)<br /> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)<br /> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)<br /> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)<br /> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)<br /> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)<br /> at java.lang.Thread.run(Thread.java:619)</p> <p>................And................</p> <p>ERROR [pool-3-thread-7] 2012-04-28 12:16:40,885 Long os-stats collector failed: Process failed: iostat -x -d -m 60 2<br /> Exit val: 130<br /> Output:<br /> Linux 2.6.21.7-2.fc8xen (xxxxxxxxxxxxxxx) 04/28/12</p> <p> Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await svctm %util<br /> sda1 0.13 24.66 0.83 3.80 0.02 0.11 58.17 0.04 8.27 1.26 0.58<br /> sdb 38.54 191.82 307.75 20.05 7.33 0.83 50.96 11.13 33.97 2.53 82.78<br /> sdc 0.00 0.00 0.00 0.00 0.00 0.00 23.52 0.00 1.03 1.00 0.00</p> <p>ERROR [pool-3-thread-3] 2012-04-28 12:16:40,885 Long os-stats collector failed: Process failed: iostat -c -m 60 2<br /> Exit val: 130<br /> Output:<br /> Linux 2.6.21.7-2.fc8xen (xxxxxxxxxxxxxx) 04/28/12</p> <p> avg-cpu: %user %nice %system %iowait %steal %idle<br /> 5.30 0.27 1.74 41.47 1.48 49.74</p> <p>**********************************************</p> <p>command df and iostat are working fine when are executed externally.</p> <p>I am running this on Amazon EC2 instance on CENTOS. </p> http://www.datastax.com/support-forums/topic/trouble-connecting-to-agent#post-1679 Wed, 18 Apr 2012 15:45:00 +0000 nickmbailey 1679@http://www.datastax.com/support-forums/ <p>The bug involved here has been fixed in the 2.0 release of OpsCenter which just came out. The workarounds mentioned above will still work fine, but shouldn't be necessary from now on.</p> <p>Let us know if you have any issues. </p> http://www.datastax.com/support-forums/topic/trouble-connecting-to-agent#post-1669 Tue, 17 Apr 2012 22:06:07 +0000 conzar 1669@http://www.datastax.com/support-forums/ <p>@jas, you need to set incoming-interface = &lt;ip address&gt; where ip address is the ip of the interface that opscenterd is running on and that the agents connect to. 0.0.0.0 isn't a valid address.</p> <p>For instance, the interface that the agents connect to for me looks like this:<br /> /sbin/ifconfig<br /> eth0 Link encap:Ethernet HWaddr **:**:**:**:**:**<br /> inet addr:10.1.1.1 Bcast:10.1.255.255 Mask:255.255.0.0</p> <p>The configuration for opscenterd.conf:<br /> [agents]<br /> use_ssl = false<br /> incoming_interface = 10.1.1.1</p> <p>Also, is there a patch to fix this problem in the code yet? </p> http://www.datastax.com/support-forums/topic/trouble-connecting-to-agent#post-1625 Sun, 15 Apr 2012 19:26:56 +0000 jas 1625@http://www.datastax.com/support-forums/ <p>thobbs,</p> <p>I thank you as well. I was having this same problem just now on some EC2 nodes I setup up and the agents just kept getting the connection refused to the opscenter (1.4.1) system. I was playing around with the AWS security groups etc., not able to figure it out. Then I did netstat -n and noticed the localhost issue. I copied interface = 0.0.0.0 from [webserver] to agents [agents] and that did not work.</p> <p>Anyway, your solution does work, and is showing up from my two DSE 2 nodes. :) </p> http://www.datastax.com/support-forums/topic/trouble-connecting-to-agent#post-1332 Wed, 07 Mar 2012 21:05:12 +0000 kireet 1332@http://www.datastax.com/support-forums/ <p>cool, i will try it out and let you know if it works. </p> http://www.datastax.com/support-forums/topic/trouble-connecting-to-agent#post-1308 Tue, 06 Mar 2012 18:17:48 +0000 thobbs 1308@http://www.datastax.com/support-forums/ <p>Thanks, that confirms what I thought was happening. Looks like there's a bug where opscenterd binds to 127.0.0.1 if ssl is disabled and the default of 0.0.0.0 isn't overridden.</p> <p>We should be able to fix this pretty easily in the next release. </p> http://www.datastax.com/support-forums/topic/trouble-connecting-to-agent#post-1301 Tue, 06 Mar 2012 04:34:25 +0000 kireet 1301@http://www.datastax.com/support-forums/ <p>yeah, i disabled ssl. i was debugging another issue and i was turning things off to see if it helped... </p> http://www.datastax.com/support-forums/topic/trouble-connecting-to-agent#post-1292 Mon, 05 Mar 2012 21:43:26 +0000 thobbs 1292@http://www.datastax.com/support-forums/ <p>Cool, glad that worked.</p> <p>Did you happen to disable SSL for communication with the agents, by chance? Did you change any other default config options? </p> http://www.datastax.com/support-forums/topic/trouble-connecting-to-agent#post-1285 Sun, 04 Mar 2012 04:15:36 +0000 kireet 1285@http://www.datastax.com/support-forums/ <p>yep that worked:</p> <p># netstat -tulpn | grep 616<br /> tcp 0 0 127.0.0.1:61619 0.0.0.0:* LISTEN 30930/python2.6<br /> tcp 0 0 66.160.192.50:61620 0.0.0.0:* LISTEN 30930/python2.6<br /> tcp 0 0 ::ffff:66.160.192.50:61621 :::* LISTEN 31064/java </p> <p>and i see in the agent log that communication was established. </p> http://www.datastax.com/support-forums/topic/trouble-connecting-to-agent#post-1284 Sat, 03 Mar 2012 19:38:42 +0000 thobbs 1284@http://www.datastax.com/support-forums/ <p>The opscenterd daemon tries to bind to 0.0.0.0:61620 by default to listen for agent communication. It seems odd that it's apparently only listening on 127.0.0.1.</p> <p>You can try overriding the default by adding an <code>[agents]</code> section to opscenterd.conf:</p> <p><code>[agents]</code><br /> <code>incoming_interface = 0.0.0.0</code></p> <p>and changing <code>0.0.0.0</code> to the correct IP address of the machine.</p> <p>If that works, let me know and we can perhaps try to figure out why it wasn't binding to the correct interface(s) in the first place. </p> http://www.datastax.com/support-forums/topic/trouble-connecting-to-agent#post-1279 Fri, 02 Mar 2012 20:27:57 +0000 kireet 1279@http://www.datastax.com/support-forums/ <p>I am trying to get datastax community (cassandra+opscenter) running all on one linux(centos) machine for testing purposes. I have everything installed and cassandra is running fine. Opscenter is communicating with cassandra fine, but it's having trouble with the agent. After debugging, I found this:</p> <p>netstat -tulpn | less<br /> ~# netstat -tulpn | grep 616<br /> tcp 0 0 127.0.0.1:61619 0.0.0.0:* LISTEN 22068/python2.6<br /> tcp 0 0 127.0.0.1:61620 0.0.0.0:* LISTEN 22068/python2.6<br /> ~# netstat -tulpn | grep 9160<br /> tcp 0 0 66.160.192.50:9160 0.0.0.0:* LISTEN 19926/java<br /> ~# netstat -tulpn | grep 7199<br /> tcp 0 0 0.0.0.0:7199 0.0.0.0:* LISTEN 19926/java </p> <p>I think the issue is cassandra is running with the public IP but somehow opscenter is opening ports using the loopback address. results from hostname -f and hostname -i look correct. How does opscenter pick an interface when it opens the agent ports? </p> http://www.datastax.com/support-forums/topic/error-installing-opscenter-agent#post-590 Thu, 27 Oct 2011 19:40:57 +0000 thobbs 590@http://www.datastax.com/support-forums/ <p>Well, I'm glad the problem resolved itself. That's pretty interesting.</p> <p>Perhaps you later changed something in the config and forgot to restart opscenter after that? </p> http://www.datastax.com/support-forums/topic/error-installing-opscenter-agent#post-589 Thu, 27 Oct 2011 19:37:28 +0000 conzar 589@http://www.datastax.com/support-forums/ <p>Thank you for your help thobbs. I restarted both Cassandra and opscenterd (with DEBUG enabled). After that, the installer worked! Maybe there was a problem with how I initially installed opscenter?</p> <p>Order of events<br /> * started Cassandra (from tarball)<br /> * intalled opscenter (through apt-get on Ubuntu)<br /> * configured and launched opscenter</p> <p>Thanks </p>