Support Forums : DataStax

DataStax Support Forums » DataStax Community

Opscenter not working on Multi-Node Cassandra and Hadoop Cluster on Linux

(8 posts) (4 voices)

Started 7 months ago by Anderverno
Latest reply from Anderverno
This topic is resolved

Tags:

No tags yet.

Anderverno
Member

Good morning.

I am testing Cassandra cluster configuration using three Oracle Enterprise Linux 6.3 32 bit virtual machines following steps provided in this tutorial: http://www.datastax.com/resources/articles/multi-node-cassandra-and-hadoop-cluster-on-linux

I´m currently trying to setup opscenter. This is what is happening. I start opscenter service from one of the nodes. Then I run the opscenter-agent script from one of the other nodes but it continuously prints this message to the agent.log file=

INFO [main] 2012-10-20 10:09:29,335 Loading conf files: ./conf/address.yaml
INFO [main] 2012-10-20 10:09:29,593 Java vendor/version: Java HotSpot(TM) Client VM/1.6.0_31
INFO [main] 2012-10-20 10:09:29,598 Waiting for the config from OpsCenter
INFO [main] 2012-10-20 10:09:29,604 SSL communication is enabled
INFO [main] 2012-10-20 10:09:29,615 Creating stomp connection to 192.168.1.3:61620
INFO [main] 2012-10-20 10:09:29,686 Attempting to determine Cassandra's broadcast address through JMX
INFO [StompConnection receiver] 2012-10-20 10:09:29,693 Reconnecting in 0s.
INFO [Initialization] 2012-10-20 10:09:29,797 Sleeping for 2s before trying to determine IP over JMX again
WARN [pool-2-thread-1] 2012-10-20 10:09:29,863 Tried to send message while not connected: /conf-request [null,["2.10","\/1898403275\/conf",["fe80:0:0:0:a00:27ff:fe0d:e126%2","0:0:0:0:0:0:0:1%1","192.168.1.4","127.0.0.1"]]]
INFO [Initialization] 2012-10-20 10:09:31,811 Sleeping for 4s before trying to determine IP over JMX again
INFO [StompConnection receiver] 2012-10-20 10:09:32,281 Connected to 192.168.1.3:61620
INFO [Initialization] 2012-10-20 10:09:35,823 Sleeping for 8s before trying to determine IP over JMX again
INFO [Initialization] 2012-10-20 10:09:43,863 Sleeping for 16s before trying to determine IP over JMX again
INFO [Initialization] 2012-10-20 10:09:59,931 Sleeping for 20s before trying to determine IP over JMX again
INFO [Initialization] 2012-10-20 10:10:19,957 Sleeping for 20s before trying to determine IP over JMX again
INFO [Initialization] 2012-10-20 10:10:40,009 Sleeping for 20s before trying to determine IP over JMX again
INFO [Initialization] 2012-10-20 10:11:00,052 Sleeping for 20s before trying to determine IP over JMX again
INFO [Initialization] 2012-10-20 10:11:20,088 Sleeping for 20s before trying to determine IP over JMX again
INFO [Initialization] 2012-10-20 10:11:40,500 Sleeping for 20s before trying to determine IP over JMX again
INFO [Initialization] 2012-10-20 10:12:00,658 Sleeping for 20s before trying to determine IP over JMX again
INFO [Initialization] 2012-10-20 10:12:20,873 Sleeping for 20s before trying to determine IP over JMX again
INFO [Initialization] 2012-10-20 10:12:40,950 Sleeping for 20s before trying to determine IP over JMX again

I was checking the forum for a similar issue and found that the it could be related to SSL. So I disabled SSL and this is the output on agent.log:

INFO [main] 2012-10-20 10:34:36,139 Loading conf files: ./conf/address.yaml
INFO [main] 2012-10-20 10:34:36,324 Java vendor/version: Java HotSpot(TM) Client VM/1.6.0_31
INFO [main] 2012-10-20 10:34:36,328 Waiting for the config from OpsCenter
INFO [main] 2012-10-20 10:34:36,343 SSL communication is disabled
INFO [main] 2012-10-20 10:34:36,346 Creating stomp connection to 192.168.1.3:61620
INFO [main] 2012-10-20 10:34:36,395 Attempting to determine Cassandra's broadcast address through JMX
INFO [StompConnection receiver] 2012-10-20 10:34:36,411 Reconnecting in 0s.
INFO [StompConnection receiver] 2012-10-20 10:34:36,535 Connected to 192.168.1.3:61620
INFO [Initialization] 2012-10-20 10:34:36,545 Sleeping for 2s before trying to determine IP over JMX again
INFO [Initialization] 2012-10-20 10:34:38,565 Sleeping for 4s before trying to determine IP over JMX again
INFO [Initialization] 2012-10-20 10:34:42,585 Sleeping for 8s before trying to determine IP over JMX again
INFO [Initialization] 2012-10-20 10:34:50,596 Sleeping for 16s before trying to determine IP over JMX again
INFO [Initialization] 2012-10-20 10:35:06,616 Sleeping for 20s before trying to determine IP over JMX again

So is still not working but I have no idea about the cause of this issue.

Details:
Cassandra Version: dsc-cassandra-1.1.5-bin.tar.gz
Opscenter Version: opscenter-2.1.2-free.tar.gz

Thanks for your assistance.

Posted 7 months ago #
nickmbailey
Key Master

That indicates that the agent is attempting to connect to cassandra over JMX and failing. Is cassandra already running on the machine the agent is running on?

Posted 6 months ago #
Anderverno
Member

Yes cassandra is running on the three servers before I start running opscenter.

Thanks.

Posted 6 months ago #
nickmbailey
Key Master

Can you change the log level for the agent to DEBUG in ./conf/log4j.properties?

Posted 6 months ago #
swoo
Member

I also have a 3 node cassandra and I'm trying to get opscenter working.
I'm getting a similar error:

I've tried to confirm if I have the right ports open:

root@cas00 opscenter]# service iptables status
iptables: Firewall is not running.
[root@cas00 opscenter]# netstat -al | grep 6162
tcp 0 0 *:61620 *:* LISTEN
tcp 0 0 localhost:61620 10.xxx.xx.141:54176 ESTABLISHED
tcp 0 0 localhost:61620 10.xxx.xx.140:38081 ESTABLISHED
tcp 0 0 localhost:61620 10.xxx.xx.142:44873 ESTABLISHED

[root@cas01 opscenter-agent]# service iptables status
iptables: Firewall is not running.
[root@cas01 opscenter-agent]# netstat -al | grep 6162
tcp 0 0 localhost:38081 ::ffff:10.xxx.xx.139:61620 ESTABLISHED

[root@cas02 opscenter-agent]# service iptables status
iptables: Firewall is not running.
[root@cas02 opscenter-agent]# netstat -al | grep 6162
tcp 0 0 localhost:54176 ::ffff:10.xxx.xx.139:61620 ESTABLISHED

[root@cas03 /]# service iptables status
iptables: Firewall is not running.
[root@cas03 /]# netstat -al | grep 6162
tcp 0 0 localhost:44873 ::ffff:10.xxx.xx.139:61620 ESTABLISHED

Posted 6 months ago #
mbulman
Key Master

swoo,

What is the exact error you're getting? Do you have SSL enabled or disabled?

Posted 6 months ago #
swoo
Member

Yes, it turned out to be my SSL settings. Disabling for all nodes made it work.

Posted 6 months ago #
Anderverno
Member

Issue resolved.

Steps: Configured Opscenter and Agents the way they will not use SSL.
Adding use_ssl: 0 in the agents and use_ssl = false in opscenterd.conf

Thanks.

Posted 6 months ago #

RSS feed for this topic

Reply

You must log in to post.