Agents not connecting to OpsCenter

(25 posts) (3 voices)

• Started 10 months ago by swoo
• Latest reply from swoo
• This topic is not resolved

Tags:

No tags yet.

1. 

   swoo
   Member

   Nevermind, found it.

   Posted 10 months ago #
2. 

   swoo
   Member

   I also get "Connection refused" when I try connecting using cassandra-cli.

   Posted 10 months ago #
3. 

   swoo
   Member

   So, should I expect that a "netstat -an | grep 9160" show the port as "LISTEN"?
   Because it does not.

   Posted 10 months ago #
4. 

   thobbs
   Administrator

   Yes, you should see something like:

   tcp 0 0 X.X.X.X:9160 0.0.0.0:* LISTEN

   on nodes that are running Cassandra. Are you sure your Cassandra nodes are actually up?

   Posted 10 months ago #
5. 

   swoo
   Member

   # ps aux | grep cas
   root 4330 0.0 0.0 106080 1284 pts/0 S+ 17:05 0:00 /bin/bash /home/cassandra/opscenter-2.1/bin/opscenter -f
   root 4362 0.0 0.0 106080 1264 pts/1 S+ 17:06 0:00 /bin/bash /home/cassandra/opscenter-2.1/agent/bin/opscenter-agent -f

   Posted 10 months ago #
6. 

   swoo
   Member

   Ok, I didn't realize cassandra had gone down. My bad, should have checked that first.

   So, now when I try to add the nodes, the error is "Error creating cluster: Call to /cluster-coinfgs timed out."

   Posted 10 months ago #
7. 

   swoo
   Member

   I'm seeing :

   2012-07-12 17:26:36+0000 [PE_Cluster] DEBUG: Attempting to load all persisted scheduled job descriptions
   2012-07-12 17:26:36+0000 [PE_Cluster] DEBUG: resubmitting get_slice request
   2012-07-12 17:26:36+0000 [PE_Cluster] DEBUG: resubmitting get_slice request
   2012-07-12 17:26:36+0000 [PE_Cluster] DEBUG: resubmitting get_slice request
   2012-07-12 17:26:36+0000 [PE_Cluster] DEBUG: resubmitting get_slice request
   2012-07-12 17:26:36+0000 [PE_Cluster] ERROR: Error while retrieving schedules from Cassandra. Trying again: UnavailableException()

   Posted 10 months ago #
8. 

   thobbs
   Administrator

   That indicates that some of your nodes are still down (or some of the nodes think that others are still down). Can you check the output of nodetool ring against each node to make sure that each node thinks all of the nodes are up?

   Posted 10 months ago #
9. 

   swoo
   Member

   # ./dse-2.1/bin/nodetool -h 10.1.18.31 ring
   Address DC Rack Status State Load Owns Token
   122825348605329854498244307355156153429
   10.1.18.31 Cassandra rack1 Up Normal 1.56 MB 50.79% 39091358306193191385997839679369834496
   10.1.18.33 Analytics rack1 Down Normal ? 2.93% 44081916082953045415693076643759127586
   10.1.18.32 Analytics rack1 Down Normal ? 40.76% 113432847117974692156422305023433173027
   10.1.18.34 Analytics rack1 Down Normal ? 5.52% 122825348605329854498244307355156153429

   Posted 10 months ago #
10. 

    swoo
    Member

    # ./dse-2.1/bin/nodetool -h 10.1.18.32 ring
    Error connection to remote JMX agent!
    java.io.IOException: Failed to retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: 10.1.18.32; nested exception is:
    java.net.ConnectException: Connection refused]
    at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:342)
    at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:267)
    at org.apache.cassandra.tools.NodeProbe.connect(NodeProbe.java:141)
    at org.apache.cassandra.tools.NodeProbe.<init>(NodeProbe.java:111)
    at org.apache.cassandra.tools.NodeCmd.main(NodeCmd.java:623)
    Caused by: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: 10.1.18.32; nested exception is:
    java.net.ConnectException: Connection refused]
    at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:118)
    at com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:203)
    at javax.naming.InitialContext.lookup(InitialContext.java:409)
    at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1902)
    at javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1871)
    at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:276)
    ... 4 more

    Posted 10 months ago #

RSS feed for this topic

Reply

You must log in to post.