We have our cassandra implementations setup so that nodes have two network interfaces, one for the gossip/jmx and one for thrift/app connections. How can I configure OpsCenter to use different IP addresses for thrift connections and jmx connections? As far as I can tell, you only specify a single IP address for cassandra nodes
OpsCenter configuration for cassandra using dual networks
(5 posts) (3 voices)- Started 7 months ago by cooptron
- Latest reply from cooptron
- This topic is resolved
-
cooptron
Membertupton,
It appears that the agents are connecting to the localhost jmx/thrift ok, but opscenter is trying to communicate to the agents on the gossip interface of our cassandra cluster instead of the thrift/rpc interface. This is causing half of the information in opscenter to be missing (storage, etc) even though all the agents are showing connected.
here is the opscenterd log output. It SHOULD be connecting to 172.23.58.80, etc, the gossip network (172.23.1.x) is unroutable on purpose, so the opscenter server can not access it.
From what it seems, you are gathering up the node IPs via thrift calls, but that call is using autoDiscoverHosts instead of requesting the thrift IP address of the nodes, so it is defaulting to the gossip interface.
What I setup in opscenter for the seed addresses:
172.23.58.80
172.23.58.81
172.23.58.82Output from opscenterd trying to connect to an agent:
WARN: HTTP request https://172.23.1.87:61621/cluster/datacenter?node_ip=172.23.1.81 failed: Connection was refused by other side: 111: Connection refused.
Posted 5 months ago # -
Some information on the 3 interfaces/IPs involved on the agent side can be seen in step 3 here: http://www.datastax.com/docs/opscenter/configure/configure_multi_region
It sounds like, since the gossip network is unroutable, you'll want to configure agent_rpc_interface on each node to point to cassandra's rpc interface (by default, that will pull from the broadcast address configured in cassandra.yaml)
If you're not able to bind to that interface on the agent side, you can configure agent_rpc_broadcast_address instead, which will allow the agent's http server to bind on a separate interface from the one opscenterd connects on.
Hope that helps. Let us know if you have any other questions.
Posted 5 months ago #
Reply
You must log in to post.