We have recently discovered that the IP address octets that our cloud provider (Joyent) offers bears no relation to the network topology. As a consequence, using a RackInferringStrategy has meant we have one server that holds the lions share of replicas as by chance it has a different 3rd octet on its IP address. We have no control of the IP addresses of new cluster nodes and wish to add six new ones - our concern is that this is going to increase the load on our lonely replica hogging node.
So the question is, can we change the snitch on a live cluster? What is the immediate consequence of this - does it start rebalancing existing replicas or just subsequent replicas? Is it possible to rebalance where the replicas are hosted once a new snitch is in place?