So it seems it's timing dependent. While trying to get a VM image, it took me 4 stop and starts to get the problem to reproduce. I'm going to first snapshot the VM and download it and see if it will reliably reproduce on windows VMWare Workstation and then also see if the state is inherent to the current data state or whether it's temporal in nature.
In an effort to consolidate free help offered for our products we have decided to move these forums to a more widely used forum. Please use one of the following queries (or any combination):
- Cassandra: tag search or plain text search
- DataStax Enterprise: tag search or plain text search
- DataStax OpsCenter: tag search or plain text search
Solr node hung on DSE start(20 posts) (2 voices)
I've been unable to export the hang. As I mentioned, I was able to reproduce the issue and once reproduced seemed to stick on the node. Deleting commit logs still resulted in the hang. I tried moving all the data directories to a different isolated node configured with the same network address and cassandra yaml file, however it starts up there and empties the queue.
Do you mean you are unable to reproduce on a different machine/vm?
Anyways, I looked at your logs and saw nothing wrong.
I'm able to reproduce the hang on multiple servers in the cluster. What I can't figure out is how to get a snapshot to you of a node in a hung condition without replicating our 9-node cluster.
The log file I sent the thing that is wrong is you don't ever see the backpressure being disabled -- the queue is hung. If you look at the com.datastax.bdp IndexPool-ism.ie QueueDepth, it sits there unchaning -- it's hung and will never start processing.
you should try to reproduce on an EC2 instance and give me access there; otherwise, I'm sorry I can't really help you more than that: we never experienced such problems on our tests or any other customers/users.