On my virtual environment I configured dse cluster with 3 Cassandra nodes and 2 Solr nodes. This is my ring info:
Address DC Rack Workload Status State Load Owns Token
113427455640312821154458202477256070485
192.168.1.61 Cassandra rack1 Cassandra Up Normal 234.94 KB 33.33% 0
192.168.1.64 Solr rack1 Search Up Normal 234.94 KB 0.00% 20
192.168.1.62 Cassandra rack1 Cassandra Up Normal 234.94 KB 33.33% 56713727820156410577229101238628035242
192.168.1.65 Solr rack1 Search Up Normal 242.31 KB 16.67% 85070591730234615865843651857942052864
192.168.1.63 Cassandra rack1 Cassandra Up Normal 234.94 KB 16.67% 113427455640312821154458202477256070485
After unexpected cluster shutdown all nodes are unable to startup. By deleting index write.lock file on Solr nodes bring them alive but 3 Cassandra nodes still cannot startup. Below is error entry from output.log.
INFO 14:42:32,218 Creating new index : ColumnDefinition{name=5f646f63426f6f7374, validator=org.apache.cassandra.db.marshal.UTF8Type, index_type=CUSTOM, index_name='TestNamespace_TestIndex__docBoost_index'}
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.apache.commons.daemon.support.DaemonLoader.load(DaemonLoader.java:160)
Caused by: java.lang.NoClassDefFoundError: org/apache/solr/common/params/SolrParams
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:186)
at org.apache.cassandra.db.index.SecondaryIndex.createInstance(SecondaryIndex.java:307)
at org.apache.cassandra.db.index.SecondaryIndexManager.addIndexedColumn(SecondaryIndexManager.java:188)
at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:254)
at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:347)
at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:319)
at org.apache.cassandra.db.Table.initCf(Table.java:369)
at org.apache.cassandra.db.Table.<init>(Table.java:318)
at org.apache.cassandra.db.Table.open(Table.java:119)
at org.apache.cassandra.db.Table.open(Table.java:102)
at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:202)
at com.datastax.bdp.server.DseDaemon.setup(DseDaemon.java:105)
at org.apache.cassandra.service.AbstractCassandraDaemon.init(AbstractCassandraDaemon.java:257)
... 5 more
Caused by: java.lang.ClassNotFoundException: org.apache.solr.common.params.SolrParams
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:334)
... 19 moreCannot load daemon
Service exit with a return value of 3
Does anybody know how to bring them alive without loosing of data stored on them?
