DataStax Support Forums » Topic: JapaneseTokenizerFactory

DataStax Support Forums » Topic: JapaneseTokenizerFactory http://www.datastax.com/support-forums/topic/japanesetokenizerfactory Software, Support, and Training for Apache Cassandra en-US Sat, 18 May 2013 17:36:57 +0000 http://bbpress.org/?v=1.0.3 <![CDATA[Search]]> q http://www.datastax.com/support-forums/search.php alexliu on "JapaneseTokenizerFactory" http://www.datastax.com/support-forums/topic/japanesetokenizerfactory#post-6646 Mon, 24 Sep 2012 20:36:26 +0000 alexliu 6646@http://www.datastax.com/support-forums/ can you try this one <a href="https://issues.apache.org/jira/browse/LUCENE-3305" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-3305</a> tomitakazutaka on "JapaneseTokenizerFactory" http://www.datastax.com/support-forums/topic/japanesetokenizerfactory#post-6561 Tue, 18 Sep 2012 17:36:47 +0000 tomitakazutaka 6561@http://www.datastax.com/support-forums/ Hi. Thank you for your information. But I would like to use not index of bi-grams but a morphological index. alexliu on "JapaneseTokenizerFactory" http://www.datastax.com/support-forums/topic/japanesetokenizerfactory#post-6542 Tue, 18 Sep 2012 00:09:57 +0000 alexliu 6542@http://www.datastax.com/support-forums/ Chinese, Japanese, Korean Lucene provides support for these languages with CJKTokenizer, which indexes bigrams and does some character folding of full-width forms. <tokenizer class="solr.CJKTokenizerFactory"/> tomitakazutaka on "JapaneseTokenizerFactory" http://www.datastax.com/support-forums/topic/japanesetokenizerfactory#post-6522 Mon, 17 Sep 2012 05:59:54 +0000 tomitakazutaka 6522@http://www.datastax.com/support-forums/ Hi. I'm checking The Dse for a document search in Japanese. I want to use fulltext search to use JapaneseTokenizerFactory in Solr 4.0 by default. but when I created a schema,that query was error. Error mesage : ERROR [http-8983-1] 2012-09-17 14:51:37,265 SolrException.java (line 114) java.lang.RuntimeException: java.lang.RuntimeException: org.apache.solr.common.SolrException: Plugin init failure for [schema.xml] fieldType:Plugin init failure for [schema.xml] analyzer/tokenizer:Error loading class 'solr.JapaneseTokenizerFactory' at com.datastax.bdp.plugin.AbstractPlugin.activate(AbstractPlugin.java:47) at org.apache.solr.core.CassandraCoreContainer.getContainer(CassandraCoreContainer.java:256) at org.apache.solr.core.CassandraCoreContainer.getCore(CassandraCoreContainer.java:181) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:189) at com.datastax.bdp.cassandra.index.solr.CassandraDispatchFilter.doFilter(CassandraDispatchFilter.java:136) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:722) schema.xml: <?xml version="1.0" encoding="UTF-8" ?> <schema name="kaiseki" version="1.1"> <types> <fieldType name="string" class="solr.StrField"/> <fieldType name="text" class="solr.TextField"> <analyzer> <tokenizer class="solr.JapaneseTokenizerFactory" mode="NORMAL" userDictionaryEncoding="UTF-8" discardPunctuation="true" /> </analyzer> </fieldType> </types> <fields> <field name="id" type="string" indexed="true" stored="true"/> <field name="title" type="text" indexed="true" stored="true"/> <field name="body" type="text" indexed="true" stored="true"/> </fields> <defaultSearchField>body</defaultSearchField> <uniqueKey>id</uniqueKey> </schema> Why I don't use JapaneseTokenizerFactory ? What should I do with the way I use it ?