Solr mapping problem

(21 posts) (7 voices)

• Started 11 months ago by
• Latest reply from Jack Krupansky
• This topic is not a support question

Tags:

• cassandra solr dse

1. 

   Anonymous
   Unregistered

   Hi,

   If I have a Cassandra ColumnFamily and I try to create a Solr schema for this column family it won't work if any of the columns has data type other than utf8. If all are utf8 then there are no problems. If there's no CF and I post the schema with fields containing other than utf8 (string) it works w/o problems, creating the corresponding Cf columns with the correct data types. The only problem is when I try to map an existing column family.
   The error is also a bit strange:
   Caused by: javax.xml.parsers.ParserConfigurationException: Current validation_class and found SolrType do not much for column column_name

   I have declared all the fieldTypes correctly (as I said, it workd if there's no column family previously declared).

   Any idea?

   Thank you,

   Cristian

   Posted 11 months ago #
2. 

   Anonymous
   Unregistered

   Testing a bit more I have discovered that IntField is the only one making problems, if the existing fields are float for example the indexes are correctly created.

   Posted 11 months ago #
3. 

   Anonymous
   Unregistered

   Please post your schema.xml file. Int should work, you can refer to this table:

   http://www.datastax.com/docs/datastax_enterprise2.0/search/dse_search_schema#wikipedia-sample-column-family-metadata

   Posted 11 months ago #
4. 

   Anonymous
   Unregistered

   To replicate the problem:

   in cqlsh:

   CREATE COLUMNFAMILY test_data
   (
   hash varchar PRIMARY KEY, received varchar, source varint
   )
   WITH comparator = varchar AND default_validation = varchar;

   then schema.xml

   <schema name="hashes" version="1.1">
   <types>
   <fieldType name="string" class="solr.StrField" />
   <fieldType name="boolean" class="solr.BoolField" />
   <fieldType name="integer" class="solr.IntField" />
   <fieldType name="long" class="solr.LongField" />
   <fieldType name="float" class="solr.FloatField" />
   <fieldType name="double" class="solr.DoubleField" />
   </types>
   <fields>
   <field name="hash" type="string" indexed="true" stored="true" />
   <field name="received" type="string" indexed="true" stored="true" />
   <field name="source" type="integer" indexed="true" stored="true" />
   </fields>
   <defaultSearchField>source</defaultSearchField>
   <uniqueKey>hash</uniqueKey>
   </schema>

   after posting the schema.xml I get the error

   <snip>
   java.lang.RuntimeException: javax.xml.parsers.ParserConfigurationException: Current validation_class and found SolrType do not much for column: source. java.lang.RuntimeException: java.lang.RuntimeException: javax.xml.parsers.ParserConfigurationException: Current validation_class and found SolrType do not much for column: source
   </snip>

   The schema works w/o problem if the column family wasn't previously defined in Cassandra.

   Posted 11 months ago #
5. 

   Anonymous
   Unregistered

   When you define the ColumnFamily in cqlsh, you add two fields "receiver" and "source" with validators "varchar" and "varint" which is causing the problem because when you post your schema.xml it tries to change the validator on those columns to "solr.StrField" and "solr.IntField" which is not a good idea and not allowed.

   Posted 11 months ago #
6. 

   Anonymous
   Unregistered

   The cqlsh syntax for declaring UTF8Type and IntegerType is varchar and varint. The ColumnFamily is created with the proper validators. The proper map for UTF8Type and IntegerType (as posted by Datastax) is solr.StrField and solr.IntField.

   There is no visible error in what I have done.

   Posted 11 months ago #
7. 

   Anonymous
   Unregistered

   Yes, there is a visibile error. IntegerType does not correspond to solr.IntField at all, IntField is Int32Type that is what it telling you. There is no reason to create columns to just redefine them later, it's not a good idea to mess with validators like that.

   Posted 11 months ago #
8. 

   Anonymous
   Unregistered

   Then please tell me what solr validator I should use and why, if I create the column family from the xml with IntField, the validator is org.apache.cassandra.db.marshal.IntegerType

   And I do have a reason to do that. I have a database with 60 millions records and I want to use solr with it.

   Posted 11 months ago #
9. 

   Anonymous
   Unregistered

   Then you should change the validator on the "source" column to Int32Type and upload the schema.xml after that.

   You can do that using CLI interface e.g. "update column family test_data with column_metadata=[{column_name:received, validation_class:UTF8Type}, {column_name:source, validation_class:Int32Type}];"

   Or when you create a column family via CQL you should use "int" instead of "varint" because IntegerType in Cassandra corresponds to BigInteger.

   Posted 11 months ago #
10. 

    Anonymous
    Unregistered

    Myeah, when I have created the cassandra database there wasn't a int32type and varint was mapping the integer type. Thx for the tip.

    Posted 11 months ago #
11. 

    Anonymous
    Unregistered

    varint is still mapping to the IntegerType but that is not what you (and Solr) want tho.

    Posted 11 months ago #
12. 

    Anonymous
    Unregistered

    It worked, thx! Any idea how to map a field that has org.apache.cassandra.db.marshal.DateType as validator? :D

    Posted 11 months ago #
13. 

    Anonymous
    Unregistered

    here is an equality table for you

    TextField = UTF8Type
    StrField = UTF8Type
    LongField = LongType
    IntField = Int32Type
    FloatField = FloatType
    DoubleField = DoubleType
    DateField = UTF8Type # C* date type encodes as long
    ByteField = BytesType
    BinaryField = BytesType
    BoolField = UTF8Type # Solr requires 'T' 'F' 'true' 'false' etc
    UUIDField = UUIDType

    Posted 11 months ago #
14. 

    Anonymous
    Unregistered

    Thank you, but isn't in that table the date from solr? There isn't any date type in the Cassandra side of the table (I have data in cassandra that is saved with DateType validator). I have tried to map it as LongField (because date is stored as long), didn't work.

    Posted 11 months ago #
15. 

    Anonymous
    Unregistered

    The table has a field "DateField = UTF8Type", try to change validator to UTF8Type on your date column.

    Posted 11 months ago #

RSS feed for this topic

Reply »

You must log in to post.