Support Forums : DataStax

DataStax Support Forums » DataStax Enterprise

POST /update/json or /update/xml does not behave as expected

(9 posts) (3 voices)

Started 4 months ago by tristanls
Latest reply from Jack Krupansky
This topic is resolved

Tags:

No tags yet.

tristanls
Member

schema.xml

<schema name="initialtiny" version="1.1"> <types> <fieldType name="string" class="solr.StrField"/> <fieldType name="text" class="solr.TextField"> <analyzer> <tokenizer class="solr.StandardTokenizerFactory"/> </analyzer> </fieldType> <fieldType name="uuid" class="solr.UUIDField" multiValued="false"/> </types> <fields> <field name="id" type="uuid" indexed="true" stored="true"/> <field name="body" type="text" indexed="true" stored="true"/> </fields>
<defaultSearchField>body</defaultSearchField> <uniqueKey>id</uniqueKey>
</schema>

solrconfig.xml (the request handler part)
<requestHandler name="/update/json" class="solr.JsonUpdateRequestHandler" />

Once these are posted and column families and such are created, this is the end of the output from cassandra-cli describe initialtiny; (this will be relevant further down)

... Column Name: body Validation Class: org.apache.cassandra.db.marshal.UTF8Type Index Name: initialtiny3_solr_body_index Index Type: CUSTOM Index Options: {class_name=com.datastax.bdp.cassandra.index.solr.SolrSecondaryIndex} Compaction Strategy: org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy Compression Options: sstable_compression: org.apache.cassandra.io.compress.SnappyCompressor

So, all good so far. I can also view this in OpsCenter, and see that column family was created, etc.

When I attempt to add a document for indexing, via:

curl http://localhost:8983/solr/resource/initialtiny.solr/update/json?commit=true -H 'Content-Type:application/json' -d '{"id":"aecfd710-5b64-11e2-bcfd-0800205c9a67","body":"This is my awesome text. I hope it gets indexed"}' -v

It returns SUCCESS. However, no data is stored in Cassandra, and searching for *:* returns 0 results.

What's odd is that running describe initialtiny; within cassandra-cli displays the following at the end:

... Column Name: body Validation Class: org.apache.cassandra.db.marshal.UTF8Type Index Name: initialtiny2_solr_body_index Index Type: CUSTOM Index Options: {class_name=com.datastax.bdp.cassandra.index.solr.SolrSecondaryIndex} Column Name: _solr_update/json Validation Class: org.apache.cassandra.db.marshal.UTF8Type Index Name: solr__solr_updatejson_idx_2 Index Type: CUSTOM Index Options: {class_name=com.datastax.bdp.util.ConfigOnlyIndex, update/json={"id":"aecfd710-5b64-11e2-bcfd-0800205c9a67","body":"This is my awesome text. I hope it gets indexed"} } Compaction Strategy: org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy Compression Options: sstable_compression: org.apache.cassandra.io.compress.SnappyCompressor

I was expecting that a document I POSTed would be indexed, and then added to Cassandra datastore.

What happened was that the document was not added to Cassandra datastore and querying for *:* results in 0 results.

Additionally, the document I posted ends up as Index Options field in column _solr_update/json for some reason.

How do I add documents to be indexed and stored by POSTing JSON to Solr?

Posted 4 months ago #
tristanls
Member

I forgot versions:

dse-2.2.1
opscenter-2.1.2

Posted 4 months ago #
tristanls
Member

Just tried it with the post.jar tool from apache-solr-4.0.0.tgz, and the exact same problem occurs (no data visible in Cassandra, no results for *:* query).

For reference, the document was stored in doc.json, the command and output were:

java -Dtype=application/json -Durl=http://localhost:8983/solr/resource/initialtiny.solr/update/json -jar post.jar doc.json SimplePostTool version 1.5 Posting files to base url http://localhost:8983/solr/resource/initialtiny.solr/update/json using content-type application/json.. POSTing file doc.json 1 files indexed. COMMITting Solr index changes to http://localhost:8983/solr/resource/initialtiny.solr/update/json..

Posted 4 months ago #
Jack Krupansky
Member

Sorry for the confusion. The field type for your "id" field should be "string", not UUID.

Posted 4 months ago #
tristanls
Member

I changed schema.xml to:

<schema name="initialtiny" version="1.1"> <types> <fieldType name="string" class="solr.StrField"/> <fieldType name="text" class="solr.TextField"> <analyzer> <tokenizer class="solr.StandardTokenizerFactory"/> </analyzer> </fieldType> </types> <fields> <field name="id" type="string" indexed="true" stored="true"/> <field name="body" type="text" indexed="true" stored="true"/> </fields>
<defaultSearchField>body</defaultSearchField> <uniqueKey>id</uniqueKey>
</schema>

No change in behavior. (no data in Cassandra, no responses for *:* query, Column Name: _solr_update/json with document contents as Index Options gets created instead)

Posted 4 months ago #
Jack Krupansky
Member

Sorry for not noticing this before... take "/resource" out of your URLs. Too bad DSE is not giving you a proper error response. Also, you need square brackets around the document list, which is a JSON array of documents. For example:

curl http://localhost:8983/solr/initialtiny.solr/update/json?commit=true -H 'Content-Type:application/json' -d '[{"id":"aecfd710-5b64-11e2-bcfd-0800205c9a67","body":"This is my awesome text. I hope it gets indexed"}]' -v

Posted 4 months ago #
tristanls
Member

Success! Thank you.

The response if everything goes well is something like (as opposed to SUCCESS):

<?xml version="1.0" encoding="UTF-8"?> <response> <lst name="responseHeader"><int name="status">0</int><int name="QTime">52</int></lst> </response>

Both of your corrections were necessary (remove "/resource" and square brackets).

I was surprised that square brackets were necessary, as the example from http://wiki.apache.org/solr/UpdateJSON for Solr 4.0 (at the bottom) shows posting without square brackets:

curl http://localhost:8983/solr/update -H 'Content-type:application/json' -d ' { "id" : "TestDoc1", "title" : {"set":"test1"}, "revision" : {"inc":3}, "publisher" : {"add":"TestPublisher"} }'

Thanks again,

Cheers

Posted 4 months ago #
Fadi
Member

What is the conclusion of this? If Cassandras table is using UUID type ID, then I cant use it as a unique id in SOLR?

Solr is able to assign/generate UUIDs, but when tested on DSE 2.2.1 we get java.lang.RuntimeException: org.apache.solr.common.SolrException: Error loading class 'solr.UUIDUpdateProcessorFactory'

The class exists in solr core jar file from apache but not the one bundled in DSE, is it OK to replace DSE solr jar files with ones ( same version ) from the net?

Regards
/Fadi

Posted 3 months ago #
Jack Krupansky
Member

For now, stick with Solr "string" and CQL "text"/UTF8Type for the primary/unique key type.

Posted 3 months ago #

RSS feed for this topic

Reply

You must log in to post.