About DataStax Enterprise Search¶

Benefits of Using Solr in DataStax Enterprise¶

DataStax Enterprise Search is built on top of Solr 4.0, which offers real-time querying of files. Search indexes remain tightly in line with live data. There are significant benefits of running your enterprise search functions through DataStax Enterprise instead of native Solr, including:

• A fully fault-tolerant, no-single-point-of-failure search architecture
• Linear performance scalability that comes from adding new search nodes online
• Automatic indexing of data stored in Cassandra
• Automatic and transparent data replication
• Isolation of all real-time, Hadoop, and search/Solr workloads to prevent competition between workloads for either compute resources or data
• The capability to read/write to any Solr node, which overcomes the Solr write bottleneck
• Selective updates of one or more individual fields instead of having to update an entire document
• Search indexes that can span multiple data centers (native Solr cannot)
• CQL supports Solr/search queries

DSE Search takes secondary indexes to a new level: data added to Cassandra is locally indexed in Solr. Data added to Solr is locally indexed in Cassandra.

Integration of Solr into DataStax Enterprise¶

DSE Enterprise supports cluster partitioning by workload as described in About Replication in Cassandra.

Using this approach, you can make some of your DSE nodes handle search while others handle MapReduce, or just act as ordinary Cassandra nodes. In production environments, do not run Solr and Hadoop on the same node. In development environments, running both is feasible.

Cassandra ingests the data, Solr indexes the data, and you run MapReduce against that data, all in one cluster without having to do any manual extract, transform, and load (ETL) operations.

Cassandra handles the replication and isolation of resources.

DataStax Enterprise to Solr Objects Map¶

Solr calls an index of documents a core. Each document in a core is considered unique and contains a set of fields that adhere to a user-defined schema. The schema lists the field types and how they should be indexed.

DSE Search links Solr cores to Cassandra column families, Solr documents to Cassandra rows, and document fields to columns. This table shows the relationship between Cassandra and Solr concepts:

Schema and Configuration Files¶

Solr has a number of required and optional configuration files. A minimal Solr installation requires these files:

• Schema.xml Describes the fields to index in Solr and types associated with them. These fields map to Cassandra columns. To route search requests to the appropriate nodes, the schema needs a unique key.
• Solrconfig.xml Holds configuration information for query handlers and Solr-specific caches.

For more information about creating the schema, see Creating a Schema.

About the DSE Search Resource REST API¶

DSE Search includes a REST API for adding and retrieving resources associated with an index. You can look at the contents of the existing Solr resource by loading its URL in a web browser or using HTTP get.

After generating valid schema.xml and solrconfig.xml files, you can create a new Solr index by posting the files through a specific HTTP endpoint. Use this format:

http://<host>:<port>/solr/resource/<keyspace>.<columnfamily>/<filename>.<ext>

Generally, you can post any resource required by Solr to this URL. For example, stopwords.txt and elevate.xml are optional, frequently-used Solr configuration files that you post using this URL.

Example of Creating an Index

For example, to create a Solr index on a column family, make two HTTP POST requests using the cURL utility as follows:

Configuration file POST request:

curl http://localhost:8983/solr/resource/keyspace.columnfamily/solrconfig.xml
  --data-binary @solrconfig.xml -H 'Content-type:text/xml; charset=utf-8'

Schema file POST request:

curl http://localhost:8983/solr/resource/keyspace.columnfamily/schema.xml
  --data-binary @schema.xml -H 'Content-type:text/xml; charset=utf-8'

DSE Search stores the files on all the Cassandra nodes and creates a new Solr core. If you HTTP post the files to a pre-existing column family, DSE Search starts indexing the data. If you HTTP post the files to a non-existing column keyspace or column family, DSE Search creates the keyspace and column family, and then starts indexing the data. For example, you can change the stopwords.txt file, repost the schema, and the index updates.

Changing the Solr schema makes reindexing necessary and reindexing can be disruptive. Users can be affected by performance hits caused by reindexing. Changing the schema is recommended only when absolutely necessary. Also, changing the schema during scheduled down time is recommended.

Unsupported Features¶

DSE Search does not support:

• Supercolumns

• Counter columns

• Timeseries type rows

• Compound columns

  Solr fields must be strings.

Links to Solr and Lucene Documentation¶

• Solr Tutorial on the Solr site
• Solr Tutorial on Apache Lucene site
• Solr data import handler
• Comma-Separated-Values (CSV) file importer
• JSON importer
• Solr cell project, which includes a tool for importing data from PDFs