Release notes¶

Enhancements and changes¶

DataStax Enterprise 3.0.1 has been enhanced or changed in the following ways:

• The default consistency level has changed from ONE to QUORUM for reads and writes to resolve a problem finding a CassandraFS block when using consistency level ONE on a Hadoop node. (DSP-1809)

• Solr type mapping to Cassandra validator types has been refactored in this release. (DSP-1876)

• The configuration files for these DSE Search/Solr demos have been modified to use new type mapping:

  • Solr wikipedia demo
  • Log search demo
  • Solr stress demo

  Running DSE Search/Solr demos using legacy data describes how to use data from an earlier release.

• DSE Search provides a new multi-threaded indexing implementation to improve performance on multi-core machines. All index updates are internally dispatched to a per-core indexing thread pool and executed asynchronously: this allows for greater concurrency and parallelism, but as a consequence, index requests will return a response before the indexing operation is actually executed. The number of available indexing threads per-core is by default equal to number of available cores times 2: it can be configured by editing the max_solr_concurrency_per_core parameter in the dse.yaml configuration file; if set to 1, DSE Search will go back to the synchronous indexing behavior of the earlier release. (DSP-1644)

  Also, DSE Search provides advanced, JMX-based, configurability and visibility through the IndexPool-ks.cf (where ks.cf is the name of a DSE Search Solr core) MBean under the com.datastax.bdp namespace.

• On the cqlsh command line, Tab completion now reveals user names when you type a CQL security command that takes a known user name as an option. (DSP-1371)

• This release includes a plugin API for Solr updates and a plugin to the CassandraDocumentReader. The plugin API transforms data from the secondary indexing API before it is submitted to Solr. The plugin to the CassandraDocumentReader transforms the results data from Cassandra to Solr. (DSP-1493)

• The deprecated Solr document cache is now disabled. (DSP-1794)

• Solr/Cassandra range manipulation and token filtering algorithms have been rewritten to improve performance and internal maintenance. This change is backward compatible with previous releases. (DSP 1708)

• This release includes two features for performing an anti-entropy node repair on a subrange of data instead of all the data in a keyspace. (DSP-1661)

  • A new dsetool command, list_subranges, estimates subranges of data in a keyspace based on a specified number of rows.
  • New nodetool repair options, start token (-st) and end token (et), designate subranges of data for distribution within those ranges.

  Using these commands, DSE Search now performs a partial re-index instead of a full re-index of Solr data after an anti-entropy repair.

• You can now track memory usage of internal Lucene and Solr data structures using OpsCenter. These metrics are among those you can track: (DSP-1617)

  • Doc values (including norms)
  • Field caches
  • Doc-set caches
  • Terms index (FST) caches
  • NRTCachingDirectory (internal RAMDirectory)
  • IndexWriter RAM usage

• Cassandra 5155 has been backported to the Cassandra component included in this release, Cassandra 1.1.9.3. With the enhancement, you can configure an Ec2Region data center name. In the same EC2 region, you can now run a real-time Cassandra data center and a DSE Search/Solr cluster. (DSE-1685)

• The Query Elevation search component now functions correctly if you upload the elevate.xml to Cassandra like you upload the solrconfig.xml. Alternatively, put elevate.xml in a directory on all the nodes. (DSP-1652)

Issues¶

• To insert data using CQL or Thrift that will be indexed by Solr, run the inserts on a Solr node. (DSP-2007)

• Use a single CQL statement or batch operation in thrift to insert data in fields that the Solr schema declares a copyField having a multi-value destination. Otherwise any, subsequent writes to those fields overwrite any values in the field being copied to. (DSP-1882)

  For example, using pycassa:

  import pycassa
  from pycassa.pool import ConnectionPool
  from pycassa.columnfamily import ColumnFamily
  
  pool = ConnectionPool('test')
  col_fam = pycassa.ColumnFamily(pool, 'copy')
  b = col_fam.batch()
  b.insert('thrift_test',
    {'text1': 'textval1',
    'text2': 'textval2'})
  b.insert('thrift_test', {'multi_text_col1': "solrjson:['foo','bar']"})
  b.send()
  
  b2 = col_fam.batch()
  

• During upgrading, you might see warnings when initially starting up a Analytics/Hadoop node. To avoid making concurrent changes to the schema, which are not fully supported in this release, nodes coordinate the configuration of the system keyspaces. When the node designated to update the schema is not fully initialized or a user runs dsetool before the schema update occurs, this type of warning occurs: (1804)

  INFO [main] 2013-04-15 19:09:22,362 CassandraFSPlugin.java (line 35)
    Found CFS filesystem in Hadoop config: cfs-archive
  WARN [RMI TCP Connection(2)-10.190.155.233] 2013-04-15 19:09:27,570
    TrackerManager.java (line 201) JobTracker location query failed with consistency level QUORUM, retrying with level ONE
  WARN [RMI TCP Connection(2)-10.190.155.233] 2013-04-15 19:09:27,571
    CassandraJobConf.java (line 358) Unable to retrieve JobTracker primary and reserve locations, will set local address as JT for Analytics-Analytics
  WARN [RMI TCP Connection(2)-10.190.155.233] 2013-04-15 19:09:27,574
    TrackerManager.java (line 157) Error writing JT location
    InvalidRequestException(why:Keyspace dse_system does not exist)
  . . .
  

  You can ignore these warnings. (DSP-1916)

• Solr can return duplicated results because Solr improperly indexes Cassandra when all of these conditions exist:

  • Primary key values contain special characters, such as #, @ and $.
  • The unique key field in the Solr schema is of field type "text" (solr.TextField).
  • The field has a tokenizer that treats such special characters as white space.

  The solution is to ensure that the Solr unique key field is of type string (solr.StrField). (DSP 882 and 839)

• When using Async or HsHa, Hadoop users may see an error that a connection failed (Failed to open server transport) and a different transport will be used (Falling back to TFramedTransport). This error is benign. To remove the error, in mapred-site.xml, set the property cassandra.client.transport.factory to org.apache.cassandra.thrift.TFramedTransportFactory". You may also need to fix the property in the dsetool, nodetool, and cassandra-cli scripts. (DSP-1844)

• The RPC Thrift server doesn't support Async or HsHa when using Kerberos. You must either change the settings in cassandra.yaml to rpc_server_type: sync, or disable Kerberos and restart the server. (DSP-1844)

Issues resolved¶

• Range query performance problems and inconsistency across nodes. (DSP-1577)
• The Query Elevation search component now works correctly. You can upload the elevate.xml to C* like solrconfig.xml or have it reside in a directory on all the nodes. (DSP-1652)
• Solr queries return inconsistent values after nodes are rebalanced. (DSP-1672)
• Solr delete by query, when the key is a UUID, creates a corrupt tombstone. (DSP-1709)
• Cassandra-5301 has been backported to the Cassandra 1.1.9.3 included in this release. This fix relaxes the consistency level for authentication queries for non-default users. The default super user, "cassandra", requires reading the dse_auth keyspace at QUORUM. Other superusers read and write at a consistency level of ONE.
• For backward compatibility, the Thrift set_keyspace behavior has been altered to allow a call to set_keyspace followed by a call to login. (DSP-1878)
• You no longer need to enable Hadoop to connect to external addresses. DSE automatically sets the listen_address:rpc_port. (DSP-1139)
• An extraneous copy of the Cassandra Command Line Interface (CLI) in <install location>/resources/cassandra/bin in a tarball installation was not correctly configured. This copy has been removed. Use the CLI utility located in <install location>/bin. (DSP-1832)

Enhancements¶

DataStax Enterprise 3.0 has been enhanced in the following ways:

• Kerberos security added to all components (Cassandra, Hadoop, Solr)
• Cassandra-based password authentication backported (Cassandra only)
• Full Cassandra-level data auditing
• At rest data encryption
• Integration of the final Solr 4.0 release
• Various Solr stability improvements:
  • Concurrent solr core reload
  • Improved handling of bad schemas
  • Improved stability under concurrent access situations
  • Proper de-duplication of search results
• Other Solr improvements
• Update to Hadoop 1.0.4.2 and other component updates
• Various Hadoop stabilization fixes
• Various improvements to stabilize the upgrade process

Changes and requirements¶

Changes to the Solr demo script

In this release, changes have been made to the Solr demo script. The scripts to run the wikipedia demo have been updated. For example, the 1_add_schema.sh script has been updated to include these lines:

CREATE_URL="http://${host}:8983/solr/admin/cores?action=CREATE&name=${KS}.${CF}"
curl -X POST $CREATE_URL
echo "Created index."

Disk full alert

DataStax assumes that the Customer's operation team monitors cluster resources to ensure that enough disk space exists. In the event of an oversight, Cassandra marks the node to be decommissioned when the disk is approaching full. The server should stop serving when the disk is almost full, the node is removed from the ring, and Cassandra issues an alert.

Java requirements

DataStax recommends using the latest 64-bit version of Java 6.

• DSE and Cassandra: JRE not supported below 1.6.0_29.
• Kerberos: The JRE should be later than 1.6.0_26 due to a Kerberos bug.

Resolved issues¶

This release fixes the following issues:

• Fields in FieldInfos that are in the SolrInputDocument when the document is sent to Solr were neither stored in Solr nor indexed. (DSP-1352)
• On node restart, errant Solr documents in commit log prevented the node from starting. (DSP-1297)
• After storing data on one Solr node, the data replication was inconsistent. Now, the data shows up in searches from other nodes. (DSP-1121)

Issues¶

This release has the following issues:

• An outdated file, SECURITY_NOTES, is included in the installation directory. This file should have been removed. For security information and procedures, use this document.
• Solr displays duplicate records when you use a unique key that you declared as type no-token. (DSP-1085)
• The Cassandra log4j appender doesn't support multiple hosts. (DSP-1601)
• The sstableloader tool does not work in an environment using Kerberos. The workaround is to run sstableloader in a ring that doesn't use Kerberos. (DSP-1168)
• When you have a multiple data center cluster with Kerberos enabled, all keyspaces accessed from Hadoop must be configured with NetworkTopologyStrategy. If configured with SimpleStrategy, Hadoop jobs will hang. (DSP-1630)
• MapReduce jobs hang before completing or finishing cleanup with older versions of Hadoop (MAPREDUCE-4560, MAPREDUCE-4299. The workaround is remove the mapred.reduce.slowstart.completed.maps parameter and restart. (DSP-1154)
• The nodetool repair -pr command does not completely repair a keyspace unless it is in every datacenter. (CASSANDRA-5424)
• In earlier releases, the authenticator allowed various Cassandra clients, such as Hector, to set a keyspace, and then login. In this release, the org.apache.cassandra.auth.PasswordAuthenticator requires that the client login and then set the keyspace. (Cassandra-5423, DSP-1878)
• Open Source Solr (OSS) supports relative paths set by the <lib> property in the solrconfig.xml, but DSE Search/Solr does not. Configuring Solr library paths describes a workaround for this issue that DataStax Enterprise will address in a future release. (DSP-1840)
• After upgrading DataStax Enterprise 2.x to 3.x, a Solr-indexed field containing an empty date causes a parse exception when encountered in search results. (DSP-1944)