DataStax Enterprise 3.1 Documentation

Expiring a DSE Search column

This documentation corresponds to an earlier product version. Make sure this document corresponds to your version.

Latest DSE documentation | Earlier DSE documentation

You can update a DSE Search column to set a time when data expires in these ways:

If you configure TTL in the solrconfig.xml, and then use the Solr HTTP API to set TTL, the latter takes precedence. If you configure TTL in the solrconfig.xml and then use CQL to set TTL, the latter also takes precedence.

After configuring a column to expire , Cassandra eventually removes the data from the database and the Solr index.

You can set a time-to-live (TTL) on an entire document or just a field. Configuring per-document TTL causes removal of the entire document. Configuring per-field TTL causes removal of the field only.

Configuring TTL in the update handler

You can configure the solrconfig.xml to include the TTL per-document or per-field on data added to the Solr index or Cassandra database. You construct a Solr HTTP API query to search the Solr index using a ttl component. Depending on the configuration, TTL then applies to the entire document or just to a named field.

To configure per-document or per-field TTL in the update handler:

  1. Configure the high-performance update handler section of the solrconfig.xml.

    • For per-document TTL, add these lines to the high-performance updateHandler section:

      <!-- The default high-performance update handler -->
       <updateHandler class="solr.DirectUpdateHandler2">
      . . .
      
      <lst name="defaults">
       <int name="ttl">1</int>
      </lst>
      
    • For per-field TTL, add these lines to the updateHandler section:

      <lst name="defaults">
       <int name="ttl.<column/field name1>">1</int>
       <int name="ttl.<column/field name2>">1</int>
       <int name="ttl.<column/field name3>">1</int>
       <int name="ttl.<column/field name4>">1</int>
       . . .
      </lst>
      
  2. Re-index the data by uploading the schema.xml and solrconfig.xml and reloading the core.

Configuring TTL using the Solr HTTP API

Use the Solr HTTP API update command to set a time-to-live (TTL). You can construct a URL to update data that includes the TTL per-document or per-field parameter:

  • Using the ttl parameter

    Specifies per-document TTL. For example:

    curl http://<host>:<port>/solr/mykeyspace.mytable/update?ttl=86400
    
  • Using the ttl.<field name> parameter

    Specifies per-field TTL. For example:

    curl http://<host>:<port>/solr/mykeyspace.mytable/update?ttl.myfield=86400
    

Configuring TTL using CQL

Using a CQL INSERT or UPDATE operation, you can set the TTL property. For example, continuing with the example in Using DSE Search/Solr, insert a 5 minute (300 seconds) TTL property on the all columns of the Einstein data:

INSERT INTO mysolr (id, name, title, body)
  VALUES ('126', 'Albert Einstein', 'Success', 'If A is success
  in life, then A equals x plus y plus z. Work is x; y is play;
  and z is keeping your mouth shut.')
  USING TTL 300;

Check the remaining time-to-live on the data:

select TTL (name) from mykeyspace.mysolr where id = '126';

After the remaining time has passed, because all columns of data were set to expire, querying the data returns no results.

Managing expired columns

After Cassandra expires a column using the time-to-live (TTL) mechanism, DSE Search/Solr can still find the expired column. The column data remains in the index until one of the following conditions is met:

Setting the ttl rebuild timeout properties is the recommended method for managing expired columns.