Full Query Logging in Apache Cassandra® 4.0

Sometimes it is helpful to be able to see the exact CQL queries that client applications are sending to your cluster. This is true when you have poorly performing queries that you need to analyze and try to optimize. Or perhaps you have queries that are producing results that are not what you expected. Full Query Logging was designed to help in these cases.

Full Query Logging is also useful as a way of capturing live query traffic against a cluster so that you can replay it at a later time, perhaps in a development or test environment.

Full Query logging is disabled by default in Cassandra. There are two ways to configure full query logging: dynamically using nodetool or statically using cassandra.yaml.

A configuration using nodetool overrides a configuration defined in cassandra.yaml and does not persist across server restarts.

The following properties under the full_query_logging_options section in the cassandra.yaml file are used to configure full query logging:

• log_dir: Enable full query logging by setting this property to an existing directory location.
• roll_cycle: Sets the frequency at which log segments are rolled - DAILY, HOURLY (the default), or MINUTELY.
• block: Determines whether writes to the full query log will block query completion if full query logging falls behind, defaults to true.
• max_queue_weight: Sets the maximum size of the in-memory queue of full query logs to be written to disk before blocking occurs, defaults to 256 MiB.
• max_log_size: Sets the maximum size of full query log files on disk (default 16 GiB). After this value is exceeded, the oldest log file will be deleted.
• archive_command: Optionally, provides a command that will be used to archive full query log files before deletion.
• max_archive_retries: Sets a maximum number of times a failed archive command will be retried (defaults to 10)

Cassandra stores full query logs in a binary format designed to be compact and very fast to write, minimizing overhead on your nodes. The full query logging tool fqltool is used to extract queries from these binary logs. It supports operations including dumping the contents of a log file to text, comparing log files, and replaying the contents of a log file into a Cassandra cluster.

This tool is located in Cassandra’s tools/bin/fqltool directory. See the documentation for more information on how to use this tool.

The full query log contains a line for each query that completes successfully. When you use the fqltool dump command, each line includes values such as the following for each query:

• Type: The type of query - single query or batch
• Query start time: Unix timestamp of the query
• Protocol version: CQL protocol version of the query
• Query: The full text of the CQL query including all literal values used