DataStax Developer Blog

Headless Solrmeter with DataStax Enterprise

By Michael Allen -  October 21, 2013 | 2 Comments

Introduction

Not too long ago, I was tasked with getting an automated regression test in place for DataStax Enterprise’s bundled version of Apache Solr. After taking a bit of time to explore the various options available, it was decided that SolrMeter would be an excellent fit for the project, except for the fact that it did not have a headless mode of operation and so controlling it from the command line would not be an easy task. After spending a few hours digging into SolrMeter’s code I came to the conclusion that, owing to the excellent architecture of SolrMeter itself, it would not be too difficult of a job to create a patch that would allow it to run in headless mode and be drivable from the command line, which is what we needed for our project. So that is exactly what I did, and in this post I will show you how to get a headless SolrMeter up and running with very little effort. I hope that you will find this useful and possibly even be able to use this in your own automation projects.

The setup

For the examples that follow, I used a single node DataStax Enterprise cluster on EC2. The single and only node of this cluster has been started up as a solr/search node, and I have taken some extra steps to load data into Solr that we will access later with SolrMeter. The steps I took were as follows.

automaton@ip-10-172-89-53:~$ cd /usr/share/dse-demos/wikipedia/
automaton@ip-10-172-89-53:/usr/share/dse-demos/wikipedia$ ls
1-add-schema.sh  2-index.sh  build.xml  download-wiki.sh  README.txt  schema.xml  set-solr-options.sh  solrconfig.xml  src  wikipedia_import.jar  wikipedia-sample.bz2

automaton@ip-10-172-89-53:/usr/share/dse-demos/wikipedia$ ./1-add-schema.sh
SUCCESS
Posted solrconfig.xml to http://ip-10-172-89-53.us-west-1.compute.internal:8983/solr/resource/wiki.solr/solrconfig.xml
SUCCESS
Posted schema.xml to http://ip-10-172-89-53.us-west-1.compute.internal:8983/solr/resource/wiki.solr/schema.xml


02880

Created index.

automaton@ip-10-172-89-53:/usr/share/dse-demos/wikipedia$ ./2-index.sh --wikifile wikipedia-sample.bz2
args: [--host, ip-10-172-89-53.us-west-1.compute.internal, --scheme, http, --wikifile, wikipedia-sample.bz2]
Start indexing wikipedia...
------------> config properties:
docs.file = wikipedia-sample.bz2
keep.image.only.docs = false
-------------------------------
13/10/21 21:30:25 INFO impl.HttpClientUtil: Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
Indexed 1000
Indexed 2000
Indexed 3000
Finished

Note: The above steps can also found in the “Running The Solr Demo” section of DataStax’s “Getting Started with Cassandra and DataStax Enterpise” documentation. Here is a link to that section of the docs for additional context: Running the Solr Demo

After having followed the steps above, we can verify that everything went well by having a look at the admin page for the wiki.solr core. Mine was available at the following URL. Yours will of course be different.

http://ec2-54-215-120-200.us-west-1.compute.amazonaws.com/:8983/solr/#/wiki.solr

From the screenshot below we can see that the wiki.solr core is now available and ready for us to use.

wiki.solr screen shot

So that’s it for the initial setup, let’s now turn our focus towards getting the headless version of SolrMeter up and running.

Patching SolrMeter

I’ve submitted the patch that I created to the SolrMeter project for discussion/review. You can obtain it directly from the issue tracker which is found here: https://code.google.com/p/solrmeter/issues/detail?id=94. Make sure to get the most recent version of the patch: solrmeter-R318-headless-v4.patch. It should apply cleanly to revision 318 of SolrMeter, like so:

[~/solrmeter-headless/solrmeter-read-only]
mallen-> ls
solrmeter-R318-headless-v4.patch sources

[~/solrmeter-headless/solrmeter-read-only]
mallen-> patch -p1 < solrmeter-R318-headless-v4.patch 
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/HeadlessModule.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/SolrMeterMain.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/SolrMeterRunModeModule.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/StatisticsModule.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/controller/StatisticDescriptor.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/model/SolrMeterConfiguration.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/model/statistic/HistogramQueryStatistic.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/model/statistic/QueryTimeHistoryStatistic.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/model/statistic/SimpleQueryStatistic.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/runMode/AbstractSolrMeterRunMode.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/runMode/SolrMeterRunMode.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/runMode/SolrMeterRunModeGUI.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/runMode/SolrMeterRunModeHeadless.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/view/HeadlessCommitConsolePanel.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/view/HeadlessConsoleFrame.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/view/HeadlessConsolePanel.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/view/HeadlessOptimizeConsolePanel.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/view/HeadlessQueryConsolePanel.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/view/HeadlessStatisticPanel.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/view/HeadlessStatisticsContainer.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/view/HeadlessUpdateConsolePanel.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/view/HeadlessUtils.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/view/QueryConsolePanel.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/view/statistic/HeadlessCacheHistoryPanel.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/view/statistic/HeadlessErrorLogPanel.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/view/statistic/HeadlessFullQueryStatisticPanel.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/view/statistic/HeadlessHistogramChartPanel.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/view/statistic/HeadlessOperationTimeLineChartPanel.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/view/statistic/HeadlessPieChartPanel.java
patching file sources/solrmeter/src/main/java/com/plugtree/solrmeter/view/statistic/HeadlessQueryTimeHistoryPanel.java
patching file sources/solrmeter/src/main/resources/StatisticDescriptorMapping.xml
patching file sources/solrmeter/src/main/resources/solrmeter.properties
patching file sources/solrmeter/src/main/resources/statistics-config.xml
patching file sources/solrmeter/src/test/java/com/plugtree/solrmeter/controller/StatisticsRepositoryTestCase.java
[~/solrmeter-headless/solrmeter-read-only]
mallen->

After successfully applying the patch, change into the “sources” directory and run “mvn package” like so:

[~/solrmeter-headless/solrmeter-read-only/sources]
mallen-> mvn package
...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Solrmeter parent POM .............................. SUCCESS [0.039s]
[INFO] StressTestScope ................................... SUCCESS [28.294s]
[INFO] SolrMeter ......................................... SUCCESS [54.670s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:23.204s
[INFO] Finished at: Mon Oct 21 16:08:32 MDT 2013
[INFO] Final Memory: 28M/175M
[INFO] ------------------------------------------------------------------------
[~/solrmeter-headless/solrmeter-read-only/sources]

That done, we can now run SolrMeter in GUI mode just as we could with the non-patched version:

[~/solrmeter-headless/solrmeter-read-only/sources]
mallen-> java -jar ./solrmeter/target/solrmeter-0.3.1-SNAPSHOT-jar-with-dependencies.jar

Solrmeter in GUI mode

With the patch applied and verification that SolrMeter starts up successfully in GUI mode, it’s time to get things setup to run in headless mode, which just takes a few more steps.

Setting up for Headless Operation

The first thing that we need to do to run in headless mode is get a valid SolrMeter configuration file. We can get that easily by running SolrMeter in GUI mode, editing the settings to point to the correct query and update URLs and to the correct query and update files, and then exporting the settings to a configuration file on the file system. We will use this config file later to start up in headless mode. Here are some screenshots of that process:

Query Settings
Solrmeter Query Settings

Update Settings
Solrmeter Update Settings

Saving the config
Solrmeter Saving Config File

You’ll notice in the screenshots above that we are using a file named queries.txt for the queries file and one named updates.txt for the updates file. Below are the contents of those two files. Note that headless mode does not require this setup. Any of the normal operating modes of SolrMeter will work in headless mode too. Fields, filter, and facet files can all be used in headless mode as well.

Contents of queries.txt and updates.txt

[~/solrmeter-headless]
mallen-> ls
my-solrmeter-config.smc.xml queries.txt  solrmeter-read-only  updates.txt

[~/solrmeter-headless]
mallen-> cat queries.txt 
Weather
"Space station"
"Train station"
Underground
"Fort Knox"

[~/solrmeter-headless]
mallen-> cat updates.txt 
id:update-number-1;name:"update number 1";title:"title for update number 1";date:"Oct 21, 2013"
id:update-number-2;name:"update number 2";title:"title for update number 2";date:"Oct 22, 2023"
id:update-number-3;name:"update number 3";title:"title for update number 3";date:"Oct 23, 2033"
id:update-number-4;name:"update number 4";title:"title for update number 4";date:"Oct 24, 2043"
id:update-number-5;name:"update number 5";title:"title for update number 5";date:"Oct 25, 2053"
[~/solrmeter-headless]
mallen->

As you can see, we are using some very simple queries and updates in this example for demonstration purposes only. In practice, we use a more involved setup to achieve more useful results. OK, we now have everything we need in place to run SolrMeter in headless mode. Next, we’ll see exactly how to do that.

Running SolrMeter in Headless Mode

Running SolrMeter in headless mode from the command line is easily done by specifying and setting values for two properties: solrmeter.runMode and solrmeter.configurationFile. We’ll set the latter to point to the configuration file that we created earlier. The former, we’ll set to “headless” which will cause SolrMeter to start up in headless mode. So let’s go ahead and start it up now and see what happens!

java -Dsolrmeter.runMode=headless \
     -Dsolrmeter.configurationFile=./my-solrmeter-config.smc.xml \
     -jar target./solrmeter/target/solrmeter-0.3.1-SNAPSHOT-jar-with-dependencies.jar

The startup process appears pretty normal. Usual text gets output to the console as follows:

mallen-> java -Dsolrmeter.runMode=headless -Dsolrmeter.configurationFile=./my-solrmeter-config.smc.xml -jar solrmeter-read-only/sources/solrmeter/target/solrmeter-0.3.1-SNAPSHOT-jar-with-dependencies.jar 
[main]| INFO|[boot]|20:45:45,881 | Adding plugins from ./plugins
[main]| WARN|[boot]|20:45:45,882 | No plugins directory found. No pluggin added
[main]| INFO|[com.plugtree.solrmeter.SolrMeterMain]|20:45:45,922 | Using module: com.plugtree.solrmeter.StatisticsModule
[main]| INFO|[org.exolab.castor.mapping.Mapping]|20:45:45,947 | Loading mapping descriptors from jar:file:/Users/mallen/solrmeter-headless/solrmeter-read-only/sources/solrmeter/target/solrmeter-0.3.1-SNAPSHOT-jar-with-dependencies.jar!/StatisticDescriptorMapping.xml
[main]| INFO|[com.plugtree.solrmeter.SolrMeterMain]|20:45:46,213 | Using module: com.plugtree.solrmeter.ModelModule
[main]| INFO|[com.plugtree.solrmeter.SolrMeterMain]|20:45:46,214 | Using module: com.plugtree.solrmeter.HeadlessModule
[main]| INFO|[com.plugtree.solrmeter.SolrMeterMain]|20:45:46,215 | Using module: com.plugtree.solrmeter.SolrMeterRunModeModule
[main]| INFO|[com.plugtree.solrmeter.model.SolrServerRegistry]|20:45:46,649 | Connecting to Solr: http://ec2-54-215-120-200.us-west-1.compute.amazonaws.com:8983/solr/wiki.solr
[main]| INFO|[boot]|20:45:46,830 | Adding Statistic histogram
[main]| INFO|[com.plugtree.solrmeter.view.I18n]|20:45:46,831 | Using Locale en_US
[main]| INFO|[boot]|20:45:46,833 | Adding Statistic timeRange
[main]| INFO|[boot]|20:45:46,835 | Adding Statistic queryHistory
[main]| INFO|[boot]|20:45:46,835 | Adding Statistic operationHistory
[main]| INFO|[boot]|20:45:46,836 | Adding Statistic fullQueryStatistic
[main]| INFO|[boot]|20:45:46,836 | Adding Statistic errorLog
[main]| INFO|[boot]|20:45:46,836 | Adding Statistic CacheStatistic
[main]| INFO|[boot]|20:45:46,836 | Adding Statistic QueryLogStatistic
[main]| INFO|[boot]|20:45:46,836 | Adding Statistic CommitHistoryStatistic
[main]| INFO|[boot]|20:45:46,836 | Adding Statistic SimpleOptimizeStatistic
[main]| INFO|[boot]|20:45:46,837 | Adding Statistic SimpleQueryStatistic
[main]| INFO|[boot]|20:45:46,837 | Adding Statistic OperationRateStatistic

Take note, however, that SolrMeter is now using a couple of new modules that it didn’t used to: SolrMeterRunModeModule and HeadlessModule. That’s your indication that SolrMeter has found your configuration file and is running correctly in headless mode. Well, that and you will surely notice that the GUI doesn’t appear on the screen!

By default SolrMeter will execute 100 queries and then exit. It should take about 5 minutes to complete with the default of 20 queries per minute.(Both of these settings are configurable so feel free to run 5,000+ queries at 1,000 per minute, for examaple) By default no updates will be performed and no commits or optimizes either.

OK, 5 minutes hence and the process has finished and returned us to the console. We have successfully run SolrMeter in headless mode. But wait! That doesn’t do us much good if we can’t see the results of the operations that have been performed like we could in GUI mode, right? Where are the results?

The headless output directory
Unless the headless.outputDirectory property has been changed in the configuration file, running SolrMeter in headless mode will result in a directory being created named solrmeter-headless in the current working directory. This is where you will find all the statistics that you are used to seeing in GUI mode. Let’s have a look in there now.

mallen-> find solrmeter-headless
solrmeter-headless
solrmeter-headless/directives
solrmeter-headless/query_console
solrmeter-headless/statistics
solrmeter-headless/statistics/error_log
solrmeter-headless/statistics/histogram
solrmeter-headless/statistics/operation_time_line
solrmeter-headless/statistics/pie_chart
solrmeter-headless/statistics/query_statistics
solrmeter-headless/statistics/query_time_history
[~/solrmeter-headless]

If you are familiar with SolrMeter, you will notice that the directory structure mostly mirrors that of the GUI component structure/layout. Most every component from the GUI version of SolrMeter has a counterpart in headless mode that maps conceptually as follows:

Main Window => solrmeter-headless
Query Console => solrmeter-headless/query_console
Query Statistics => solrmeter-headless/statistics/query_statistics

The information that you are used to seeing in GUI mode is available to you on the file system now in headless mode. The information in this directory refreshes continuously as operation results become available. You may have noticed that there is no update_console file. This is because no updates were performed. Let’s update our config file so that updates will be performed along with queries the next time we run SolrMeter in headless mode:

mallen-> cat my-solrmeter-config.smc.xml | grep headless | grep -v "guice"
<entry key="headless.performQueryOperations">true</entry>
<entry key="headless.numUpdates">100</entry>
<entry key="headless.outputDirectory">./solrmeter-headless</entry>
<entry key="headless.performUpdateOperations">false</entry>
<entry key="headless.numQueries">100</entry>
[~/solrmeter-headless]
mallen->

Above are the settings relevant to headless mode that you can adjust.

Edit my-solrmeter-config.smc.xml and change performUpdateOperations from “false” to “true”. Now, we run SolrMeter again in headless mode just like we did before. Another 5 minutes or so later, we inspect the solrmeter-headless directory again:

mallen-> find solrmeter-headless
solrmeter-headless
solrmeter-headless/commit_console
solrmeter-headless/directives
solrmeter-headless/query_console
solrmeter-headless/statistics
solrmeter-headless/statistics/error_log
solrmeter-headless/statistics/histogram
solrmeter-headless/statistics/operation_time_line
solrmeter-headless/statistics/pie_chart
solrmeter-headless/statistics/query_statistics
solrmeter-headless/statistics/query_time_history
solrmeter-headless/update_console
[~/solrmeter-headless]

Largely the same as before but this time you’ll notice that there are two new files present: update_console and commit_console. These two new files contain the results of those operations (updates and commits).

Everything seems to be operating as it should be. Let’s take a look now at the contents of some of the files in the output directory to see what information is available to us.

mallen-> cat query_console 
queries per minute:     20
total queries:  101
total query time:       59
total client time:      331542
total errors:   0
average query time:     0
average client time:    3282
intended queries per minute:    20
actual query rate:      36.0
[~/solrmeter-headless/solrmeter-headless]

mallen-> cat update_console 
added documents:        100
errors on update:       0
intended updates per minute:    20
update rate:    12.0
[~/solrmeter-headless/solrmeter-headless]

mallen-> cat commit_console 
not committed documents:        2
last commit:    10/21/13 9:23 PM
total commits:  29
errors on commit:       0

As you can see from above, the headless mode of operation attempts to make the same information that is available to you in GUI mode available to you in headless mode too. Excellent!

Final Words

Well, that about covers it, but there is just one last operation that we haven’t talked about yet. How do you perform an optimize in headless mode, like you do in GUI mode? In GUI mode, when you want to perform an optimize operation you click the “optimize now” button. In headless mode there are no buttons to push but we can still achieve our goal of ‘on-demand’ optimize operations as follows. When in headless mode, SolrMeter will listen for changes to the “directives” directory. This is our way of communicating with SolrMeter. When you want to perform an optimize operation in headless mode create a file in that directory named “optimize”. Example:

mallen-> touch directives/optimize

SolrMeter will then schedule on optimize operation to be performed and delete that file. You can repeat this process as often as you’d like to. Take note that after an optimize operation has been performed you will then see a new file created inside the solrmeter-headless directory named (you guessed it!) optimize_console, which is the counterpart of the “Optimze Console” panel from the GUI version.

I hope that you have found the information in this post useful and/or interesting. We have been using the headless mode of SolrMeter for a few months now with good results in our test harness here at DataStax. On a personal note, I enjoyed working on the patch for SolrMeter and I want to thank the members of that project for providing such a useful tool and a well designed system, which ultimately made my job easier!



Comments

  1. Matt Lieber says:

    Thanks Mike –
    It seems like as of now (Nov 2013), this patch is included OOTB in SolrMeter.
    Also of note: in addition I had to install an X11 package in order to get to the SolrMeter Java UI.

  2. Tomás says:

    Thanks for adding this feature. I just added a wiki page in SolrMeter referencing this blog for using the headless mode:
    http://code.google.com/p/solrmeter/wiki/HeadlessMode

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>