Using the OpsCenter API with Python

DataStax OpsCenter provides an outstanding browser-based user interface to manage and monitor DataStax Enterprise and Cassandra clusters. In addition to the browser-based UI, OpsCenter also has a full API that can be used with your clusters, and this API can easily be accessed using Python. With a few simple operations, you could develop many different websites or programs. In DataStax Test Engineering, we use this API extensively in our automated tests against OpsCenter.

What Can I Do Through the API?

Some of the operations that can be performed through the API include: administering users; configuration of clusters, nodes, and OpsCenter itself; performing operations on clusters and nodes (including start/stop/restart, rebalancing, repair, backup, and more); managing alerts, best practice rules, and backup schedules. The API can access the full range of performance metrics data available through the OpsCenter UI, including forecasts. It can even be used to provision existing nodes with DSE/DSC instances, including launching new AWS EC2 instances.

Using the API with Python

The code examples here are intended only to be an introduction; see the API documentation for full details on the API's capabilities. We're using the Python Requests library to handle HTTP interaction.^{[1]For these examples, I have two 2-node clusters (named 'pool1' and 'pool2') running DataStax Enterprise 4.5 connected with my OpsCenter 5.0.0 instance.}

As with most RESTful interfaces, OpsCenter API requests use the various HTTP request methods; the Requests library has separate functions for each. For example, when handling cluster configurations you could:

• retrieve cluster configuration information with the GET method
• update a cluster configuration with PUT
• add a cluster to OpsCenter management with POST, and
• remove a cluster from OpsCenter management with the DELETE method.

The API calls all return JSON data; the content of the returned JSON will depend on the request. The Requests library has a built-in JSON parser to turn the JSON result string into a Python dict. When needed, data is passed to PUT and POST requests in JSON format.^[2]

Queries

Simple Query Example: OpsCenter Version

What version of OpsCenter is running? To find this, we'll use the 'meta' request with the GET method. The request returns a JSON string with a number of meta-information values about the OpsCenter instance; the json() method of the Requests library gives us an easy-to-use dict of these values. You could use "pprint" to see all the other "meta" information this call returns.

# These statements apply to all following snippets as well.

import requests

import time

opsc_url = '10.0.0.1:8888'

meta = requests.get("http://{url}/meta".format(url=opsc_url)).json()

print("OpsCenter Version: {version}".format(version=meta['version']))

OpsCenter Version: 5.0.0

Retrieving Cluster Configurations and Node Properties

A slightly more involved example gets cluster configurations and node properties from OpsCenter.

clusterconf = requests.get("http://{url}/cluster-configs".format(url=opsc_url)).json()

for cluster in clusterconf.keys():

    print("\nJMX Port for cluster '{cluster_id}': {jmx_port}".format(cluster_id=cluster,

                                                                     jmx_port=clusterconf[cluster]['jmx']['port']))

    nodes = requests.get("http://{url}/{cluster_id}/nodes".format(url=opsc_url, cluster_id=cluster)).json()

    for idx, node in enumerate(nodes):

        print("\tIP for node {index}: {node_ip}".format(index=idx, node_ip=node['node_ip']))

JMX Port for cluster 'pool2': 7199
	IP for node 0: 10.0.10.121
	IP for node 1: 10.0.10.63

JMX Port for cluster 'pool1': 7199
	IP for node 0: 10.0.10.212
	IP for node 1: 10.0.10.80

This example first fetches information about all clusters being managed by OpsCenter from '/cluster-configs'. We use the response dictionary's keys to get a list of the Cluster IDs. This gives us various information about each cluster, and in turn we can get properties of each of the cluster's nodes with '/nodes'. (Node information could be retrieved directly if the node IPs were already known.)

Long-Running Action Example: Cluster Rolling Restart

In this example we are going to tell OpsCenter to perform rolling restart of all nodes in a cluster. This operation can, of course, take a long time to complete. Certain long-running requests like this are handled asynchronously, returning a Request ID that is used to check the status of the request. We could do any number of things while waiting such as updating a dynamic web page, broadcasting periodic notifications, or simply blocking in a wait loop until the request completes.

print("\nRestarting cluster {cluster_id}".format(cluster_id=cluster))

restart_reqid = requests.post("http://{url}/{cluster_id}/ops/restart".format(url=opsc_url, cluster_id=cluster),

                              data='{"sleep": 10, "drain_first": false}').json()

print("Restart Request Id: {req_id}".format(req_id=restart_reqid))

while True:

    restart_status = requests.get(

        "http://{url}/request/{req_id}/status".format(url=opsc_url, req_id=restart_reqid)).json()

    if restart_status['state'] != 'running':

        break

    time.sleep(2)

if restart_status['state'] == 'success':

    print("Cluster restarted.")

else:

    print("Cluster restart error?")

Restarting cluster pool1
Restart Request Id: 41e8946d-a647-4d64-a1c7-832e69b5977a
Cluster restarted.

This is a POST method call, and it takes some optional arguments in the "data=" parameter: 'sleep' is a number of seconds to sleep between node restarts, and 'drain_first' is a boolean to drain the nodes before restarting. (You could also pass a list of specific node IPs to restart.) The data passed to this API call must be in JSON format; in this example I use a simple string: '{"sleep": 10, "drain_first": false}'. You could also build a Python dict with the data, and use the standard library "json.dumps()" to convert it to JSON before the call.

In this example we sit in a simple 'wait loop' until the activity finishes, checking the response of the '/status' API call every two seconds. The content of the response depends on the API call, but it usually contains a very fine-grain detailed account; in the case of this rolling restart it gives the status and timing of each phase of the operation for each node involved. At the top level is the overall operation's 'state' element, which is one of three values: running, success, or error. Again, the exact details of the response can be inspected with a 'pprint' statement or in a debugger.

Conclusion

The OpsCenter API is a very flexible tool for programmatically monitoring and maintaining your clusters. And, as we have seen, it is also easy to use. I have demonstrated this using Python and the Requests library, but this can be done in any language capable of communicating over HTTP.

We have covered most of the actual mechanics used to access the API; beyond these simple examples, the majority of the complexity comes in knowing the specifics of your own needs. The API documentation shows the necessary details of what information you need to provide in order to perform sophisticated operations such as changing cluster configurations or node properties, provisioning existing clusters with the database software, or even launching new clusters in AWS.

[1] With appropriate modifications the Python standard 'urllib2' library could be used as well.
[2] An alternative way to work with the API is through "curl" or similar commands. The OpsCenterAPI documentation shows many examples of how to do this.

DISCLAIMER: Certain API requests that alter the parameters of your instances could potentially be harmful. Make sure you know what you are doing: read the documentation, and test thoroughly on a development system before using it in production. To simplify matters, I ignore exceptions and errors, and I'm not using any kind of security or authentication in these examples. When enabled, OpsCenter currently uses HTTP Basic Authentication, and the Requests methods readily accept authentication credentials.