How to Connect Temporal.io to Astra DB in Just 5 Easy Steps with CQL-Proxy

Temporal.io is an open source microservice orchestration platform that assists in tracking workflows in your application development. It provides the user with a plug-and-play persistence layer that lets the user choose and configure their Temporal Server with their preferred backend. Currently, Temporal is compatible with Postgres, MySQL, CockroachDB and Apache CassandraⓇ as backend dependencies. 

In Part 1 of our blog series on using Temporal.io with Astra DB, we showed you how Cassandra is the ideal choice as a backend database for apps that need scale. In addition to offering high performance and reliability, Cassandra’s architecture makes it possible to scale horizontally to handle very high loads. Astra DB simplifies cloud-native Cassandra application development. When you combine Temporal with Astra DB, you get all the benefits of Cassandra with the ease of use that comes with a multi-cloud, serverless database-as-a-service (DBaaS). 

In this post, we’ll show you how to connect Temporal with Astra DB to easily manage workflows at scale. This runbook is going to utilize DataStax CQL-proxy which is a sidecar designed to forward your application’s CQL traffic to an appropriate database service. You may want to bookmark this one – it provides a runbook that can take you from start to finish in five steps. Let’s get started!   

Note that this runbook was written using Mac OS Monterey but it will also work with Windows. Any Windows-specific instructions will be noted as such.  

Step 1: Astra Prerequisites

Before connecting to Temporal, you need to first create an Astra database instance and gather all your credentials. If you don’t already have an Astra DB, you can sign up for a free account that will give you 80 GB monthly to work with. Creating an Astra account and spinning up your first database is really quick and easy following the instructions we provide here: 

1. Create an Astra Database
1. When asked for the keyspace name, name it temporal. 
2. Once your database is created, navigate back to your database dashboard and click Add Keyspace.
3. Name this second keyspace temporal_visibility
4. The status of your database will temporarily go to Maintenance but after a couple seconds you can refresh your screen and status should go back to Active.

Figure 1. Screenshot of Astra DB dashboard and where to add the keyspaces.

2. Create an Astra Token
   
   Tokens are required to authenticate against Astra with APIs or Drivers. These tokens can be used with multiple databases and can be configured to have specific permissions. In this example, you will create a token with Admin Role. Temporal uses this token to receive credentials and permission to access your database in a similar way to how Cassandra has a “user” and “password”, which we’ll discuss in more detail in Step 4 where you will configure the Persistence Layer in Temporal.
   e) When you create your tokens, download the CSV file to keep these credentials. 

3. Find your Database ID

Lastly, you want to get your Database ID. Find this by navigating to your Astra DB Dashboard.

Step 2. Temporal Pre-setup 

1. Clone this GitHub repository
2. Navigate to your cloned repository and using your preferred text editor (e.g. VisualStudio or Sublime), update the .env file with your Astra Token and Astra Database ID that you obtained above. 

ASTRA_TOKEN=<your Astra token>
ASTRA_DATABASE_ID=<your DB ID>

Step 3. Temporal schema migration to Astra DB

In this step, you will create and set up the keyspaces you created earlier in Astra earlier with temporal-cassandra-tool. This tool is part of the Temporal repo and it relies on the schema definition. 

1. Navigate to your cloned temporal-astra-cql-proxy directory
2. Run the following commands to initialize the keyspaces that we created through Astra. Note that there are two sets of commands, one for temporal keyspace and one for temporal_visibility keyspace:

docker-compose -f docker-compose-schema.yaml run temporal-admin-tools \
-ep cql-proxy -k temporal setup-schema -v 0.0
docker-compose -f docker-compose-schema.yaml run temporal-admin-tools \
-ep cql-proxy -k temporal update-schema -d schema/cassandra/temporal/versioned/

docker-compose -f docker-compose-schema.yaml run temporal-admin-tools \
-ep cql-proxy -k temporal_visibility setup-schema -v 0.0
docker-compose -f docker-compose-schema.yaml run temporal-admin-tools \
-ep cql-proxy -k temporal_visibility update-schema -d schema/cassandra/visibility/versioned/

 Once the process is completed, you should see a message similar to this:

2022-03-02T22:23:27.618Z INFO Validating connection to cassandra cluster. {"logging-call-at": "cqlclient.go:112"}

2022-03-02T22:42:53.526Z INFO Connection validation succeeded. {"logging-call-at": "cqlclient.go:118"}
2022-03-02T22:42:53.526Z INFO Starting schema setup {"config": {"SchemaFilePath":"","InitialVersion":"0.0","Overwrite":false,"DisableVersioning":false}, "logging-call-at": "setuptask.go:57"}
2022-03-02T22:42:53.526Z DEBUG Setting up version tables {"logging-call-at": "setuptask.go:67"}
2022-03-02T22:42:54.120Z DEBUG Current database schema version 1.6 is greater than initial schema version 0.0. Skip version upgrade {"logging-call-at": "setuptask.go:116"}
2022-03-02T22:42:54.120Z INFO Schema setup complete {"logging-call-at": "setuptask.go:131"}

Great! Your schemas have been migrated with Astra DB. You can double-check to make sure the correct tables have been created by querying your database in Astra DB’s CQL Console. 

(See troubleshooting section if you are seeing errors)

3. If you do a quick DESCRIBE tables; in both your temporal and temporal_visibility keyspaces, you should see there are tables loaded in that were created by the schema migration:

token@cqlsh:temporal> DESCRIBE tables;

history_node            tasks              cluster_metadata_info
cluster_membership      namespaces         cluster_metadata
schema_version          namespaces_by_id   schema_update_history
executions              queue_metadata  
queue                   history_tree

token@cqlsh:cadence> use temporal_visibility ;
tokencqlsh:temporal_visibility> desc tables;

open_executions schema_update_history schema_version closed_executions

Step 4. Run Docker Compose

In this step, the docker-compose.yaml file is already provided for you in the temporal-astra-cql-proxy repo. This file creates different docker containers to run Temporal server. The persistence layer is configured for you to connect with cql-proxy, and it should pull your Astra credentials from when you set it earlier:

services:
 cql-proxy:
   container_name: cqlproxy
   image: datastax/cql-proxy:v${CQL_PROXY_VERSION}
   ...
   environment:
     - ASTRA_TOKEN=${ASTRA_TOKEN}
     - ASTRA_DATABASE_ID=${ASTRA_DATABASE_ID}
     - HEALTH_CHECK=true

Now you can run the docker-compose command to start up Temporal:

docker-compose up

Give docker compose a couple minutes to complete start up. You can navigate to localhost:8088 on your browser to bring up the Temporal UI. A successful deployment of the Temporal UI should look like the following:

Step 5. Test and validate

Now it is time to test and validate your Temporal and Astra integration. You can test your connection and run some sample Temporal projects with these instructions. You can choose between both Go and Java samples to test and run workflows with Temporal and Astra DB.

Using Temporal’s Command Line tool, tctl, you will be able to interact with your local Temporal server. Use the following steps to set up a namespace where your workflows will appear. Temporal already creates a default namespace for you, however you can use the following to create a new one to organize your workflows. 

1. Create an alias for tctl:

% alias tctl="docker exec temporal-admin-tools tctl"

2. The following is an example of how to register a new namespace dedicated to certain workflows: test with one day of retention, -rd 1.

% tctl --ns test namespace register -rd 1
% Namespace test successfully registered.

Run the following queries in the Astra CQL Console to confirm that your namespace has successfully been created and appears in Astra. You should receive an output similar to this:

token@cqlsh:temporal> select * from namespaces where name='test' and namespaces_partition=0;

@ Row 1
----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------
namespaces_partition | 0
name                 | test
detail               | 0x0a2e0a2435323131643236322d316630662d343531622d383263622d36653732313739343432623210011a0474657374120c0a040880a3051a00200130011a100a066163746976651206616374697665
detail_encoding      | Proto3
id                   | 5211d262-1f0f-451b-82cb-6e72179442b2
is_global_namespace  | False
notification_version | 2

(1 rows)

3. Make sure to use tctl when using the sample apps. Also, keep in mind that you want to modify the starter and worker code so that it points to this specific Temporal deployment. For example:

c, err := client.NewClient(client.Options{HostPort: "127.0.0.1:7233", Namespace: "test"})

Once you get your sample project of choice up and running, you should be able to see your workflows reflect on both the Temporal UI and Astra UI (Figure 2).

Figure 2: Verifying that your workflows show up on both the Temporal UI and Astra UI.

1. Obtain your: 
1. namespace_id - by running tctl namespace list in your terminal 
2. workflow_id - from the Temporal UI 
3. run_id - from the Temporal UI

1. Run the following query to confirm that the workflow has reflected in Astra UI CQL Console (Keep in mind that you should only use ALLOW FILTERING in development and never in production. In this case, we are using ALLOW FILTERING to just confirm that the workflow has reflected into our Astra UI)

select * from executions where namespace_id=<NAMESPACE_ID> and  workflow_id=’<WORKFLOW_ID>’ and run_id=<RUN_ID>ALLOW FILTERING;

The output should look like this:

… 

 shard_id                      | 2
type                          | 1
namespace_id                  | 5211d262-1f0f-451b-82cb-6e72179442b2
workflow_id                   | hello_world_workflowID
run_id                        | 30000000-0000-f000-f000-000000000001
visibility_ts                 | 2000-01-01 00:00:00.000000+0000
task_id                       | -10
...

Congratulations! You have successfully integrated Temporal UI with Astra DB, executed a workflow, and confirmed that your results are present on both ends. 

Troubleshooting

The following are a few common errors that you may encounter when setting up Temporal with Astra.

1) Error during schema migration:

2022-07-19T21:55:22.853Z ERROR Unable to update CQL schema. {"error": "error reading current schema version:Failed to get current schema version from cassandra", "logging-call-at": "handler.go:82"}

If you receive the above error, double check your .env file and make sure that your credentials are correct. Is your token an Admin Token? Is your Database ID correct? (Note that these credentials don’t have to be in brackets or quotes). Once you have confirmed all of this, try doing the following before running the schema migration again:

• Deleting / Re-adding the temporal and temporal_visibility keyspace in the Astra UI
• Deleting / Re-adding the Docker container in your Docker desktop
• Restarting Docker desktop
• Re-run the scripts once again

Conclusion

If you need a microservice orchestration platform backed by a world-class database for high-volume workflows but don’t have the time to manage it all yourself, give Astra DB and Temporal a try. With this runbook and your free Astra DB account, you can see just how easy it can be to connect Temporal to Astra with all the power of Apache Cassandra and none of the complexities.