DataStax News: Astra Streaming now GA with new built-in support for Kafka and RabbitMQ. Read the press release.

Toggle Menu

The DataStax Blog

Easily Manage Workflows at Scale with Temporal.io and Astra DB

Managing your workflows has never been easier using open source Temporal.io as your microservices platform. With Temporal’s quick start installation, getting started is simple plug-and-play for anyone wanting to use it with their own personal workflows.

Currently, Temporal provides different configurations using different databases and dependencies, including Apache Cassandra. DataStax Astra DB, which is built on Cassandra, allows you to add a managed, cloud-native database backing your Temporal platform that will support your heavy workloads while providing high performance and reliability. 

In this post, the first of a two part series, we want to introduce you to Temporal and Astra DB and illustrate the advantages of using these two powerful technologies together to manage your workflows reliably and with less effort. 

What is Temporal?

Temporal is an open source, distributed and scalable workflow orchestration engine capable of running millions of workflows. Workflows can hold state and describe which activities (workflow tasks) should be carried out. 

Figure 1. Illustration of a Temporal Cluster, which consists of four independently scalable services. (Source: Temporal.io)

Illustration of a Temporal Cluster The Temporal “system” consists of a Temporal Server (either the Temporal Cloud service or self-hosted) orchestrating work with a fleet of Temporal workers (operated by application developers), and Temporal Clients (embedded in applications) over gRPC. 

These workflows are carried out by the Temporal (or self-hosted) server which consists of four independently scalable services: 

  • Frontend gateway (rate limiting, routing, authorizing) 
  • History subsystem to maintains data (mutable state, queues, and timers)
  • Matching subsystem to host task queues for dispatching
  • Worker service to handle the internal background workflows

Temporal Clients embedded within your app (route handlers or serverless functions) can start, cancel, signal, and query individual workflow executions. Activities are distributed using task queues and executed on worker nodes organized in a cluster. 

A Temporal cluster is the Temporal Server paired with a persistence layer (i.e. the data access layer). The workflow data is stored in its respective backend depending on what you choose to configure Temporal with. Supported databases include PostgreSQL, MySQL, CockroachDB, and Cassandra.

One aspect of the Temporal system is that it abstracts the complexity of a distributed system. Distributed systems are known to scale computation across multiple machines and handle the potential load of system changes. In theory, a distributed system facilitates a reliable and highly performant application. Application developers don’t have to worry about handling failures because that’s handled by the engine. However any failure that leaves the downstream part of the application waiting for a response can make things very complicated, especially at a large scale.

Temporal Persistence Layer 

Temporal supports Cassandra, MySQL, and PostgreSQL schemas and can be used as the server's database. The database stores the following types of data:

  • Tasks to be dispatched
  • The state of Workflow Executions
  • The mutable state of Workflow Executions
  • Event History, which provides an append-only log of Workflow Execution History Events
  • Namespace metadata for each Namespace in the Cluster
  • Visibility data, which enables operations like "show all running Workflow Executions"

Figure 2. Parts of a Temporal Cluster stored in a database (Source: Temporal.io).

Why Cassandra and Temporal work so well together

In addition to the fact that Temporal supports Cassandra schema, Temporal is write-intensive and can leverage the lightweight transactions (LWT) feature of Cassandra to provide significantly more database capacity. With Temporal, 50% of the writes are LWT making it ideal for heavily distributed applications that experience high load. The architecture of Cassandra makes it easily scaled horizontally to handle very high loads.

Cassandra provides unparalleled performance and reliability and is a highly scalable, fault-tolerant database. When you combine Temporal with Astra DB, you get all the benefits of Cassandra with the ease of use that comes with a multi-cloud, serverless database-as-a-service (DBaaS). 

Astra DB simplifies cloud-native Cassandra application development

Cassandra is the open-source NoSQL database behind some of the largest applications in the world, including Netflix and Instagram. Astra DB is built on Cassandra to simplify cloud-native Cassandra application development. Using Astra DB takes care of the heavy lifting often associated with using a powerhouse database like Cassandra, reducing deployment time from weeks to minutes. 

Astra DB creates the Cassandra database for you. Pairing Astra with your Temporal workflow will make carrying out your application development much easier, increase performance, and optimize your time. Astra also provides you with a convenient dashboard where you can visualize your database usage, health and other metrics.

Ready to dig in?

Astra DB makes leveraging the benefits of Temporal easy. In our next post in this series, we’ll show you how you can connect Temporal with Astra DB in just five easy steps:  

  1. Create your free Astra DB and gather your credentials.
  2. Install and deploy the Temporal Server with Docker and Helm charts.
  3. Migrate your Temporal schema to your Astra DB.
  4. Configure the Persistence Layer in the Temporal Server.
  5. Test and validate.

You don’t have to wait until our next post to get started. Sign up for your free Astra DB now and start exploring how it can benefit your application development. And, be sure to check back here to learn more and get the details for connecting your Astra DB with Temporal.

Authored by

Sign up for our Developer Newsletter

Get the latest articles on all things data delivered straight to your inbox.

Open-Source, Scale-Out, Cloud-Native NoSQL Database

Astra DB is scale-out NoSQL built on Apache Cassandra™. Handle any workload with zero downtime and zero lock-in at global scale.

Get Started For FreeSchedule Demo
Open-Source, Scale-Out, Cloud-Native NoSQL Database