Building Microservices with a Cloud-Native, Scalable, and Highly Available Database
date: September 26, 2019
Modern application development dictates agility. According to O'Reilly's “State of Microservices Maturity” survey, over 50% of respondents stated that more than half of their new development is based on a microservices architecture, and 86% of respondents highlighted migration to microservices as at least partially successful.
Many enterprises are moving from monolithic applications to microservices. The main driver behind this move is not just to follow the latest trend, but rather to achieve high agility and expand application capabilities by decoupling and breaking down monolithic systems to independent units, aka microservices, which allow businesses to address new, rapidly changing requirements and roll out new changes faster, in a shorter cycle. By adopting cloud-native patterns and best practices, enterprises aim to achieve the next level of application development that’s dynamic in nature, distributed, and highly available by design, and can be seamlessly deployed to any cloud.
But what about databases; how does the database fit into the new cloud-native and microservices paradigm? Can it be highly available, distributed, and cloud native? Let’s explore.
Databases and Microservices
While microservices are stateless by design, they still need to be able to interact with a database to store or retrieve transactional data. However, in order to follow a microservices architecture pattern, each microservice’s persistent data must be private and no other microservice can access that data directly. There are several ways to achieve this: create a set of tables for each service, create a dedicated private schema, or create a dedicated database for each service. While private tables or schema have the lowest overhead, some high throughput services might benefit by having their own private database. In general, using a private schema or database per service is most appealing since it clearly defines ownership and provides encapsulation.
Some practical examples of microservices backed by a database per service may include ecommerce or customer 360 applications where separate microservices are responsible for customers, products, orders, and payments data management. Those microservices can reside across multiple data centers or cloud providers and still be able to provide instant access to customer order data or shopping carts. Additionally, they can address high availability with no latency, and be in compliance with local regulations by keeping data within a certain geography.
Traditional relational databases (RDBMS) are not really a fit for clouds, because they bring a lot of legacy pain points such as fixed schema, master-slave architecture, fragile replication, low scalability, and high latency. With unpredictable payloads and rapid changes in data and global deployment models in mind, it would be difficult to achieve key goals and have agile and highly available microservices with legacy database software.
Further, traditional databases can only scale vertically for both read and write operations, meaning that CPU, memory, and storage have to be added to an existing host machine to accommodate growth, and even then, it will eventually hit its limits. Horizontal scaling can be achieved through sharding and a master-slave architecture, however, it introduces additional complexity and increases chances for failure. Achieving high availability with a traditional database, again, is not so simple and is not future proof. Keeping replicas in sync, dealing with anticipated downtime when switching out a master node, having replicas that are read-only, and more makes for a management nightmare.
Only a modern, cloud-native database is fit to serve as database as a service (DBaaS) for a microservices architecture. DataStax Enterprise (DSE) is a modern, truly scalable, distributed and resilient database platform based on the best distribution of Apache Cassandra™ with a masterless architecture at its core. The Apache Cassandra project was born at Facebook, was open sourced in July 2008, and was built on Amazon’s Dynamo and Google’s Big Table, so it is designed to be a truly distributed database that’s capable of managing very large amounts of data across multiple commodity servers and provide high availability services without a single point of failure.
Cassandra is masterless by design, so there is no concept of a master node; all nodes in the cluster are equal, with the data being evenly distributed across the nodes by a consistent hashing algorithm. Continual availability and resilience is achieved through continuous data replication, which ensures fault tolerance and reliability in the cluster. For example, if the replication factor for the cluster is set to 3, then each node of the cluster will have two replicas of other nodes, and if one of the nodes goes down, the cluster will still be operational and the data preserved.
A transparent and configurable replication strategy allows a Cassandra cluster to span across servers, racks, and data centers. And if you want to translate it to a cloud-native language, the cluster can operate across compute, availability zones, or regions regardless of cloud providers.
This concept is perfectly aligned with modern application development patterns and microservices architectures where services are usually deployed across multiple availability zones, regions, and geographies to provide high availability and low latency to client applications.
Building a modern cloud-native, microservices-driven application with DataStax and Cassandra couldn't be easier given that the database can reside in the same availability zone or region and close to microservices, which reduces latency and traffic overhead. Data can be easily replicated across multiple geographies where other instances of microservices are deployed to provide a resilient, always available, and highly scalable DBaaS.
DataStax is committed to providing support to polyglot programming and offers a wide variety of database drivers for seamless integration. Moreover, DataStax provides multiple cloud deployment options from self-managed installation and Docker images to fully managed services through cloud marketplaces, DataStax Managed Services (DMS), and DataStax Constellation.
Solutions from DataStax have been proven to support large enterprise applications at global scale. With Cassandra and DataStax, enterprises are able to unlock the full potential of microservices by backing them with a highly available, scalable, distributed, and modern cloud-native database.