DataStax News: Astra Streaming now GA with new built-in support for Kafka and RabbitMQ. Read the press release.
Four Reasons We Enjoy Developing and Deploying on Google Cloud Platform
Choice is important when it comes to building an enterprise data architecture. It’s one of the reasons why we deploy DataStax Astra DB, a database-as-a-service built on the highly scalable, high-throughput NoSQL database Apache Cassandra®, on the three major cloud platforms across an aggregate 28 regions.
In some ways, the cloud has commoditized and standardized how we use and deploy infrastructure. Yet each cloud provider—and not just the big three—has its own strengths and produces its own pleasant surprises. We’ve learned a lot about the subtleties of each particular cloud by deploying Astra DB across all of them.
We’ve been developing our cloud service on Google Cloud Platform (GCP) for some time now; here, we’ll highlight four GCP features that stand out and just make our lives easier.
There's no debate. Kubernetes is the number one thing to be grateful to GCP for.
But wait, all cloud platforms offer Kubernetes, right? This isn’t a GCP specific thing!
Yes—and that's exactly the point. If you're old enough to remember when Amazon first announced a public compute cloud (and if you were as passionate about open source as I was, and am), it was revolutionary but also troubling. We had just arrived in a world where Linux was the free and open source operating system, and all your middleware, frontend, and even databases were open source. Here comes this revolutionary thing, infrastructure-as-a-service, and it wasn't open source at all!
Yet it was too compelling not to use it. The advantages of IaaS were too great, so even those of us who were conscious about things like single-vendor lock-in, lack of choice, monoculture—we still used the cloud, for fear of becoming irrelevant.
Of course the open source community did what we always do: we launched an open source alternative to the closed-source market leader: OpenStack. But this time, it didn't work. Sure, OpenStack is there and we even have an old private cloud running here at Datastax. But OpenStack did not take over the IaaS space like Linux did for operating systems. There could be many reasons why, but one reason is that IaaS is, by definition, more of a hardware business than a software one. No matter how much open source software you'd write, the economies of scale on the hardware side would be what drove users to choose one vendor over another one. The fact that software cost nothing was just a marginal advantage.
Eventually Kubernetes, launched and open sourced by Google, arrived to fill the need that OpenStack didn't. It enables us to build our DBaaS against a standard (and open source!) cloud platform. Even if the underlying actual infrastructure is neither free nor open source, Kubernetes fills the need we had for a "Linux for cloud."
In my long career with open source databases, I've also been part of an effort to create a DBaaS that had to build a separate backend for each cloud it wanted to support. That entailed a significant amount of overhead in terms of extra development time—with no meaningful benefit. When Kubernetes eventually became popular, we had to build yet another backend to support running on Kubernetes.
At Datastax, on the other hand, we were lucky with our timing; we built Astra DB to target Kubernetes from day one. And we're grateful for it!
The developer experience
There’s a lot of reasons why developers appreciate GCP over other public clouds. There are some well-known advantages: the opportunity to experiment with AI and machine learning big data, Google’s consistent willingness to experiment and innovate, and great documentation and explainers that range from basic concepts to more detailed tutorials, to name a few. At DataStax in particular, we’ve run into a handful of others, including:
- A great command line tool – The gcloud CLI enables every developer to quickly create disposable dev and test environments in the cloud.
- A simple authentication handshake – It’s easy to do on a laptop or cloud VM.
- A collaborative project structure – This really simplifies how developers work together on shared resources with easy-to-understand authorization.
- Flexible machine sizes – GCP is great about enabling developers to pick a dynamic amount of local SSDs and change the CPU-to-memory ratio to suit our workloads.
- Workable quotas – Quotas can be very frustrating to deal with at all cloud providers, but GCP does the best job of rolling up the information and showing where we’ll need to request an increase.
A browser-based cloud shell
This is almost embarrassing to admit, because of course we use GitOps, DevOps, and infrastructure-as-code. But in many situations, the browser-based cloud shell is an embarrassingly quick and easy way to log in to your cloud VMs. While this feature has recently been added also by some other cloud providers, GCP introduced it in 2015, and the GCP implementation is still superior in simplicity: just a click and seconds later you are logged into a shell on your VM.
I've been using Unix as long as I've been growing beards. I've used command-line SSH since it was invented by a fellow student in my university—and I still do. But increasingly when working on GCP I've found myself just clicking on that button that's sitting there right next to the IP address information I came there to copy paste. If all I wanted was to have a quick look (Okay, I’ll just say it: to fix a bug in production—with hobby projects only, of course!), then the Cloud Shell lets me access the VM and do what I need to do, right there, from the GCP console where my VM is listed (Figure 1).
Figure 1: Screenshot of the GCP Cloud Shell, which launches an SSH connectionright in your browser.
The sustained use discount
Is billing a feature? You might think it's not, but the reality is that we have people working full-time on managing cloud costs. There's even a small industry of consultants available to optimize your cloud bill. Understanding what details drive the total cost of your cloud use requires digging into detailed components, like disk usage, network use, NAT routers, inter-region versus intra-region traffic, and more.
The cost savings from such optimization projects can often be millions of dollars per year, which means they are definitely worth the effort. For a SaaS vendor like DataStax, the cost savings are not just a matter of avoiding waste in our own costs, but being disciplined about the cost structure of our own cloud platform. It’s a competitive advantage that directly affects our own ability to set the price on our products. This multiplies the value of cost optimization compared to being merely a cost savings task.
As is typical with most other optimization efforts, typically you start paying attention to optimizing cloud costs fairly late. All your initial energy is typically spent on deploying a working product and building the team. Once we finally got around to seriously thinking about our cloud costs, GCP provided us with a pleasant surprise: the platform had been providing us with a discount all the time—before we even asked!
There are typically two ways to get discounts when procuring software or IT infrastructure—and the same rules apply for IaaS: You get volume discounts, or you get discounts for committing to some level of consumption up front. On GCP this is known as a “committed use” discount. The discounts are significant (other cloud platforms offer this, too), and range from 50% to 70%.
Google also applies a volume discount to all users, automatically. This is known as a sustained use discount – and it’s not offered on other cloud platforms. The sustained use discount is applied every month, with the general idea that it's cheaper (on an hourly basis) to run an instance for 30 days compared to just running one day. Considering that we don't need to make any commitments at all, I think the discounts are quite generous: They can be up to 30%.
Sustained use discounts reward use cases that run resources continuously compared to just using a VM for a few hours or a couple days. At 20% to 30% it's a reasonably generous discount and something I would love to see in other clouds as well.
We develop and deploy DataStax Astra DB on the three major cloud platforms. While they all provide fairly similar "infrastructure services," there are still differences in the details of the user experience, contributions to open source, and even discounting.
For us, a team of cloud developers, GCP provides a smooth user experience (and a great discounting program!). We’re grateful for Google's investment in developing and sharing Kubernetes as an open source platform to develop cloud native applications in a standard way on any cloud. We are all in on Kubernetes, and it can't be overstated how much it simplifies our task of offering Astra DB in multiple clouds.
If you want to see for yourself how easy it is to work with Astra DB in GCP and the other major cloud platforms, or in multiple clouds, sign up for a free account today.