Introduction to the DataStax Enterprise Capacity Service
A major iceberg that sits waiting to crash into many data-driven online applications is running out of capacity. Sometimes it happens during an initial app launch (e.g. the Obamacare website). But, most times it occurs later; either during periods of key business spikes like Christmas or when an app or service suddenly becomes extremely popular.
If there’s one phrase I’ve heard repeated more than any other by the founder of a successful business it’s: “I can’t believe how fast we grew!”
That being the case, the key question is: how can you ensure your business success doesn’t outgrow your database?
The answer is two part. First, you have to choose a database whose architecture future-proof’s your expected growth – one that allows easy online capacity additions like Cassandra.
Second, you need to practice capacity planning so you understand where your system has been, where it is today, and where it’s headed tomorrow.
Most operations staff understand this, but few actually practice the discipline of capacity planning because, well, it’s oftentimes not easy. You have to smartly collect metrics concerning your database infrastructure in a way that doesn’t cause much overhead. Then you need to be able to perform historical trend analysis so determinations can be made concerning causes and occurrences of system stress. Lastly, you have to be able to take this data and forecast into the future to know if and when your database will hit the wall.
Maybe this is why, when I talked to our customers about our new DataStax Management Services, one of the most requested services out of all we have on our drawing board was the Capacity service. The good news is, the Capacity service is available now in DataStax Enterprise and helps automate everything for you where the key tasks in capacity planning are concerned.
How the Capacity Service Works
The Capacity service works in a very straightforward manner. Our DataStax Agent is bundled with each server download, and works with our central management service in DataStax OpsCenter to collect key database and server metrics for capacity planning on each node in a DataStax Enterprise cluster in a manner that causes very little to no overhead.
Next, within OpsCenter you can perform historical trend analysis on any metric within a customizable timeframe so you can spot exactly when a node or cluster is being stressed and what the general historical trend looks like:
Finally, the forecasting capability allows you to predict future needs by selecting a date/time in the future and examining where the system is expected to go:
The Business Value of the Capacity Service
No longer do you have to be in the dark about questions like “When will my cluster hit 30TB?”, “When will this node run out of space?”, or “When will we need to add new capacity?” Instead, you can work proactively and understand when you’ll need to add additional capabilities, and put them in place far ahead of that time so your database can stand up under what’s coming in the future and your business will remain open for business.
The service also helps IT management out with budgeting activities, because they can know in advance when they’ll need to purchase new servers or add existing disks/RAM, etc., to existing nodes.