This is an excerpt from the DataStax whitepaper Moving to a Modern Architecture, which delves into the eight key ways to prepare for and manage a modern data architecture. Click here to download the full whitepaper.
The database world is changing fast, and the coming decade—the 2020s—will mean even more change. Open source solutions and frameworks have become the norm. More services and data now reside in the cloud. Today’s enterprises are looking to data managers to be able to respond to business challenges with scalable and responsive systems that deliver both structured and unstructured data—and accompanying insights—at a moment’s notice, with the ability to respond to any and all queries. What’s needed is a modern data architecture that is built on flexible, modular technology, either from open source frameworks and software or through cloud services. Here are eight key ways to prepare for and manage a modern data architecture:
Focus First and Foremost on Customer Experience
All data management initiatives should ultimately focus on one thing: serving the customer or end user. In today’s digital economy, competitive advantage comes from delivering a superior customer experience, and a superior experience is only possible with enough of the right data at the right time. Data architects need to work with business owners to determine customer requirements, and work backward from there.
Ensure Data Portability
The goal of a modern data architecture is to abstract all underlying infrastructure—servers, storage, and networks—from the applications and data that are being used. Thus, data should be capable of being moved expediently between platforms with little-to-no disruption to the end user. This may be from an on-premise system to the cloud, from cloud to cloud, or from the cloud back to an on-premise environment. While cloud platforms offer many compelling advantages to data managers and developers, they also pose the risk of vendor lock-in by potentially making it too costly and messy to attempt to switch providers. It may not even be possible to extract databases from cloud providers. That’s why data portability needs to be an element of a modern data architecture—data and applications will inevitably be moved to the platform that makes the most sense to the business. Just as important is enabling data analytics and storage anywhere it is requested. Gone are the days when data was stored in a central data center in storage arrays, to be made accessible to end users within the walls of the enterprise. Today’s users may be accessing data and applications from anywhere in the world.
Build a Data Service Layer
Fundamental to data architecture success is a data infrastructure that is agile and flexible. Data and insights should be available to end users at a moment’s notice. A modern data architecture should support a highly networked business, marked by a continuous flow of information across all boundaries and organizational walls. Establishing a standardized interface that can access all applications and data brought into the infrastructure is the essence of a modern data architecture. This helps to meet the goals of data portability, mentioned earlier, while enabling end users to quickly gain access to information from all parts of their enterprises. Data will reside in many places in the years ahead—in on-premise legacy systems, in private clouds, and in public clouds. The experience in using this data needs to be the same, with the underlying platform virtually invisible to the end user. In addition, IT departments should be able to deliver services through an interface that is as easy to use as public a cloud service.
Reach Out to the Edges
A concept that is gaining traction in enterprises is “fog computing,” in which access, storage, and processing can take place anywhere between a centralized server and edge device. This is critical, as much of the information that will drive businesses—operational issues, product usage, and customer trends—will come from devices and sensors situated at the edges of enterprises. Sensors in production systems may be streaming time series data on the health of the system, for example. A modern data architecture needs to support edge computing, as well as streaming data.
Be Highly Available
A modern data architecture needs to ensure that data and applications are available to end users at all times, even if there are disruptions or outages to the system. This is accomplished through a well-defined resiliency strategy that, in the past, typically required a secondary data center, but today may involve cloud resources. Disaster recovery, failover, load balancing, backup, and data restoration processes need to be built into architectural planning. This requires greater planning than was necessary in the simpler days of data mirroring or replication between primary and hot standby systems. Today’s data environments may be dependent on various cloud services, in conjunction with on-premise environments. While cloud providers are very adept at maintaining highly resilient systems, data architects need to look at the whole picture of how data is stored and used across their enterprises.
There are many database tasks and operations that take up inordinate amounts of staff time that could be applied to higher-level tasks, such as backups, patching, security, and anomaly detection. A modern data architecture incorporates automation of such tasks. AI and machine learning can play important roles in increasing the ability of databases to manage themselves.
Promote Collaboration in Design and Implementation
Importantly, a modern data architecture is not solely the domain of the IT department. It is a set of services and capabilities that are built and used by just about every department across the organization. A modern data architecture needs to be closely tied to the needs of the business, and needs to have the flexibility to adapt to and meet business changes. While data managers and IT departments will continue to oversee infrastructural and security matters, their business partners in their organizations will help set the agenda for the types of analytics delivered, the speed of delivery, and the capabilities offered.
Ensure a Highly Secure and Compliant Infrastructure
Cybersecurity needs to be baked into a modern data architecture right from the beginning. The goal of a modern data architecture should be to reduce the potential attack surface as much as possible. In addition, as data is subject to many regulations and mandates, the data architecture needs to build in checks and balances, as well as transparency to ensure that data is well-maintained and secured. A modern data architecture is a living, breathing entity that grows and adapts as the enterprise itself evolves. Today’s enterprises run on data, and their success depends on how well they leverage their data assets.
Thanks for reading this excerpt from the DataStax whitepaper Moving to a Modern Architecture, tune in next week when we'll release another excerpt or click here to download the full asset.