Artem Chebotko

<img src="https://www.datastax.com/sites/default/files/inline-images/data%20modeling%20image.png" alt="data modeling image" data-entity-type="file" data-entity-uuid="498c726d-c8c7-4326-9c32-d1a7d7689ba0" />
What does it take to build an efficient and sound data model for <a href="https://cassandra.apache.org">Apache Cassandra</a>&reg; and <a href="https://dtsx.io/3DFpMka">DataStax Astra DB</a>? Where would one start? Are there any data modeling rules to follow? Can it be done consistently time and time again? The answers to these and many other questions can be found in the Cassandra data modeling methodology.
In this post, we present a high-level overview of the data modeling methodology for Cassandra and Astra DB, and share over half a dozen complete data modeling examples from various real-life domains. We apply the methodology to create Cassandra and Astra DB data models for IoT, messaging data, digital library, investment portfolio, time series, shopping cart, and order management. We even provide our datasets and queries for you to try.
As a side note, if you are new to Cassandra or if the terms <a href="https://dtsx.io/3FL2GKO">single-row partitions</a> and <a href="https://dtsx.io/3FFWRyv">multi-row partitions</a> sound unfamiliar, we recommend taking a closer look at <a href="https://dtsx.io/30n7LJj">Cassandra Fundamentals</a> before deep diving into data modeling.
Data modeling and the methodology
Data modeling is a process that involves many activities:
<ul>
<li>Collecting and analyzing data requirements&nbsp;</li>
<li>Understanding domain entities and relationships&nbsp;</li>
<li>Identifying data access patterns&nbsp;</li>
<li>Organizing and structuring data in a particular way</li>
<li>Designing and specifying a database schema&nbsp;</li>
<li>Optimizing schema and data indexing techniques</li>
</ul>
Data modeling can have a profound effect on data quality and data access. For data quality, think about data completeness, consistency and accuracy. With respect to data access, think about queryability, efficiency and scalability. An efficient and sound data model is crucial for both data and applications.
Our methodology defines how the data modeling process can be carried out in a well-organized and repeatable fashion. In particular, the Cassandra data modeling methodology is based on four objectives, four models, and two transitions; along with specific modeling, visualization, mapping, and optimization techniques and methods.
<img src="https://www.datastax.com/sites/default/files/inline-images/data%20modeling%20table.png" alt="data modeling table" data-entity-type="file" data-entity-uuid="193916d1-080c-4e0a-adc9-e9849c12879a" />
Figure 1: Cassandra data modeling methodology.
Four objectives
The Cassandra data modeling process, when discussed at a high level, can be distilled into these four key objectives:
<ol>
<li>Understand the data: Whether starting from scratch or dealing with an existing dataset, do you understand data that needs to be managed? Things like entities, relationships, and key constraints come to mind.&nbsp;</li>
<li>Identify data access patterns: Do you have a good idea of what a data-driven application should be able to do? Think of tasks (or microservices) and their required data access patterns, execution sequences and workflows, and how data retrieved in one task is used by the next one.</li>
<li>Apply the query-first approach: Do you know how to design Cassandra tables to support specific queries? It is called a query-first or query-driven approach because designing table schemas depends on both data and queries.&nbsp;</li>
<li>Optimize and implement: How do you verify that both database tables and application queries are efficient and scalable? For example, large partitions and queries that access many partitions may require additional optimizations.&nbsp;</li>
</ol>
Four models
The four models directly correspond to the four objectives and are meant to make the process more concrete, manageable, repeatable, documentable, collaborative, and shareable. They are:
<ol>
<li>Conceptual data model: A technology-independent, high-level view of data. Its purpose is to understand the data in a particular domain. While there are a number of conceptual data modeling techniques, we use the Entity-Relationship Model and Entity-Relationship Diagrams in Chen&rsquo;s Notation to document entity types, relationship types, attribute types, and cardinality and key constraints.</li>
<li>Application workflow model: A technology-independent, high-level view of a data-driven application, consisting of application tasks, execution dependencies, and access patterns. Its purpose is to identify data access patterns and how they may be executed in sequences. These include queries, inserts, updates and deletes required by different data-driven tasks. We use simple graph-like diagrams to represent application workflows.</li>
<li>Logical data model: A Cassandra-specific data model featuring tables, materialized views, secondary indexes, user-defined types and other database schema constructs. It is derived from a conceptual data model by organizing data into Cassandra-specific data structures based on data access patterns identified by an application workflow. This is where the query-first approach is applied. Logical data models can be conveniently captured and visualized using Chebotko Diagrams that can feature tables, materialized views, indexes and so forth.</li>
<li>Physical data model: A Cassandra-specific data model that is directly derived from a logical data model by analyzing and optimizing for performance. Physical data models can be conveniently captured and visualized using Chebotko Diagrams and implemented in Cassandra using CQL.</li>
</ol>
Two transitions
To complete the picture, the methodology must define the transitions between the models:
<ul>
<li>Mapping a conceptual data model and an application workflow model to a logical data model</li>
<li>Optimizing a logical data model to produce a physical data model</li>
</ul>
In many aspects, the transitions are the most interesting and profound components of the methodology. To carry out the first transition, the methodology defines mapping rules and mapping patterns. For the second transition, some common optimization techniques include splitting and merging partitions, data indexing, data aggregation, and concurrent data access optimizations.
You can find more information about the Cassandra data modeling methodology in the <a href="https://www.dropbox.com/s/4bu0dy0ayrqygei/cassandra-data-modeling-methodology-paper.pdf">original paper</a>, <a href="https://www.dropbox.com/s/3cul3hqzr84bark/cassandra-data-modeling-methodology-presentation.pdf">conference presentation</a>, or <a href="https://dtsx.io/3iWLIzz">DataStax Academy video course DS220</a>.
Data modeling in action
One of the best ways to become skilled in data modeling is to explore concrete examples. We maintain <a href="https://dtsx.io/3mRP9IO">this growing collection of data modeling examples</a> from various domains to help you get started with Cassandra and Astra DB data modeling. Each example applies the Cassandra data modeling methodology to produce and visualize four important artifacts: conceptual data model, application workflow model, logical data model and physical data model.&nbsp;
Moreover, each example has a hands-on portion with practice questions and solutions. The hands-on scenarios make it straightforward to implement a data model in Cassandra, express data access patterns as CQL queries and run the queries against our sample datasets.
<img src="https://www.datastax.com/sites/default/files/inline-images/data%20modeling%20investment%20portfolio.png" alt="data modeling investment portfolio" data-entity-type="file" data-entity-uuid="4b491f84-163c-47e1-a895-e0f47c7c5429" /> &nbsp;
Figure 2: Example hands-on scenario with schema, data and queries.&nbsp;
Go ahead and explore these data models, and execute real queries against them in your browser:
<ul>
<li><a href="https://dtsx.io/30hFV0V">Sensor data model</a>: Modeling sensor networks, sensors, and temperature measurements. The resulting database schema has four tables supporting four data access patterns.</li>
<li><a href="https://dtsx.io/3mNYZLN">Messaging data model</a>: Modeling users, email folders, emails and email attachments. The resulting database schema has five tables supporting four data access patterns.</li>
<li><a href="https://dtsx.io/3lBqR6v">Digital library data model</a>: Modeling performers, albums, album tracks and users. The resulting database schema has eight tables supporting nine data access patterns.</li>
<li><a href="https://dtsx.io/3lE82jj">Investment portfolio data model</a>: Modeling users, investment accounts, trades and trading instruments. The resulting database schema has six tables supporting seven data access patterns.</li>
<li><a href="https://dtsx.io/3ADnvnY">Time series data model</a>: Modeling IoT data sources, groups of related sources, metrics, data points, and time series with higher or lower resolution. The resulting database schema has seven tables supporting seven data access patterns.</li>
<li><a href="https://dtsx.io/2YQwsNF">Shopping cart data model</a>: Modeling users, items and shopping carts. The resulting database schema has three tables and one materialized view supporting seven data access patterns, including updates that use batches and lightweight transactions.</li>
<li><a href="https://dtsx.io/3oU4w6l">Order management data model</a>: Modeling users, payment methods, addresses, items, shopping carts, orders, delivery options, and order statuses. The resulting database schema has four tables supporting five data access patterns, including multi-step updates that use lightweight transactions.</li>
</ul>
Data modeling and Astra DB
<a href="https://dtsx.io/3DFpMka">Astra DB</a> is a cloud database service built on Apache Cassandra. It is a serverless and multi-region service that works in AWS, Azure and GCP. If you haven't already, you should take advantage of <a href="https://dtsx.io/3lABoyM">Astra DB&rsquo;s free tier</a> to create your own fully managed Cassandra database in the cloud. After all, how many multi-cloud, multi-region, serverless databases built on open-source technologies do you know? Astra DB is the first one.
Astra DB databases are Cassandra databases. The same data modeling methodology applies and the above example data models can be instantiated in Astra DB. However, there are a couple of minor differences that you may want to be aware of:&nbsp;
<ul>
<li>Astra DB does not support materialized views. <a href="https://dtsx.io/3DGsxC7">Materialized views</a> are experimental in Cassandra and the use of regular tables is usually recommended instead.&nbsp;</li>
<li>Astra DB does not support user-defined functions. Strictly speaking, user-defined functions are not data modeling constructs. They usually can be readily replaced with computation outside of a database.</li>
<li>Astra DB supports Storage-Attached Indexing or SAI. <a href="https://dtsx.io/3lzQ9lp">Storage-attached indexes</a> in Astra DB are secondary indexes with better performance, space efficiency and more capabilities than <a href="https://dtsx.io/30dDP24">regular secondary indexes or experimental SASI</a> in Cassandra. With that said, it is important to understand that SAI and other secondary indexes still have the same <a href="https://dtsx.io/30dDP24">use cases and limitations</a>, and should be used with caution.&nbsp;</li>
</ul>
The Astra DB and Cassandra differences with respect to materialized views, user-defined functions, and secondary indexes should not have any profound effect on data modeling.&nbsp;
Data modeling and K8ssandra
<a href="https://dtsx.io/3oUzIm1">K8ssandra</a> is a cloud-native distribution of Cassandra that runs on Kubernetes. Besides Cassandra, the distribution also includes several integrated components that enable richer data APIs, and provide better automation for observability, metrics monitoring, backup and restore, and data anti-entropy services.&nbsp;
K8ssandra is open-source, free to use, and data modeling in K8ssandra is identical to data modeling in Cassandra. &nbsp;
Data modeling and Stargate
<a href="https://dtsx.io/3AF9cPH">Stargate</a> is an open-source data gateway deployed between applications and a database. It supports different API options for an application to interact with Cassandra, Astra DB and K8ssandra. Stargate&rsquo;s API extensions include CQL, REST, GraphQL, and Document APIs. &nbsp;
The use of CQL, REST and GraphQL APIs has no effect on data modeling: the same data modeling methodology applies.
The use of Document API has a significant impact on data modeling. With Document API, the focus shifts from organizing data as rows, columns and partitions to structuring data as JSON documents. Stargate then uses the <a href="https://dtsx.io/3aDCMuc">predefined mapping</a> to shred JSON documents and store them as rows in Cassandra tables. The topic of data modeling for document databases is beyond the scope of this article. &nbsp;
Conclusion
Data modeling in Cassandra and Astra DB is a very important topic and we just scratched the surface in this post. We presented a high-level overview of the Cassandra data modeling methodology and urged you to sharpen your skills by exploring the <a href="https://dtsx.io/3mRP9IO">data modeling examples</a>. We also established that data modeling in Cassandra, Astra DB, and K8ssandra are practically identical; with <a href="https://dtsx.io/3lABoyM">Astra DB</a> having a significant advantage of being serverless and fully managed. Finally, we briefly discussed how Stargate APIs &mdash; namely CQL, REST, GraphQL, and Document APIs &mdash; can affect data modeling.&nbsp;
Explore <a href="https://dtsx.io/3iWLIzz">DataStax Academy</a> to get certified in Apache Cassandra with hands-on courses. You can also <a href="https://dtsx.io/3awm1Bl">subscribe to our event alert</a> to get notified about our latest developer workshops. Lastly, follow <a href="https://dtsx.io/3mSlW0o">DataStax on Medium</a> for exclusive posts on all things Cassandra, streaming, Kubernetes, and more.
Resources
<ol>
<li><a href="https://cassandra.apache.org">Apache Cassandra</a></li>
<li><a href="https://dtsx.io/3DFpMka">Astra DB</a>&nbsp;</li>
<li><a href="https://dtsx.io/3oUzIm1">K8ssandra</a>&nbsp;</li>
<li><a href="https://dtsx.io/3AF9cPH">Stargate</a>&nbsp;</li>
<li><a href="https://dtsx.io/30n7LJj">Cassandra Fundamentals</a>&nbsp;</li>
<li><a href="https://dtsx.io/3mRP9IO">Data Modeling by Example</a></li>
<li><a href="https://www.dropbox.com/s/4bu0dy0ayrqygei/cassandra-data-modeling-methodology-paper.pdf">A Big Data Modeling Methodology for Apache Cassandra</a></li>
<li><a href="https://dtsx.io/3iWLIzz">DataStax Academy DS220: Data Modeling with Apache Cassandra</a>&nbsp;</li>
<li><a href="https://www.dropbox.com/s/3cul3hqzr84bark/cassandra-data-modeling-methodology-presentation.pdf">Using the Chebotko Method to Design Sound and Scalable Data Models for Apache Cassandra</a></li>
</ol>

Build and scale cloud native apps

Discover the benefits of DBaaS and why your apps deserve an upgrade.

Learn about NoSQL Databases

Learn about NoSQL databases with Apache Cassandra and Astra DB.

2022 Recap: A Year for Customers, Community, and Real-Time Data

Highlights from 2022 and a glimpse into the year ahead.

ViewingTechnical How To’s

Data Modeling in Cassandra and Astra DB

Sign up for our Developer Newsletter