Artem Chebotko

Welcome to Part 2 of our&nbsp;<a href="https://www.datastax.com/cassandra">Apache Cassandra</a>&reg; series. In&nbsp;<a href="https://www.datastax.com/blog/introduction-to-apache-cassandra-the-lamborghini-of-the-nosql-world">part 1</a>&nbsp;you read about the features of the powerful database Cassandra trusted by thousands of global enterprises. In this post, we&rsquo;ll show you how to build advanced data models on Cassandra so you can use them in successful applications.
Cassandra is a distributed NoSQL database well known for its big data scale and disaster tolerance capabilities. It offers great features like elastic linear scalability, a high level of performance, and the ability to handle millions of queries while keeping petabytes of data. This is why the vast majority of Fortune 100 companies use it.
But remember: this powerful capacity is a shared responsibility between developers and Cassandra. It&rsquo;s not enough to just launch a cluster. It&rsquo;s crucial to develop a proper data model to take advantage of all Cassandra has to offer. Otherwise, these powerful features will fall short.
In this post, we go beyond queries, partitions and tables to design an efficient data model for highly-loaded applications. We&rsquo;ll discuss the following:
<ul>
<li>How Cassandra organizes data</li>
<li>Data modeling methodologies</li>
<li>Three data modeling use-cases for IoT, investment portfolio, and order management</li>
<li>Work through hands-on scenarios to represent the data models on Cassandra</li>
</ul>
<h2>How Cassandra organizes data</h2>
In order to deliver successful and efficient applications, you need to know how Cassandra organizes data. This is essential to building a correct data model. The main components of Cassandra&rsquo;s data structure include:
<ul>
<li>Keyspace:&nbsp;A container of tables, similar to a PostgreSQL schema</li>
<li>Table:&nbsp;A set of columns, primary key, and rows storing data in partitions</li>
<li>Partition:&nbsp;A group of rows together with the same partition token (a base unit of access in Cassandra)</li>
<li>Row:&nbsp;A single, structured data item in a table</li>
</ul>
<img src="https://cdn.sanity.io/images/bbnkhnhl/production/70eeac603733b9ee0eb0e58686c7912fcd83dcd1-512x282.png" alt="Layout of Cassandra Data in Tables" width="512" height="282" />
Figure 1. Cassandra stores data in tables.
Let&rsquo;s look at each of the components in more detail.
<h3>Keyspace in Cassandra</h3>
<img src="https://cdn.sanity.io/images/bbnkhnhl/production/45ff011a909677b532cab8afaa2b86963560c1ef-512x163.png" alt="Code to Create a Keyspace " width="512" height="163" />
Figure 2. A keyspace.
<a href="https://dtsx.io/3K9lbcW">A keyspace</a>&nbsp;is a data container on Cassandra. It holds tables, materialized views, user-defined types, functions and aggregates. A Cassandra cluster typically has one keyspace per application.
A keyspace determines how data replicates on multiple nodes. The total number of replicas across the cluster is known as the replication factor.&nbsp;
In Figure 2, <code>3</code> and <code>5</code> are the replication factors for <code>DC-West</code> and <code>DC-East</code> respectively. Data replication is an innate feature of Cassandra to ensure reliability and fault tolerance.
When creating or modifying a keyspace, you need to specify a replication strategy that determines the nodes where replicas are placed. There are two kinds:
<ol>
<li>Simple strategy:&nbsp;This strategy should never be used in production. It is useful for a single data center and one rack for development and testing environments only. If you intend to have more than one data center, use the next strategy</li>
<li>Network topology strategy: This strategy is good for production or staging environments. It&rsquo;s network topology aware, so it understands your servers, server racks, and data centers. This strategy is preferred for most deployments because it&rsquo;s much easier to expand to multiple data centers when needed</li>
</ol>
<h3>Tables in Cassandra</h3>
Once you&rsquo;ve created a keyspace, you can&nbsp;<a href="https://dtsx.io/3K3pExR">create a table</a>. Every keyspace has multiple tables to store data. Each table consists of rows and columns.
<img src="https://cdn.sanity.io/images/bbnkhnhl/production/5fb5b45ac63264aaa42bb75ff0b64c816534f54c-512x213.png" alt="Code to Create a Table" width="512" height="213" />
Figure 3. A table in Cassandra.
To create a table, you must define a&nbsp;<a href="https://docs.datastax.com/en/dse/5.1/cql/cql/cql_using/useSimplePrimaryKeyConcept.html#useSimplePrimaryKeyConcept">primary key</a>. A primary key is a combination of&nbsp;<a href="https://docs.datastax.com/en/dse/5.1/cql/cql/cql_using/wherePK.html">partition key</a>&nbsp;and&nbsp;<a href="https://docs.datastax.com/en/dse/5.1/cql/cql/cql_using/whereClustering.html">clustering columns</a>&nbsp;(if any). In Figure 3 above, the <code>year</code> in this example is the partition key. There&rsquo;ll be as many partitions as there are distinct partition keys.
A partition key specifies which node will hold a particular table row. Clustering columns ensure data uniqueness and establish sorting order. Note that once you&rsquo;ve set a primary key for your table, it can&rsquo;t be changed.
<h3>Primary key and partitions in Cassandra</h3>
All data in Cassandra is organized with&nbsp;<a href="https://docs.datastax.com/en/dse/5.1/cql/cql/cql_using/wherePK.html">partition keys</a>. Without understanding partitions, there&rsquo;s little chance of building an efficient data model.
There are two kinds of partitions:
<ul>
<li>Single-row:&nbsp;there&rsquo;s only a partitioning key on a single column without clustering keys. Each partition can only store one row because the partition key uniquely identifies that row is a primary key</li>
<li>Multi-row:&nbsp;has both a partition key and a clustering key. This means you can have many different rows in a partition. Cassandra uses a static column to describe the whole partition, not individual rows</li>
</ul>
Depending on the number of users, you can have as many partitions as you want. It&rsquo;s a common misconception that a large number of partitions make your data model inefficient. Even&nbsp;billions&nbsp;of partitions won&rsquo;t affect Cassandra&rsquo;s performance. However, there are some limitations on the rows inside a partition. You usually don&rsquo;t want to have more than a hundred thousand rows in a single partition.
Once you set a partition key for your table, a partitioner transforms the value in the partition key to tokens (also called hashing). It then assigns every node with a range of data called a token range. Then, Cassandra automatically distributes each row of data across the cluster by the token value.
Watch this video to&nbsp;<a href="https://dtsx.io/3vEhqXV">learn more about Cassandra data structure</a>&nbsp;through an example.
<h2>Cassandra Query Language</h2>
All above is done using&nbsp;<a href="https://docs.datastax.com/en/dse/5.1/cql/cql/cqlAbout.html">Cassandra Query Language (CQL)</a>, which is the primary language for communicating with the database. There are two parts to CQL:
<ol>
<li>Data definition:&nbsp;includes language to create keyspaces, tables, indexes, custom secondary indexes, and materialized view</li>
<li>Data manipulation:&nbsp;includes language to run standard operations such as select, insert, update, and delete</li>
</ol>
<img src="https://cdn.sanity.io/images/bbnkhnhl/production/bdd6625e9958cbd455f6a8bbe939c297afaad9f3-512x286.png" alt="Sample CQL Query Code" width="512" height="286" />
Figure 4. Sample CQL Queries on Cassandra.
Figure 4 represents typical sample CQL queries on Cassandra. Everything works here because the queries match the clustering key (C) and partition keys (K).
<img src="https://cdn.sanity.io/images/bbnkhnhl/production/be0aafe915189b5ea253ab2299b23a5bca86627d-512x281.png" alt="Sample of an Invalid Query" width="512" height="281" />
Figure 5. Invalid CQL queries.
In Figure 5 you can see invalid CQL queries. The first two don&rsquo;t work because they only specify half of the partition key <code>venue</code> so Cassandra can&rsquo;t calculate the token. For the query to be valid, you need both <code>venue</code> and <code>year</code>.
The fourth query has both <code>venue</code> and <code>year</code> but still doesn&rsquo;t work. This is because title is a data column, and not part of the primary key. The fifth query has a similar problem. <code>Country</code> is a static field and you&rsquo;ll still need to calculate the partition using the partition key and clustering key.
To sum up, there are some important implications to keep in mind when working with data on Cassandra:
<ul>
<li>Primary keys define data uniqueness</li>
<li>Partition keys define data distribution</li>
<li>Partition keys affect partition sizes</li>
<li>Clustering keys define row ordering</li>
</ul>
When working with queries, consider the following:
<ul>
<li>Primary keys define how data is retrieved from tables</li>
<li>Partition keys allow equality predicates</li>
<li>Clustering keys allow inequality predicates and ordering</li>
<li>Only one table per query, no joins</li>
</ul>
These implications ensure that Cassandra can handle petabytes of data and answer your queries within milliseconds, while still being globally available with multiple data centers.
<h2>Cassandra data modeling methodology</h2>
<a href="https://docs.datastax.com/en/dse/5.1/cql/cql/ddl/dataModelingApproach.html">Data modeling on Cassandra</a>&nbsp;is a process to define and analyze data requirements. It lets you access data patterns needed to support a business process. It&rsquo;s more than just schema design. It&rsquo;s about:
<ul>
<li>Collecting and analyzing data requirements</li>
<li>Identifying participating entities and relationships</li>
<li>Identifying data access patterns</li>
<li>Organizing and structuring data in a particular way</li>
<li>Designing and specifying a database schema</li>
<li>Schema optimization and data indexing techniques</li>
</ul>
All of these points are essential. They will affect data quality (completeness, consistency, accuracy) as well as data access (queryability, efficiency, scalability). For a more detailed explanation, see our&nbsp;<a href="https://dtsx.io/3xPwCnE">YouTube Tutorial: Data Modeling Methodology</a>.
Moving on, there are four objectives of the Cassandra data modeling methodology:
<ul>
<li>Conceptual Data Model: understand the data</li>
<li>Application Workflow Model: identify access patterns</li>
<li>Logical Data Model: apply the query-first approach and design tables based on queries</li>
<li>Physical Data Model: optimize and implement</li>
</ul>
<img src="https://cdn.sanity.io/images/bbnkhnhl/production/2f763702dade0c5673d769c0c9c17cc2e7b5dcb5-512x276.png" alt="Diagram of Cassandra Data Modeling Methodology" width="512" height="276" />
Figure 6. Cassandra data modeling objectives and tools.
Each model has certain tools that support it, as illustrated in Figure 6. The transitions between the models are usually the most difficult parts of data modeling. We&rsquo;ll briefly touch on this below. If you want a deeper dive, we recommend taking one of our&nbsp;<a href="https://www.datastax.com/dev/academy">Cassandra courses on DataStax Academy</a>.
<h2>Data modeling use cases in Cassandra</h2>
Let&rsquo;s look at the data modeling process with a few Cassandra data modeling examples. We&rsquo;ll focus on&nbsp;<a href="https://www.datastax.com/learn/data-modeling-by-example/sensor-data-model">IoT sensor data modeling</a>&nbsp;in particular. If you&rsquo;re interested in other use cases, check out our&nbsp;<a href="https://www.datastax.com/learn/data-modeling-by-example/investment-data-model">investment portfolio data modeling</a>&nbsp;and our&nbsp;<a href="https://www.datastax.com/learn/data-modeling-by-example/order-management">order management data modeling</a>&nbsp;tutorials.
<h3>IoT sensor data modeling</h3>
<img src="https://cdn.sanity.io/images/bbnkhnhl/production/e7e40fa130af2c98098e87342352945221f6674f-512x288.jpg" alt="Diagram of Entity Relationships for Sensor Data Model" width="512" height="288" />
Figure 7. Entity-relationship diagram for IoT sensor data model.
The first example is sensor network or IoT data. It&rsquo;s similar to the data you would see for a smart home system. This is how you would model it on Cassandra:
<ol>
<li>Conceptual data model: this model features sensor networks, sensors, and temperature measurements. Each network has a unique name, description, region, and number of sensors. The sensor network can have many sensors, but each sensor can belong to only one network. Similarly, each sensor can take many temperature measurements, but each temperature measurement is taken by exactly one sensor</li>
</ol>
<ol>
<li>Application workflow: this workflow is designed with the goal of understanding data access patterns for a data-driven application. There are four data access patterns for a database to support:</li>
</ol>
<ul>
<li>Q1: Show all sensor networks</li>
<li>Q2: Display a heat map for a network</li>
<li>Q3: Display all sensors in a network</li>
<li>Q4: Show raw temperature values for a sensor</li>
</ul>
<ol>
<li>Logical data model: This model organizes data into Cassandra-specific data structures based on data access patterns. In this example, there are four tables designed to specifically support data access patterns Q1, Q2, Q3 and Q4, respectively:</li>
<ol>
<li>Networks</li>
<li>Temperatures_by_network</li>
<li>Sensors_by_network</li>
<li>Temperatures_by_sensor </li>
</ol>
<li>Physical data model: You can create a physical data model directly from a logical data model by analyzing and optimizing for performance. This model defines data types and determines if we need secondary indexes or materialized views. The most common type of analysis is identifying potentially large partitions. Common optimization techniques include&nbsp;</li>
<ol>
<li>Splitting and merging partitions</li>
<li>Data indexing</li>
<li>Data aggregation</li>
<li>Concurrent data access optimizations</li>
</ol>
</ol>
In this example, the model optimizes data retrieval by creating a new partition key &ldquo;bucket&rdquo;. It also limits the growth of the partition size of a table by introducing a new partition key &ldquo;week.&rdquo; This technique is known as packeting.
Watch our&nbsp;<a href="https://dtsx.io/38cOaiI">YouTube video</a>&nbsp;and visual representations of&nbsp;<a href="https://www.datastax.com/learn/data-modeling-by-example/sensor-data-model">sensor data modeling</a>&nbsp;for in-depth explanations. Then, run the&nbsp;<a href="https://www.datastax.com/learn/data-modeling-by-example/sensor-data-model">sensor data modeling scenario</a>&nbsp;to instantiate this data model on Cassandra.
The next two examples of data modeling deserve more time. So please watch this&nbsp;<a href="https://youtu.be/u6pKIrfJgkU?t=4210">video</a>&nbsp;or read all about them here:
<ul>
<li><a href="https://www.datastax.com/learn/data-modeling-by-example/investment-data-model">Investment portfolio data modeling</a></li>
<li><a href="https://www.datastax.com/learn/data-modeling-by-example/order-management">Order management data modeling</a></li>
</ul>
<h2>Conclusion</h2>
Now you have an overview of the data structure on Cassandra and the process to create advanced data models crucial to building successful, global applications.&nbsp;
Don&rsquo;t forget to check out&nbsp;<a href="https://www.datastax.com/blog/introduction-to-apache-cassandra-the-lamborghini-of-the-nosql-world">Part 1</a> for an introduction to Cassandra. In Part 3 and Part 4 we'll discuss benchmarking your database and Storage-Attached Indexes. Part 5 will explain how you can migrate your SQL applications to NoSQL, then Part 6 will walk you through an exercise to put it all into practice.
For more workshops on Cassandra, check out&nbsp;<a href="https://dtsx.io/3uRldBT">DataStax&rsquo;s Academy on GitHub</a>&nbsp;and our&nbsp;<a href="https://dtsx.io/3vEE9mw">DataStax Devs YouTube channel</a>&nbsp;where we regularly host live workshops. If you have any specific questions about Cassandra, join the&nbsp;<a href="https://community.datastax.com">DataStax Community</a>&nbsp;and get answers from the experts.
<h2>Resources</h2>
<ol>
<li><a href="https://dtsx.io/3vEhqXV">YouTube Tutorial: Advanced Data Modeling in Apache Cassandra</a></li>
<li><a href="https://dtsx.io/3v4LLzP">DataStax Academy on GitHub</a></li>
<li><a href="https://www.datastax.com/dev/academy">Cassandra Courses on DataStax Academy</a></li>
<li><a href="https://dtsx.io/3vBnmAR">Data Modeling in Cassandra and DataStax Astra DB</a></li>
<li><a href="https://www.datastax.com/learn/data-modeling-by-example">Data Modeling by Example</a></li>
<li><a href="https://www.dropbox.com/s/3cul3hqzr84bark/cassandra-data-modeling-methodology-presentation.pdf">Using the Chebotko Method to Design Sound and Scalable Data Models for Apache Cassandra</a></li>
<li><a href="https://www.dropbox.com/s/4bu0dy0ayrqygei/cassandra-data-modeling-methodology-paper.pdf">A Big Data Modeling Methodology for Apache Cassandra</a></li>
<li><a href="https://dtsx.io/3k2otEn">A Beginner&rsquo;s Guide to Benchmarking with NoSQLBench</a></li>
</ol>

Advanced Data Modeling on Apache Cassandra

Artem ChebotkoContributor

Discover more

Share

Share

How Cassandra organizes data

Keyspace in Cassandra

Tables in Cassandra

Primary key and partitions in Cassandra

Cassandra Query Language

Cassandra data modeling methodology

Data modeling use cases in Cassandra

IoT sensor data modeling

Conclusion

Resources

More Technology

How Winweb Built its AI Assistant with DataStax Astra DB and LangChain

Vercel + Astra DB: Get Data into Your GenAI Apps Fast

Simplifying Agent Development with Astra DB Connector for Vertex AI Search

Making Astra DB easier for MongoDB developers

One-stop Data API for Production GenAI