Accelerate Rewind: How Instagram Uses Cassandra to Operate on a Global Scale
DataStax
Interested in more talks like these? Register today for Accelerate: A NoSQL Original Series. We’d love to see you online!
When Instagram was introduced all the way back in 2010, the social network stored all of its data in Apache Cassandra™ in a single data center located in the United States. As the company began to scale, they added a second data center to the mix. A little while later, they added a third.
Every time they added a new data center, they expanded replication.
Fast-forward to today, and Instagram serves 1 billion daily active users all over the world.
But to accommodate all of those users while maintaining high performance, they needed to adjust their data placement strategy.
Cassandra on a global scale
At DataStax Accelerate 2019, Andrew Whang, an engineer at Facebook, explained how the company solved data placement challenges as it expanded.
“It became clear we needed to scale our hardware resources to support the increased workload,” Whang explained. “Ultimately, this meant more data centers in more countries.”
Sure enough, Instagram added more data centers in other countries. But they realized that their replication strategy fell short at this scale.
When expanding to multiple countries, Instagram ran into two challenges:
- Too many replicas: With three data centers, Instagram had three replicas. With five data centers, they had five replicas. This problem would only compound further over time.
- Decreased performance: Quorum requests involved response from two replicas, located nearby one another. Over time, quorum requests required three replicas, and when they came from other continents, those replicas were not close to each other. This suboptimal configuration hurt performance.
To solve these problems, Instagram decided to stop duplicating data everywhere. By only storing data closest to the region where it is generated, the company could build a faster, more efficient network that offered the performance benefits that come with local data access. After all, most users use Instagram from the same geographic region for the majority of the time.
By storing U.S. data in data centers based in the U.S. and European user data in data centers based in Europe, the company was able to achieve optimal performance.
Instagram also uses a tool developed by Facebook called Akkio to manage local data at scale, delivering stronger user experiences because of it.
To learn more about Instagram’s journey with Cassandra, how they overcame challenges, and the lessons they’ve learned from it, check out Andrew’s session from Accelerate 2019: Solving Optimal Data Placement for Instagram’s Global Scale.
