Yelp has built a robust stream processing ecosystem called Data Pipeline. As part of this system we created a Cassandra Source Connector, which streams data updates made to Cassandra into Kafka in real time. We use Cassandra CDC and leverage the stateful stream processing of Apache Flink to produce a Kafka stream containing the full content of each modified row, as well as its previous value.
Speakers
Andrew Prudhomme
Distributed Systems Engineer at Yelp
Abrar Sheikh
Data Engineer at Yelp
You May Also Like:
Podcast
Securing Data Pipelines with Mike Loukides | Ep. 142 Distributed Data Show
Continuing our discussion with Mike Loukides of O'Reilly Media from last week's episode, we dive into data privacy and how when we know we've crossed the line of violating the…
Learn More
Blog
Accelerate Rewind: How Instagram Uses Cassandra to Operate on a Global Scale
Interested in more talks like these? Register today for Accelerate: A NoSQL Original Series. We’d love to see you online! When Instagram was introduced all the way back in 2010,…
Learn More
Blog
Leading with Code for a Better Apache Cassandra and Kubernetes
As a part of our ongoing commitment to open source and the Apache Cassandra™ community, DataStax is opening a version of a Kubernetes Operator for Cassandra. The code is freely…
Learn More