After a busy fall of travel doing advocacy at Strata NYC and several DataStax Developer Day events, I had the privilege of a month with no travel. I figured this was the perfect time to take on a development project I’d been thinking about for a while: creating an implementation of the KillrVideo reference application in Python — specifically, the microservices tier.
If you’re not familiar, KillrVideo is a reference application for learning Apache Cassandra and DataStax Enterprise created by Luke Tillman, Jon Haddad, and others. You can find the killrvideo-python repository on GitHub.
Why Python?
First of all, I thought I’d articulate some of my reasons for kicking off this project. Here they are quite simply:
- I’ve never done development on a deep level in Python — growth opportunity!
- Python is currently #3 on the TIOBE Index… and rising. The Python drivers are also among the top downloaded Cassandra / DSE Drivers. However, there aren’t nearly as many examples using these drivers as I would hope.
- I have secret plans to incorporate serverless capabilities into KillrVideo at some point. Well, maybe not so secret anymore. We could potentially use Python (or of course JavaScript) for this.
Python seems to have somewhat of a mixed reputation in the computing industry. On the one hand, it’s growing greatly in popularity, especially in data science and data engineering fields, I’m assuming because of its flexibility and ease of use. For many of us in the rapidly growing group of people who call ourselves software developers, it’s one of the first languages we encounter.
On the other hand, I know engineers who would look down on using Python in a production system for complaints such as performance. Personally, my number one struggle in learning Python (and JavaScript for that matter) is the dynamic typing, which is, of course, related to its being an interpreted language. I’ll confess I’ve found myself using lots of print() and type() calls to navigate type mismatches as I’ve worked on this project. My other big learning is that you should prototype things at the command prompt as much as possible.
Application Development Approach
I’ll freely admit that there’s one factor that greatly simplifies this application development effort: the availability of the service tier implementations in other languages such as Java, Node.js and C#. For this effort I’ve been leaning heavily on the great work done previously on the Java implementation by DuyHai Doan and David Gilardi.
So my development process has been to implement one service at a time, leveraging the same algorithms and queries used by the Java implementation. As I started working and reading the Java code, I figured that it would be really useful to create functional specifications for the services to make things easier for the next person to create an implementation in a new language. I’m using this to try to bait Adron Hall into doing a golang implementation ;).
By the way, I’m working on the services one at a time, in the order implied by the KillrVideo integration tests. Since most actions in the KillrVideo application are associated with a user, the first service to implement is the User Management Service, followed by the Video Catalog Service.
After 3 weeks of work on KillrVideo Python services, it finally occurred to me to run the Webapp, which of course uncovered a bug — no preview images!
More to come
Here are some of the aspects that I plan to cover in this series, without committing to any particular order:
- Using the DataStax Python Driver for Cassandra, Search, and Graph queries, including pros and cons of the cqlengine mapper
- Using GRPC in Python
- Service discovery and Etcd
- Leveraging Python libraries to get things done quickly — email address validation, stop word removal, etc.
- The KillrVideo integration test approach
- Implementing KillrVideo events via Kafka
- KillrVideo Docker deployment and the move to Kubernetes that Aleksandr Volochnev is working on.
Thanks for reading. I’m excited that Amanda Moran has agreed to collaborate with me on this project, and would love to have contributions from anyone that’s interested in helping out!
This article is cross-posted from Jeff's personal blog on Medium.
To hear about the development of this Python application in chronological order, read Part 2 of the KillrVideo Python series here.
