Support Forums : DataStax

DataStax Support Forums » Apache Cassandra

Why a 3rd column family when creating a custom index on Cassandra?

(6 posts) (2 voices)

Started 3 months ago by Janbalik
Latest reply from Janbalik
This topic is resolved

Tags:

No tags yet.

Janbalik
Member

As I always say, sorry for my English. I'm working on creating some manual indexes for some column families in Cassandra. I have read everything I could about this but I have found something I'm not able to understand properly.

In this presentation (http://www.slideshare.net/edanuff/indexing-in-cassandra), pages 36 to 45, done by Ed Anuff, I have seen his simple example for creating an index for a Users column family. He uses the 2 obvious CFs and another one to deal with concurrency. This third CF is "my problem". If I'm not wrong, Cassandra will always store the most recent value for each column. If this value is indexed, I have to update it in the Index CF (delete old index and create the new one), but why it is necessary the third CF? When I think about that and the concurrency, what my understanding says is: ok, many people updating a value which is indexed. It will mean a lot of work updating the index, but finally the last value will be in the Users CF and also in the Index CF, that's why there is a timestamp per column, so what's the matter with the concurrency? Even more, if the value can be updated only by one user (the owner of the data), there will be no concurrency...

I know I am a big ignorant in Cassandra affairs, but I don't see the reason behind the third CF. Ed Anuff explains that using this third column family you can restore the indexes to a consistent status (http://www.anuff.com/2010/07/secondary-indexes-in-cassandra.html), but, why are them going to fall into an inconsistent status? And, if this happens, the Users CF could be enough to restore the index, or am I wrong?

Please, could someone explain me this? What is/are my error/s?

Posted 3 months ago #
khahn
Member

Does the last paragraph (below) of this InfoQ article help: http://www.infoq.com/news/2011/07/cassandraindexing This article also contains a link to sample code: https://github.com/edanuff/CassandraIndexedCollections

"In general, he (Anuff) said that reading before writing can be an issue with Cassandra. Rather than doing locking (e.g., with ZooKeeper), Anuff presented a technique that uses three Column Families. For example, in a table with a users Column Family and an indexes Column Family, there will be a third Column Family Users_Index_Entries. Updates first read the previous index values from this column family to avoid concurrency issues and both it and Users use timestamped columns to avoid the need for locking. ..."

Posted 3 months ago #
Janbalik
Member

Hi Khahn,

Thanks for your answer. I had read the links you recommend. And I have read them again (a few times), but I still don't understand the problem. Probably it is obvious, but I don't see it. Why there could be a concurrency issue if I read from users CF instead of reading from Users_Index_Entries? Both CF must have the most recent value due to the timestamp which is part of each column. What is the difference? Following the process I don't see why the value from Users CF could be not aligned with the index CF if I use only these 2 CFs. Do you (or anyone) know what is the problem that could happen? Have any example of the issue?

Thank you very much!

Posted 3 months ago #
khahn
Member

Sorry the link didn't answer your question. I learned a little more after researching your question, but I'm afraid it's not a complete answer. I hope an expert gives you a better explanation soon.

The third column family is needed because simply reading the previous value from the users CF before updating it and then removing the index entry for that value from the Users_Index_Entries will not reliably work due to Cassandra's model of eventual consistency and lack of transactions. When creating your own index, a node might index data held by another node. Updates to the index and data are not atomic. To overcome these problems, you maintain a third column family to list previous values for the property of an item and use the list to remove these values from the index before adding the new value to the index.

Posted 3 months ago #
Janbalik
Member

Hi Khahn,

Don't say sorry friend ;) You are helping me! Thank you very much. I'm still tring to understand this. I think a key is this: "Updates to the index an data are not atomic". Another one is "a node might index data held by another node". This is aligned with what I thought: you can have an index to a data which does not exist or a data not indexed. But, why using a third with the values indexed we can avoid this problem? We could say updates to the index, data and values indexes are not atomic, I mean, from my understanding, there could be concurrency and consistency problems also between the third CF (values indexed) and data. Or am I wrong? If so, why? Can you (or anyone) explain me why I'm wrong?

Thanks!!

Posted 3 months ago #
Janbalik
Member

Since I think someone else could have the same doubt than me, I'm going to answer my own question with the things I have found:

As I supposed the main problem is the concurrency. If we assume that many users at the same time can be changing the same indexed value, as you have to read the index before updating, between the time you read a value and the time you update a value in the index another user could have changed again that value. As well as, from the moment the value is updated until you update the index the system could crash. Then, after a few concurrent changes the index could have old values that point to rows that have not that value.

By adding the third column family this process is safer but NOT 100% SAFE.

And a last thing: from my understanding, if there is no concurrency when updating the values, then there must be no problem. Let's supose you are indexing some user data. If only the owner of the data is allowed to modify the data, there is no concurrency at all. The unique risk is the system crashes before you finish the process to align the index with the value, but this operation is idempotent, so you can repeat it until success.

Hope this explain what I have understood and help others.

Posted 3 months ago #

RSS feed for this topic

Reply

You must log in to post.