Latency in reading data from cassandra

(12 posts) (3 voices)

• Started 9 months ago by Srini
• Latest reply from Srini
• This topic is not resolved

Tags:

• african

1. 

   Srini
   Member

   Hi,

   I have 2 DC cassandra setup. Dc1 has nodes a&b. Dc2 has nodes c&d. My keyspace uses NTS and my strategy option is dc1:2,dc2:2. I did not specify consistency so it must be Quorum.

   Initial token:
   A 0
   B 85070591730234615865843651857942052864
   C 1
   D 85070591730234615865843651857942052865

   I have configured A&C as seed nodes in all the 4 Cassandra.yaml.

   i have a script that puts data into cluster and get the data back. This script gets executed for ever 5sec. The script is configured in such a way that it can put/get from any of the 4 nodes.

   I see there are too many misses in reading.. For 400 req fired. I see at least 50 read misses...

   i could understand that the Cassandra writes are faster than reads. But there should be a way to over come. Can someone suggest me ?

   Posted 9 months ago #
2. 

   thobbs
   Administrator

   The default is not QUORUM on all clients; the default for a few of them, at least, is ONE. I suggest explicitly setting your CL to QUORUM and testing again.

   Posted 9 months ago #
3. 

   Srini
   Member

   Thanks for your quick reply.

   I have few doubts.

   1) As i am trying multiple clusters, should i set consistency to LOCAL_QUORUM ?

   2) Is it fine to set only Reads to QUORUM and dont worry about Writes ?

   3) How do i do i set consistency level ?

   i see examples for setting it during client requests. something like

   "SELECT * FROM users WHERE state='TX' USING CONSISTENCY QUORUM;"

   i am trying out HTTP Session management. I use 3rd party jars which does these Cassandra inserts/updates etc. i am trying out from below link.

   http://code.google.com/a/apache-extras.org/p/tomcat-cassandra/

   Posted 9 months ago #
4. 

   Srini
   Member

   My questions is, is it possible to set Consistency level during keyspace or column creation ?

   If yes, how ?

   Posted 9 months ago #
5. 

   Srini
   Member

   Can someone help me.. i am stuck up with this..

   Posted 9 months ago #
6. 

   Srini
   Member

   I tried setting the consistency at the hector client by using the below code snippet

   Create a customized Consistency Level
   ConfigurableConsistencyLevel configurableConsistencyLevel = new ConfigurableConsistencyLevel();
   Map<String, HConsistencyLevel> clmap = new HashMap<String, HConsistencyLevel>();

   clmap.put("MyColumnFamily", HConsistencyLevel.LOCAL_QUORUM);

   configurableConsistencyLevel.setReadCfConsistencyLevels(clmap);
   configurableConsistencyLevel.setWriteCfConsistencyLevels(clmap);

   HFactory.createKeyspace("MyKeyspace", myCluster, configurableConsistencyLevel);

   But still i see latency. Can someone pls help me out ?

   Posted 9 months ago #
7. 

   thobbs
   Administrator

   With multiple datacenters, you do usually want to use LOCAL_QUORUM instead of QUORUM, so you are correct there. If you want strong consistency within a datacenter (which it sounds like you do), you need to both read and write at LOCAL_QUORUM. However, this does not give you any guarantees about reading the most recent data if it was written to another datacenter.

   I'm not very familiar with Hector, so I don't know for sure if you are setting the consistency levels correctly. You may want to check with the hector users mailing list: https://groups.google.com/forum/?fromgroups#!forum/hector-users

   Can you describe in more detail what you mean by "I still see latency"? Do you mean that you're performing a write, and then a read, but the read does not get the data that was inserted with the write?

   Posted 9 months ago #
8. 

   Srini
   Member

   Yes right. i am performing a write, and then a read, but the read does not get the data that was inserted with the write.

   As per hector support forum, EACH_QUORUM should solve the issue. But i tried that as well. It dint help.

   Posted 9 months ago #
9. 

   nickmbailey
   Key Master

   If you are correctly both writing and reading at EACH_QUORUM then the best guess is that you reads aren't actually waiting for writes to completely return before executing. If possible you might try posting your entire sample application to one of the hector support areas.

   Posted 9 months ago #
10. 

    Srini
    Member

    I have following code which sets EACH_QUORUM as write and LOCAL_QUORUM as read consistency level. I am unable to set EACH_QUORUM for read. I get exception saying we cant set EACH_QUORUM for reads. So i have set LOCAL_QUORUM for Reads alone.

    ConfigurableConsistencyLevel consistency = new ConfigurableConsistencyLevel();

    consistency.setConsistencyLevelForCfOperation(HConsistencyLevel.EACH_QUORUM, SESSIONS, OperationType.READ);
    consistency.setConsistencyLevelForCfOperation(HConsistencyLevel.EACH_QUORUM, SESSIONS, OperationType.WRITE);

    consistency.setConsistencyLevelForCfOperation(HConsistencyLevel.EACH_QUORUM, SESSIONOBJECTS, OperationType.READ);
    consistency.setConsistencyLevelForCfOperation(HConsistencyLevel.EACH_QUORUM, SESSIONOBJECTS, OperationType.WRITE);

    Map<String, HConsistencyLevel> clmap = new HashMap<String, HConsistencyLevel>();
    Map<String, HConsistencyLevel> clmapp = new HashMap<String, HConsistencyLevel>();

    clmap.put(SESSIONS, HConsistencyLevel.EACH_QUORUM);
    clmapp.put(SESSIONOBJECTS, HConsistencyLevel.EACH_QUORUM);

    consistency.setReadCfConsistencyLevels(clmap);
    consistency.setWriteCfConsistencyLevels(clmap);

    consistency.setReadCfConsistencyLevels(clmapp);
    consistency.setWriteCfConsistencyLevels(clmapp);

    consistency.setDefaultReadConsistencyLevel(HConsistencyLevel.LOCAL_QUORUM);
    consistency.setDefaultWriteConsistencyLevel(HConsistencyLevel.EACH_QUORUM);

    return HFactory.createKeyspace(config.getKeySpace(), cluster, consistency, failover, credentials);

    i dont see any errors in the logs as well.

    One thing to note in my setup is,

    i have 4 appserver running in all the 4 boxes. each appserver is configured to use the cassandra running in that box.

    I have a script which does something as below

    Req1 --> Appserver1 --> Grid1 in DC1
    Req2 --> Appserver2 --> Grid2 in DC1
    Req3 --> Appserver3 --> Grid1 in DC2
    Req4 --> Appserver4 --> Grid2 in DC2
    Req1 --> Appserver1 --> Grid1 in DC1
    Req2 --> Appserver2 --> Grid2 in DC1
    ...it goes on.

    Posted 9 months ago #
11. 

    Srini
    Member

    Onething i noted now is, my test app i see concurrent request competing with each other trying.

    Posted 9 months ago #
12. 

    Srini
    Member

    This is related to my tester. Closing this Thread. Thx all for your help.

    Posted 9 months ago #

RSS feed for this topic

Reply

You must log in to post.