not a support question
Anonymous
Unregistered
Hi,
How can I get the total number of records in column family/table under a given keyspace?
I tried this in cql
SELECT count(*) FROM abc_dimension;
but the value is not correct.
[Nodetool -h localhost cfstats] did not help me neither. Any better ideas?
Thanks
Nish
What makes you think that 'SELECT count(*) FROM abc_dimension;' is incorrect? That query should return the total number of rows in the column family, although it will be a very slow query and should not be run often.
Anonymous
Unregistered
Hey Nick,
I am trying load data into Cassandra from another datasource (relational database).
I need to ensure the data has been loaded correctly and is consistent.
True, it's a slow query. But cfstats gives only the number of keys as an estimate.
Any other ideas?
Well like I said before, why do you think that the 'select count(*)...' query is incorrect? The number of rows doesn't match the number of rows in the relational database?
The 'select count(*)...' query might be slightly inaccurate if you are writing new rows while doing the query. It scans through all your rows in order to compute the count (which is why it is slow), so updates that occur while it is running may not be accurate.
Also, what version of cassandra are you on?
Anonymous
Unregistered
Well, you guessed it right. The numbers are not matching. The data has about 18K rows in the RDB, whereas in Cassandra SQL shows 10,000!. (The data has been completely loaded)
Am loading into a cassandra keyspace having replication factor of 3 and using simple strategy. And I am using the latest AMI(2.1) of DataStax, Cassandra 1.0 using 8 node cluster.
What and Why Am I missing here? Any clues..
Anonymous
Unregistered
I logged into the cassandra client and did a get for the the max key (some 18105) for the CF. Surprisingly, the key values was present there!
I no longer trust CQL now. The CQl "select count(*)" query shows correct values for tables/CF whose row count is under 10,000. Once the number of rows exceeds 10000, it shows constant 10,000!
Has someone faced this before?
Apparently there is a default limit of 10,000 rows for cql queries. In this specific case you can do 'SELECT count(*) from abc_dimension limit 20000;' in order to increase the limit manually. It is not a good idea to do this for larger numbers of rows though since all the rows need to be pulled into memory.
I've filed a bug to allow paging for count() queries, although due to limitations in thrift it's not certain when it will be fixed.
Anonymous
Unregistered
Thanks Nick. You are an awesome Admin!
What else can I do to ensure correct data flowing into cassandra, lets say suppose, a million records are scheduled daily to write into cassandra. T
It sounds like the best way to monitor that is application level monitoring. Cassandra will throw an exception if errors occur during writing. Ideally you should be processing any failures encountered by your application when writing to cassandra. That way you can monitor how many failures there were, which inserts failed, and potentially retry them at a later date.
You must log in to post.
