http://www.datastax.com/support-forums/topic/number-of-map-tasks Software, Support, and Training for Apache Cassandra en-US Mon, 20 May 2013 04:28:05 +0000 http://bbpress.org/?v=1.0.3 q http://www.datastax.com/support-forums/search.php http://www.datastax.com/support-forums/topic/number-of-map-tasks#post-1507 Mon, 02 Apr 2012 16:59:04 +0000 tjake 1507@http://www.datastax.com/support-forums/ <p>Make it part of the TBLPROPERTIES</p> <p>The smaller the split size is the more mappers it will create. </p> http://www.datastax.com/support-forums/topic/number-of-map-tasks#post-1506 Mon, 02 Apr 2012 16:55:01 +0000 Anonymous 1506@http://www.datastax.com/support-forums/ <p>Is it the same as setting HIVE_OPT while invoking hive? Or should we make it part of TBLPROPERTIES while defining an external table?</p> <p> <code>export HIVE_OPT=&quot;-Dcassandra.input.split.size=268435456&quot;</code></p> <p>we are trying to increase the number of map tasks that get kicked of as a result of a simple query like </p> <p> <code>select count(*) from tablefoo;</code> </p> http://www.datastax.com/support-forums/topic/number-of-map-tasks#post-1504 Mon, 02 Apr 2012 16:32:54 +0000 tjake 1504@http://www.datastax.com/support-forums/ <p>You can set it there or as part of the job using -D in hadoop cmd </p> http://www.datastax.com/support-forums/topic/number-of-map-tasks#post-1503 Mon, 02 Apr 2012 16:31:17 +0000 Anonymous 1503@http://www.datastax.com/support-forums/ <p>Thanks. Where do we set this parameter? In mapred-site.xml? </p> http://www.datastax.com/support-forums/topic/number-of-map-tasks#post-1501 Mon, 02 Apr 2012 16:21:01 +0000 tjake 1501@http://www.datastax.com/support-forums/ <p>Yes, The setting is "cassandra.input.split.size" its default is 64k rows </p> http://www.datastax.com/support-forums/topic/number-of-map-tasks#post-1497 Mon, 02 Apr 2012 15:23:53 +0000 Anonymous 1497@http://www.datastax.com/support-forums/ <p>While using DSE 2.0, how can we get hadoop/hive to increase the number of map tasks for maximum performance? We are trying to run map reduce with the input data coming from cassandra. Are there things like cassandra input buffer size or blocks that map reduce can see and increase the number of map tasks? </p>