DataStax Support Forums » Forum: Apache Cassandra

DataStax Support Forums » Forum: Apache Cassandra - Recent Posts http://www.datastax.com/support-forums/forum/cassandra Software, Support, and Training for Apache Cassandra en-US Sun, 26 May 2013 03:37:13 +0000 http://bbpress.org/?v=1.0.3 <![CDATA[Search]]> q http://www.datastax.com/support-forums/search.php Sven on "File offset bug ?" http://www.datastax.com/support-forums/topic/file-offset-bugs#post-11261 Mon, 20 May 2013 14:50:33 +0000 Sven 11261@http://www.datastax.com/support-forums/ For Cassandra questions, this forum is no longer the best place to look for help, it's mostly kept around for old references. I suggest subscribing to the Cassandra users list linked at the bottom of <a href="http://cassandra.apache.org/" rel="nofollow">http://cassandra.apache.org/</a>, visiting <a href="http://http://planetcassandra.org/" rel="nofollow">http://http://planetcassandra.org/</a> and using <a href="http://stackoverflow.com/search?q=cassandra" rel="nofollow">http://stackoverflow.com/search?q=cassandra</a> Thanks, Sven yienyien on "File offset bug ?" http://www.datastax.com/support-forums/topic/file-offset-bugs#post-11260 Mon, 20 May 2013 14:37:33 +0000 yienyien 11260@http://www.datastax.com/support-forums/ Could I have some feedbacks please ? joaquin on "Amazon AMI Setup Video" http://www.datastax.com/support-forums/topic/amazon-ami-setup-video/page/2#post-11153 Thu, 16 May 2013 18:25:53 +0000 joaquin 11153@http://www.datastax.com/support-forums/ Ash, That is correct these AMIs are built on instance-stores and will not allow you to stop the instance. You can research more on the differences between the two that caused us to choose the instance-store AMIs: <a href="http://www.mail-archive.com/user@cassandra.apache.org/msg11022.html" rel="nofollow">http://www.mail-archive.com/user@cassandra.apache.org/msg11022.html</a> <a href="http://blog.scalyr.com/2012/10/16/a-systematic-look-at-ec2-io/" rel="nofollow">http://blog.scalyr.com/2012/10/16/a-systematic-look-at-ec2-io/</a> If you would like to purchase DataStax Enterprise, which comes with support, you can submit this form and our sales team will get back to you: <a href="http://www.datastax.com/what-we-offer/products-services/datastax-enterprise/production" rel="nofollow">http://www.datastax.com/what-we-offer/products-services/datastax-enterprise/production</a>. Thanks, Joaquin blueash on "Amazon AMI Setup Video" http://www.datastax.com/support-forums/topic/amazon-ami-setup-video/page/2#post-11097 Thu, 16 May 2013 00:18:36 +0000 blueash 11097@http://www.datastax.com/support-forums/ Joaquin, Do you know anyone who could offer me phone support. I can pay for the time spent. I will greatly appreciate it. Thanks, -ash blueash on "Amazon AMI Setup Video" http://www.datastax.com/support-forums/topic/amazon-ami-setup-video/page/2#post-11095 Wed, 15 May 2013 23:49:17 +0000 blueash 11095@http://www.datastax.com/support-forums/ Hi Joaquin, this seems to be a bigger topic. because this is an instance store ami - I am not able to stop the instance I am forced to terminate the instance everytime and lose all the work what is the work around? Thanks, -ash blueash on "Amazon AMI Setup Video" http://www.datastax.com/support-forums/topic/amazon-ami-setup-video/page/2#post-11093 Wed, 15 May 2013 23:31:27 +0000 blueash 11093@http://www.datastax.com/support-forums/ Joaquin, This is supposed to be instance store AMI. I am not able to stop and restart the instance at will. Having to terminate it. Have you guys figured out a way to attach EBS to it? <a href="https://aws.amazon.com/amis/datastax-auto-clustering-ami-2-2" rel="nofollow">https://aws.amazon.com/amis/datastax-auto-clustering-ami-2-2</a> Thanks, -ash joaquin on "Amazon AMI Setup Video" http://www.datastax.com/support-forums/topic/amazon-ami-setup-video/page/2#post-11037 Wed, 15 May 2013 04:30:33 +0000 joaquin 11037@http://www.datastax.com/support-forums/ Glad to see you got this sorted out! blueash on "Amazon AMI Setup Video" http://www.datastax.com/support-forums/topic/amazon-ami-setup-video/page/2#post-11033 Wed, 15 May 2013 02:45:22 +0000 blueash 11033@http://www.datastax.com/support-forums/ Dear Joaquin, Got past. ubuntu@ip-10-147-192-93:/usr/bin$ cassandra-cli Connected to: "myDSCcluster" on 127.0.0.1/9160 Welcome to Cassandra CLI version 1.2.4 Type 'help;' or '?' for help. Type 'quit;' or 'exit;' to quit. [default@unknown] Thanks for you help. Hope I dont bother you from here on. Thanks, -ash. blueash on "Amazon AMI Setup Video" http://www.datastax.com/support-forums/topic/amazon-ami-setup-video#post-11032 Wed, 15 May 2013 01:17:56 +0000 blueash 11032@http://www.datastax.com/support-forums/ ubuntu@ip-10-147-192-93:/usr/share/cassandra$ cassandra -cli xss = -ea -javaagent:/usr/share/cassandra/lib/jamm-0.2.5.jar -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms1862M -Xmx1862M -Xmn200M -XX:+HeapDumpOnOutOfMemoryError -Xss180k getopt: invalid option -- 'c' getopt: invalid option -- 'l' getopt: invalid option -- 'i' ubuntu@ip-10-147-192-93:/usr/share/cassandra$ Error: Exception thrown by the agent : java.rmi.server.ExportException: Port already in use: 7199; nested exception is: java.net.BindException: Address already in use ubuntu@ip-10-147-192-93:/usr/share/cassandra$ blueash on "Amazon AMI Setup Video" http://www.datastax.com/support-forums/topic/amazon-ami-setup-video#post-11031 Wed, 15 May 2013 01:13:39 +0000 blueash 11031@http://www.datastax.com/support-forums/ Joaquin, back to you. How can create a keyspace? which folder should I go to.? ubuntu@ip-10-147-192-93:/usr/share/cassandra$ show cluster name The program 'show' is currently not installed. You can install it by typing: sudo apt-get install nmh ubuntu@ip-10-147-192-93:/usr/share/cassandra$ create keyspace No command 'create' found, did you mean: Command 'mcreate' from package 'lustre-utils' (universe) create: command not found ubuntu@ip-10-147-192-93:/usr/share/cassandra$ how do i know if cassandra is running or not? could you help PLEASE I just want to get started. Thanks, -ash joaquin on "Amazon AMI Setup Video" http://www.datastax.com/support-forums/topic/amazon-ami-setup-video#post-11029 Wed, 15 May 2013 00:54:10 +0000 joaquin 11029@http://www.datastax.com/support-forums/ Glad to hear! blueash on "Amazon AMI Setup Video" http://www.datastax.com/support-forums/topic/amazon-ami-setup-video#post-11028 Wed, 15 May 2013 00:37:18 +0000 blueash 11028@http://www.datastax.com/support-forums/ Joaquin, Thanks that worked. -ash joaquin on "Amazon AMI Setup Video" http://www.datastax.com/support-forums/topic/amazon-ami-setup-video#post-11002 Tue, 14 May 2013 04:11:41 +0000 joaquin 11002@http://www.datastax.com/support-forums/ I'll work on these cases having a cleaner return, sorry about that. Here's the issue: [ERROR] m1.small and m1.medium instances are not supported. At minimum, use an m1.large instance. Aborting installation. Could you try the same with an m1.large? Thanks, Joaquin blueash on "Amazon AMI Setup Video" http://www.datastax.com/support-forums/topic/amazon-ami-setup-video#post-10993 Tue, 14 May 2013 02:35:50 +0000 blueash 10993@http://www.datastax.com/support-forums/ launcher.py ubuntu@ip-10-80-58-176:~/datastax_ami$ cat ds1_launcher.py #!/usr/bin/env python ### Script provided by DataStax. import os import subprocess import shlex import time import urllib2 import logger import conf # Temporary fix until next rebake def fix_profile(): # Setup a link to the motd script that is provided in the git repository file_to_open = '/home/ubuntu/.profile' logger.exe('sudo chmod 777 ' + file_to_open) with open(file_to_open, 'r+') as f: profile = f.read() f.seek(0) f.write(profile.replace('jdk1.6.0_31', 'jdk1.6.0_38')) logger.exe('sudo chmod 644 ' + file_to_open) def initial_configurations(): # Begin configuration this is only run once in Public Packages if conf.get_config("AMI", "CurrentStatus") != "Complete!": # Configure DataStax variables try: import ds2_configure ds2_configure.run() except: conf.set_config("AMI", "Error", "Exception seen in %s. Please check ~/datastax_ami/ami.log for more info." % 'ds1_launcher.py') logger.exception('ds1_launcher.py') # Set ulimit hard limits logger.pipe('echo "* soft nofile 32768"', 'sudo tee -a /etc/security/limits.conf') logger.pipe('echo "* hard nofile 32768"', 'sudo tee -a /etc/security/limits.conf') logger.pipe('echo "root soft nofile 32768"', 'sudo tee -a /etc/security/limits.conf') logger.pipe('echo "root hard nofile 32768"', 'sudo tee -a /etc/security/limits.conf') # Change permission back to being ubuntu's and cassandra's logger.exe('sudo chown -hR ubuntu:ubuntu /home/ubuntu') logger.exe('sudo chown -hR cassandra:cassandra /raid0/cassandra', False) logger.exe('sudo chown -hR cassandra:cassandra /mnt/cassandra', False) else: logger.info('Skipping initial configurations.') def write_bin_tools(): with open('/usr/bin/datastax_support', 'w') as f: f.write("""#!/usr/bin/env python\nprint '''DataStax Support Links: Cassandra Cluster Launcher: <a href="https://github.com/joaquincasares/cassandralauncher" rel="nofollow">https://github.com/joaquincasares/cassandralauncher</a> Documentation: <a href="http://www.datastax.com/docs" rel="nofollow">http://www.datastax.com/docs</a> AMI: <a href="http://www.datastax.com/ami" rel="nofollow">http://www.datastax.com/ami</a> Cassandra client libraries: <a href="http://www.datastax.com/download/clientdrivers" rel="nofollow">http://www.datastax.com/download/clientdrivers</a> Support Forums: <a href="http://www.datastax.com/support-forums" rel="nofollow">http://www.datastax.com/support-forums</a> For quick support, visit: IRC: #cassandra channel on irc.freenode.net ''' """) with open('/usr/bin/datastax_demos', 'w') as f: f.write("""#!/usr/bin/env python\nprint '''DataStax Demo Links: Portfolio (Hive) Demo: <a href="http://www.datastax.com/demos/portfolio" rel="nofollow">http://www.datastax.com/demos/portfolio</a> Pig Demo: <a href="http://www.datastax.com/demos/pig" rel="nofollow">http://www.datastax.com/demos/pig</a> Wikipedia (Solr) Demo: <a href="http://www.datastax.com/demos/wikipedia" rel="nofollow">http://www.datastax.com/demos/wikipedia</a> Logging (Solr) Demo: <a href="http://www.datastax.com/demos/logging" rel="nofollow">http://www.datastax.com/demos/logging</a> Sqoop Demo: <a href="http://www.datastax.com/demos/sqoop" rel="nofollow">http://www.datastax.com/demos/sqoop</a> ''' """) with open('/usr/bin/datastax_tools', 'w') as f: f.write("""#!/usr/bin/env python\nprint '''Installed DSE/C Tools: Nodetool: nodetool --help DSEtool: dsetool --help Cli: cassandra-cli -h <code>hostname</code> CQL Shell: cqlsh Hive: dse hive (on Analytic nodes) Pig: dse pig (on Analytic nodes) ''' """) os.chmod('/usr/bin/datastax_support', 0755) os.chmod('/usr/bin/datastax_demos', 0755) os.chmod('/usr/bin/datastax_tools', 0755) def restart_tasks(): logger.info("AMI Type: " + str(conf.get_config("AMI", "Type"))) # Create /raid0 logger.exe('sudo mount -a') # Disable swap logger.exe('sudo swapoff --all') # Ensure the correct blockdev readahead since this sometimes resets after restarts if conf.get_config('AMI', 'raid_readahead'): logger.exe('sudo blockdev --setra %s /dev/md0' % (conf.get_config('AMI', 'raid_readahead')), expectError=True) def wait_for_seed(): # Wait for the seed node to come online req = urllib2.Request('http://instance-data/latest/meta-data/local-ipv4') internalip = urllib2.urlopen(req).read() if internalip != conf.get_config("AMI", "LeadingSeed"): logger.info("Waiting for seed node to come online...") nodetoolStatement = "nodetool -h " + conf.get_config("AMI", "LeadingSeed") + " ring" logger.info(nodetoolStatement) while True: nodetool_out = subprocess.Popen(shlex.split(nodetoolStatement), stderr=subprocess.PIPE, stdout=subprocess.PIPE).stdout.read() if (nodetool_out.lower().find("error") == -1 and nodetool_out.lower().find("up") and len(nodetool_out) > 0): logger.info("Seed node now online!") time.sleep(5) break time.sleep(5) logger.info("Retrying seed node...") def launch_opscenter(): logger.info('Starting a background process to start OpsCenter after a given delay...') subprocess.Popen(shlex.split('sudo -u ubuntu python /home/ubuntu/datastax_ami/ds3_after_init.py &')) def start_services(): # Wait for system setup changes to settle time.sleep(5) # Actually start the application if conf.get_config("AMI", "Type") == "Community" or conf.get_config("AMI", "Type") == "False": logger.info('Starting DataStax Community...') logger.exe('sudo service cassandra restart') elif conf.get_config("AMI", "Type") == "Enterprise": logger.info('Starting DataStax Enterprise...') logger.exe('sudo service dse restart') def run(): fix_profile() initial_configurations() write_bin_tools() restart_tasks() if conf.get_config("AMI", "LeadingSeed"): wait_for_seed() launch_opscenter() start_services() ubuntu@ip-10-80-58-176:~/datastax_ami$ blueash on "Amazon AMI Setup Video" http://www.datastax.com/support-forums/topic/amazon-ami-setup-video#post-10992 Tue, 14 May 2013 02:31:05 +0000 blueash 10992@http://www.datastax.com/support-forums/ log contents ubuntu@ip-10-80-58-176:~/datastax_ami$ cat ami.log [EXEC] 05/14/13-01:42:04 git pull: Updating b9eca2b..f4a706a Fast-forward ds1_launcher.py | 26 ++++++- ds2_configure.py | 211 ++++++++++++++++++++++++++++++++++++++++++-------- ds4_motd.py | 50 ++++++++++-- presetup/launch_amis | 166 +++++++++++++++++++++++++++++++++++++++ presetup/pre_2.py | 7 +- presetup/setup.md | 97 +++++++++++++++++++++++ 6 files changed, 513 insertions(+), 44 deletions(-) create mode 100755 presetup/launch_amis create mode 100644 presetup/setup.md [EXEC] 05/14/13-01:42:04 sudo chmod 777 /home/ubuntu/.profile [EXEC] 05/14/13-01:42:04 sudo chmod 644 /home/ubuntu/.profile [EXEC] 05/14/13-01:42:04 sudo rm ds2_configure.py [INFO] Deleting ds2_configure.py now. This AMI will never change any configs after this first run. [EXEC:E] 05/14/13-01:42:05 gpg --keyserver pgp.mit.edu --recv-keys 40976EAF437D05B5: gpg: directory `/root/.gnupg' created gpg: new configuration file `/root/.gnupg/gpg.conf' created gpg: WARNING: options in `/root/.gnupg/gpg.conf' are not yet active during this run gpg: keyring `/root/.gnupg/secring.gpg' created gpg: keyring `/root/.gnupg/pubring.gpg' created gpg: requesting key 437D05B5 from hkp server pgp.mit.edu gpg: /root/.gnupg/trustdb.gpg: trustdb created gpg: key 437D05B5: public key "Ubuntu Archive Automatic Signing Key <ftpmaster@ubuntu.com>" imported gpg: no ultimately trusted keys found gpg: Total number processed: 1 gpg: imported: 1 [PIPE] 05/14/13-01:42:05 gpg --export --armor 40976EAF437D05B5 | sudo apt-key add -: OK [PIPE] 05/14/13-01:42:05 gpg --export --armor 40976EAF437D05B5 | sudo apt-key add -: OK [EXEC] 05/14/13-01:42:05 sudo rm -rf /etc/motd [EXEC] 05/14/13-01:42:05 sudo touch /etc/motd [INFO] Started with user data set to: [INFO] --clustername myDSCcluster --totalnodes 6 --version community [INFO] Using instance type: m1.small [ERROR] m1.small and m1.medium instances are not supported. At minimum, use an m1.large instance. Aborting installation. Please verify your settings: --clustername myDSCcluster --totalnodes 6 --version community [ERROR] Exception seen in ds1_launcher.py: Traceback (most recent call last): File "/home/ubuntu/datastax_ami/ds1_launcher.py", line 31, in initial_configurations ds2_configure.run() File "/home/ubuntu/datastax_ami/ds2_configure.py", line 932, in run File "/home/ubuntu/datastax_ami/ds2_configure.py", line 117, in get_ec2_data File "/home/ubuntu/datastax_ami/ds2_configure.py", line 55, in exit_path AttributeError ubuntu@ip-10-80-58-176:~/datastax_ami$