Exception during bulk loads

(5 posts) (3 voices)

• Started 5 months ago by tomekkup
• Latest reply from nickmbailey
• This topic is not a support question

Tags:

No tags yet.

1. 

   tomekkup
   Member

   I'm getting exception while bulk loads with sstableloader. I'm using JDK 1.6.0_25 64bit, Ubuntu 12.04 server. Ipv6 is turned off. Network communication between hosts works correctly. I'm going crazy ;-(

   Exception in thread "Streaming to /192.168.219.36:1" java.lang.RuntimeException:
   java.net.SocketException: Invalid argument or cannot assign requested address
       at org.apache.cassandra.utils.FBUtilities.unchecked(FBUtilities.java:628)
       at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
       at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
       at java.lang.Thread.run(Thread.java:662)
   Caused by: java.net.SocketException: Invalid argument or cannot assign
   requested address
       at java.net.PlainSocketImpl.socketConnect(Native Method)
       at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
       at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
       at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
       at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
       at java.net.Socket.connect(Socket.java:529)
       at java.net.Socket.connect(Socket.java:478)
       at java.net.Socket.<init>(Socket.java:375)
       at java.net.Socket.<init>(Socket.java:276)
       at org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket(OutboundTcpConnectionPool.java:96)
       at org.apache.cassandra.streaming.FileStreamTask.connectAttempt(FileStreamTask.java:245)
       at org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:91)
       at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)

   All hosts runs Cassandra 1.1 (datastax edition). Ports 7000,7199,9160 opened. Any ideas ??

   Posted 5 months ago #
2. 

   Sven
   Administrator

   I found some references to other folks upgrading the OS (although that was RHEL 5 to RHEL 6). One thing to try is running the upload on a Cassandra node directly (if you are not already).

   So you can telnet to the ports from the remote machine? Can you see your remote machine from the Cassandra ring (can you ping it)? Are the machine on the same subnet?

   If possible, would you be able to upgrade to a newer Java version on your remote machine (just poking around on this one, but it's something to try).

   Is there a firewall in the mix? If so, maybe temporarily disable that.

   Thanks,
   Sven

   Posted 5 months ago #
3. 

   tomekkup
   Member

   Of course I can telnet to remote hosts on both ports (7000 and 9160)

   Posted 5 months ago #
4. 

   tomekkup
   Member

   my netstat:

   Active Internet connections (only servers)
   Proto Recv-Q Send-Q Local Address Foreign Address State
   tcp 0 0 0.0.0.0:54383 0.0.0.0:* LISTEN
   tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
   tcp 0 0 192.168.219.35:7000 0.0.0.0:* LISTEN
   tcp 0 0 0.0.0.0:40094 0.0.0.0:* LISTEN
   tcp 0 0 0.0.0.0:7199 0.0.0.0:* LISTEN
   tcp 0 0 0.0.0.0:9160 0.0.0.0:* LISTEN

   Posted 5 months ago #
5. 

   nickmbailey
   Key Master

   It looks like the loader is failing because it can't bind the local socket to a local interface/port. Cassandra will use the listen_address value from cassandra.yaml as the local interface if one is available. Do you have a listen_address set in cassandra.yaml? Perhaps one for another node?

   If there isn't a listen_address set cassandra will call InetAddress.getLocalHost() which should return the correct value. Does running 'hostname' on your machine return a valid interface to assign?

   Posted 5 months ago #

RSS feed for this topic