Connection timeout between (7199) Management server and Cassandra

[ashokhein]

Hi Team,

We are doing the 10 hosts customized installation in that 3 CZ and 1 Management Server.

Management server is getting the connection timeout error for 7199 port when doing the installation. we have checked the firewall matrix and all ports are listening well. we are not able to find any clues.

[root@xxxx ~]# /opt/apigee/apigee-cassandra/bin/nodetool status
Datacenter: dc-1
================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address         Load       Tokens  Owns (effective)  Host ID                               Rack
UN  x.x.x.x  47.33 KB   1       66.7%             e2ce4251-b762-41c1-a315-39c57c99b211  ra-1
UN  x.x.x.x  71.4 KB    1       66.7%             e2544148-102a-4579-b7a2-58fa96a9bde1  ra-1
UN  x.x.x.x  56.73 KB   1       66.7%             c405205c-2e42-4348-adff-96ea7e06c08f  ra-1

Checking for optional variables
Found optional variable CASS_USERNAME...OK
Found optional variable CASS_PASSWORD...OK
updated property schema_cassandra_keyspace_strategy_options in file /opt/apigee/token/application/management-server.properties
apigee-configutil: edge-management-server: # OK
nodetool: Failed to connect to 'x.x.x.x:7199' - ConnectException: 'Connection timed out (Connection timed out)'.
Error: Unable to connect to x.x.x.x
Error Processing schema file: /opt/apigee/edge-management-server-4.17.01-0.0.967/schema/cassandra/analytics-schema.txt
Error: setup.sh: /opt/apigee/apigee-service/bin/apigee-service exited with unexpected status 1

Note: 7199 is listening on 0.0.0.0 in Cassandra node.

[rtalanki]

Ashok, From Management server node do the following to each of the cassandra nodes.

telnet cass_ip 7199

You should be able to connect on this port. If not, then you know where the problem is.

[ashokhein]

its open and we can do the telnet, but we can't make it from nodetool

[ashokhein]

Our infrastructure is based on multiple network interface with restrictions so if the Cassandra is bind with 0.0.0.0 we can't able to reach from neighbour server. We mimic this and update the default.sh to add this line -Djava.rmi.server.hostname=IP and restart now after these changes I can able to connect.

[aruljothi]

Currently -Djava.rmi.server.hostname=<cassandra node ip> has to be updated on <apigee cassandra root folder>/conf/cassandra-env.sh file, but this will be overwritten on cassandra service restart.

To make this change persistent after restart, update cassandra-env.sh file under <apigee cassandra root folder>/source/conf/cassandra-env.sh