Management Server setup fails (4.51.00-0.0.60169)

[ChristopheG]

Hello,

I am following the 3 node topology for this onPrem deployment.

I installed Cassandra and ZooKeeper on 3 servers. All validations seem to show that Cassandra is up an running properly.

During the setup of the edge-management-server using "setup.sh -p ms" with a silent file, it fails with the below error :

http://157.164.164.21:8080/v1/servers/self......
Unable to authenticate with Management Server
Error: setup.sh: /opt/apigee/apigee-service/bin/apigee-service exited with unexpected status 1

In the Management Server Log, I see these errors :

2022-08-04 14:16:38,444 org: env: target: contextId: action: qtp1219553025-63 ERROR DATASTORE.CASSANDRA - AstyanaxCassandraClient.getColumnValueAsString() : Exception while getColumnValueAsString  for rowKey :support.esb.digital@spw.wallonie.be  & columnName : setupdone in columnFamilyName : totp
2022-08-04 14:16:38,445 org: env: target: contextId: action: qtp1219553025-63 ERROR DATASTORE.CASSANDRA - AstyanaxCassandraClient.logHostPoolInCaseOfErrors() : Cassandra Host Pool under use - All Hosts: 157.164.164.21(157.164.164.21):9160. Active Hosts: 157.164.164.21(157.164.164.21):9160
2022-08-04 14:16:38,446 org: env: target: contextId: action: qtp1219553025-63 ERROR REST - Authorizer.beforeInvocation() : Authentication Failure : Error while accessing datastore;Please retry later
com.apigee.datastore.DataAccessException: Error while accessing datastore;Please retry later
        at com.apigee.datastore.client.astyanax.AstyanaxCassandraClient.getColumnValueAsString(AstyanaxCassandraClient.java:2140)
        at com.apigee.rbac.datastore.RBACDataStore.isMFAEnabled(RBACDataStore.java:159)
        at com.apigee.rbac.impl.AuthenticationServiceImpl.isMFAEnabled(AuthenticationServiceImpl.java:372)
        at com.apigee.security.providers.TOTPAccessController.authenticate(TOTPAccessController.java:43)
        at com.apigee.rest.framework.security.authentication.ApigeeLoginModule.login(ApigeeLoginModule.java:50)

 Using the localhost, I can see some datastores registered in the central pod:

curl http://localhost:8080/v1/servers/self
{
  "buildInfo" : {
    "buildNumber" : "60169",
    "buildTimestamp" : "1655419092799",
    "rPMName" : "apigee-rpm-1.0.0.60169.d53296a2b.2206162235-045100_01",
    "releaseID" : "045100_01",
    "sCMBranch" : "origin/OPDK_5100",
    "sCMRevision" : "d53296a2b4fc79adcc93ff5bdd7d8861d0d44464"
  },
  "externalHostName" : "sd5323v.spw.dev.wallonie.be",
  "externalIP" : "157.164.164.21",
  "internalHostName" : "sd5323v.spw.dev.wallonie.be",
  "internalIP" : "157.164.164.21",
  "isUp" : true,
  "pod" : "central",
  "reachable" : true,
  "region" : "dc-1",
  "tags" : {
    "property" : [ {
      "name" : "config_transport",
      "value" : "HTTP1"
    }, {
      "name" : "dp.color",
      "value" : "green"
    }, {
      "name" : "cache_update_transport",
      "value" : "HTTP1"
    }, {
      "name" : "release.id",
      "value" : "045100_01"
    }, {
      "name" : "http.management.protocol",
      "value" : "HTTP2"
    }, {
      "name" : "default.profile.type",
      "value" : "management-server"
    }, {
      "name" : "http.management.port",
      "value" : "8080"
    }, {
      "name" : "rpm.name",
      "value" : "apigee-rpm-1.0.0.60169.d53296a2b.2206162235-045100_01"
    }, {
      "name" : "started.at",
      "value" : "1659613489975"
    }, {
      "name" : "startup.interval",
      "value" : "15166"
    }, {
      "name" : "rpc.port",
      "value" : "4526"
    }, {
      "name" : "Profile",
      "value" : "ManagementServer"
    }, {
      "name" : "startup.time",
      "value" : "15 seconds "
    }, {
      "name" : "up.time",
      "value" : "1 hour 56 minutes 9 seconds"
    } ]
  },
  "type" : [ "audit-datastore", "management-server", "application-datastore", "auth-datastore" ],
  "uUID" : "090641df-c8f2-451d-b8bc-1b7c7c39e9a5"
}

Can you help me to overcome this problem ?

Thanks and regards,

Christophe

[ChristopheG · Accepted solution]

In the setup logs, I noticed :

Datacenter: dc-1
================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address         Load       Tokens  Owns (effective)  Host ID                               Rack
UN  157.164.164.21  96.52 KB   1       66.7%             a0fd4dfb-5177-4fb3-8112-4237a5ab8d2b  ra-1
UN  157.164.164.22  98.26 KB   1       66.7%             0daec12f-1c3e-4511-bca1-74a1e615b8b6  ra-1
UN  157.164.164.23  130.16 KB  1       66.7%             cc3aec9d-ab65-4fef-b217-cbbfe80d370b  ra-1

Exception during authentication to the cassandra node, Verify the keyspace exists, and that you are using the correct credentials.

In the silent file, I activated the Cassandra authentication and changed the default password.

When I reverted to the default "cassandra" password, the installation of the Management Server passed.