Troubleshooting Checklist for Private Cloud Manag...

Report Inappropriate Content · ‎07-21-2016

Intro

Management Server issues typically manifest itself as login problems which may have been triggered by a Private Cloud migration, reinstallation, bugs, or changes done during maintanence.

Checklist

Products: Private Cloud (applies specifically to 15.07 or 16.0x)

Get the following info from the customer:

Customer topology (ie. how many nodes and what Apigee components are installed in these nodes
What changed that may have introduced the problem?
Are they able to run management API calls? and as which user and roles?
Specifically what error are they seeing in the UI or management server system.log.

UI Login Problems - Apigee OpenLDAP Setup

What to do...

Check if the sysadmin user can run any management API calls, test with:

curl -v -u sysadmin@email.com http://{MS}:8080/v1/servers

If Management call doesn’t work:

Check the sysadmin password

The credentials should be the sysadmin user and password that is in Apigee OpenLDAP.

If the sysadmin users password needs to be changed, check here.

Check if the user is in Apigee OpenLDAP look for the user where “dn: cn=sysadmin,ou=userroles,ou=global,dc=apigee,dc=com” and see if the users uid is a roleOccupant in the ldap.txt output:

ldapsearch -w <ldap_password> -D "cn=manager,dc=apigee,dc=com" -b "dc=apigee,dc=com" -LLL -h localhost -p 10389 > ldap.txt

If there is a second sysadmin user to run API calls, check the users userroles for authorization to run the API call:

curl -v -u sysadmin@email.com http://{MS}:8080/v1/users/<user@email.com>/userroles

If Management calls work:

Check the UI’s configuration

Check the value for apigee.mgmt.credential and apigee.analytics.credential has been properly encrypted by EncryptUtil tool. They values for these properties need to be the same on all UI servers.

15.07: /opt/apigee4/conf/ui/apigee.conf
16.0x and newer: /opt/apigee/customer/application/ui.properties

Check UI logs:

15.07: /opt/apigee4/var/log/apigee/ui/start.out and /opt/apigee4/share/ui/logs/application.log
16.0x and newer: /opt/apigee/var/log/edge-ui/edge-ui.log

EncryptUtil tool usage here.

UI Login Problems - Externalized Auth Setup

What to do...

Check the sysadmin users credentials. The credentials to use should be the sysadmin user and the users external auth password. They should be able to check these if the user credentials work fine with any other SSO client applications that is being used with their Corporate LDAP server.

If a management API call does not work:

Test management calls with a sysadmin user and with orgadmin users.

Check Externalized Auth configuration

15.07: /opt/apigee4/conf/apigee/management-server/security.properties

16.0x: /opt/apigee/customer/application/management-server.properties

If External LDAP is non-SSL, you can check the LDAP calls by collect tcpdump:

tcpdump -s0 -i any port 10389 or 389 -XXvvv -w /tmp/ldap.cap

Check management logs:

15.07: /opt/apigee4/var/log/apigee/management-server/logs/system.log

16.0x or newer:

/opt/apigee/var/log/edge-management-server/system.log

If management API calls are working:

That means that externalized auth setup for management server is working, there may be a problem with the credentials being used for the UI only.

Check the UI’s configuration

Check the value for apigee.mgmt.credential and apigee.analytics.credential has been properly encrypted by EncryptUtil tool. They values for these properties need to be the same on all UI servers.

15.07: /opt/apigee4/conf/ui/apigee.conf
16.0x and newer: /opt/apigee/customer/application/ui.properties

Check UI logs:

15.07: /opt/apigee4/var/log/apigee/ui/start.out and /opt/apigee4/share/ui/logs/application.log
16.0x and newer: /opt/apigee/var/log/edge-ui/edge-ui.log

Run a management call to check if the user has sysadmin credentials.

curl -v -u sysadmin@email.com http://{MS}:8080/v1/userroles/sysadmin/users

If the user needs to be made a sysadmin, run:

curl -v -u sysadmin@email.com -X POST http://{MS}:8080/v1/userroles/sysadmin/users

Check if they encrypted the sysadmin user into the ui.properties by using:

16.0x or newer:/opt/apigee/apigee-setup/bin/setup.sh -p ui -f configFile

(also check if configFile has the correct sysadmin user credentials that are being used by the externalized LDAP)

15.07:

EncryptUtil tool usage here.

Login issues with Multiple Management Servers being Load Balanced

What to do...

NOTE: The application.conf should have a parameter application.secret, which needs to be the same for all UI servers behind a load balancer because the Play server signs a cookie with the secret. This parameter value is used as a salt for encryption of SSL keystore passwords and sysadmin credentials. The application.secret is randomly generated upon install of the management server node and would differ from each management node install.

Make sure the value for application.secret is the same on all UI servers.
- 15.07: /opt/apigee4/conf/ui/application.conf
- 16.0x: /opt/apigee/customer/application/ui.properties

Check the value for apigee.mgmt.credential and apigee.analytics.credential has been properly encrypted by EncryptUtil tool. They values for these properties need to be the same on all UI servers.
- 15.07: /opt/apigee4/conf/ui/apigee.conf
- 16.0x: /opt/apigee/customer/application/ui.properties
Check if SSL is enabled on the UI, if so the SSL keystore password used for the SSL certificate needs to be encrypted by EncryptUtil. Use the following utility to configure SSL:
- 15.07: /opt/apigee4/bin/ui-configure-ssl.sh
- 16.0x: /opt/apigee/apigee-service/bin/apigee-service edge-ui configure-ssl
Check if the customer is using Externalized Auth Integration. If so, check if they are using the indirect bind method or the direct bind method.
For Externalized Auth: Check the configuration here
- 15.07: /opt/apigee4/conf/apigee/management-server/security.properties
- 16.0x: /opt/apigee/customer/application/management-server.properties
Check the sysadmin password
- Externalized Auth: The credentials to use should be the sysadmin user and the users external auth password.
- Apigee OpenLDAP: The credentials should be the sysadmin user and password that is in Apigee OpenLDAP.
Check if the user that is being configured for the UI is a sysadmin by running management API call:

curl -v -u sysadmin@email.com http://{MS}:8080/v1/userroles/sysadmin/users

If the user needs to be made a sysadmin, run:

curl -v -u sysadmin@email.com -X POST http://{MS}:8080/v1/userroles/sysadmin/users -H “Content-type: application/x-www-form-url-encoded” -d ‘id=user@email.com’ -Q -v

Check if the user is in Apigee OpenLDAP:

ldapsearch -w <ldap_password> -D "cn=manager,dc=apigee,dc=com" -b "dc=apigee,dc=com" -LLL -h localhost -p 10389 > ldap.txt

cmbrown · ‎05-16-2017

@cocoandjan or @sgilson we should post one for 17.xx