Apigee Private Cloud Upgrade from 4.15.04.00 to 4.15.07.00 fails with errors indicating analytics upgrade is not completed

We have completed upgrading Postgres and while we are trying to upgrade the RMPs, we get the below error:

/app/apigee4/share/installer/check-ax-upgrade.sh 
Checking to ensure analytics is upgraded to 4.15.07.00 before upgrading run time components... 
Enter global Apigee Edge admin password: 
...
 
==> 000 [ERROR]: Could not find ax_execution_fault_policy_name in analytics postgres schema 
Ensure analytics is upgraded before upgrading run time components. 
After Confirming that the analytics components are upgraded, re-run /app/apigee4/share/installer/apigee-upgrade.sh

Can someone please help me in resolving this issue ?

Solved Solved
0 5 334
1 ACCEPTED SOLUTION

Not applicable

Please check Disabling the check on upgrade order in the upgrade script section in the installation guide. If it is not related to the scenarios mentioned in the documentation, then it could be:

  • The postgres upgrade might have failed to upgrade the schema. Check the upgrade-<version>.log file in <install_root>/apigee4/var/log directory for errors.
  • The cluster might have multiple org-envs and not all of them might be linked to the postgres server. An example of this scenario is a test org that was deployed but not completely wired or removed. Such orgs will report the schema as missing and cause the check to fail. The check in place ensures all the org-envs in the cluster have the updated schema before proceeding with the upgrade.

To check if the postgres schema has been updated run the following command against each org-env combination:

Note: Replace <..> with appropriate values.

curl -u "<admin_user>:<admin_pwd>" "http://<management_server_ip>:<management_server_port>/v1/o/<org>/e/<env>/analytics/admin/schema?type=fact"

Look for the entries: ax_execution_fault_policy_name, ax_execution_fault_flow_state, ax_execution_fault_flow_name and ax_dn_region.

If these entries are present, it is safe to proceed by disabling the check as mentioned in the Disabling the check on upgrade order in the upgrade script section of the installation guide.

If these entries are not present, it is recommended NOT to proceed with the router/message-processor upgrade till schema is updated.

View solution in original post

5 REPLIES 5

sgilson
Participant V

Is there any chance that the Router and Message Processor nodes cannot connect to the Management Server over port 8080? If so, you can enable access from them to port 8080.

Or, if that is not possible, there is a section in the upgrade doc, "Disabling the check on upgrade order in the upgrade script", about how to disable that check on upgrade.

Stephen

@sgilson,

Thanks for your suggestions.

Not applicable

Please check Disabling the check on upgrade order in the upgrade script section in the installation guide. If it is not related to the scenarios mentioned in the documentation, then it could be:

  • The postgres upgrade might have failed to upgrade the schema. Check the upgrade-<version>.log file in <install_root>/apigee4/var/log directory for errors.
  • The cluster might have multiple org-envs and not all of them might be linked to the postgres server. An example of this scenario is a test org that was deployed but not completely wired or removed. Such orgs will report the schema as missing and cause the check to fail. The check in place ensures all the org-envs in the cluster have the updated schema before proceeding with the upgrade.

To check if the postgres schema has been updated run the following command against each org-env combination:

Note: Replace <..> with appropriate values.

curl -u "<admin_user>:<admin_pwd>" "http://<management_server_ip>:<management_server_port>/v1/o/<org>/e/<env>/analytics/admin/schema?type=fact"

Look for the entries: ax_execution_fault_policy_name, ax_execution_fault_flow_state, ax_execution_fault_flow_name and ax_dn_region.

If these entries are present, it is safe to proceed by disabling the check as mentioned in the Disabling the check on upgrade order in the upgrade script section of the installation guide.

If these entries are not present, it is recommended NOT to proceed with the router/message-processor upgrade till schema is updated.

@sukruth , Welcome to Apigee Community 🙂 Would you like to use Apigee account to get addition privileges moving forward. It will also helps others recognize Apigeeks easily.

Dear @sukruth,

Thanks for your suggestion. We ran the curl command to check the schemas and the ax_* entries were not present. Looks like there was a disk space issue due to which the schemas did not get updated completely.

We re-ran the upgrade script and were able to get the schemas. And then we were able to do the router/message-processor upgrade successfully 🙂