Unable to update a proxy that was previously deployed and activated

[shrinivaskulkar]

We have an Apigee on-prem installation running 4.18.05.00 and have noticed constant issues when trying to push a code update to an existing proxy that is already deployed.

The first issue is that when we undeploy an active proxy, we are still able to hit it and get a valid response back. After restarting the message processors, this issue goes away and we get a 404 response as expected. However, once we deploy the new proxy revision (through source code using Maven plugin) we do not see the Java callout code changes that were made. Checking the proxy in the UI we see policies that were removed in source control but they still show up in the proxy as unlinked policies not attached to any flow.

When I download the same revision from the UI, I do not see the extraneous policy and after decompiling the Java callout jar, I can see the code changes. I saw similar posts around these issues but without any answers. Is there any tier that caches the actual proxy bundle/artifacts that is causing these issues? I am running out of ideas on how to further troubleshoot this issue, appreciate any inputs.

[dchiesa1]

You've got a bunch of problems

• zombie proxies. after undeploy, the proxy appears to still run, and handle requests.
• deploying new proxies does not show updates in the Java JARs
• the UI displays policies that were removed in source control
• downloading the proxy and examining it, the proxy appears to have all the updates

It's possible (likely?) that these are all related.

The fact that undeploy and redploy is not working correctly suggests to me that you have a data problem. Maybe the MPs are not retrieving the correct data from the backing store, which in your case is Cassandra. It could be that the MPs cannot reach C*. It could be that the C* nodes cannot contact each other.

Have you checked the C* logs and the MP logs for errors? If there are data retrieval errors, quorum errors (failure to read consistently) or C* synchronization errors, you could get weird symptoms. That's the first thing I would check. If you have more than one MP, you need to check logs for them all . Likewise C*.

Spelunking through your log files is likely to provide clues. (Also look in the MGMT server log file). When you find an error message, diagnose and solve THAT, and then test whether the undeploy really works.

Another thing I suggest trying is inquiring the state of deployment and undeployment of your proxy or proxies with the Admin API.

curl -i -u name@example.com $mgmtserver/v1/o/ORGNAME/apis/APINAME/deployments

Many times, the API call response will provide additional information that you do not get to see in the UI.

Another thing I suggest: insure you are up to date on the latest Apigee Edge OPDK. And latest OS patches, etc.

And check all the basic stuff: you have enough file descriptors, you have enough disk space.

Good luck.

[ashokhein]

We are experiencing the same issues in 4.18.05. After undeploy, we are still able to hit context path and get a valid response back. Restarting MP is not a good idea.