Downtime during deployment of revision of proxy

[guy_hagemans]

During deployment of a large(?) proxy (20+ resources, 40+ policies) I face an issue where the proxy (or rather API) is not available on the production environment for about 2 minutes.

During tests with other proxies and from my experience a proxy response when deployed from the Apigee management UI responses might be delayed by a few seconds but an error is usually not the default response. Is it because of the rather large size of the proxy that a deployment takes too much time so that an error response is sent?

In the documentation I found that there is an option to first deploy the new revision and only then undeploy the previous one ( named as seamless deployment). I assumed this is the default deployment method of also the Apigee Management UI, but I might be wrong?

thanks in advance,

Guy

[sudheendras]

Proxy deployment could take time depending upon the policies and resources present. Did you try setting the "override" attribute to "true"? When you set this to 'true', it forces deployment of the new revision by overriding conflict checks between revisions. You can set this parameter to 'true' when using the 'delay parameter' to minimize impact on in-flight transaction during deployment. You are right.. Apigee Management UI uses this parameter while deploying.

[#3 · Accepted solution]

Thanks @GuyH for posting your question. I was just looking into this a few days trying to narrow the cause of this issue. You're right about the issue you're currently facing, in which there's some downtime when deploying through the UI in certain cases. Please allow me to describe these two scenarios and ways to try to fix it:

Scenario #1: API traffic with downtime

This issue occurs when you make some changes and deploy the same API Revision. AFAIK, currently, deploying the same API Proxy revision without zero downtime is currently not supported. So, essentially this happens when you just make some changes through the UI and hit save button.

Scenario #2: API traffic with zero downtime

In my experience, I was able to workaround the issue above by saving my changes as a new revision, and subsequently deploying the new revision. What this effectively does is to import a new revision and then overriding the previous deployment with the last revision. In this case, if you turn on developer tools, you'll observe that the resource /{apiproxy_name}/revision/{revision_id} with method POST will include the variable mentioned above by @sudheendra1. Check image below.

Other ways to fix this issue in elegant ways:

Since you might be wondering, that eventually you may forget to follow steps from scenario #2. There are ways to always making sure that doesn't happen through usage of offline tools. Enter Apigee Deploy Grunt and Maven Plugins, which currently support this behavior. I'd encourage you to try these plugins, they come with a few more utilities aiming to improve the API Lifecycle and offline development. Please let me know if you have any questions.

@arghya das please correct if I'm missing anything above.

[guy_hagemans]

Thanks! Based on this I also did some testing;

Without using the management UI, I deployed my proxy via the Management API;

curl -X POST -H "Content-type:application/x-www-form-urlencoded" \ https://api.enterprise.apigee.com/v1/o/{org_name}/environments/{env-name}/apis/{api_name}/revisions/... \-u myname:mypass

as explained here: http://apigee.com/docs/api-services/content/deploy-api-proxies-using-management-api

In the meantime, I ran Siege to provide a request to the proxy every half a second. Using the API with both 'override' and 'delay' there was no downtime detectable. The only thing is that the delay parameter must be set long enough so that the new revision gets a chance to be deployed. For instance it might take 2 minutes for the proxy to get deployed, so it's safe to set the delay parameter to 3 minutes so that the 'old' revision keeps servicing the requests.

[#5]

Thanks @GuyH . This is a good test, I'll compile your findings and run an internal research and explain it further.

[adas]

@dzuluaga you are absolutely correct. To further clarify the point about deployment with override flag, it would be good to set the delay query parameter as well, so that the older revision is undeployed only after the specified delay. This ensures that the in-flight transactions are not getting a classification failure because the proxy is undeployed and the new revision is not yet deployed.

[#7]

There is no 'delay' parameter that is being set by the UI when deploying a new revision . It does only override . Even in the case of saving as a new revision there will a minimal downtime and its not 100 % zero downtime but using APIs and offline tools we can achieve 100 % cc @dzuluaga , @sudheendra1

[ceber]

This might be worthy of an article out here explaining how it works in detail.

[#9]

@Carlos Eberhardt , thats a good idea , we can probably use @dzuluagas answer with small change (as I pointed out in my answer) to Scenario #2 .

[#10]

Definitely, thank @Carlos Eberhardt and @Maruti Chand to point this out. Let me fix it and send it to you for review.

@oponce@apigee.com and @argo do you think this is something that can be improved on the UI? I think many customers still want to deploy from the UI, with zero downtime.