Strategies for restarting lots of Edgemicro clients when Edge Cloud configuration changes

[#1]

We're going to be using Edgemicro for a number of deployments. There's a good chance that we will need to push updates to the Edge Cloud configuration, and perhaps products, to Edge Cloud for a given Org+Env combination.

If I understand Edgemicro correctly, it needs to be restarted to get any changed configuration within Apigee Edge Cloud.

Edgemicro will be running on-premise on machines managed by vendor partners and we need some way to ensure that those instances get the updated Edge Cloud configuration. I'm not sure we'll have sufficient control of the cloud and hybrid outage windows (or access to Edgemicro instances) to be able to rely on manual restarts of Edgemicro to get the new configuration.

We've talked about this internally regarding options and come up with the following as a starter for 10, which still has gaps you could drive a bus through.

1. Create a plugin that periodically communicates with the standard configuration endpoints in Apigee Edge Cloud.
2. Compare the configuration to the stored configuration.
3. If it's changed, generate a random number between 0 and 10 minutes (so all the machines in a cluster don't all turn off at the same time). Or we randomize the initial check interval.
4. Turn off the health check endpoint that the load balancer is hitting (don't have this yet)
5. Somehow detect when there are no more requests coming through edgemicro, e.g. the load balancer has switched the node to inactive.
6. Identify the edgemicro node process ID and then send it an appropriate signal, probably SIGHUP.
7. Rely on upstart to restart the edgemicro process based on getting that SIGHUP. We're using upstart to respawn if edgemicro is killed, e.g. for out-of-memory issues that we've seen.
8. edgemicro is restarted, picks up the new configuration and then continues on it's merry way.

Has anyone else solved this automatic update of edgemicro problem? What did you do?

[Former Community Member]

@David Teirney

This is a feature that is under development. I'll ask @vinothapigee comment on it.

[#3]

We had a rather long conversation about this 2 quarters ago with Apigee - and while we went ahead and built a system to handle this activity for us (a bit of a sledgehammer approach) i cant wait to see what Apigee comes up with.

[vinoth]

@David Teirney, We are in the process of shipping this as a product capability. We will keep you posted as we release it. As far as how we did it, ill give an overview here.

Edgemicro will keep polling (interval is configurable) for config changes in Edge(Cloud/OnPrem). If any change is detected, It gracefully reloads the edgemicro cluster without dropping the runtime messages. We rely on IPC for communicating with edgemicro cluster.

Along with this change, there is a capability to manually and gracefully reload the edgemicro cluster, if you do not want to wait till the reload happens automatically.

Let us know if you have more questions.

Thanks

[#5]

@vinothapigee great to hear. Is there any timeframe indication for when this will be available? If this is coming I might delay some prod deploys so we don't need to upgrade them later.

Is there a roadmap of upcoming features for Edge Microgateway published somewhere?

[vinoth]

@David Teirney, We do not have a definitive date in hand. It will have to pass the beta phase and enter GA. Hopefully it should be available before 4 weeks from now.Again, this is tentative. So, I would suggest you to take a call based on these. We do not have it published anywhere as of now. But, beta should give an idea on whats new.

[#7]

We'd be very keen to be part of the beta testing. When might the beta be available?

[vinoth]

We will target for beta within a week's time.