Auto scaling on Apigee Cloud - How does it work ? Find out here...

Cross posting from original blog here for community audience,

We run a complex cloud infrastructure at Apigee, with hundreds of paid and tens of thousands of free and trial customers, all on shared, multi-tenant infrastructure that supports their API management needs. Our customers’ API needs are diverse—and while the law of large numbers tends to smooth out variability within our customer base, it still is the case that the workload of any one customer varies tremendously.

There are two patterns of variability that are pretty common:

The pattern on the left represents “flash sales.” While not very common, they contribute to a significant uptick in load, and, if we don’t scale up and down appropriately, result in wasted capacity and money being donated to Jeff Bezos. The pattern on the right is very common (daily and weekly variations) and might result in 50% wasted capacity if not scaled up or scaled down appropriately.

As an Apigee customer, why would you care? You’re most likely paying by call volume* (the area under the curve) and not by peak. There are two reasons why this matters to you:

  1. A better experience As we detect load increases, we scale it for you. You don’t have to watch, inform us, create a support ticket, whatever. Over time, for predictable increases (like the flash sales on the left in the image above), you can even self-service an increase in capacity in anticipation of the sales.
  2. Cleaner whitelisting for the backends A big reason to avoid auto-scaling is that now new IPs will pound on the backends, and your security teams will disallow them. However, we’ve introduced a NAT (network address translation) layer, which pushes the scaled MPs (our API run time workhorses) one level behind—so backends only see NATs, and these NATs need to be scaled up and down much more slowly. So now you get whitelisting that is stable, and auto-scaling that is transparent. Win-win :).

We are rolling out these changes for our cloud customers now—soon, all of our customers should have auto scaling groups transparently.

Keep us posted if you have any feedback / queries.

Version history
Last update:
‎08-31-2016 07:35 PM
Updated by: