Cross posting from original blog here for community audience,
We run a complex cloud infrastructure at Apigee, with hundreds of paid and tens of thousands of free and trial customers, all on shared, multi-tenant infrastructure that supports their API management needs. Our customers’ API needs are diverse—and while the law of large numbers tends to smooth out variability within our customer base, it still is the case that the workload of any one customer varies tremendously.
There are two patterns of variability that are pretty common:
The pattern on the left represents “flash sales.” While not very common, they contribute to a significant uptick in load, and, if we don’t scale up and down appropriately, result in wasted capacity and money being donated to Jeff Bezos. The pattern on the right is very common (daily and weekly variations) and might result in 50% wasted capacity if not scaled up or scaled down appropriately.
As an Apigee customer, why would you care? You’re most likely paying by call volume* (the area under the curve) and not by peak. There are two reasons why this matters to you:
We are rolling out these changes for our cloud customers now—soon, all of our customers should have auto scaling groups transparently.
Keep us posted if you have any feedback / queries.