organization rate limit

[#1]

In a private cloud, on-premise deployment of Apigee Edge, I will have several organizations. Let's assume there will be three message processor units, each capable of actually 750 requests per second maximum given the proxies in use. Let's assume each organization has created one API.

Can each organization create rate plans that allow up to 3x750 requests per second? If so, what happens if a consumers of one organization's API drive 3x750 requests per second, do consumers of the other organizations' APIs effectively get locked out?

Put another way, how is it possible to ensure one particular organization's SLAs are met, regardless of the rates the other organizations permit by their plans?

[sarthak]

To answer your question - Yes. Each Organization will be able to make 750 requests per second thus in total 3*750 requests per second in total.

A consumer is defined by the api key issued. Each app in each org will have a different API key (or you are planning to have the same API key across orgs?). So if app1 in org1 hits the limit of 750 calls per second then it will not impact the the apps on org 2.

It will boil down to how the apps and API products are defined. Since you are going by org level segragation I am assuming that you are having completely separate products,app registered separately at each org, apps getting different credentials each org. So every limit enforcement will be limited to the scope of the organization.

Does that make sense? Or I misunderstood your question?

[mschreuder]

Hi @Frank Weiss Welcome to Apigee Community!

Managing and tuning your Private Cloud to ensure all SLAs will require an understanding of the traffic patterns, the request / response processing load and the backends, in other words you need to look at the whole end to end experience.

Traffic Patterns:

If your organizations are each going to be processing peak traffic at the same time, then you will need to make sure your Private Cloud total capacity, not just your message processors (MPs) but also routers, cassandra, qpid and postgres can handle the combined volumes. It also matters whether there are momentary spikes or sustained high traffic.

Request / Response Processing Load:

It is also important to understand the nature of the payloads, are you doing lightweight small payloads or is there a lot of data in each request. And how complex are the proxies, is there lots of logic, possibly lots of service callouts / heavy us of java, etc or are they pretty straightforward.

Backend Servers:

And the performance of your backend servers also matters, for example if you have three different organizations, two of which are connected to fast backends and the third is connected to a slow backend then if you are using the same MPs for all three orgs, you will see the one connected to the slow backend will cause system impacts, e.g. consuming connections for longer, that could start to impact the other orgs.

If you do have the same traffic patterns and you will be seeing sustained traffic rates, then the best way to ensure you don't have such issues is to ensure each org has two MPs (at least two for high availability) that are not shared.

However, if your traffic patterns are spread, for example if one org would be serving peak traffic at midday and one in the afternoon and one at midnight then that traffic spread could enable you to make optimum utilization of your MPs and you can put all of your orgs on the same MPs.

And also make sure you have sized your cassandra ring, qpid and postgres accordingly to handle the traffic volumes.

The best way to work this all out is to start modelling all or as many of the variables as possible to flesh out your specific use case, work out the answers to the above and then develop some performance testing scenarios to validate that the system as a whole can handle the expected load.

To directly answer your question: If traffic for one org is 2,250 requests per second (3x 750) and you also have requests coming in for the two other orgs, then what happens next depends on the combination of timeout settings and how long the total traffic volumes are above 2,250 requests per second.

If it is short, then you would see latency increasing but all requests would still be serviced.

However, if the total volume of requests were sustained at above 2,250 requests per second for long enough, you would start to see timeout errors happening.

All three orgs would likely see those timeouts, because the system won't give specific preference to the 2,250 requests coming in for the one org, it will process the requests in the order received.

If this has answered your question, please click the Accept link, below; alternatively let us know how we can help further on this one.