How to avoid over subscription using Quota for a proxy across multiple consumer apps

[#1]

@Dino-at-Google@Anil Sagar @ Google In reference to your answer here.

We have backends certified for a certain TPS. We have decided to have 1:1 mapping for Api proxy and respective backend. This proxy can be included in multiple products, which in turn can be subscribed by multiple consumer apps.

The use case we have is to track the tps subscription for a proxy by all its consuming apps via any number of products, so as to avoid over subscription. So, lets say, we have P1 proxy pointing to backend B1 which is certified for 100tps. The p1 proxy can be included in multiple products PR1 & PR2 . And these products PR1, PR2 can be subscribed by multiple apps, like A1 & A2 subscribes to PR1 and A3 & A4 subscribes to PR2.

We want to allow all A1, A2, A3 & A4 apps not to oversubscribe above certified 100TPS for B1 exposed through P1.

I see you and @Dino mentioned that we can use custom attributes at app level, and use them in the quota policy in proxy to implement the different quota for each consumer. Assuming the dev apps are created by the consumer itself, and so as those custom attributes for defining their respective quota which we cannot track as part of subscription process, we will have to track not only that consumers have to define those standardized custom attributes but also what values they are filling in for their respective quota.

This will not allow us to achieve full self service model where we can track a particular proxy is subscribed for how much tps by different consuming apps. I hope I understood it right way.

[dchiesa1]

It sounds to me that you would like to rate limit the AGGREGATE number of requests to the backend, is that right?

And you don't have concern about apportioning the requests fairly across all the apps. Is that right?

So 100TPS in aggregate, and it does not matter if 90 tps are generated by 1 app and 10tps are generated by some other app. Is that right? As long as the backend does not experience >100tps.

Look at the SpikeArrest policy.

<SpikeArrest name="Spike-Arrest-1">
    <Rate>100ps</Rate>
    <Identifier>Target-Server-Name</Identifier>
    <UseEffectiveCount>true</UseEffectiveCount>
</SpikeArrest>

By using a name for the target server in the Identifier element, and by attaching that SpikeArrest policy in the request flow you should get what you want: No more than 100 requests per second will be sent to the target, regardless of the clients that are using the API.

If you want to restrict by a particular client, then you can use an additional SpikeArrest policy, chained. For example to restrict each particular client to 20 requests per second, use this:

<SpikeArrest name="Spike-Arrest-2">
    <Rate>20ps</Rate>
    <Identifier ref='client_id'/>
    <UseEffectiveCount>true</UseEffectiveCount>
</SpikeArrest>

A client_id would reflect a particular credential embedded into the app, eg, your iOS app version 123.45.6. Therefore this SpikeArresy would apply a limit on an aggregate count of requests across all instances of that particular client. Eg, all instances of your iOS app version 123.45.6.

Regarding custom attributes, I'm not clear what problem you are trying to solve by discussing custom attributes. If the rate limit you want to enforce on the backend is 100ps and it does not matter if the requests are sent on behalf of product A, B, or C; and it does not matter if app X, Y or Z; and it does not matter if the requests are sent on behalf of proxy 1, 2, or 3, then you don't need custom attributes.

[#3]

I will put it in terms of requirements :

- Req 1 - Restrict overall traffic to target server as per certified TPS, which can be achieved using SpikeArrest as you mentioned in #1.

- Req 2 - Dynamically restrict respective consumer apps to their requested TPS. So, if App 1 requests 50 & App 2 requests 20, then traffic from App 1 should not go beyond 50tps, and for App 2 it should not go beyond 20. Thats where I referred to custom attributes within apps after reading the other article.

- Req 3 - To have a way so as to track how much TPS is subscribed by existing consumers apps. So, out of 100 TPS in this case, if 3rd consumer expects to have 40tps traffic which is > 30 leftover TPS, looking for how to handle such situation.

[dchiesa1]

Restricting consumer apps could be done with custom attributes on the API product, App , or developer. I don't understand your concern about making it self-service, or enabling developers to specify their rate limits. YOU are the API Program manager so obviously you have the ability to set the rate limit attributes as needed on each app or product.

I don't understand your 3rd requirement.

But I think if you consider what I've written you'll be able to figure it all out!

[#5]

Sure, thanks Dino.