Is there a proxy to slow down the requests in apigee

Not applicable

We have a requirement where our third party vendor can send too many requests to our Apigee gateway. We definitely would like to process these requests and not discard them.

We have concurrent rate limit policy in place, for example let say the limit is 5.

Now when the third party vendor sends 10 requests at the same time.

5 requests would get processed and 5 gets discarded due to concurrent rate limit.

In this case we do not want to reject those 5 requests rather wait for some time till the resources are available and process it next.

0 5 652
5 REPLIES 5

sjm2000
Participant V

hmmm.

so you mean queuing the requests . I guess there is no way for this . Similar was asked multiple times in community forum if I remember .

Having said that you can make custom program in java and use it as a queuing system and add it as a Java policy . I'm not sure it's a good approach though and never tried it.

Regards

Sujith Mathew

Sujith is correct. There is no queue within Apigee Edge. If you have some external store you could use it as a queue. A BaaS collection could act as a queue.

If you write a nodejs target, it can periodically "wake up" (via setTimeout()) and read from a BaaS Collection, then send out any queued requests.

But this is a D-I-Y approach. There is nothing in Apigee Edge that says "queue this request".

Another way to do it, also with nodejs, is to update a counter in the Cache, with each inbound request. And if the counter is higher than 5 (or whatever) then the nodejs request just waits.... And re-checks every 3 seconds , or whatever.

Then when the count returns to below threshold, the nodejs logic processes the request.

But I have a question - why is 10 "too many" ? What is the problem here?

The normal HTTP thing to do is issue a 503 and the client must retry. The ConcurrentRateLimit will issue the 503 appropriately.

It sounds like the client (third party vendor) is not behaving very well. Can you not fix THAT?

And why is the limit so low ? Maybe you should fix THAT?

@dchiesa

I liked your logic using nodejs. Now a use case -

Let's say the target endpoint is having some downtime (scheduled/ unscheduled) for 10 minutes. In this case can I make use of your logic with a slight change-

storing the inbound proxy messages in cache (L1 cache) and nodejs to "send"/"trigger" to target endpoint when target is up. Do you think it is feasible?

Regards

Sujith Mathew

Although, it was meant for @Dino , since this is an open community 🙂

My answer would be - It depends. Are you trying to implement a Queuing Buffer or Perform Async Processing

Let's assume you were operating in the request-response paradigm (where the client must be sent a processed response for every request they make). If your backend takes a 10 minute downtime, you are asking clients to wait for a response during that downtime. So, this approach will most probably not work in such cases (it still could, depending on the specifics of your business use case).

If you were operating in a "fire-and-forget" paradigm, where the client fires a request, but doesn't really care about the response beyond a basic acknowledgment of the request, then you could meet your requirement more easily. You will basically end up implementing a "Service Activator" pattern by draining from a Queue and posting it to your backend. If this is acceptable to you, you can get pretty sophisticated with such an implementation - you could limit the rate of the drain depending on the throughput and availability supported by your backend.In this model, i'd write two API's

- An API which listens to client requests and publishes it onto a Queue, sends clients a basic acknowledgement of request (202 ACCEPTED)

- An API which is activated by a poller listening to the queue, depending on your queue/poller implementation/choice , you will be able to limit the polling rate/processing rate and you can switch off the polling completely during the backend downtime.

@rmishra , it was meant for all 🙂 . I appreciate your response :).

Yes, its sort of - "fire-and-forget" - more or less like an "asynchronous api ".

I guess i never tried the second API you mentioned. Technically it should work . Maybe need to double check the sizing .

Anyways, i will try it as a POC this week.

Thanks & Regards

Sujith Mathew