Increased Request Processing latency under load

karliso · 07-21-2020 09:08 PM

Hello Apigee team. We have been load testing our proxy. Under load we are seeing increased request processing latency. Our target latency looks good, but our request processing latency is spiking. Any idea what the issue could be or how to trouble shoot the issue? None of the polices I have are call out policies. Below is a list of the policies I am using.

AccessControl

AccessEntity

XMLToJSON

AssignMessage

ExtractVariables

RaiseFault

AssignMessage

Javascript - only buisness logic

AssignMessage

SpikeArrest

OAuthV2

Report Inappropriate Content

I also have faced the same during load testing. During the monitor, I found the CPU utilization is high in RMP reaching more than 90%. In that case we did set 3 parameters to find the limit.

1. increased error response

2. cpu utilization reaching 90%

3. response time more than 2 secs

Anyone we reach first will consider as the limit.

Also, the traffic will be in the queue to get processed, which increases the processing latency as well. Another reason could be the number of Cassandra queries becomes high.

dchiesa1

@steve richardson may have an opinion.

The first thing I would look at is AccessEntity. The AE policy in Apigee does not utilize caching very well. This means if you use AE in your proxies, you will be putting a lot of load on the storage nodes (possibly Cassandra if you are using OPDK).

To avoid that, wrap a cache around the AE policy.

Try that and re-test.

karliso

Thanks for your response. I opened a ticket with our the Apigee support team. Hopefully they can help getting some better analytics as to what is happening. I was wondering if the AccessEntity might be an issue. How do I wrap the access entity policy to cache it?

dchiesa1

to "wrap it", I would:

create a SharedFlow
insert the AccessEntity into the SharedFlow
insert a LookupCache prior to the AE in the SharedFlow. If this succeeds, then the item you need was available in the cache, and has now been loaded into a context variable that you specify.
apply a condition to the AccessEntity which tests the context variable from the prior step. If null, you need the AccessEntity. If non-null, skip the AccessEntity.
Insert a PopulateCache step after AccessEntity again checking the context variable from the prior step. If non-null, skip the PopulateCache step. (It means the cache has the item, so no need to populate the cache). If null, then execute the PopulateCache step. Use as the source, the variable populated by AccessEntity. Make sure to use the same key here as in the LookupCache step.
Now, replace all calls to AccessEntity in your proxy, with a FlowCallout to the SharedFlow.

karliso

Thanks! I am assuming the shared flow is about code reuse, and the same pattern could be used within the proxy if it is only needed in one place? What is the backend for the cache?