Spike Arrest Policy - Spike arrest violation error

[balaji17]

Hello,

We are experiencing a spike arrest violation error in Apigee when the request count is not exceed the rate limit configuration.

The SA policy is defined in the sharedflow and the sharedflow is attached in different proxies.

SA Policy Configuration:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<SpikeArrest async="false" continueOnError="false" enabled="true" name="SA-Spike-KVM-Custom">
<DisplayName>SA-Spike-KVM-Custom</DisplayName>
<FaultRules/>
<Properties/>
<Rate ref="KVM-Spike-Rates-Proxies">5ps</Rate>
<UseEffectiveCount>true</UseEffectiveCount>
</SpikeArrest>

"KVM-Spike-Rates-Proxies" is a reference variable and the value comes from KVM, current value is 100ps

Below are properties that captured from the trace during the 429 error occurred in SA policy and as you can see the allowed count is 100, and available count is 93, but still SA policy raised an exception as follows.

Error: Spike arrest violation. Allowed rate : MessageRate{messagesPerPeriod=100, periodInMicroseconds=10000000, maxBurstMessageCount=1.0}

ratelimit.SA-Spike-KVM-Custom.used.count	7
ratelimit.SA-Spike-KVM-Custom.expiry.time	0
ratelimit.SA-Spike-KVM-Custom.failed	true
ratelimit.SA-Spike-KVM-Custom.class.exceed.count	0
ratelimit.SA-Spike-KVM-Custom.available.count	93
ratelimit.SA-Spike-KVM-Custom.allowed.count	100
ratelimit.SA-Spike-KVM-Custom.fault.name
ratelimit.SA-Spike-KVM-Custom.class.available.count	0
ratelimit.SA-Spike-KVM-Custom.class.total.exceed.count	0
ratelimit.SA-Spike-KVM-Custom.fault.cause
ratelimit.SA-Spike-KVM-Custom.identifier	****@@@***@@@***@@@SA-Spike-KVM-Custom@@@nondistributed@@@_default
ratelimit.SA-Spike-KVM-Custom.class.allowed.count	0
ratelimit.SA-Spike-KVM-Custom.class.used.count	0
ratelimit.SA-Spike-KVM-Custom.total.exceed.count	0
ratelimit.SA-Spike-KVM-Custom.datastore.fail.open	false
ratelimit.SA-Spike-KVM-Custom.exceed.count	0

Can someone please help to find out why this error is happening? even the requests is not exceeding the threshold limit which is 100ps, and still there was 93 available count.

Please let me know if you need any other details for this issue.

Thanks,

Balaji Govindasamy

[balaji17 · Accepted solution]

This is a SA policy configuration issue,  while troubleshooting the issue we found that we have 10 MPs configured in the Apigee (SaaS - Cloud Env), so based on 100ps rate limit config, each MP process only 1 request per 10 milliseconds timeframe when spread linearly, since there were more then one requests reached within 10 milliseconds timeframe to 1 MP, the MP set SA policy failed = true, hence SA policy raised a fault with "429 - Too many requests".

Resolved this issue by increased the rate limit and defined identifier in the SA policy configuration.