api monitoring alerts not behaving as expected
I am trying to setup alerts within the 'api monitoring' feature so I get notified if there is a platform-wide issue. My hope was to get an alert if >10% of all traffic in the environment is getting 5xx status codes.
I set it up as shown here:
This has been creating a lot of notifications. When I investigate, it looks like "false alarms" because I could have total traffic in the thousands and only 1 failed requests with 'Status Code=5xx'. Certainly not >10% of all the traffic across all proxies.
What I think is happening, is even though I've said 'Proxy:All' in the Dimension, the 'Threshold' is being calculated on each proxy individually.
This means proxy A can have 100k successful requests, but then if proxy B only gets 1 request, and that request fails with status '5xx', an alert is sent because >10% of proxy B's traffic had status 5xx.
Is there some other way to set it up so the 'Threshold' is calculated across all proxies as a group?
@Mark Hammelman Can you please provide more details like org, env and alert info so that we can investigate and come back. if you do not like to share the info here, please raise a support ticket and we'll address the same.
