Apigee trace - filter on HTTP status codes

Oliver_Ogg
Participant II

The trace tool is extremely useful in both debug, and Ops situations.

The limit of just 20 requests makes filtering specific production traffic volumes hard, if not impossible.

I would dearly like to have this limit increased, but appreciate this has a potential knock on performance. 20 does seem particularly ungenerous 🙂

What would be _really_ useful is being able to filter on proxy and/or target HTTP status codes. I find I am often just looking for a particular "type" of response.

5 18 2,399
18 REPLIES 18

Hi @Oliver.Ogg I agree that would be very helpful.

Unfortunately there is a very hard technical problem to solve to enable that. Trace is very expensive and collects a very large amount of log data from every policy in the request/response processing. This is why it's limited to 20.

The trace starts when the request hits Apigee. To support a response status code filter, we would need to trace every single request because we won't know at the point where we need to start collecting the deep details what that request's response status code will be. This would put an enormous load on the system. In fact the very time when something like this would be most helpful, when you are trying to track down a very low occurrence of a specific response status code, is the hardest of all because the trace might need to process many thousands or even tens of thousands of requests before it could catch the one response you're looking for. The load that this put on the system would be very noticeable.

In the meantime, while this is not real-time, I suggest working with custom reports in analytics to post-process the responses. You can use the extensive set of drill downs to dig into the data. You can also add custom statistics collection to store additional specific data that could also help troubleshoot deeper.

Hi Marc, I understand that capturing all traffic in Trace for high volume Proxies may be overkill... but instead, the Apigee Trace tool should offer more options to do filtering.

Today it only supports HTTP Headers and Query Parameters

but I often find myself in the need to filter by URI Path where (in REST APIs) is the main indication of the resource, operation and mandatory parameters (like a UserId or ResourceId) you often are trying to troubleshoot.

If Trace Filters supported URI Path with simple pattern matching (* or **) that would make the Trace tool really useful in high volume traffic.

Thank you for great feedback @roberto.navas@millicom.com , You can also post same as an idea here for better visibility. We will keep you posted if any update regarding same in future.

Any update on this or plans to introduce more filters on your product road map?

Yes, we do have plans to extend those filters. Coming soon!

This is the greatest Christmas present I could have ever asked for.

@Dino-at-Google any update on the status of new filters you can provide?

I can't make any commitments, but I'm hoping and expecting that new feature to be delivered to the cloud, soon. This month. !!

Amazing!!!

@Dino-at-Google I'm guessing the release got delayed?

Yes, indeed. (Just saw this).

We put a hold on the release because of various external events, unfortunately.

Bump! Please this feature is desperately needed to offer support to our customers more efficiently. @Dino-at-Google

This capability is now available in Apigee X?
Have you checked it out?

I think it would be valuable to have a screencast video on this. I'll see if I can produce one, or convince one of my colleagues to produce one.

Apigee X? What's Apigee X?

Dino@Google

Any update on adding new filters?

This is now available in ApigeeX !

Is there any plan to provide the same to on premise ? 

No, that is not on the roadmap. The Apigee Product Management team in the past has stated that there will be no new features added to the Edge OPDK, aside from security updates. 

If you look at the new features list for the most recent releases (4.52.00, 4.52.00.01, 4.52.00.02) , the only new features are security/scale features. And I suppose that will continue. In the same timeframe, there have been lots of new features added to Apigee X.