Performance implication of using service callout to overcome shortcoming of Message Logging Policy

sunin_shah
Participant II

Since the Apigee message logging policy does not allow the severity of a message to be specified when using syslog, I would need to drop out of the API Proxy into custom code to do this. The message logging policy is Asynchronous and therefore does not add much (if any) latency to the API flow. I am not sure if a service callout would behave in the same way. My concern here is what is the performance impact if any of adding custom code that the API proxies will have to call out in order to do the logging? The problem will also be compounded by the fact that the logging will be using SSL over TCP. Will there be an added cost in performance of connection setup and tear down for each message?

2 6 886
6 REPLIES 6

Not applicable

@Sunin Shah , There will definitely be a performance impact if you are using servicecallouts for logging instead of message logging . You can only get the actual numbers once you test it out on your environment.

If you really want to achieve severity of mesages with message logging policy, you can have 1 message logging policy for each severity (1 for info and 1 for debug for example).

You can enable or disable the policies based on the type of severity you want at any point using the management API and no redeployments are required.

<MessageLogging enabled="false"continueOnError="true"async="true"name="Transaction_Logger">

Can you please point me to the specific management API that you are referring to, to dynamically enable/disable policies? I glanced over management API index but couldn't pinpoint any API that could do it.

edit: cc @Maruti Chand @Diego Zuluaga @Anil Sagar - Can anyone answer this pls?

sunin_shah
Participant II

Thanks @Maruti Chand for confirming my concerns. Just to put it in context this was a follow on from a previous questions I had on whether it is possible to supply the severity value when using syslog in a message logging policy. It is apparent from use of this policy that we cannot do this. In my opinion this is quite a big shortcoming on Apigee implementation of message logging policy. A workaround suggested was to use a callout policy. Whilst this may be the only option, it would be preferable for the Message logging policy to be enhanced so that it does allow the message severity to be specified.

The PRIVAL - which is derived from facility code and severity is a very useful number in syslog.

I am not sure if multiple policies will help with this.

Let me know if you have any more thoughts on this.

Not applicable

Hi @Sunin Shah, another alternative is to leverage JavaScript Callout in async mode. So, the event happens in fire-and-forget fashion. The example provided in this page sends the request and continues execution of the flow. http://apigee.com/docs/api-services/content/httpclient-javascript-object

sunin_shah
Participant II

Thanks @Diego Zuluaga. I was having a look at this and definitely worth considering. Java script will give a good workaround I think - however still feel that this is something Apigee Edge should provide out of the box - especially since syslog is the only viable option for logging unless you are on private cloud.

johnferguson
Participant II

Sorry for resurrecting an old post, but I have a similar issue that I wanted to get some advice on.

The log destination for me is an ELK stack within my corporate back-end. I am sending the logs to a RESTful endpoint via HTTP. But I also found the MessageLogging policy very limiting in terms of the material I wanted to log and how I wanted to structure the payload.

I have built a log payload through JavaScript, which works well and contains a wealth of information... but it means I can't put it in the ClientPostFlow, as it doesn't appear you can execute scripts in this flow.

So I have put it in the Response PostFlow instead. I have bundled the JS step and a Service Callout into a Shared Flow, and set the whole lot on "Continue on Error". OK so far...

But... Even with the Service Callout running asynchronously, the average proxy response time has jumped from an average of 15ms to an average of 60ms. When you look at the Trace, it's difficult to see where this extra overhead has come from.

7691-trace.png

If you take this Trace above, it shows the JS step taking < 1ms, and the SC step taking < 1ms. The overall Flow Callout is taking < 1ms. BUT... the "Response Sent to Client" overall is showing an elapsed time of 58ms. That doesn't seem to tally-up with the response times of each of the steps within the flow.

I guess there's a few questions here:

  1. Any idea where my overhead is coming from, or how I figure it out?
  2. Is there any plan to allow scripting in the ClientPostFlow, as I know this is the best place to put the logging?
  3. Would I gain any benefit from building my payload in Javascript, but then using a MessageLogging policy rather than a Service Callout policy to actually make the call?