Message Logging policy sometimes drop off some data sent to Syslog Server

Not applicable

Hi Team,

For the issue report by a customer I tested and confirmed on Edge Public Cloud.

If the API call coming around 1 to 2RPS or higher rate the data to Syslog server is lost and empty for the {response.content} part as follows.

raw message
<14>Thu Jul 27 10:53:54 UTC 2017Info: [LogglyToken tag="tshibamoto.ml-3.test"] response =


Tis happens once in a every 3 to 4 calls.

The Message Logging policy I tried against httbin.org target end is;

<MessageLogging async="false" continueOnError="false" enabled="true" name="Message-Logging-1">
<DisplayName>Message Logging-1</DisplayName>
<Syslog>
<Message>[logglyToken tag="{organization.name}.{apiproxy.name}.{environment.name}"] response = {response.content}</Message>
<Host>logs-01.loggly.com</Host>
<Port>514</Port>
</Syslog>
</MessageLogging>


Is this a known issue or limitation? And please let me know if any configuration to fix this issue is available?

Regards,
Toshi

0 4 652
4 REPLIES 4

Make sure you specify <Protocol>TCP</Protocol> to ensure message delivery. When it is not present, UDP is the default.

http://docs.apigee.com/api-services/reference/message-logging-policy

You could also consider using a service callback for logging services that offer an API for logging.

Thank you so much, @Ricardo de Andrade.

I added <Protocol>TCP</Protocol>, but the result is the same.
Sometimes the log data received on syslog server has incomplete data as mentioned above.

Then the customer says when they configured logsessions on MP

> curl -v -X POST {mp-ip}:8082/v1/logsessions?session=test --noproxy {mp-ip}

The result has changed and seems ok without missing the data of {response.content} part.
Is this something related?

I suppose that in any case there seems to be certain limit of requests handled by message logging especially for large size of message passed to syslog server, right? Is the timing of the data for {response.content} to become ready also related?

If it's the case, do we have any points of configuration for handling higher rate of incoming requests without losing the message data?

Regards,
Toshi

Check this https://community.apigee.com/articles/42504/tcpsyslogsocket-logic.html which talks about what happens in the background.

@Maruti Chand Thank you for the useful info.
In your post,

Dropped messages : No of messages that were dropped as the queue was full and this has nothing to do with the HTTP response codes > 4XX transactions.

In this case is the dropped message entire message?

In our case as I wrote only the part of the message where {response.content} is specified is dropped.
If so, the situation seems to be different. Please advise me.