How to hide the Loggly customer token from logs?

Not applicable

When I use the Message Logging policy to send the logs to Loggly, the Loggly customer token is also getting saved in the logs. How can I hide this from the logs?

Here is the policy that I am using:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<MessageLogging async="false" continueOnError="false" enabled="true" name="Message-Logging">
    <DisplayName>Message Logging</DisplayName>
    <Syslog>
        <Message>[xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx@xxxxx Hello World!]</Message>
        <Host>logs-01.loggly.com</Host>
        <Port>514</Port>
        <Protocol>TCP</Protocol>
    </Syslog>
</MessageLogging>

Corresponding log in Loggly:

[xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx@xxxxx Hello World!]
Solved Solved
1 22 1,310
1 ACCEPTED SOLUTION

Issue with Message Logging Policy & Syslog format has been resolved. Tags feature / customer token formatting should work now while using message logging policy. For more details, check article here.

View solution in original post

22 REPLIES 22

Not applicable

Hi @Bibin Kurian

Could you try by changing the content of <Message> tag as below:

<Message>[xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx@xxxxx] Hello World!</Message>

I guess, "]" bracket should close before the message content as specified in the example:

http://apigee.com/docs/api-services/reference/message-logging-policy#thirdparty

Hi @rdoda

I tried this but didn't work. The new log is as below:

[xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx@xxxxx] Hello World!

Not applicable

Hi @Bibin Kurian

One another aspect is pending and that is Loggly requires message to be RFC5424 compliant. ref Loggly documentation @

https://www.loggly.com/docs/streaming-syslog-without-using-files/

RFC5424 requires header and structured data before the message. Ref section 6 of the specification:

http://tools.ietf.org/html/rfc5424#page-191

Based on these requirements; could you specify the message tag as:

<Message><14>1 - - - - - [xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx@xxxxx] Hello World!</Message>

Note that blank space is required between all fields except first and second. There is no blank space between ">" and "1".

Cheers,

Rajesh Doda

@rdoda This also didn't work.

Could you copy new message tag and the log content.

Hi @bibin_kurian,

Are you able to resolve this issue? If yes, could you share the solution. If not, could you copy the message tag and content at log.

Regards,

Rajesh

Hi @rdoda,

I couldn't solve the issue yet. Here is my Message attribute and log:

<Message><14>1 [xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx@xxxxx] Hello World!</Message>

Log:

<14>1 [xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx@xxxxx] Hello World!

Not applicable

@Bibin Kurian - Karen from Loggly here. Try moving the closing bracket before the message. Let us know if that helps!

@Karen Sowa I tried this, but not working.

@bibin_kurian Please open a ticket, our support team is here to help you out! http://bit.ly/1TRBZ7k - Karen

Not applicable

Hi @bibin_kurian,

you forget to specify 5 "-" separated by space character as mentioned in my earlier post. Note that there is a space between "1" and first "-" as well as last "-" and "[". These are required as specified in the RFC5424 (Ref section 6).

<Message><14>1 - - - - - [xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx@xxxxx] Hello World!</Message>

Give a try and let us know.

Cheers, Rajesh

Hi @rdoda,

Surely the above example is malformed XML? Why is the <14> required?

Thanks,

Sean

Hi @rdoda,

Thanks for pointing out the mistake. But no luck with this also. Here is my new message and log:

Message:

<Message><![CDATA[<]]>14<![CDATA[>]]>1 - - - - - [xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx@xxxxx] Hello World!</Message>

Loggly log:

<14>Sun Aug 16 22:09:59 UTC 2015Info: <14>1 - - - - - [xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx@xxxxx] Hello World!

When I send the below data using curl, Loggly is properly hiding the customer token. But when I use the same data in Apigee message logging policy, the customer token is getting added to the logs.

curl -H "content-type:text/plain" -d "<14>1 2015-16-08T10:14:15.003Z - - - - [xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx@xxxxx] Hello World!" http://logs-01.loggly.com/inputs/xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx/tag/http/ 

Hi @bibin_kurian,

Some points:

1. Curl command for loggly works fine as now your message is RFC5424 compliant.

2. Latest log specified by you has "<14>Sun Aug 16 22:09:59 UTC 2015Info: " as prefix. Was it not coming in the earlier logs? This text is missing in the log contents specified earlier.

3. Try as below and let us know:

<Message>[xxxxx] HelloWorld!</Message>

Note: "xxxxx" are corresponding to yours Loggly private enterprise number.

Cheers, Rajesh

@rdoda,

Regarding second point, this prefix was there in earlier logs as well. Is this something that Apigee adding as header? If so, is this a RFC5424 compliant header?

Regarding third point, I don't think the messages will get logged to my Loggly account without my customer token.

Thanks,

Bibin

Apigee's syslog policy adds the timestamp, so as @rdoda pointed out, all you need is <Message>[xxxxx] HelloWorld!</Message>, and you will see that loggly has received:

<14>Tue Aug 18 09:59:47 UTC 2015Info: [xxxxx] HelloWorld!

To answer your original question - I have been unable to hide the customer token from the logs, and am not sure it is possible when using Apigee's Message Logging Policy.

I can able to reproduce same, Any pointers? @rdoda @krupa @sriki77

@Anil Sagar, could you please get this logged as an issue with the apigee product team?

@Cladius Fernando ,Sure i will do same. I will keep you posted.

Issue with Message Logging Policy & Syslog format has been resolved. Tags feature / customer token formatting should work now while using message logging policy. For more details, check article here.

Thanks for letting us know Anil.

sunin_shah
Participant II

A year too late, but have you tried adding the following tag:

<FormatMessage>true</FormatMessage>