Log Messages into Loggly  

One of the best ways to track down problems in APIs is through logging. The easiest way to do that In Apigee Edge is to attach a Message Logging policy on your API and log custom messages to a local disk or to syslog. When configured for syslog, it will forward log messages from Apigee Edge to a remote syslog server including external log management services, such a Splunk, Sumo Logic, and Loggly.

This article covers some of the common methods to log into Loggly from Apigee Edge. If you are interested in logging to Splunk, then jump to this article.

Pre-requisites

You should have an account with Loggly. I am using my free trial account for creating this article. In this post I will discuss two approaches to build this integration. However both the approaches require you to have an unique Consumer Token. You can get your Consumer Key from the Loggly management UI, under Source Setup tab.

Approach 1 : Using Message Logging Policy

You can attach a Message Logging policy either on the request or response path of your API. A simple Message Logging policy will look like this -

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<MessageLogging async="false" continueOnError="false" enabled="true" name="Message-Logging-1">
    <DisplayName>Message Logging-1</DisplayName>
    <Syslog>
        <Message>[YOUR_LOGGLY_CONSUMER_TOKEN@41058 tag="{organization.name}.{apiproxy.name}.{environment.name}"] MESSAGE_YOU_WANT_TO_LOG </Message>
        <Host>logs-01.loggly.com</Host>
        <Port>514</Port>
    </Syslog>
</MessageLogging>

Let's take a closer look at this policy, especially the section under <Syslog>.

<Message> : Represents the syslog message to be logged. This includes the header, tag (s) and the actual content. Loggly only accepts syslog events with a RFC5424 compliant header. Within the header there must be a structured data set that includes your Customer Token and the Loggly private enterprise number (PEN) which is 41058 (a constant). Tags are great for adding organization to your log events to aid in segmentation & filtering. They are metadata you can set on the source side, and can be included with any event that is transmitted to Loggly.

<Host> & <Port> : Loggly provides two syslog endpoints that accept both UDP and TCP syslog. For standard syslog please use:

 logs-01.loggly.com:514 

For secure syslog encrypted with TLS please use:

 logs-01.loggly.com:6514. 

In the Message Logging policy, these are represented under <Host> and <Port> sections.

Approach 2 : Using ServiceCallout Policy

Loggly provides standard HTTP based API interfaces for sending data. You can send a single line, multiple lines or bulk event data using these interfaces. With Apigee Edge, you can call this interface using a Service callout policy. A sample policy will look like this -

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ServiceCallout async="false" continueOnError="false" enabled="true" name="Service-Callout-1">
    <DisplayName>Service Callout-1</DisplayName>
    <Properties/>
    <Request variable="myRequest">
        <Set>
            <Headers>
                <Header name="content-type">text/plain</Header>
            </Headers>
            <Payload>
                MESSAGE_YOU_WANT_TO_LOG 
            </Payload>
        </Set>
    </Request>
    <Response>calloutResponse</Response>
    <HTTPTargetConnection>
        <Properties/>
        <URL>http://logs-01.loggly.com/inputs/YOUR_LOGGLY_CONSUMER_TOKEN/tag/http/</URL>
    </HTTPTargetConnection>
</ServiceCallout> 

NOTE : While using these interfaces you don't have to append **@41058** to your Consumer Token.

Viewing Logs in Loggly

The logs in Loggly can be viewed either on the Dashboard or by searching based on the tag.

NOTE : This article focus only on Apigee Edge and Loggly integration. Please refer to Loggly documentation if you need additional information of their capabilities.

References -

• Loggly API reference - https://www.loggly.com/docs/api-overview/
• Loggly Tutorial - https://www.loggly.com/docs/loggly-tutorial/

UPDATE :

We have fixed issue with Loggly message formatting while using message logging policy, for more details please refer article here.