How can I Send logs to AWS CloudWatch from Apigee PostClientFlow?

[dorianblanc]

Hello,

How to send logs to AWS (CloudWatch) from Proxy's Postclientflow ?

CloudWatch cannot take syslog as input, it has REST API or SDK to push logs but how to use it with MessageLogging policy ?

Thanks.

[dchiesa1 · Accepted solution]

If CloudWatch has a REST API, then you can use ServiceCallout in PostClientFlow to send the message to the endpoint. The endpoint needs to be publicly accessible, publicly routable.

I think Apigee documentation states that the only policy permitted for use within PostClientFlow is the MessageLogging policy. BUT, Apigee operations can also configure your organization to allow ServiceCallout in PostClient Flow. To make this change, contact your Apigee support rep. There's no charge for that.

So if you can figure out how to make a POST call to your CloudWatch endpoint, with the appropriate credentials and payload, then ... you should be able to use ServiceCallout to emit that call for every request that Apigee handles.

[epbgonzalez]

Hi, like Dino mentioned, I would recommend using AWS Rest API. I was able to get it working using this Java Callout policy implemented by @miguelmendoza, since you need to add AWS Signature V4 headers to the HTTP requests. Check out his post: https://community.apigee.com/articles/76109/calling-amazon-web-services-aws-rest-apis-from-api.html

Notice that there are some limitations when you are trying to log from multiple proxies, you need to make sure there are no collisions.

From AWS API Documentation here are some limitations:

• The maximum batch size is 1,048,576 bytes. This size is calculated as the sum of all event messages in UTF-8, plus 26 bytes for each log event.
• None of the log events in the batch can be more than 2 hours in the future.
• None of the log events in the batch can be older than 14 days or older than the retention period of the log group.
• The log events in the batch must be in chronological order by their timestamp. The timestamp is the time the event occurred, expressed as the number of milliseconds after Jan 1, 1970 00:00:00 UTC. (In AWS Tools for PowerShell and the AWS SDK for .NET, the timestamp is specified in .NET format: yyyy-mm-ddThh:mm:ss. For example, 2017-09-15T13:45:30.)
• A batch of log events in a single request cannot span more than 24 hours. Otherwise, the operation fails.
• The maximum number of log events in a batch is 10,000.
• There is a quota of 5 requests per second per log stream. Additional requests are throttled. This quota can't be changed.

Hope this helps!