How to Monitor apigee logs with Elasticsearch and logstash

[sarthak]

Are you using Apigee in the cloud or on-prem ? What kind of logs are you asking about ? i.e. system logs or audit logs ?

[gnanasekaran]

you could write logs form Apigee to a syslog destination using the MessageLogging policy

http://apigee.com/docs/api-services/reference/message-logging-policy

So logstash already supports syslog as input,

https://www.elastic.co/guide/en/logstash/current/plugins-inputs-syslog.html

so it should be fairly straightforward to configure syslog input in logstash and write logs from Apigee

Thanks,

[#3]

Hi Mukundha,

I am basically new to apigee and kinda learning how the MessageLogging policy works. I have created a simple yahoo weather API Proxy and trying to forward the logs to our syslog server. The syslog server i created is working fine as i have a rsyslog client that can send data to it and i can view them in kibana. (ELK setup).

Below is the message policy i am using.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <MessageLogging async="false" continueOnError="false" enabled="true" name="rsyslog"> <DisplayName>rsyslog</DisplayName> <Syslog> <Message>Message.id = {request.header.id}</Message> <Host>xx.xx.xx.xx</Host> <Port>5544</Port> <Protocol>TCP</Protocol> </Syslog> </MessageLogging>

But i don't seem to find any logs been received and seen in Kibana.

[gnanasekaran]

isn't syslog UDP?

plus, also check if firewall rules has been set to open port 5544 [either tcp or udp, based on your config]

[#5]

It supports both UDP and TCP. And the example in Apigee Message policy documentation also describes it.

[gnanasekaran]

not sure if you missed this -"plus, also check if firewall rules has been set to open port 5544 [either tcp or udp, based on your config]"

also, are you trying this in the cloud? can you share which org it is?

[#7]

Yes i have opened the 5544 port to allow traffic from the Public Message Processor which we have got.

This is Apigee Edge , and the ELK setup is on AWS.

The org is hpcorp.

[gnanasekaran]

can you share your proxy? could be a proxy config issue - lets try to rule that out

[#9]

What do you want me to share in that. ? It is a yahoo weather API.

Is this what you require.

https://hpcorp-dev.apigee.net/weather/forecastrss?w=2502265

[#10]

@mukundha@apigee.com I can receive the syslog messages now. Can you guide me on how do we record the API status codes. Like 200 OK or a 404 error in the messages. Something like APIResponseCode.

[gnanasekaran]

can you also post what was the issue you were facing earlier?

[#12]

@mukundha@apigee.com The problem was something weird. I had the same rsyslog policy been applied to request and response and when i removed one of them. It started working. Don't know but from the ELK side i didn't had anything been changed.

Can we apply the same policy to both request and response?

[#13]

@Ozan Seymen Sorry my bad. I had a filter been set on my dashboard and i really didn't noticed it. I am able to view the status code and other fields perfectly fine. Thanks for the help.

[ozanseymen]

@Niraj - the variable that you need to use is response.status.code. So in message logging policy, Message tag, include this in curly brackets (as you are in interested in the value of the variable):

<MessageLogging name="LogToSyslog">>
  <Syslog>
    <Message>{response.status.code}...</Message>

Here is the full documentation of message logging policy: http://apigee.com/docs/api-services/reference/message-logging-policy

Obviously in order for this to work, you will need to put this policy in response flow - otherwise response variables will not be available for you.

All other variables that you might want to use is explained here: http://apigee.com/docs/api-services/reference/variables-reference. Just reference them in curly brackets and you are good to go.

[#15]

@Ozan Seymen Thanks for the wonderful answer. I really helped me a lot to gather all the variables information at a single page. But i was unable to collect the response code as mentioned by you. I am using a single policy for both request and response. Is that something supported or we need to configure two different policy for the same.

[ozanseymen]

Hi @Niraj - can you elaborate on what you mean by "unable to collect the response code" please? Are you getting an error from Apigee or you are unable to see correct data?

It would also help if you can paste your message logging policy and the resulting data you are seeing on your syslog servers.

[#17]

@Ozan Seymen I don't see the status code in my kibana dashboard. There is no data been populated and the message field is blank.

[gnanasekaran]

Hi Niraj, I guess your policy is getting executed on the request flow, you just need one policy attached at the Response flow to send all the information to syslog. If you see issues, check the Trace in apigee as well, to make sure if those variables are populated.

[#19]

Hi Mukunda, Today i tried adding a new message logging policy and attached that to the response flow as well. Saved it and tried firing the URL , but i see no response code in my kibana. Am i doing something wrong here. Apologies for my basic questions...

[ozanseymen]

@Niraj Does it work when you don't use any variables in <Message> field in Message Logging Policy?

E.g. - does this work?

<MessageLogging name="LogToSyslog">>
  <Syslog>
    <Message>Hello from Apigee</Message>
    ... 
</MessageLogging>

[#21]

I added the following snippet to my policy in response section but i do not get any Logs back to my logstash. There is no data been populated.

i am using postman to submit the GET request.

[#22]

Niraj, can you please provide more info when you post the question. This will help people understand your thought process and provide you a good answers much faster.

[#23]

Hi Sriki77, It was more of question rather than problem. So i had no more info to post. Let me know if you need more info on this.

[#24]

@Niraj

I am actually doing this right now w/ an internal set of systems. I cant promise a lot of useful data out of this - bu when i am finished i will publish some cleansed logstash configs and some brief instructions if you wish. I will NOT cover building an ELK stack - just how to get some basic and decent info out of the existing logs.

Will that help?

[#25]

@Benjamin Goldman Hey !! Today i finally was able to receive the messages properly to my logstash and the same been shipped to kibana to view the dashboard data. Do let me know the setup you anywhere i can help you around.

[birute]

@Benjamin Goldman @Niraj It would be awesome if you could share your learning with the community. Might save a ton of time to the next person trying to solve the same problem. Seems like a great material deserving a dedicated thread 🙂

[#27]

@Birute Awasthi

i will document the same and share across the community soon. 🙂

[#28]

@Birute Awasthi @Benjamin Goldman

Attached is the documentation for the same.

[#29 · Accepted solution]

Documentation for setting this up.

[#30]

This is a great answer. If anyone is on prem and doesnt want to use syslog - i have a bunch of lessons learned about logging correctly out of the components and out of policies that im willing to share.