Access logs for every request in apigee

[zubermohammad]

Hi Apigeeks,

I was having a hard time figuring out where do we have logs for request that are received from client.

I checked router/message-processor access.log but for both components it showed this file as empty.

Even the nginx access.log were empty.

I want to know the source IP of every request and what URL was requested by that IP.

Could someone please help with this?

Thanks..

[rdeandrade]

The best bet is for you to instrument your proxies with a messagelogging policy (http://docs.apigee.com/api-services/reference/message-logging-policy) which will log the IP and URL.

A list of variables that can be used to log the data can be found here: http://docs.apigee.com/api-services/reference/variables-reference

You log message could contain lots of other data as well depending on your requirements.

Depending on your architecture and configs, the client's IP address will be in the X-Forwarded-For header.

I hope this helps.

[zubermohammad]

Hi @Ricardo de Andrade, message logging policy will only work when someone is calling a proxy which is deployed to some environments. What I want is any request that is made to a router should be logged along with the IP of client making request.

Say for example http://router.api.com/lakjsdklfja/ajsfldaj/ should also be logged to the router in the access log. For this kind of requests, the MPs will show me an error saying no proxy found for lakjsdklfja/ajsfldaj/. There is no information as to from where this request is being triggered.

recently we faced a similar problem: MP error log

2017-08-01 17:26:41,928  NIOThread@3 ERROR ADAPTORS.HTTP.FLOW - AbstractRequestListener.onException() : Request: GET, uri: /, message Id:null, exception:com.apigee.rest.framework.ResourceNotFoundException{ code = messaging.adaptors.http.flow.ApplicationNotFound, message = Unable to identify proxy for host: vhdef-in and url: /, associated contexts = []} context:ClientInputChannel(ClientChannel[A:10.19.3.234:8998 Remote host:10.13.151.13:45442]@37 useCount=1 bytesRead=0 bytesWritten=0 age=0ms  lastIO=0ms ))

I understand that logging this information will be an overhead, but how can we avoid such attacks? Or at least some way to know the source IPs so that we can block list the IPs.

[rdeandrade]

Hi Mohammed,

If you create a proxy with '/' as the base path it will work as a 'default' clause in a case/switch type of statement, meaning that all requests that do not match another deployed proxy, will be routed to this 'catch-all' proxy.

You can also use this technique to overwrite the Apigee's default 404 response besides doing the logging.

[zubermohammad]

This sounds good. I tried this option but again it will fail when the URL for proxy doesn't mache '/'. I am looking at something at router level where using Nginx modules I should be able to log these things and when required we can take necessary actions using Nginx logs.

If I am creating a 'catch-all' proxy then complete gateway pod will be prone to attacks where as if I capture this only in logs and router somehow rejects the request before sending it to MP.

Thanks for help 🙂

[#6]

Be design Edge do not log API request/response. You can use Message Logging Policy if required.

I recommend avoiding all requests/responses. Increasing I/O and storing sensitive information on log files is never a good idea.

http://docs.apigee.com/api-services/reference/message-logging-policy

[zubermohammad]

@Maudrit, I understand the use of Message logging policy. But my use case is not to log requests/responses I just want to log every request be that a legit request to an API proxy deployed on edge or to a garbage URL on VHOST.

[rblewitt]

Hi,

Every request inbound to the router should be logged even if it's an invalid. Did you check the vhost logs?

The default location for this is /opt/apigee/var/log/edge-router/

the vhost files are

<org-name>~<env_name>.<port>_access_log.

If a request does not hit a specific vhost then check /opt/nginx/logs.

If connection is made but no HTTP request is sent it won't arrive in any access log.

Check each router in turn.