Collecting Data for Custom Analytics

[#1]

We would like to implement a solution for our APIs on Apigee that will allow us to collect Data for logging and analytics usage.

We would like to collect all available data - application name, user id, full request data, full response data.

• What are the best practices for implementing a solution for collecting analytics data with Apigee? The solution doesn’t have to be real time but it should not cause performance degradation or a failure point in the process.
• In all of our api proxies we are setting “response.streaming.enabled” to ‘true' for performance reasons, since disabling this will introduce slowness/memory consumption on the gateway as the request/response needs to be completely buffered in the apigee layer before being sent to the backend or client. This prevents us from accessing request and response bodies in the api proxy flow. Is there a way to have access to request and response data without changing “response.streaming.enabled” to false?
• We believe that disabling the streaming property in one api proxy can affect performance of other api proxies, since the apigee api gateway is a shared resource for all our services. Is there a way to prevent this? How would it affect performance?

Thanks in advance for your help,

Michael

[sudheendras]

Hi @michael.rozman There are many built-in reports under Analytics menu on the management UI, that can help you. They collect more than hundred metrics and dimensions as part of every API call automatically. This does not affect your API performance as they are collected asynchronously, after the response is sent to the client. However these built-in reports does not collect request or response payload attributes. You can change that by adding Statistics collector policy in your API proxy flows. If streaming is enabled on your proxies, you have limited options in collecting entire response body in analytics.

The streaming properties can be enabled at individual proxy level. See the docs here. They are configured as part of the <TargetEndpoint> configuration of your API proxy. So disabling streaming property in one API proxy does not affect the performance of other proxies.

Hope this helps!