Exporting data from Apigee Edge for real-time analytics

Hi everyone,

I am trying to export analytics data such as performance metrics from Apigee Edge for use in a visualization tool such as Grafana. Ideally, data is gathered and made available in as close to real time as possible. Does anyone have experience with something like this?

Thank you!

0 1 231
1 REPLY 1

I'd advise that you re-think slightly.
Apigee collects Analytics data automatically, for every API call, and then on a repeating schedule, computes elaboration on that data, for example computing a latency percentile (TP50, TP95, etc) for a lookback period of 5 minutes, computing averages and sums, and so on. This allows you to compare this week's performance with last week's, or error rates this month vs last month, or latency percentiles along similar periods, and so on. 

Apigee exposes the elaborated data for retrieval via the documented stats API. But because there is a queue and a data pipeline between the event (the API call) and the data collection and elaboration, the data you get from the stats API can be ~5-10 minutes old.  What this means is, while "exporting analytics data" is perfectly possible, it won't meet your goal of being "available in as close to real time as possible."

For that you need a different plan,  one that does not involve the analytics data stream.  You yourself can instrument your API proxies to emit a log record for every call. An easy way to insure this is happening for all your API calls is to register a PostProxy Flowhook.  In the sharedflow that you register, log out the data you want, via ServiceCallout, to your own database - maybe influxdb, or maybe some other DB.  This is a database that is outside the Apigee service or installation - you would be responsible for provisioning it and connecting your Apigee proxies into it.  

Then just hook up Grafana to that database as you normally would. 

This will give you near real-time data on all your API calls.