Custom Reports : How to Automate CSV File Download through API call or Management API

Is there any way to get the CSV of Custom Reports automatically througha job through any API call or Management API?

1 13 1,090
13 REPLIES 13

Hi,

If I understood you correctly, I think you can call the another endpoint through JavaScript Callout. The best approach is to use Callback functions, but you can to use Anti-pattern as well.

https://docs.apigee.com/api-platform/samples/cookbook/implementing-http-clients-javascript

https://docs.apigee.com/api-platform/antipatterns/wait-for-complete

Actually I want to download the CSV file of custom reports on APIGEE on prem via an automated API or Management API call without manually downloading it.

Yes, you can do it.

I don't know if it is documented, but, here's how you can do it:

First query the set of "custom reports" you have in your org:

GET ${mgmtserver}/v1/organizations/${ORG}/reports

From that you will get a list of mappings from name to id. The friendly human-readable name of the custom report, to the ID, which is a uuid. Suppose you want to download the data for the custom report with ID da94c72e-e028-43cc-b4ea-ba241e703a69 . You first must get the details for the report:

GET ${mgmtserver}/v1/organizations/${ORG}/reports/${reportid} 

The result of that is something like this:

{
  "chartType" : "column",
  "comments" : [ ],
  "createdAt" : 1574475520000,
  "createdBy" : "dchiesa@google.com",
  "dimensions" : [ "apiproxy" ],
  "displayName" : "testReport3",
  "environment" : "prod",
  "lastModifiedAt" : 1574475520000,
  "lastModifiedBy" : "dchiesa@google.com",
  "lastViewedAt" : 1574475520000,
  "metrics" : [ {
    "function" : "avg",
    "name" : "target_response_time"
  }, {
    "function" : "avg",
    "name" : "total_response_time"
  } ],
  "name" : "da94c72e-e028-43cc-b4ea-ba241e703a69",
  "organization" : "gaccelerate3",
  "properties" : [ {
    "property" : "__ui__",
    "value" : [ {
      "name" : "description"
    }, {
      "name" : "accuracy"
    } ]
  } ],
  "sortbyCols" : [ ],
  "tags" : [ ],
  "timeUnit" : "hour"
}


Then you must submit an async query for that report . To do so you must map the metrics and dimensions into a new payload, and supply the time range The result is a POST, like this:

POST ${mgmtserver}/v1/organizations/${ORG}/environments/${ENV}/queries \
-H content-type:application/json \
-d '
{
    "metrics": [
        {
            "function": "avg",
            "name": "target_response_time"
        },
        {
            "function": "avg",
            "name": "total_response_time"
        },
        {
            "function": "sum",
            "name": "message_count"
        }
    ],
    "dimensions": [
        "apiproxy"
    ],
    "timeRange": {
        "start": "2020-02-03 22:12:00",
        "end": "2020-02-03 23:12:00"
    },
    "name": "testReport3",
    "outputFormat": "csv",
    "reportDefinitionId": "da94c72e-e028-43cc-b4ea-ba241e703a69",
    "groupByTimeUnit": "minute"
}'

That kicks off an "Asynchronous query". The response from that is a payload that tells you where to look. It's like this:

{

  "self" : "/organizations/ORG/environments/ENV/queries/QUERYID",
  "state" : "enqueued",
  "created" : "2020-02-03T23:23:55Z"
}

Then, just poll that query to get the results:

GET ${mgmtserver}/v1//organizations/ORG/environments/ENV/queries/QUERYID

Eventually you will see a payload that tells you the report is ready:

{
  "self" : "/organizations/ORG/environments/ENV/queries/QUERYID",
  "state" : "completed",
  "result" : {
    "self" : "/organizations/ORG/environments/ENV/queries/QUERYID/result",
    "expires" : "2020-02-10T23:24:09Z"
  },
  "resultFileSize" : "0.08 KB",
  "resultRows" : "0",
  "executionTime" : "6 seconds",
  "created" : "2020-02-03T23:23:55Z",
  "updated" : "2020-02-03T23:24:09Z"
}


And then finally just retrieve the "result.self" URL, and you will get a ZIP containing .... uh.... a gzip... which contains.... a file which contains a set of JSON records. You will need to unpack and parse that yourself.

Keep in mind this would count against your entitled allotment of async reports for a given month/quarter, etc.

@Dino-at-Google I have tried the first command through the browser and also through the POSTMAN, but I get 404 error- this is when I tried for the private on prem installation.

But for a cloud based organization account I was able to get the success response.

Please let me know what I can do for the on prem APIGEE?

curl -X GET --header "Authorization: Basic <encoded username and pwd>" "https://api.enterprise.apigee.com/v1/organizations/{orgname}/reports"
I also tried with mgmt server url instead of "api.enterprise.apigee.com"

@Dino-at-Google can u please help me with this. Thanks in advance.

You cant use api.enterprise.apigee.com for on premise apigee, you need to refer to your onprem management server. In that instance, what issue are you encountering?

@dane knezic I tried with mgmt server host name via postman and passed the basic auth headers. I am getting 404 error.

Check your endpoint. And port. If you get a 404, it may not be the management server that you are contacting. I suggest you pare the URL down; start with /v1/o/ORGNAME, and see if the mgmt server responds then. If still 404, you are not using the correct server name, port, and scheme.

We can't tell you the right answer. Connect with the people who installed your Apigee. They will tell you the URL to use for the Apigee management server.

I am able to access all the proxies and components of the APIGEE installation. There is no issue as such.

Just I am able to hit the /reports endpoint. I am using everything ok.

@dane knezic Please help me with this if possible. Thanks in advance.

@dane knezic I am getting 404 error. when trying through postman or browser. annotation-2020-02-07-170411.png

@dane knezic I am getting 404 error. when trying through postman or browser. annotation-2020-02-07-170411.png

annotation-2020-02-07-170411.png (30.0 kB)

There is not enough information to provide any useful help or to understand what you're doing and what you've tried so far.

Again as Dino has suggested, double check your management server url, and before trying the reporting APIs try just the generic org level resource /v1/o/ORGNAME