Trouble retrieving all transactions via Analytics API call. Please help.

[#1]

Hi,

I am having issue retrieving all transactions that has been run in a day via Analytics API call. I am trying to test with large data set of 5.6M transactions that were requested for an API Proxy.

Below is the API call , I am requesting. This returns multiple dimensions in response along with time units. The maximum it returns is 14400. I think there is some kind of limit when the response is returned.

https://api.enterprise.apigee.com/v1/organizations/nonprod/environments/staging/stats/proxy_pathsuff...


Response returned:

{
  "Response" : {
    "TimeUnit" : [ 1500302820000, 1500307500000, 1500307620000, 1500308880000, 1500308940000, 1500309240000, 1500309300000, 1500309360000, 1500309420000, 1500310980000, 1500311700000, 1500311760000 ],
    "metaData" : {
      "errors" : [ ],
      "notices" : [ "query served by:475fbc6a-2de2-4749-947f-3262193444ea", "Spark engine used" ]
    },
    "stats" : {
      "data" : [ {
        "identifier" : {
          "names" : [ "proxy_pathsuffix", "username", "vin" ],
          "values" : [ "/v1/path1", "user1", "XXXXX" ]
        },
        "metric" : [ {
          "env" : "staging",
          "name" : "sum(message_count)",
          "values" : [ 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0 ]
        } ]
      }, {
        "identifier" : {
          "names" : [ "proxy_pathsuffix", "username", "vin"        ],
          "values" : [ "/v1/path2", "user1", "XXXXX" ]
        },
        "metric" : [ {
          "env" : "staging",
          "name" : "sum(message_count)",
          "values" : [ 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0 ]
        } ]
      }
      ......
      and so on 

I looked at the API documentation and its been mentioned that we can get additional data by applying pagination. For this I am supposed to add limit and offset as query params, which I did and and submitted the request. But I don't see any change in response. It returns same response as above, which is incorrect.

I am expecting to retrieve next 14400 stats results.

https://api.enterprise.apigee.com/v1/organizations/nonprod/environments/staging/stats/proxy_pathsuff...

one observation:

If i am running query with no multiple dimensions , i see limit and offset working.

https://api.enterprise.apigee.com/v1/organizations/nonprod/environments/staging/stats/apiproxy?selec...

Multiple dimensions are proxy_pathsuffix,username,vin.

Please help with this issue.

[#2]

Adding some more notes here.

The idea of getting multiple dimensions with analytics API was provided by Floyd Jones in my previous post. This has helped me a lot in my use case.

https://community.apigee.com/questions/42259/how-to-get-multiple-dimensions-with-analytics-api.html?...

So current post is an extension to the previous post. Here i am trying to retrieve huge transactions by using pagination, which is not working.

[spadmanabhan]

Hi, the offset feature is not available when the query is executed by the Spark query engine as per below.

"notices":["query served by:475fbc6a-2de2-4749-947f-3262193444ea","Spark engine used"]
The reason is that Spark does not support it. We are looking at ways to support this differently.