Cache update time

[#1]

Do Populate Cache and Lookup Cache policies work asynchronously? How long does it take apigee to update cache value?

I'm sending 3 requests to my proxy one after another within short period of time.

The first request updates environment cache value using PopulateCache policy(async parameter set to "false"). The second request reads that cache value using LookupCache policy(async="false") but cached value was not updated by previous request. The third request goes after second is finished and it retrieves cache value using the same LookupCache policy as 2nd request but it get's updated value.

I've added js policy that prints current timestamp to my flows and as far as I can see processing of request2 and request3 start after request1 is finished so all cache values should be gresh.

[#2]

UPD: We have created a simple proxy with two operations:

/store - stores post body into a cache (PopulateCache policy)

/retrieve - reads information from the cache (LookupCache policy)

max-test-rev3-2016-06-24.zip

sequential steps:

/store "testCache"

/store "testCache2"

/retrieve -> "testCache"

/retrieve -> "testCache2"

/retrieve -> "testCache"

trace: trace-1466770342039xml.zip

question:

why we are receiving different answers from LookupCache policy?

It seems that something has broken...

[adevegowda]

trace-1466790978260xml.zipDear @MSaichuk, @Dmitry Lukyanov,

I had a look at the trace file provided and confirm with your observations. Next, I loaded your proxy bundle into my Public Cloud org and I tested the API calls and got the following results:

/store "testCache"
/store "testCache2"
/retrieve -> "testCache2"
/retrieve -> "testCache2"
/retrieve -> "testCache2"
/retrieve -> "testCache2"
/retrieve -> "testCache2"


/store "oldCacheValue"
/store "newCacheValue"
/retrieve -> "newCacheValue"
/retrieve -> "newCacheValue"
/retrieve -> "newCacheValue"
/retrieve -> "newCacheValue"
/retrieve -> "newCacheValue"

I have uploaded the trace file from my org for your reference.

Did you test your API in a Private Cloud setup or Public Cloud ? If it's in Private Cloud setup, can you please let me know the version of the same ? If you are on Public Cloud, can you please let me know the org name ?

This will help me to check if this is an issue that's fixed in the newer release or not. If required, I can investigate the issue further.

Thanks,

Amar

[#4]

thanks for your answer

we are still in the public cloud org=tn

[adevegowda]

@Dmitry Lukyanov,

Thanks for sharing the org name. I will investigate and get back to you.

[adevegowda · Accepted solution]

@Dmitry Lukyanov,

As per the documentation in Cache internals, this is how the PopulateCache works:

When a policy writes to the cache (such as with the PopulateCache policy):

1. The system puts the entry in both the in-memory cache and the persistent cache.
2. The system sends all other message processors a request to create a cache entry with the same key, cache name, and value. This ensures that only fresh data is stored.

I investigated the issue further and found that there was an issue with one of the MPs not able to talk to another MP. Hence, when a new value was updated to the cache using PopulateCache policy it was getting updated into only one of the MPs (let's call it as MP 1). However, the new value was not getting propagated to the other MP (let's call it as MP 2). Due to this whenever the "retrieve" request went to the MP 2, it used to return back with the old cache value. And when the request went to MP 1, we used to get the new cache value.

I restarted the MP 1 that had the issue and connection issue with MP 2 was resolved. I ran the APIs in your org several times and confirmed that we are now getting the latest value always.

Can you please verify at your end and let me know it works ?

Thanks,

Amar

[#7]

Now it works fine. Thank you!

And what if we go to apigee edge start ( http://apigee.com/about/pricing/apigee-edge-pricing-features ) -

how this kind of problem will be solved?

Could we detect by ourselves this MPs communication problem and restart them?

[anilsr]

@Dmitry Lukyanov , Apigee Edge Trial Orgs are running in the separate shared pod compared to Paid Orgs. Trial Org Pod is shared & holds multiples of Trial Org Accounts.

When you buy Apigee Edge Paid Plans your org will be provisioned in a separate pod which will have dedicated MPs & Apigee SLAs. Apigee Enterprise Support will be available on call whenever you see any issues related to production traffic getting disrupted. I don't think you will get access to MPs on cloud since it's very dynamic & Apigee takes care of Uptime, Scaling, etc. Apigee Support team will do it for you & available on call for P1s.

[adevegowda]

@Dmitry Lukyanov,

Glad to know that the problem is resolved now. @Anil Sagar has answered your question. On cloud you will not have access to MPs. But the Apigee Support team will be help you when you have any issues and restart any components if needed.

Please do accept the answer as it has resolved your issue as it will be helpful for anyone else seeing the same problem.

[#10]

Hmmm....

Our test service works fine, but our main service MobileGW still has this problem.

Could you help us?

[adevegowda]

@Dmitry Lukyanov,

What does MobileGW service mean ? Is that service in a different org ? If so, please provide the org name, api bundle name that shows the problem.

[#12]

It's the same org, but different proxy service called 'MobileGW'.

We are trying to get cache data by the key specified in request header and invoke Raise Fault policy if there is no data in cache. After 5 identical requests I got different responses. you can see trace in attachment:

trace-1467205589860.zip

[#13]

@AMAR DEVEGOWDA,

Looks like PopulateCache policy doesn't populate value if we invoke InvalidateCache policy previously. But if we are not clearing cache before population - cache is always updated with new value. See attachments:

max-test-rev3-2016-06-29.zip

max-test-rev4-2016-06-29.zip

traces.zip

rev3 doesn't call InvalidateCache policy and it works fine.

rev4 calls InvalidateCache before PopulateCache policy and we can't always retrieve cache value.

In trace for rev4 3rd and 4th POST request for /retrieve operation get no data from cache

[adevegowda]

@MSaichuk,

Thanks for providing more details, traces and policy. I will look into the issue and get back to you. However, it will be good if you can start a new thread/community post for this issue. It is better to discuss one issue in one thread, so that it's easy for everyone to understand what's being discussed. So, can you please open a new question for this issue ?

Thanks,

Amar

[#15]

Thanks for your replies. I've created a new community post for cleaning cache problem as you asked.