Apigee x MP node in-memory cache limits

[kmohit520]

Hi All

We are checking some performance aspects of Apigee x cache internals. while checking the below page, got to know the short-lived time of cache entry in MP node is 1 second.

https://cloud.google.com/apigee/docs/api-platform/cache/cache-internals

This means, after each 1 second the entry will be removed from MP in-memory cache and will refer Cassandra for further request.

Now we have following queries regarding same:

1. Doesn't it increase Cassandra calls which may increase cache lookup time?
2. Does MP removes cache entry each 1 second even if lookup requests are coming in-between OR it removes after 1 second of not use?

[ganadurai · Accepted solution]

1. Doesn't it increase Cassandra calls which may increase cache lookup time?
   Its a trade-off on how long the data is stored in L1 for faster access, the memory size limits within the MP nodes and the overall performance of the MP as runtime processor. On these considerations the product has  fixed on the 1-second evict policy and yes, for data lookup beyond 1 second it has to fallback on cassandra with the addtnl latency.
   
   
2. Does MP removes cache entry each 1 second even if lookup requests are coming in-between OR it removes after 1 second of not use?
   MP evicts its data only when the memory limit is reached. The eviction are on a least-recently-used basis. (removes after 1 second of no use).
   

Thanks

[kmohit520]

Hey @ganadurai 
Thanks for quick reply!

Got your points regarding internal cache working.

Just for clarification we are trying to find which one to be used between KVM & Cache. Like which one will give efficient performance with respect to loading data from Cassandra. We can consider the scenarios when Cache is 50% or 100% loaded (5-10 million entries) & so as KVM (2.5 - 5 million) then which would perform better.

[ganadurai]

KVM also caches its values. Checkout the docs
For your scenario you have to tune to the available parameters to pick the right one. On the usability standpoint, KVM is better for grouping by key/value pairs.

[kmohit520]

Yes already gone thru this doc, but still not very clear about <ExpiryTimeInSecs> working.

Document says:

The <ExpiryTimeInSecs> element lets you control how long the keys/values used in the policy are stored in cache before they're refreshed again from the KVM.

To let be more specific let's take an example.

We have 100 entries in KVM & a GET policy with <ExpiryTimeInSecs> 60.

Now proxy executes GET policy on API hits with 2 different keys and data for those 2 keys/values now loads in cache.

Now after 60 seconds, what happens:

Case 1: Cache only clears its data i.e. now keys will be loaded in cache based on further API hits. (might be different keys from earlier 2 hits)

Case 2: Cache refresh again only 2 values from KVM which were there earlier.

Case 3: Cache refresh all 100 values from KVM into cache.

[ganadurai]

For KVMs with "ExpiryTimeInSecs" defined, the data is loaded into cache on the GET operation, just the key/value pair thats looked up via GET not the entire KVM. 

From the docs:
GET - The very first time a KVM GET operation executes, the requested keys/values from the KVM are loaded into cache.

When the number of seconds specified in <ExpiryTimeInSecs> expires, the data is evicted and re-loaded when there is a GET operation for the key/value.

Case2 and Case3 are not correct.

[kmohit520]

Thanks! for clarification