Unreliable KVM return values

[peterlambrechts]

I have a pipeline script that deploys KVMs using apigeetool onto our edge instance and it removes the whole KVM then creates it and adds the KVM values for all environments.

A very weird problem I have found is afterwards even if I undeploy and redeploy the latest or previous revisions of my API Proxy the old KVM value still keeps on being passed into the policy flow.

I thought it was the lookup cache but I removed that condition from the policy, I also thought it was the ExpiryTimeInSecs value from the KeyValueMapOperations but even after I set it down to 10 seconds the problem still persisted.

The only way I have found to reliably get the KVM to apply is to assign it a new value in the KVM itself, then update the KeyValueMapOperations to pull out the new value.

Has anyone else seen this before?

[dchiesa1]

I also thought it was the ExpiryTimeInSecs value from the KeyValueMapOperations but even after I set it down to 10 seconds the problem still persisted.

You may be operating on a mistaken understanding of how KVM cache works.

Changing the cache TTL (time-to-live, aka ExpiryTimeInSecs) in the policy does not affect the currently-cached values. It affects the entry TTL, the next time the cache entry is WRITTEN.

Suppose that there is a KVM policy with a cache ExpiryTimeInSecs set to 86400. This is one day. You send a request through the API Proxy, the policy executes, the value gets cached, and has an expiry of 1 day . Now you change the value in the KVM. You send another request. The cache is still warm. The old value is returned. Not surprising, right?

Now you change the policy, using ExpiryTimeInSecs = 60. You send another request. Still you get the old value. What's going on?

The previously cached value is still in cache! It doesn't matter if the policy says "use ExpiryTimeInSecs = 60". The previously cached item has its TTL set from the previous policy configuration, 86400 seconds. That cache item will live for a full day, regardless of how you change ExpiryTimeInSecs in the policy.

The logic of the KeyValueMapOperations policy with a Get operation is:

• check the cache, and
• if the entry is present in the cache, return that entry
• if the entry is not present in the cache, read the entry from persistent store, and store it in the cache with the given key, using the configured TTL value from the policy

Changing the policy configuration does not affect the existing elements present in runtime cache. It does not affect the "check the cache" step. It can only change TTL of items cached in the future.

To affect the elements present in the runtime cache, you need to PUT a new entry using the KeyValueMapOperations policy. Or use the InvalidateCache policy. You can also "work around" this behavior by using a different cache key.

More on this here.

---

The only way I have found to reliably get the KVM to apply is to assign it a new value in the KVM itself,

I'm not certain what you mean by "assign it a new value in the KVM itself". But I guess maybe you're talking about the same thing I mentioned above. The KVM PUT policy affects the cache.

[peterlambrechts]

Right, that makes a lot of of sense in regards to the TTL on the Expiry time. Yes I had Expiry time set to 86400 and then set it down to 10 seconds assuming that after that change the next request would expire the entry in 10 seconds.

Is there any way to manually expire the cache entry as using apigeetool to remove and re-add the KVM entry doesn't expire it nor does undeploying and redeploying the particular revision of the api proxy.

If using a PUT in KeyValueMapOperations is the best and only way to expire a KVM that previously had a long TTL then it's good to know.

It would be useful if there was a way via the Edge UI or API to expire everything in cache for a particular KVM and make the api proxy read the current value. Or have low TTL values during the development cycles and bump the numbers once you get into production and values are static.

Many thanks for the clarification. It makes sense as to why the only work-around I could figure out was creating new a new KVM entry and updating the policy to point to the new name.

[dchiesa1]

Using a PUT in KeyValueMapOperations is a good way to expire a KVM that previously had a long TTL. The other option is the InvalidateCache policy, which is dedicated to this purpose.

We have no plans to change that, or to introduce an ability to reset the cache from within the Edge management API, or UI.

I think it's probably a good practice to use low TTL values during the development and test phases, and then raise the TTL once you get into production and the cached values are more stable.