KeyValueMap expiration

[#1]

I have an app where every time someone logs in, I'm going to temporarily persist a small amount of information in a KeyValueMap. After a reasonable amount of time, I want that information to go away. In the documentation, it says:

If the amount of data might grow beyond 15 MB, be sure to use the ExpiryTimeInSecs element to expire entries before exceeding the limit.

However, looking at this parameter, it says:

For a GET from the KeyValueMap, specifies the duration in seconds after which Edge refreshes its cached value from the KeyValueMap.

A value of -1 means the GET value never expires and, because the value never expires, Edge never refreshes its cached value from the KeyValueMap.

This looks more like a duration of caching value, not a time-to-live. Am I just misinterpreting the documentation or does this not do what I need?

[remeeshnair]

You are correct, it look like a value for apigee edge(message processor) to do a refresh on its on internal cache, it will not actually expire the KeyValue. For your requirement i guess you should consider Populate Cache Policy

[Abhishek_Subram]

Is there any specific reason you are using KeyValueMap? Else please use caching policies as @Remeesh suggested.

[#4]

Cache policy was something I had not seen before. I looked at the documentation, but I didn't understand exactly how it worked, while the KeyValueMap was much easier to understand.

What I'm doing is getting some sensitive data from a target endpoint that I don't want to return to the client. Instead, I want to store that data and return a token to the client. When the client calls other services, they will pass that token and Edge will use it to retrieve the sensitive data and pass it to subsequent endpoints.

Is that something that can be done with the cache?

[Abhishek_Subram]

OK. You need a temporary data store which can be achieved either way.

[anilsr]

@Abhishek Subramanya , Yep , Description is not clear. Please check my answer below for more details. @docs team , Can we improve ExpiryTimeInSecs element description with more details. Thank you 🙂

KVM Expiry Time In Secs

• You can reduce requests to the datastore (cassandra) by leveraging the built-in cache functionality of KVM lookups with a simple configuration in the policy: <ExpiryTimeInSecs>300</ExpiryTimeInSecs>.This setting only pertains to the lookup's cache expiry as KVM data does not expire in the datastore.

[anilsr · Accepted solution]

Dear @David Scott ,

Welcome to Apigee Community 🙂

KVMs provide long-term data persistence. It doesn't go away after certain time automatically. You can delete same using KVM Policy itself. Check delete element. You can use one KVM Policy to populate, other to retrieve, & one more KVM policy to delete when some condition matches.

You can also delete them manually using Apigee Edge Management API call.

Regarding, KVM Expiry Time In Secs

• You can reduce requests to the datastore (cassandra) by leveraging the built-in cache functionality of KVM lookups with a simple configuration in the policy: <ExpiryTimeInSecs>300</ExpiryTimeInSecs>. This setting only pertains to the lookup's cache expiry as KVM data does not expire in the datastore.
• For more detail about how caching works, see the following article: Apigee Edge Caching In Detail

Regarding, Populate Cache Policy,

Configures how cached values should be written at runtime. You can cache individual values from response using Populate Cache Policy. You can cache the response from target for specific period of time. Once the time expires, data will be deleted from Apigee Edge Platform. If a cache entry expires there will be no data to return. You need to make sure to populate cache. Populate cache works if it's same API Proxy Endpoint. In your case seems like you will be using data stored in apigee edge in another api call. It may result you will end up sending empty data to other API call (since you mentioned other services) if cache is expired.

I think KVM best suites your use case. You can also refer this article for more information.

[#8]

Thanks for the clarification. Looks like I will need to provide a logout function that deletes these keys or maybe use the ManagementAPI to retrieve them all and clear out old ones.

[#9]

Most of the points are already answered . Just one more comment - the 15MB limit is a physical size limit for an map. Current Edge platform does not support a map whose size (all the keys and values) is > 15MB. Here putting in ExpityTimeInSecs is not going to help. The lookup or populate entry operation will fail with a 5XX error.

[anilsr]

Welcome to Apigee Community @Vedanth Bhangale 🙂 Thank you for deeper insights. I learnt a new thing today. I am sure it will be helpful for many others.