A pattern for caching KVM values

Update: Apigee Edge release on June 15, 2016 added the capability to cache KVM key/values in KeyValueMapOperations policy. This is controlled by the ExpiryTimeInSecs configuration element. Recommendation is to consider that in-built method first before applying the pattern described here bearing in mind the following caveats/differences:

  1. KeyValueMaps policy cache expiry time configuration does not allow any variables yet so expiry value needs to be hardcoded in the policy.
  2. Updating a KVM with the management API will not reset the KVM cache. So if you are updating KVM values via CI automation, e.g. apigee-config-maven-plugin, in-built expiry method will not work for you - yet.
  3. Updating a KVM with the management UI will not reset the KVM cache. So If admins are accustomed to modify KVM values from UI, in-built expiry method will not work - yet.

It is important to cache data retrieved from KVM in order to increase performance of the proxy runtime - Apigee hits the persistent datastore for KVM get/set operations.

One thing I always move to KVM is api proxy configuration. It contains any data I can externalise, e.g. access/refresh token TTLs, traffic management values for different quota uses, feature toggles, ip restrictions, data around logging (loggly url), etc.

You can create a key value map and a cache resource to store and cache configuration data. I always choose to include the API version number in the naming convention of KVM map and cache resources so I can configure data independently, can support both API versions at the same time for a period and can easily deprecate the previous version without impacting the new version. So I create "myapi-v1-configuration" KVM and "myapi-v1-configuration" cache resource.

One of the benefits of using an explicit cache resource is "control": I can control when data will expire and I can force-invalidate the whole cache when needed via Apigee Enterprise UI (APIs -> Environment Configuration). If you are using automation to maintain configuration, you can use Apigee Management APIs to update KVM and invalidate cache. Don't forget to think about the cache expiry based on the nature of the data you will be caching. For api configurations I prefer to cache for 1 week+ as the data doesn't change too often.

The Pattern

The pattern I follow is as follows:

<Step>
    <Name>LookupCache.GetApiConfiguration</Name>
</Step>
<Step>
    <Condition>lookupcache.LookupCache.GetApiConfiguration.cachehit = false</Condition>
    <Name>KeyValueMapOperations.GetApiConfiguration</Name>
</Step>
<Step>
    <Condition>lookupcache.LookupCache.GetApiConfiguration.cachehit = false</Condition>
    <Name>JavaScript.ConvertApiConfigurationToJson</Name>
</Step>
<Step>
    <Condition>lookupcache.LookupCache.GetApiConfiguration.cachehit = false</Condition>
    <Name>PopulateCache.SetApiConfiguration</Name>
</Step>
<Step>
    <Name>JavaScript.SetApiConfigurationVariables</Name>
</Step>

This flow can be visualised as follows:

2745-screen-shot-2016-05-23-at-134717.png

Lookup/PopulateCache and KeyValueMapOperations policies should be straightforward but let me explain the two javascript policies I am using.

I store each configuration name/value pair as a separate entry in the same KVM map but what I want to cache is the whole KVM map. Current KVM implementation will retrieve the whole key-value map even if you are accessing a single entry by its key (https://community.apigee.com/articles/1396/apigee-developer-heres-the-stuff-you-should-know.html). Therefore it makes sense to read the whole map and cache the full data once rather than reading piecemeal from KVM.

Apigee LookupCache policy can read one key at a time. In order to prevent having one LookupCache policy per configuration item, you can store a single compact data containing all configuration name/value pairs. This will enable you to read the whole configuration with a single LookupCache policy. ConvertApiConfigurationToJson policy constructs this compact JSON data structure where each configuration entry is a property.

So let's assume I have the following configuration data stored in KVM:

oauth.accesstoken.ttl = 1200
oauth.refreshtoken.ttl = 2628000
toggle.countryvalidation = enabled

ConvertApiConfigurationToJson policy could create the following JSON object and set it as a flow variable:

{
	accessTokenTtl: 1200,
	refreshTokenTtl: 2628000,
	countryValidationToggle: "enabled"
}

The responsibility of SetApiConfigurationVariables policy then is to read this JSON object and set individual flow variables out of it so we can use configuration data during runtime, e.g. in conditional flows. Note that if KVM data is read from cache (i.e. cached data hasn't expired yet), ConvertApiConfigurationToJson policy will not execute but the same JSON structure will now be read from cache which needs to be parsed.

So if cache hasn't expired yet, only two policies will execute: LookupCache and SetApiConfigurationValues which is very efficient during runtime.

The code for the individual policies are as follows:

LookupCache.GetApiConfiguration

<LookupCache name="LookupCache.GetApiConfiguration">
    <CacheKey>
        <KeyFragment>configuration</KeyFragment>
    </CacheKey>
    <CacheResource>myapi-v1-configuration</CacheResource>
    <AssignTo>flow.myapi.configuration.json</AssignTo>
</LookupCache>

KeyValueMapOperations.GetApiConfiguration

<KeyValueMapOperations name="KeyValueMapOperations.GetApiConfiguration" mapIdentifier="myapi-v1-configuration">


    <Scope>environment</Scope>


    <Get assignTo="flow.myapi.configuration.accessTokenTtl" index="1">
        <Key>
            <Parameter>oauth.accesstoken.ttl</Parameter>
        </Key>
    </Get>
    <Get assignTo="flow.myapi.configuration.refreshTokenTtl" index="1">
        <Key>
            <Parameter>oauth.refreshtoken.ttl</Parameter>
        </Key>
    </Get>
    <Get assignTo="flow.myapi.configuration.countryValidationToggle" index="1">
        <Key>
            <Parameter>toggle.countryvalidation</Parameter>
        </Key>
    </Get>
</KeyValueMapOperations>

ConvertApiConfigurationToJson.js

var accessTokenTtl = context.getVariable('flow.myapi.configuration.accessTokenTtl');
var refreshTokenTtl = context.getVariable('flow.myapi.configuration.refreshTokenTtl');
var countryValidationToggle = context.getVariable('flow.myapi.configuration.countryValidationToggle');

var configuration = {
    "accessTokenTtl": accessTokenTtl,
    "refreshTokenTtl": refreshTokenTtl,
    "countryValidationToggle": countryValidationToggle
};

context.setVariable('flow.myapi.configuration.json', JSON.stringify(configuration));

Note that you can go fancy here by storing keys in an array and doing forEach to construct the object as the only change between the first 3 lines is the name of the configuration key.

You can also construct a fancier configuration object grouping related properties together - I am leaving this to your needs and imagination:

{
	"oauth": {
		"accessTokenTtl": 1200,
		"refreshTokenTtl": 2628000
	},
	"toggles": [
		"countryValidation",
		"samlSupport"
	]
}

PopulateCache.SetApiConfiguration

<PopulateCache name="PopulateCache.SetApiConfiguration">
    <CacheKey>
        <KeyFragment>configuration</KeyFragment>
    </CacheKey>
    <CacheResource>myapi-v1-configuration</CacheResource>
    <ExpirySettings>
        <TimeoutInSec>3600</TimeoutInSec>
    </ExpirySettings>
    <Source>flow.myapi.configuration.json</Source>
</PopulateCache>

SetApiConfigurationVariables.js

var configuration = JSON.parse(context.getVariable('flow.myapi.configuration.json'));

context.setVariable('flow.myapi.configuration.accessTokenTtl', configuration.accessTokenTtl);
context.setVariable('flow.myapi.configuration.refreshTokenTtl', configuration.refreshTokenTtl);
context.setVariable('flow.myapi.configuration.countryValidationToggle', configuration.countryValidationToggle);

Now that we have every configuration entry as a flow variable, we can use it within conditions or custom code where necessary:

<Step>
	<Condition>flow.myapi.configuration.countryValidationToggle = "enabled"</Condition>
	<Name>JavaScript.CountryValidation</Name>
</Step>
Comments
Not applicable

Love it. It is the same implementation as Java/Database caching template.

Thanks,

davidmehi
Staff

Hi Ozan, do you know what the time improvement is by caching the KVM values? I thought the KVM was pretty much like a cache, so I'm wondering why we would need to cache the KVM values?

ozanseymen
Staff

Hi @dmehi@apigee.com - KVM data is coming from the data store. Data is not cached (apart from the internal cache that product has which is in seconds and invalidation is out of our control). Data you put in cache comes from the MP memory - apart from the first-ever-read per MP in which case it is either coming from C* L2 or target endpoint.

What I am talking about here is to cache data like api configuration for x hours (or days) and the ability to trigger invalidation whenever you want to refresh. If you use KeyValueMapOperations policy in the response flow without implementing some caching like this, you are basically going to the data store for each request for data that is potentially haven't changed for a long time.

I am seeing considerable improvement on latency when KVM values are cached like this, especially for 100+TPS usage.

Hope this helps?

sunandita_dam
Participant V

Hi @oseymen@apigee.com , I found your article on exception handling pattern very useful . Another interesting pattern you have come up with and so far my understanding was that KVMs are supported with in-built caching but it looks to me that's not the case.My question is can we separate out this piece to utility proxy and invoke them as needed as in our proxy we've 3-4 KVMs which are used and introducing these many policies for each KVM will make code difficult to understand. What do you say?

ozanseymen
Staff

Hi @Sunandita Dam

If you are using maven to do your deployments, I would consider using flow fragments to keep the xml readable with Proxy Dependency Plugin (https://github.com/apigee/proxy-dependency-maven-plugin).

Also see this for artefact reuse: https://community.apigee.com/content/kbentry/26266/best-practices-for-artefact-reuse-in-proxy-develo...

Another proxy to implement KVM access - in some form of internal configuration api - would also work. You will need to restrict access to it from external clients either using transport level (internal domain/port) or application level (authorization, oauth scopes).

sunandita_dam
Participant V

Thanks @oseymen@apigee.com for your reply.

Version history
Last update:
‎05-23-2016 05:28 AM
Updated by: