Cache all sub variables

olofhaglund
Participant III

I want to cache all sub variables. From what I have understod it is not possible from the built in policies but are it possible to do that from a JS script? Hopefully when we extract more data we would not need to update the policy/script.

If we take the following extraction as an example I would like to automatically cache a new variable that I add to this list to be extracted.

    <JSONPayload>
        <Variable name="var.id" type="string">
            <JSONPath>$.id</JSONPath>
        </Variable>
        <Variable name="var.id2" type="string">
            <JSONPath>$.roles[0].id2</JSONPath>
        </Variable>
        <Variable name="var.id3" type="string">
            <JSONPath>$.roles[0].id3</JSONPath>
        </Variable>
        <Variable name="var.id4" type="string">
            <JSONPath>$.roles[0].id4</JSONPath>
        </Variable>
    </JSONPayload>
0 4 180
4 REPLIES 4

What do you mean by "cache all sub variables"?

I think what you are saying is... you want to start wuth a JSON hash, something like this:

{
  "id" : "837iuhdjk",
  "roles" : [
    "id1" : "98ui4ej",
    "id2" : "4768yr",
    "id3" : "37yur",
    "id4" : "9t8yuh"
  ]
}

And... you would like to extract all of that data into discrete context variables, so that you have context variables like

something.id = 837iuhdjk
something.roles[0].id1 = 98ui4ej

...and so on.

Is that right?

The reason for my question asking for clarification: There is a thing called a "Cache" in Apigee Edge, and what you are describing does not seem to be related to the cache. I want to make sure.

Not applicable

If you are asking to extract a dynamic number of variables to be cached in your api flow, you can use java script policy. JSON Payload can be easily parsed inside the policy and dynamically values can be assigned to variables. I am not talking about the caching of data into apigee cluster.

olofhaglund
Participant III

@Dino-at-Google I do a service callout to a service that holds all IDs and so on in our company. The result is a JSON string and I extract some of those IDs to var.*

Those extracted IDs I want to cache so I don't need to run the extract JSON code on each request.

ok, be careful.

I think a cache wrapped around a network call makes sense. The network call can take 50, 100, 200ms. Using a cache lookup in lieu of making that network call, means you will spend <1ms. Obviously a huge performance win.

I think you are talking about wrapping a cache around the "extract JSON" call. That's not a huge performance win. Extracting variables from json will be fast. There's no I/O, there's no network or disk access. This will take <1ms also. The cache lookup will take about the same about of time. Except the management of N items of already-extracted data in he cache will be more complicated to describe in the DSL that is Apigee Edge flow logic.

The PopulateCache can take a single value, and place it into cache. If you want to cache N distinct values, you need N distinct PopulateCache policies. Likewise for LookupCache. It feels to me that caching N items separately, in order to avoid an ExtractVariables call which is fast anyway, is not a good idea.

I suggest that you wrap the Cache around the ServiceCallout call, and always run ExtractVariables whether you obtain the value from the network or the cache. That makes the most sense.

Ok, so what are you stuck on? What is the obstacle?

I think the PopulateCache policy would look like this:

<PopulateCache name='PC-1'>
  <CacheResource>cache1</CacheResource>
  <Source>serviceCalloutResponse</Source>
  <Scope>Application</Scope>
  <CacheKey>
    <Prefix>fixedPrefix</Prefix>
    <KeyFragment ref='variable.containing.keyfrag' />
  </CacheKey>
  <ExpirySettings>
    <TimeoutInSec ref='duration_variable'>120</TimeoutInSec>
  </ExpirySettings>
</PopulateCache>
<br>

...where serviceCalloutResponse is the name of the variable that you specified in the ServiceCallout policy to hold the response.

The LookupCache is similar. The flow logic will look something like this:

        <!-- 1. check cache for the data -->
        <Step>
          <Name>CacheLookup-MyData</Name>
        </Step>
        <!-- 2. make ServiceCallout if cache miss -->
        <Step>
          <Name>ServiceCallout-GetMyData</Name>
          <Condition>cached.data = null</Condition>
        </Step>
        <!-- 3. populate cache if previously there was a cache miss -->
        <Step>
          <Name>CacheInsert-MyData</Name>
          <Condition>cached.data = null</Condition>
        </Step>
        <!-- 4. assign the cached data to the variable we will use -->
        <Step>
          <Name>AV-MyData</Name>
          <Condition>cached.data != null</Condition>
        </Step>
        <!-- 5. extract the data items from the json -->
        <Step>
          <Name>ExtractVariables-MyData</Name>
        </Step>
         ...

The ExtractVariables is always executed, whether the proxy uses cached data or not.