How can you Extract all values from a JSON hash into context variables?

Context variables in Apigee Edge are the things that hold state, or information about the message being processed. You use a context variable to read anything about the message, including any header, any query parameter or form parameter, the uri, or the message body (or content).

If you need to read a portion of the URI or message payload, you can't do it directly via context variables. For example if you want to read just one segment in the uri, or one element in an XML document, you cannot do it directly.

Apigee Edge includes a handy policy called ExtractVariables which lets you extract a portion of a JSON or XML or URLpath into a new variable . For example, starting with the uripath of /v1/foo/1234, you could extract the 1234 part into a context variable named "extracted.fooid". This might look like this:

<ExtractVariables name='EV-UriElement'>
   <DisplayName>Extract a portion of the url path</DisplayName>
   <Source>request</Source>
   <VariablePrefix>extracted</VariablePrefix>
   <URIPath>
      <!-- the extracted value gets placed into extracted.fooid -->
      <Pattern ignoreCase='true'>/v1/foo/{fooid}</Pattern>
   </URIPath>
   <IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
</ExtractVariables>

A similar extraction is possible with JSON payloads. If you have { "property1" : 76.23, "prop2" : false } , you can use JSONPath to specify that you'd like to extract the value of property1 into a context variable called "extracted.property1". The policy configuration looks like this:

<ExtractVariables name='EV-1'>
  <Source>message</Source>
  <VariablePrefix>extracted</VariablePrefix>
  <JSONPayload>
    <Variable name='property1'>
       <JSONPath>$.property1</JSONPath>
    </Variable>
  </JSONPayload>
</ExtractVariables>

It works similarly with XML payloads and xpath.

This is nice for targeted purposes. Especially when used with AccessEntity, which gives you access to all the custom properties and attributes associated to a Developer, or API Product, or an App. The typical scenario is, use AccessEntity to get the entity in XML format into a context variable, then use ExtractVariables to extract just one small piece of that information, one atom, into a separate context variable.

Examples of configurations for these policies:

<AccessEntity name='AE-1'>
  <EntityType value='app' />
  <EntityIdentifier type='consumerkey' ref='client_id' />
</AccessEntity>
<ExtractVariables name='EV-1'>
  <Source>AccessEntity.AE-1</Source>
  <VariablePrefix>entity</VariablePrefix>
  <XMLPayload>
    <Variable name='extracted_app_display_name' type='string'>
      <XPath>/App/Attributes/Attribute[Name='DisplayName']/Value/text()</XPath>
    </Variable>
    <Variable name='extracted_consumer_secret' type='string'>
      <XPath>/App/Credentials/Credential[0]/ConsumerSecret/text()</XPath>
    </Variable>
  </XMLPayload>
</ExtractVariables>

But sometimes you don't want to extract one or two elements from a JSON. Sometimes you have a JSON payload and you want to have all of the information items just available for access from any policy. In other words, you'd like to have some sort of way to read any part of the JSON tree, just by reading a context variable.

Here's a way that I know of to do this.

Bulk Extraction

The first way is to walk the JSON object and extract each item into its own context variable. Do this with a JavaScript policy. You just use the introspection capability of JS to do the right thing. The code is re-usable, works with any JSON hash. It looks like this:

var variableName = "message.content";
var hash = JSON.parse(context.getVariable(variableName));
walkObj(hash,
        'json',
        function(name, value) {
              context.setVariable(name, value);
        });

What's going on there? It's pretty simple: Parse the string as JSON, then ... recursively walk the tree of the object and call a function to set a context variable for each item in the tree. For an input JSON like this:

{
  "name": "rooms/ABBa-BE7/conversations/qo6NAAE/messages/Kokoisk334",
  "sender": {
    "name": "users/11388443469853495",
    "displayName": "Dino Chiesa",
    "avatarUrl": "https://lh4.googleusercontent.com/ADg/hxfWDAGRhDY/photo.jpg",
    "email": "dchiesa@google.com"
  },
  "createTime": "2017-03-07T23:53:04.504207Z",
  "text": "/jira MGMT-3909"
}

...and using 'json' as the prefix as in the sample code, I get context variables like this:

json.name = "rooms/ABBa-BE7/conversations/qo6NAAE/messages/Kokoisk334"
json.sender.name = "users/11388443469853495"
json.sender.displayName = "Dino Chiesa"
json.sender.avatarUrl = "https://lh4.googleusercontent.com/ADg/hxfWDAGRhDY/photo.jpg"
json.sender.email = "dchiesa@google.com"
json.createTime = "2017-03-07T23:53:04.504207Z"
json.text = "/jira MGMT-3909"

The magic is all in that walkObj function. That's just a recursive fn that walks the entire graph in the JSON hash. It works with arrays and nested hashes.

Here's a gist containing all the code necessary for this approach, and an example policy configuration. After you extract all the json data into variables, you can just access them like any other context variable.

JSONPath within MessageTemplate

Another approach is more targeted. Rather than extracting everything, you could extract just the thing you want, within a "Message Template". You use a static function called jsonPath to look into a JSON string. Suppose your input JSON is as above. You could then use a jsonPath like so to get the avatarUrl:

{jsonPath($.name.sender.avatarUrl, message.content)}

This is lots cleaner but: it works only within a Message Template. This is the thing that gets expanded in the AssignMessage / Set / Payload , or in lots of other places inside policies.

Comments
jonesfloyd
Staff

This is brilliant, Dino. Thanks!

Version history
Last update:
‎04-20-2018 09:04 AM
Updated by: