Accessing children json element without ExtractVariable

[patryklapiezo]

I'm facing the following issue:

I'm calling a service that returns xml.
Then I use XMLToJson policy to transform the response to json and store it in GetResponseJson variable.
Then I use AssignMessage to create json payload.

The relevant part of AssignMessage policy is

        <Payload variablePrefix="#" variableSuffix="$" contentType="application/json">
            {
                "policyNumber" : "#apigee.policyNumber$",
                "retrievedPolicy": #GetResponseJson$
            }
        </Payload>

This works (returns expected response), however, I do not really want to do exactly that.

Instead, I want to return a children (Envelope) element of GetResponseJson, like that:

        <Payload variablePrefix="#" variableSuffix="$" contentType="application/json">
            {
                "policyNumber" : "#apigee.policyNumber$",
                "retrievedPolicy": #GetResponseJson.Envelope$
            }
        </Payload>

Now, this returns empty retrievedPolicy element.

Why is that?

I know that there is a workaround with ExtractVariable policy, however it only makes sense for cases as simple as that (it's very simple PoC), in any real situation it would be ridiculous to use.

If I had to return 20 different children from different nesting levels of GetResponseJson with different names I would have to create 20 ExtractVariables as workarounds.

[#2]

can you share the actual XML or json response?

[dchiesa1 · Accepted solution]

First thing. FYI. Back in the day, the message template in Apigee did not support using curly braces to denote variable references, when assigning to a payload that used curlies otherwise, like a json payload. That's why you see older examples that use the # and $ as prefix and suffix. This is no longer necessary. You can now assign a JSON payload like this:

<Payload contentType="application/json">
{
  "policyNumber" : "{apigee.policyNumber}",
  "retrievedPolicy": {GetResponseJson}
}
</Payload>

To me that is easier on the eyes.

But that doesn't answer your question. Basically, if I understand your question correctly, you have a JSON (GetResponseJson), and you want to convert it to a different JSON. It just so happens that the "source" JSON is derived from an XMLtoJSON policy, but that is somewhat irrelevant for our purposes. You wrote:

I think you want to extract arbitrary fields, at arbitrary nesting depth, from the source JSON and embed those into a different JSON. Is that right?

Essentially that is a JSON-to-JSON policy. With XML, there is a tool that can do that: XSLT. It accepts an XML document as input, and then produces another document (typically an XML document; more about this in a moment) as output. With JSON, there is no JSON-to-JSON transform language. There is no specific language designed for transforming JSON to another JSON.

Option 1

The good news is there is a general-purpose language that you can use for transforming JSON to another JSON: JavaScript. It's really straightforward and pretty easy.

You didn't give an example of the source JSON with the "up to 20" fields and the target destination JSON. So I will contrive an example for the purposes of this answer. Suppose the source JSON is like this.

{
      "policy" : {
        "number" : "82901-jshs",
        "subject" : {
          "name" : {
            "given": "Chris",
            "surname": "Okanawee"
          },
          "ratePlan": "A89"
        },
        "inception": "20210524",
        "package" : {
          "id" : "78Y",
          "coverage": "pq23"
        }
      },
      "agent" : {
        "id" : "abc123"
      }
    }

And you want the destination, the target JSON, to look like this:

{
  "policyNumber": "82901-jshs",
  "coveredName": "Chris Okanawee",
  "inception": "20210524",
  "packageId": "78Y"
}

The JS code that you can execute in a JS callout within Apigee to produce this is:

var originalJson = context.getVariable('GetResponseJson');
var source = JSON.parse(originalJson);
var transformed = {
      policyNumber: source.policy.number,
      coveredName: source.policy.subject.name.given + " " + source.policy.subject.name.surname,
      inception: source.policy.inception,
      packageId: source.policy.package.id
    };
var destination = JSON.stringify(transformed, null, 2);
context.setVariable('transformedJson', destination);

I hope you can see what's going on there. If you want a different transform, then just use different JavaScript. The reason you need to resort to JavaScript here, is that "GetResponseJson" is just a string. It happens to contain JSON, but the context variable in Apigee is a plain string, and therefore referring to a field within the JSON, like GetResponseJson.Envelope... is not going to work. That isn't a known context variable to Apigee, and Apigee doesn't know to automatically treat a String which happens to contain curly braces as a JSON. But within the JS callout, you invoke JSON.parse(), and that does what you want - it makes all the child fields available for reference.

---

Option 2

There is an alternative. If you DO want to rely on AssignMessage, because it's easier to read and maintain, then you may just want to shred ALL of the JSON into distinct context variables. You can do that, too, but it requires (again) JavaScript. In this case it would be a general-purpose JavaScript.

As an example, for the above "source" JSON, you can us JS to set all of these context variables:

json.policy.number = "82901-jshs"
json.policy.subject.name.given = "Chris"
json.policy.subject.name.surname = "Okanawee"
json.policy.subject.ratePlan = "A89"
json.policy.inception = "20210524"
json.policy.package.id = "78Y"
json.policy.package.coverage = "pq23"
json.agent.id = "abc123"

At that point you would be able to use AssignMessage and use variable references specifying any of the things in the list above, to create your new message or string.

To make this magic happen you can use a JS policy like this:

<Javascript name='JS-ShredJSON' timeLimit='200' >
  <Properties>
    <Property name='output-prefix'>json</Property>
    <Property name='source'>ContextVariableContainingJSON</Property>
  </Properties>
  <ResourceURL>jsc://extractJsonToContextVars.js</ResourceURL>
</Javascript>

The magic happens in the extractJsonToContextVars.js file, which is pretty simple: it just walks the JSON structure, setting a context variable for each field. You can find usable code here. This will work without modification in Apigee.

---

Option 3

I can think of a third option. Use XML to transform to JSON itself. You said you used the XMLToJSON policy to convert the XML (I suppose it is SOAP, since there is an Envelope field) into JSON. If you are savvy with XSLT, you can use an XSLT script to do the transform in one step. Your XSLT script would emit JSON, not XML.

This will probably perform pretty well, but for some people XSLT is not super readable or maintainable, so they avoid it. Anyway it's an option. If you want this option, Google around for hints on how to use XSLT to JSON. This is not an Apigee-specific technique; any XSLT that does this, will work in Apigee.

Here is an example XSLT from Stackoverflow that transforms XML to JSON.

[patryklapiezo]

Hi, thank you for the answer. Option #1 seems most universal to me (well #2 is similar but pollutes variable namespace). To be honest I'm comparing different tools for api management / mediation and so on, and the expectation I had was that corresponding tools between mulesoft and apigee would have similiar functionalities (Assign Message <-> Set Payload), even though admittedly mule uses its own language in that case (DataWeave).
I tried avoiding falling back to writing code, as I'm building really basic PoC right now, but I guess creating JS script would not actually be that much different than writing dataweave transformation.