Challenges getting elements from xml payload using ExtractVariables policy/xpath

[scase]

We have xml requests coming through something like this:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
    <soapenv:Body>
        <lookupCodeRequest xmlns="http://schema.llbean.com/xml/ns/marketing">
            <codes xmlns:a="http://www.w3.org/2001/XMLSchema-instance" a:type="PromotionCode">
                <codeType>PROMOTION_CODE</codeType>
                <code>CARD25</code>
                <isIntegration>false</isIntegration>
            </codes>
            <asOfTimestamp>2019-10-07T13:35:42.836-04:00</asOfTimestamp>
            <includeActive>true</includeActive>
            <includeGracePeriod>false</includeGracePeriod>
            <includeInactive>false</includeInactive>

I am trying to capture all of the individual elements within <codes>. To get it to work at all--and not error--had to add the namespace. Here is an example from part of the policy:

<ExtractVariables name="ExtractVariables-XMLCacheVars">
   <Source>request</Source>
   <XMLPayload stopPayloadProcessing="false">
      <Namespaces>
            <Namespace prefix="soapenv">http://schemas.xmlsoap.org/soap/envelope/</Namespace>
      </Namespaces>
        <Variable name="codeType" type="string">
            <XPath>//*[local-name()='codeType']</XPath>
        </Variable>

This allows us to get values, but to get at the elements, I have only been able to get it via local-name. I tried using an explicit xpath like this example, but it didn't get the values

<Variable name="isIntegration" type="string">         <XPath>/soapenv:Envelope/soapenv:Body/lookupCodeRequest/codes/isIntegration</XPath>      </Variable>

The problem is that there may be multiple "code" elements, so we would like to be able to extract them by index value (ie ../lookupCodeRequest/codes[1]/isIntegration which doesn't work when using the local-name method in xpath as shown above.

Other than transforming the whole payload or using an XMLtoJSON policy, is there any way to accomplish this? I admit that I am not the most savvy when it comes to xpath, so it may be simple, but I haven't been able to figure it out.

[dchiesa1 · Accepted solution]

Your question seems to be mostly about Xpath, and not related to Apigee at all.

First, this xpath won't work:

/soapenv:Envelope/soapenv:Body/lookupCodeRequest/codes/isIntegration

The reason: you have a namespace prefix for the SOAP things, the Envelope and the Body. But the lookupCodeRequest is also contained within an XML Namespace, so you need a prefix for that too. There is no such thing as a "default namespace" in xpath. You have to use a prefix if there is a namespace.

For a simplified illustration, consider these example xml documents:

<root xmlns="foo"> <bar>A</bar> </root>	The root element and its child, bar, are both in the XML namespace "foo". The root because the namespace is declared with the xmlns="..." attribute on the element itself. The bar element inherits the namespace from its parent.
<root> <bar xmlns="foo">A</bar> </root>	The root element is in the default (blank) XML namespace. The bar element is in the foo namespace because the namespace is declared on that element with the xmlns="..." attribute on the bar element itself.
<root xmlns:f="foo"> <f:bar>A</f:bar> </root>	The root element is in the default (blank) XML namespace. The bar element is in the foo namespace because it uses the f: prefix, and the f: prefix is mapped to the "foo" namespace by the declaration on the root element.
<f:root xmlns:f="foo"> <bar xmlns="bax">A</bar> </f:root>	The root element is in the "foo" XML namespace, because it bears a prefix which is mapped to that namespace. The bar element is in the "bax" namespace because it has an xmlns="..." attribute.

The last case is most analogous to your SOAP document. The lookupCodeRequest element is analogous to the bar element in my example. It's in a namespace! Also, all of its children are in the same namespace.

The correct xpath for the isIntegration element is something like this:

/s:Envelope/s:Body/m:lookupCodeRequest/m:codes/m:isIntegration
			

...given prefixes like this:

s	http://schemas.xmlsoap.org/soap/envelope/
m	http://schema.llbean.com/xml/ns/marketing

Note: the presence of the trailing slash on the first namespace string and the absence of the trailing slash on the second string is significant. These are not urls, though they look like urls. They're just strings. XML uses exact string comparison to evaluate namespaces. If there's a trailing slash, then you need to always use the trailing slash. http://foo/ is not equivalent to http://foo.

Also: you may be aware but the prefix itself is not semantically significant. You can use SOAPENV or SOAP-ENV or soap or soapenv or S or s or bar or anything. The prefix is just a "variable name". The important thing is the thing, the namespace string, that the prefix refers to. In our xpath, we want s to refer to the well-known soap namespace. And we want m to refer to the marketing namespace. It does not matter that the original XML document used soapenv for the prefix referring to the soap namespace, and used no prefix at all for the marketing namespace. (Take note: XML allows a "default" (unprefixed) namespace, while xpath does not!)

The way you declare namespace and prefix mappings within the ExtractVariables policy is with the Namespaces element, like so:

<ExtractVariables name='EV-1'>
  <Source>request</Source>
  <IgnoreUnresolvedVariables>false</IgnoreUnresolvedVariables>
  <XMLPayload>
    <Namespaces>
      <Namespace prefix='s'>http://schemas.xmlsoap.org/soap/envelope/</Namespace>
      <Namespace prefix='m'>http://schema.llbean.com/xml/ns/marketing</Namespace>
    </Namespaces>
    <Variable name='isIntegration' type='string'>
      <XPath>/s:Envelope/s:Body/m:lookupCodeRequest/m:codes/m:isIntegration</XPath>
    </Variable>
  </XMLPayload>
</ExtractVariables>
			

---

OK, what do you mean by "extract them by index value" ?

Do you mean extract ALL of the codes irrespective of how many there are? That you cannot do with the ExtractVariables policy. For something like extracting an indeterminate number of things from an XML, I would suggest an XSLT; or a JSONToXML first, followed by a JavaScript policy.

Now if you know you will have 2 codes, or you know you only want to extract the FIRST discount code, and the SECOND discount code, you could do THAT with the ExtractVariables. And you can go up to any determinate number. For example given this policy

<ExtractVariables name='EV-1'>
  <Source>contrivedMessage</Source>
  <IgnoreUnresolvedVariables>false</IgnoreUnresolvedVariables>
  <XMLPayload>
  <Namespaces>
      <Namespace prefix='s'>http://schemas.xmlsoap.org/soap/envelope/</Namespace>
      <Namespace prefix='m'>http://schema.llbean.com/xml/ns/marketing</Namespace>
  </Namespaces>
    <Variable name='code1' type='string'>
      <XPath>/s:Envelope/s:Body/m:lookupCodeRequest/m:codes[1]/m:code</XPath>
    </Variable>
    <Variable name='code2' type='string'>
      <XPath>/s:Envelope/s:Body/m:lookupCodeRequest/m:codes[2]/m:code</XPath>
    </Variable>
    <Variable name='code3' type='string'>
      <XPath>/s:Envelope/s:Body/m:lookupCodeRequest/m:codes[3]/m:code</XPath>
    </Variable>
  </XMLPayload>
</ExtractVariables>

..and this SOAP request:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
    <soapenv:Body>
        <lookupCodeRequest xmlns="http://schema.llbean.com/xml/ns/marketing">
            <codes xmlns:a="http://www.w3.org/2001/XMLSchema-instance" a:type="PromotionCode">
                <codeType>PROMOTION_CODE</codeType>
                <code>CARD25</code>
                <isIntegration>false</isIntegration>
            </codes>
            <codes xmlns:a="http://www.w3.org/2001/XMLSchema-instance" a:type="DiscountCode">
                <codeType>DISCOUNT_CODE</codeType>
                <code>DISC27</code>
                <isIntegration>true</isIntegration>
            </codes>
            <asOfTimestamp>2019-10-07T13:35:42.836-04:00</asOfTimestamp>
            <includeActive>true</includeActive>
            <includeGracePeriod>false</includeGracePeriod>
            <includeInactive>false</includeInactive>
        </lookupCodeRequest>
    </soapenv:Body>
</soapenv:Envelope>

Then you would get these variables :

• code1 => CARD25
• code2 => DISC27
• code3 (unset)

[scase]

@Dino-at-Google

Thanks, that was exactly what I wanted. You were right about what I meant with regard to "extract them by index value". Basically, I need to know if there is just one codes element or multiples.