About patten matching in ExtractVariables policy

[sgilson]

The ExtractVariables policy extracts information from a request or response and writes that information to a variable. For each type of information that you can extract, such as URI path or XML data, you specify the pattern to match and the name of the variable used to hold the extracted information. However, the way pattern matching works depends on the source of the extraction.

The following sections describe the two basic categories of information that you can extract.

Matching URI paths, query parameters, headers, form parameters, and variables

When extracting information from a URI path, query parameters, headers, form parameters, and variables you use the <Pattern> tag to specify one or more patterns to match.

For example, the following policy example shows a single matching pattern for the URI path:

<ExtractVariables name="ExtractVariables-1">
   <Source>request</Source>
   <URIPath>
      <Pattern ignoreCase="true">/a/{pathSeg}</Pattern>
   </URIPath>
   <VariablePrefix>urirequest</VariablePrefix>
   <IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
</ExtractVariables> 

In this example, the urirequest.pathSeg variable is set to whatever appears in the URI path after "/a/". For example, for the following request URL, the variable is set to "b":

http://myCo.com/a/b

You can specify multiple patters to match, where:

• All patterns are tested for match.
• If none of the patterns match, the policy does nothing and the variable(s) contain no value.
• If more than one pattern matches, the pattern with longest path segments is used for extraction.
• If two matched patterns has same longest path segments, then the pattern specified first in the policy is used for extraction.

In the next example, you create a policy that contains three matching patterns for the URI path:

<ExtractVariables name="ExtractVariables-1">
   <Source>request</Source>
   <URIPath>
      <Pattern ignoreCase="true">/a/{pathSeg}</Pattern>
      <Pattern ignoreCase="true">/a/b/{pathSeg}</Pattern>
      <Pattern ignoreCase="true">/a/b/c/{pathSeg}</Pattern>
   </URIPath>
   <VariablePrefix>urirequest</VariablePrefix>
   <IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
</ExtractVariables> 

The request URL to the API proxy is in the form:

http://myCo.com/a/b

In this example, the first pattern matches the URI and the urirequest.pathSeg variable is set to "b". If the request URL is:

http://myCo.com/a/b/c/d 

Then the third pattern matches and the urirequest.pathSeg variable is set to "d".

You can specify multiple variables in the matching pattern. For example, you specify a matching pattern with two variables:

<ExtractVariables name="ExtractVariables-1">
   <Source>request</Source>
   <URIPath>
      <Pattern ignoreCase="true">/a/{pathSeg}</Pattern>
      <Pattern ignoreCase="true">/a/b/{pathSeg}</Pattern>
      <Pattern ignoreCase="true">/a/{pathSeg1}/c/{pathSeg2}</Pattern>
   </URIPath>
   <VariablePrefix>urirequest</VariablePrefix>
   <IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
</ExtractVariables> 

For the request URL:

http://myCo.com/a/b/c/d 

The urirequest.pathSeg1 variable is set to "b" and the urirequest.pathSeg2 variable is set to "d".

You can also match patterns when there are multiple instances of an item with the same name. For example, you can make a request that contains multiple query parameters or multiple headers with the same name. The following request contains two query parameters named "w":

http://myCo.com/a/b/c/d?w=1&w=2 

To reference these query parameters in the ExtractVariables policy, you use indexes, where the first instance of the query parameter has no index, the second is at index 2, the third at index 3, etc.

For example, the following policy extracts the value of the second query parameter named "w" in the request:

<ExtractVariables name="ExtractVariables-1">
   <Source>request</Source>
   <QueryParam name="w.2">
      <Pattern ignoreCase="true">{secondW}</Pattern>
   </QueryParam>
   <VariablePrefix>urirequest</VariablePrefix>
   <IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
</ExtractVariables> 

The urirequest.secondW variable is set to "2". If the second query parameter is omitted from the request, then the urirequest.secondW variable is empty. Use indexing any time there are multiple items with the same name in the request.

Using special characters in the pattern

When matching URI paths, you can use the "*" and "**" wildcard characters in the pattern, where:

• "*" matches any one segments of the path
• "**" matches multiple segments of the path

For example, you specify patterns to the <URIPath> element as shown below:

<URIPath>
  <Pattern ignoreCase="true">/a/*/{id}</Pattern>
  <Pattern ignoreCase="true">/a/**/{id}</Pattern>
</URIPath> 

The first pattern matches URIs such as "/a/b/c", "/a/foo/bar", etc. The second pattern matches any number of path segments after "/a/", such as "/a/foo/bar/baz/c".

When specifying patterns to query parameters, headers, and form parameters, the "*"character specifies to match any number of characters. For example, when matching a header, specify the pattern as:

*;charset={encoding} 

This pattern matches the values "text/xml;charset=UTF-16" and "application/xml;charset=ASCII".

If the value passed to the ExtractVariables policy contains a special character, such as "{", use the "%" to escape it. The following example escapes the "{" and "}" characters in the pattern because they are used as literal characters in the value of the parameter:

<QueryParam>
  <Pattern ignoreCase="true">%{user%} {name}</Pattern>
</QueryParam> 

In this example, the pattern matches the value "{user} Steve" but not the value "user Steve".

Matching JSON and XML

When extracting data from JSON and XML, you specify one or more <Variable> tags in the policy. The <Variable> tag specifies the name of the destination variable where the extracted information is stored, and the JsonPath (JSON) or XPATH (XML) to the extracted information.

All <Variable> tags in the policy are evaluated, so that you can populate multiple variables from a single policy. If the <Variable> tag does not evaluate to a valid field in the JSON or XML, then the corresponding variable is not set.

The following example shows an ExtractVariables policy that populates two variables from the JSON body of a response:

<ExtractVariables name="ExtractVariables-3">
   <Source>response</Source>
   <JSONPayload>
      <Variable name="latitude" type="float">
         <JSONPath>$.results[0].geometry.location.lat</JSONPath>
      </Variable>
      <Variable name="longitude" type="float">
         <JSONPath>$.results[0].geometry.location.lng</JSONPath>
      </Variable>
   </JSONPayload>
   <VariablePrefix>geocoderesponse</VariablePrefix>
</ExtractVariables>

[#2]

Thanks for this wonderful article and explanation. Was really very helpful to me although I had some issues with the path in URIPath/Pathern in which I was also mentioning the Basepath. You need to omit Basepath. Please also refer to https://stackoverflow.com/questions/21191759/extract-variable-policy-not-working

Its something like this:

<URIPath><PatternignoreCase="true">/pathsuffix/{pathVariable_1}/.../{pathVariable_n}/...</Pattern></URIPath>

Thanks,

Sanjay