ExtractVariables with line break in json is not working.

[#1]

Hi ,

My application sends json payload with line break in value.ex.

{

"refid":"1234";

"mailingstreet":"asjdhwiiwjd

ajshjahahhak

ajshajs"

"number":"1234"

}

I use extract variable policy to fetch the refid value.But when I am using the above json content it fails due to a line break in the request .

Do we have any other option to make it work?

Even tried in JS still the same issue exists.

Thanks,

Sumiya.

[dubeysiddharth · Accepted solution]

Hello @sumiya try putting JS policy before extract variable policy:

JS Code

var _json = JSON.parse(JSON.stringify(context.getVariable('request.content').replace(/\s/g, ''))); 

context.setVariable('request.content', _json);

[dchiesa1]

You could do it more simply. You don't need to do JSON.parse and JSON.stringify.

var re1 = new RegExp('\\s', 'g');
context.setVariable('request.content', context.getVariable('request.content').replace(re1,' '));

But while that will remove the disallowed newlines in strings, it will not fix the field separators. The result is like this:

{  "refid":"1234";  "mailingstreet":"asjdhwiiwjd  ajshjahahhak  ajshajs"  "number":"1234"  }

In one case there is a semicolon as a field separator. In another case there is no field separator.

The original is not JSON. That is hard to fix with a regex.replace.

[dubeysiddharth]

You are right @DinoChiesa-at-Google. My original answer was based on the assumption that JSON's semicolon and missing comma was typo error.

[dchiesa1]

I suppose the reason your ExtractVariables policy is not working is, your JSON is not valid.

in JSON, strings cannot contain un-escaped newlines . Also, the field separator is a comma; your input contains a semicolon separator.

Whatever system is generating that "JSON" (but It's not actually JSON), needs to be fixed. That's where the bug is. When you get valid JSON, then the ExtractVariables will work properly.

[#6]

Hi Dino and siddharth,

Sorry for the confusion.

There was a typo in the provided request.

{

"refid":"1234",

"mailingstreet":"asjdhwiiwjd

ajshjahahhak

ajshajs",

"number":"1234"

}

Corrected it now.

Tried with the provided soluction.

var re1 = new RegExp('\\s', 'g');
context.setVariable('request.content', context.getVariable('request.content').replace(re1,' '));

As you mentioned @DinoChiesa-at-Google mailingstreet value has spaces after patching up the line spaces.

Still it acts like a proper json right?but Extract variable is not working as expected after using JS before the EV policy.

[dchiesa1]

Still it acts like a proper json right?

Yes, it does if you use the JS to fix it up. When I try this, it works. This is what the transformed (legal) JSON looks like:

{ "refid":"1234", "mailingstreet":"asjdhwiiwjd  ajshjahahhak  ajshajs", "number":"1234" }

And this is the ExtractVariables policy:

<ExtractVariables name='EV-AddressElements'>
  <Source>tempMessage</Source>
  <JSONPayload>
    <Variable name='number'>
      <JSONPath>$.number</JSONPath>
    </Variable>
    <Variable name='refid'>
      <JSONPath>$.refid</JSONPath>
    </Variable>
    <Variable name='mailingstreet'>
      <JSONPath>$.mailingstreet</JSONPath>
    </Variable>
  </JSONPayload>
</ExtractVariables>

And this is the result in trace. As you can see, the expected extracted variables get set .

There is a bug with ExtractVariables, in which it does not set Trace statements for the variables set by the policy, if you use a VariablePrefix. The variables actually get set, but you don't see them in the trace buffer. Is it possible you are experiencing this bug? You will hit this problem if you use a VariablePrefix . The fix is in testing now.

Attached is my API Proxy.

apiproxy-extract-variables-broken-json.zip

[#8]

@DinoChiesa-at-Google

Yes ,I experience this bug using EV.

Thanks Siddharth and Dino,

It works for me know.

@Dino I had posted one more question about JWT ,would you be able to help me on it.?

Thanks,

sumiya.