Re: request.content showing data elements which ar...

scase · 07-19-2021 12:02 PM

With assistance, I was able to mask request xpath elements. However, now the elements are still showing with their respective values in request.content. Of course, I know that I can set request.content or message.content to be masked within data masking, but this blocks out the entire payload. We were hoping to try and keep much of the rest of the request viewable for monitoring and troubleshooting.

Is there a way to have it show the request payload in trace, but only mask the desired elements?

Former Community Member

I'm not sure I fully understand the use case. The debugsession API supports XPath and JSONPath for Requests and Responses. Can you please help me understand with an example, why this doesn't meet your use case?

scase

@Former Community Member So, the XPath is working fine for data masking in "Body" under the "Request Content" section. However, up above under "Variables" in the trace, the full value of "request.content" is visible including the xml node which was masked under "Request Content".

If I set "request.content" to be data masked, then the full request payload everywhere is masked which prevents us from seeing the rest of the payload which we didn't want to have masked.

Former Community Member

I wonder if you can use JSON Path and XPath to your advantage like this:

debugMask

And this shows up in debug session as:

scase

Unfortunately, that didn't seem to do it, perhaps because the request payload is SOAP and includes a customer namespace.

dchiesa1

This shouldn't be an impediment. You need to just declare a prefix for the namespace in the maskconfigs, and then use that prefix in your xpath. You probably need to declare the SOAP namespace too.

For example I used this maskconfig

{
  "name": "default",
  "namespaces": [
    {
      "prefix": "ns",
      "value": "http://abc.com/Message"
    },
    {
    "prefix" : "ns1",
    "value": "urn://C593C08C-A6D3-4806-A3C6-BE232825D97A"
     },
    {
    "prefix" : "soap11",
    "value": "http://schemas.xmlsoap.org/soap/envelope/"
     },
    {
    "prefix" : "wsse",
    "value": "http://schemas.xmlsoap.org/ws/2003/06/secext"
     }
  ],
  "variables": [
  ],
  "xPathsRequest": [
    "//child/@sensitiveInfo",
    "/ns:Message/ns:SensitiveInfo",
    "/soap11:Envelope/soap11:Header/wsse:Security/wsse:UsernameToken/wsse:Username",
    "/soap11:Envelope/soap11:Header/wsse:Security/wsse:UsernameToken/wsse:Password",
    "/soap11:Envelope/soap11:Body/ns1:Operation1"
  ]
}

And this payload:

<soap:Envelope xmlns:ns1="urn://C593C08C-A6D3-4806-A3C6-BE232825D97A"
               xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">

 <soap:Header>
  <wsse:Security xmlns:wsse="http://schemas.xmlsoap.org/ws/2003/06/secext">
   <wsse:UsernameToken wsu:Id="element-123"
       xmlns:wsu="http://schemas.xmlsoap.org/ws/2003/06/utility">
    <wsse:Username>person@example.com</wsse:Username>
    <wsse:Password Type="wsse:PasswordText">VerySecret!</wsse:Password>
    <wsu:Created>2020-10-02T08:44:51Z</wsu:Created>
   </wsse:UsernameToken>
  </wsse:Security>
 </soap:Header>

  <soap:Body>
   <ns1:Operation1>hello</ns1:Operation1>
  </soap:Body>
</soap:Envelope>

And I see these results, as expected:

dchiesa1

I provided an earlier response that was flat wrong, so I've deleted it.

Here's what I found: I can set a maskconfig that specifies xpath queries, and it works on the Body (as shown in the Trace UI) as well as on the variable "request.content".

Here's my maskconfig:

{
  "name": "default",
  "namespaces": [
    {
      "prefix": "ns",
      "value": "http://abc.com/Message"
    }
  ],
  "variables": [],
  "xPathsRequest": [
    "//child/@sensitiveInfo",
    "/ns:Message/ns:SensitiveInfo"
  ],
  "xPathsResponse": [
    "//child/@sensitiveInfo",
    "/ns:Message/ns:SensitiveInfo",
    "/ns:Message/ns:Id/@*"
  ]
}

What that says is... for any XML request payload, check it for the given xpaths, and mask the matching nodesets. And similarly, for different xpaths, on the response.

For this request payload:

<Message xmlns='http://abc.com/Message'>
  <Id source='id-source-1'>43AC497B-4C6A-47DA-B823-29E095BFDF13</Id>
  <SensitiveInfo>something-here</SensitiveInfo>
</Message>

...then I get this in the Trace UI.

If I highlight a policy that READs the context variable "request.content", then... the contents of that variable is also masked. See here:

A similar thing happens with response.content.

Some things you need to be careful of:

after updating the maskconfigs, you must start a new trace session in order to get the new behavior.
If you READ the request.content variable and write it into some other variable, that other variable will not be masked. (Unless it is explicitly on the masked variables list)

Let me know if further questions.

scase

@dchiesa1 et al, I think I found the issue. The credentials were coming out of an encrypted kvm and were being held into a variable named

private.MerchantKey. This variable was not visible in other places and within the content, the xpath was masking it. However, in order to make it not visible within the request.content variable in the top section (under variables) I needed to add this to my data mask definition:

<Variables>
    <Variable>private.MerchantKey</Variable>
</Variables>

I was surprised that in this situation, a private. variable from an encrypted kvm would need to be data masked, but that seemed to resolve the issue!

sillan_dt

you cannot to partial masking if you are not recreating the payload. If you want to hide that in trace then you need to store it in private variable and remove the original payload.

If you want to show masked in the trace then you can try printing that within javascript policy.

scase

@sillan_dt I was storing it in a private variable. Please see the solution I found, above.

request.content showing data elements which are being masked.