Data Masking not working as expected at AssignMessage policy

[Prashant819]

Hello,

I am facing an issue with data masking when AssignMessage policy is used to set a payload.

Here is my example scenario -

1. API receives Json payload request with 'Password' field in it.

{
	"credentials": {
		"UserId":"1234",
		"password":"MyPassword"
	}
}

2. API extracts the password field from request json into a private variable.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ExtractVariables async="false" continueOnError="false" enabled="true" name="EV-FetchPassword">
    <DisplayName>EV-FetchPassword</DisplayName>
    <Properties/>
    <IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
    <JSONPayload>
        <Variable name="private.password">
            <JSONPath>$.credentials.password</JSONPath>
        </Variable>
    </JSONPayload>
    <Source clearPayload="false">request</Source>
</ExtractVariables>

3. Private variable for password is then used in Assign Message policy to build a SOAP XML payload to be sent to back-end target.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<AssignMessage async="false" continueOnError="false" enabled="true" name="AM-BuildBackendSoapRequest">
    <DisplayName>AM-BuildBackendSoapRequest</DisplayName>
    <Properties/>
    <Set>
        <Payload contentType="application/xml">
            <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns="http://abc.com/Request">
                <soapenv:Header/>
                <soapenv:Body>
                    <ns:Request>
                        <ns:UserId>5678</ns:UserId>
                        <ns:Password>{private.password}</ns:Password>
                        <ns:OtherFields>OtherFields</ns:OtherFields>
                    </ns:Request>
                </soapenv:Body>
            </soapenv:Envelope>
        </Payload>
    </Set>
    <IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
    <AssignTo createNew="false" transport="http" type="request"/>
</AssignMessage>

4. As password is sensitive information, it is supposed to be masked and hidden everywhere in the api trace. Here is the masking configuration set in the environment.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<MaskDataConfiguration name="default">
    <JSONPathsRequest>
        <JSONPathRequest>$.credentials.password</JSONPathRequest>
    </JSONPathsRequest>
    <JSONPathsResponse>
        <JSONPathResponse>$.credentials.password</JSONPathResponse>
    </JSONPathsResponse>
    <Namespaces>
        <Namespace prefix="ns">http://abc.com/Request</Namespace>
    </Namespaces>
    <Variables>
        <Variable>request.content.credentials.password</Variable>
        <Variable>response.content.credentials.password</Variable>
    </Variables>
    <XPathsRequest>
        <XPathRequest>//ns:Password</XPathRequest>
    </XPathsRequest>
    <XPathsResponse>
        <XPathResponse>//ns:Password</XPathResponse>
    </XPathsResponse>
</MaskDataConfiguration>

When request is executed to the API endpoint -

1. Password field value is masked at each stage in the API trace except at AssignMessage Policy.

2. At AssignMessage stage, payload has password masked but trace also shows message.content variable with Password value unmasked.

Why does it show message.content flow variable? If it is there to display the newly constructed payload, it should use the data masking config available in the environment to mask the the password value.

Please note that masking whole message.content as a variable is not a good approach as we might need to check few fields in payload at various policies.

Thanks. @Dino-at-Google @Anil Sagar @ Google

[sidd-harth]

It is the expected behavior of Assign Message policy to display both masked data and encrypted values.

BTW message.content is again a flow variable and as it is not defined in MaskConfig, nothing is masked. AFAIK, there is no way to mask a specific field using message.content flow variable.

Generally speaking, sending plain passwords in the payload is also not recommended, we encrypt/hash them using an algorithm on the frontend side and pass the encrypted value to the next component.

+ @Dino-at-Google

[Prashant819]

Hello Siddharth,

Password is just an example in this case. There can be many other fields which need to be masked in trace.

If Apigee data masking config supports fields from request and response JSON/XML payload , then I think it should have handled the message.content variable based on request/response context flow.

Thanks.

[dchiesa1 · Accepted solution]

Can you please try using this in AssignMessage ?

 <AssignTo>request</AssignTo>

...in place of

 <AssignTo createNew="false" transport="http" type="request"/>

Because you are not explicitly specifying a message variable with AssignTo, the AssignMessage policy is assigning to "message", and this variable is not covered by the <XPathsRequest> element in the MaskDataConfiguration .

I just tested this, and it works for me. When I use the former configuration, the masking works as desired.

When I use the latter, the password is not masked.

[Prashant819]

Thanks Dino . It works.

However I still think that it should have resolved the AssignTo variable as per execution flow i.e request in this case. and should have applied the mask config accordingly.

https://docs.apigee.com/api-platform/reference/policies/assign-message-policy#assignto

[venkatvenkatsiv]

Hi Dino,

I'm trying to create a new DataMask Configuration. This is API proxy specific Data mask configuration. Below is the mask configuration XML.

<MaskDataConfiguration name="CustomName"> <JSONPathsRequest> <JSONPathRequest>$.password</JSONPathRequest> </JSONPathsRequest> <JSONPathsResponse> <JSONPathResponse>$.store.book[*].author</JSONPathResponse> </JSONPathsResponse> <Variables> <Variable>requestPath.private.password</Variable> </Variables> </MaskDataConfiguration>

How ever while creating mask configuration through enterprise API, I'm getting below error response. I have checked the API url and it is accurate.

{ "code": "distribution.InvalidMaskConfigurationId", "message": "Invalid mask configuration id", "contexts": [] }

I'm able to successfully create mask configuration using the name "default". That is below XML does not have any issue.

<MaskDataConfiguration name="default"> <JSONPathsRequest> <JSONPathRequest>$.password</JSONPathRequest> </JSONPathsRequest> <JSONPathsResponse> <JSONPathResponse>$.store.book[*].author</JSONPathResponse> </JSONPathsResponse> <Variables> <Variable>requestPath.private.password</Variable> </Variables> </MaskDataConfiguration>

Are there any naming constraints for creating Data mask configuration?

[dchiesa1]

Yes, that is a confusing part of the Apigee admin api. The only name permitted there is “default”.

[venkatvenkatsiv]

ok. Thanks for the update Dino!

[dchiesa1]

Really good question, thanks for all the detail. I salute you for taking the time to include the relevant policy information and even trace UI screenshots. This tells us exctly what you're doing. Also you explained clearly what you are seeing and what you are expecting to see.

This makes it really easy to have a conversation.

Many times I see people saying "Mask data config is not working. Please help!"

And then we have 15 back-and-forth Q&A to get relevant information. But your question is clear and complete. I hope we'll be able to resolve and explain it quickly.