XML threat Protection getting failed with XML data if prefix information not specified

Not applicable
<par:synchronized></par:synchronized> XML Threat Protection failed
<par:synchronized xmlns:par=""></par:synchronized> XML threat protection passed

Client doesn't send us the in the latter format, hence is there any way to make XML threat protection works for above payload. Also, I noticed, if I didn't pass Content-Type Header the above condition works ok!!

Please let us now if this is encountered and how to resolve it.

0 10 664
10 REPLIES 10

I think you are using the XML threat protection policy. Can you show how you have it configured?

Also, the data the client is sending - is that ALL OF IT? What you showed, is that the entire document? Or is that just a fragment of a document?

Finally, if the client does not pass an XML content-type, then the XML Threat Protection policy will not apply. This is as documented and as expected. And as desired. If you want to harden your interface, you should reject requests (via a RaiseFault wrapped in a Condition) that do not have the appropriate content-type. This is a best practice.

<Step>
  <Name>RaiseFault-BadRequest</Name>
  <Condition>NOT (request.header.content-type =| "application/xml")</Condition>
</Step>

Not applicable

Hi Dino,

we are using xml threat protection, and I have just given the fragment of payload.The issue here is when Conetnt-Type as application/xml , XML threat protection policy is getting failed.

It was identified that as namespace definition was not given in payload, so it was getting failed. As soon as I give the namespace definition, it got passed.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<XMLThreatProtection async="false" continueOnError="true" enabled="true" name="XMLThreatProtection_Threat">
    <DisplayName>XMLThreatProtection_Threat</DisplayName>
    <Properties/>
    <NameLimits>
        <Element>60</Element>
        <Attribute>60</Attribute>
        <ProcessingInstructionTarget>60</ProcessingInstructionTarget>
    </NameLimits>
    <Source>request</Source>
    <StructureLimits>
        <NodeDepth>1</NodeDepth>
        <AttributeCountPerElement>4</AttributeCountPerElement>
        <NamespaceCountPerElement>3</NamespaceCountPerElement>
        <ChildCount includeComment="true" includeElement="true" includeProcessingInstruction="true" includeText="true">3</ChildCount>
    </StructureLimits>
    <ValueLimits>
        <Text>2048</Text>
        <Attribute>2048</Attribute>
        <NamespaceURI>256</NamespaceURI>
        <Comment>300</Comment>
        <ProcessingInstructionData>60</ProcessingInstructionData>
    </ValueLimits>
</XMLThreatProtection>

Not applicable
<NodeDepth>1</NodeDepth> mentioned in comment is 15, not 1

hi!

Well listen, I cannot tell for sure, but it may be that the missing namespace is causing the xml document to be treated as not well-formed. You gave me only a fragment so I cannot tell, for sure. But certainly, this is invalid XML:

<par:synchronized></par:synchronized>

The reason is that the par namespace prefix is not defined.

I think what is happening is: during execution of the XML Threat Protection policy, Apigee Edge READS the XML document. But this fragment, if included within a document that does not define par as a namespace prefix, will cause an error during reading. The XML Reader will be reject that as invalid XML. If you modify the fragment to define the namespace, Apigee will be able to read the XML as desired.

I don't know this for sure; I'm only surmising based on your description, that this is what is happening.

The solution is to pass well-formed XML.

Hmmmm, but shouldn't the XML Threat Protection policy reject the input (eg, throw a fault) when the inbound document is not well-formed? That seems like reasonable behavior to me. I'll have to investigate it a little more before I understand the situation completely... stay tuned.

Nope - My XMLThreatProtection policy throws a fault with an undefined prefix, as expected:

$ curl -i https://ORG-ENV.apigee.net/xmlthreat-1 -H content-type:text/xml -d '<root><par:synchronized></par:synchronized></root>'
HTTP/1.1 500 Internal Server Error
Date: Mon, 23 Oct 2017 22:30:53 GMT
Content-Type: application/json
Content-Length: 218
Connection: keep-alive
Server: Apigee Router

{"fault":{"faultstring":"XMLThreatProtection stepDefinition XMLTP-1: Execution failed. reason: Unresolved Prefix at line 1(possibly  around char 24)","detail":{"errorcode":"steps.xmlthreatprotection.ExecutionFailed"}}}

I think there is something else going on with your situatin, that I don't yet understand. If you could simplify the scenario - test simpler documents and try to narrow down what is going wrong, that would be helpful. It's obvious that it's /not/ simply the presence of an element like this:

<par:synchronized></par:synchronized>

By the way, how does the policy behave, when the document does and does not include the namepace prefix? Specifically what result do you see? Be explicit. In other words, please explain what you mean by

XML threat Protection getting failed

It may be that I have been misunderstanding you all along. When you say "getting failed", maybe you mean "The XMLThreatProtection policy is throwing a fault." If that is what you are reporting, then ... my response is... "that is expected behavior".

The XMLThreatProtection policy should throw a fault when passed XML that is not well-formed. This includes XML documents that include references to undefined namespace prefixes.

You understood it correctly, with undefined namespace in the payload, XML threat protection is throwing fault. Is it the expected behavior as you said.

actually the namespace prefix in payload is added in APIGEE policy when we prepare the SOAP request, but from client get the payload with namespace definition. Is there any way in which XML threat protection doesn't fail if client doesn't pass the namespace definition

If the client is not passing namespace info, you probably can use the following property

<NamespaceURI>0</NamespaceURI>.

This will validate an xml payload with no namespace.

@Dino, is this right?

This doesn't work on validating XML payload without namespace.

This doesn't work on validating XML payload without namespace.

I think we need to be careful about the words we're using. With the words, "This doesn't work" I think you mean "the policy is behaving exactly as documented and as designed."

Also, and I know this is going to sound pedantic, but we cannot say that the XMLThreatProtection policy is "validating" an XML document. The "validate" action is strictly defined in XML; it refers to verifying that a document complies with an XML schema, usually described in a .xsd file or a .dtd. There is no schema in this example, therefore it is not possible to "validate" the document.

I think what you are imagining is, when the client sends something that is not well-formed XML (for example missing the definition of a namespace prefix), you would like the XMLThreatProtection policy to treat it as well formed XML. And that's not possible. There are formal rules regarding the format of XML, and "namespace prefixes must be declared" is one of those rules. When a document violates any of the rules, the document is "not well formed", and in fact the document is not XML. XML parsers do not and cannot treat those documents as XML.

By the way, I am still not clear on the exact situation. This set of statements is not clear to me:

actually the namespace prefix in payload is added in APIGEE policy when we prepare the SOAP request, but from client get the payload with namespace definition. Is there any way in which XML threat protection doesn't fail if client doesn't pass the namespace definition

I cannot make out whether the client is sending well-formed XML or not. But anyway, I'm not sure it matters.

Step back and think about what the policy should do. If a client sends "ba.5985-+lsjkwefsijs" and says "this is XML", what should happen? The content of the message is obviously not well-formed XML. In this case the XMLThreatProtection policy should reject the message, right? We would want that to happen, am i correct? This seems like the appropriate, correct, desired behavior. And that is what the policy does, that's how it behaves.

Now suppose the message content is "<root>" . That is also not well-formed XML. It looks a little more like XML, in fact it looks like an "open element". But there is no closing element, and for that reason, it's not well-formed XML. (not well-formed XML = not XML). Again, we would want the XMLThreatProtection policy to throw an error. And it does.

Now suppose the message content is "<par:root>foo</par:root>". Once again, it looks like XML, but ... there's no declaration for that namespace prefix. And for that reason, according to the rules around XML, this is not well-formed XML. Therefore it cannot be treated as XML. And once again, we would expect the policy to throw a fault. And it does.

This seems like what we want to happen.

You may be thinking "Well, this XMLThreatProtection policy is too strict. The policy is the problem." OK, suppose you eliminate the XMLThreatProtection policy. I think your proxy also has an XSL policy in it. I would expect the subsequent XSL policy that you mentioned, to also throw a fault when the content uses undeclared xml prefixes, because in that case the input is not XML.

I think the solution is:

  • clients must send well-formed XML
  • if the client does not send well-formed XML, you should send a fault back to the client app saying "please send well-formed XML". You can use a FaultRule to handle the fault thrown by the XMLThreatProtection policy to send back a custom message to the client.

Thats correct Dino, we should be informing client about invalid XML. As we don't have any XSLT policy, we are just enclosing the invalid XML in SOAP envelopw with namespace definition, so that it becomes valid.

Also, I found the issue during testing error handling scenario. The issue I notice even though XML threat protection failed, it is not entering the fault rules section until some other policy fails.

So, why does XML threat protection even on failure continues with the flow. The ideal behaviour should be it should enter fault rules section.

Please ignore as I noticed continueonerror is true for this. I would like to now to handle this scenario if we enclose this in SOAP envelope and then do XML threat protection on the valid request, will that work