I need to convert Xop include into binary format in Apigee. How can we do it?

[TejpalSingh]

l had a requirement where I will be having one xml payload with XOP include object (Not a multipart request). I need to convert that XOP object into Binary format (base64 encoded string) and update the xml payload. Can you please help how we can do this in Apigee?

@dchiesa1 : Could you please suggest how this can be implement either with javacallout or Javascript

[dchiesa1 · Accepted solution]

ok Tejpal

I've updated the Java callout. It now handles multiple attachments with the TRANSFORM_TO_EMBEDDED action. You can get the latest from the repo, and try it. 

But before you try it with your message, one note.

The XML document in your example input, which begins like this:

<soapenv:Body>
<ns3:process
xmlns:ns3="http://abc.com/"
xmlns:ns2="http://ACORD.org/Standards/Life/2">
<ns2:withMessg>
 ...

...is not well-formed. The opening element there, soapenv:Body , refers to a namespace prefix soapenv which is not declared. Normally I would expect an Envelope element surrounding the Body. Like this:

<soapenv:Envelope xmlns:soapenv='http://schemas.xmlsoap.org/soap/envelope/'>
<soapenv:Body>
<ns3:process
xmlns:ns3="http://abc.com/"
xmlns:ns2="http://ACORD.org/Standards/Life/2">
<ns2:withMessg>
 ...

[TejpalSingh]

@dchiesa1 Appreciate your Support

[dchiesa1]

Show me an example? In my experience, XOP involves  a multi-part request. the xop:Include elements refer to MIME attachments. What is the packaging that you have? What does the xop:Include refer to? 

You could probably get pretty far by starting with this: https://github.com/DinoChiesa/Apigee-Java-XOP-Handler

[TejpalSingh]

@dchiesa1 : I have added the sample xml payload (Highlighted the XOP object). I need to convert that object into binary data. How can we achieve it in Apigee? 

<soapenv:Body>
<ns3:process
xmlns:ns3="http://abc.com/"
xmlns:ns2="http://ACORD.org/Standards/Life/2">
<ns2:withMessg>
<ns2:Life>
<ns2:LifeRequest PrimaryObjectID="An7485">
<ns2:SupportMultipleResponsesInd tc="1">True</ns2:SupportMultipleResponsesInd>
<ns2:OLifE>
<ns2:FormInstance id="FormInstance_477364">
<ns2:FormName>Required Forms For Client Signature</ns2:FormName>
<ns2:ProviderFormNumber>PDFmanifest</ns2:ProviderFormNumber>
<ns2:Attachment id="xx4704">
<ns2:DateCreated>2022-02-10</ns2:DateCreated>
<ns2:AttachmentBasicType tc="3">File</ns2:AttachmentBasicType>
<ns2:AttachmentData64Binary>
<xop:Include href=cid:b68d5d837c4a0f3d30c410@apache.org
xmlns:xop=http://www.w3.org/2004/08/xop/include/>
</ns2:AttachmentData64Binary>
<ns2:AttachmentType tc="1">Document</ns2:AttachmentType>
<ns2:MimeTypeTC tc="17">File/PDF</ns2:MimeTypeTC>
<ns2:TransferEncodingTypeTC tc="4">Base 64</ns2:TransferEncodingTypeTC>
<ns2:AttachmentLocation tc="5">XOP Include</ns2:AttachmentLocation>
<ns2:AttachmentHashValue>abc</ns2:AttachmentHashValue>
<ns2:AttachmentHashType tc="2">2</ns2:AttachmentHashType>
</ns2:Attachment>
</ns2:FormInstance>
</ns2:OLifE>
</ns2:LifeRequest>
</ns2:Life>
</ns2:withMessg>
</ns3:process>
</soapenv:Body>

[dchiesa1]

ok I see that, thanks, that's helpful.

Where is the rest of the data? Where is the thing with the href cid:b68d5d837c4a0f3d30c410@apache.org ?

How could a reader of that XML resolve that reference (href) into actual data?

You said

I need to convert that object into binary data.

Normally that data is sent in a multipart/related package, which look something like this:

Content-Type: multipart/related; boundary=MIME_boundary; start='<d19096ed-b2d9-4c72-8e87-e4f76b2534b8>'

--MIME_boundary
Content-Type: application/soap+xml; charset=UTF-8
Content-ID: <d19096ed-b2d9-4c72-8e87-e4f76b2534b8>

<S:Envelope xmlns:S='http://schemas.xmlsoap.org/soap/envelope/'>
 ...
  <xop:Include 
     xmlns:xop='http://www.w3.org/2004/08/xop/include' 
     href='cid:0b83cd6b-af15-45d2-bbda-23895de2a73d'/>
   ...
</S:Envelope>

--MIME_boundary
Content-Type: application/pdf
Content-Transfer-Encoding: binary
Content-ID: <0b83cd6b-af15-45d2-bbda-23895de2a73d>

...binary pdf data...

--MIME_boundary--

What is the "package" for your case? the XML you showed must be packaged with the other stuff it refers to.  The attachment referenced by this id: b68d5d837c4a0f3d30c410@apache.org Where is that other data?

[TejpalSingh]

I am trying to create the XOp package with xml and PDF binary data.

Below is the XOP Package look like:

--MIME_boundary
Content-Type: application/soap+xml; charset=UTF-8;
type="application/soap+xml;
Content-Transfer-Encoding: binary;
Content-ID: <d19096ed-b2d9-4c72-8e87-e4f76b2534b8>

<soapenv:Body>
<ns3:process
xmlns:ns3="http://abc.com/"
xmlns:ns2="http://ACORD.org/Standards/Life/2">
<ns2:withMessg>
<ns2:Life>
<ns2:LifeRequest PrimaryObjectID="An7485">
<ns2:SupportMultipleResponsesInd tc="1">True</ns2:SupportMultipleResponsesInd>
<ns2:OLifE>
<ns2:FormInstance id="FormInstance_477364">
<ns2:FormName>Required Forms For Client Signature</ns2:FormName>
<ns2:ProviderFormNumber>PDFmanifest</ns2:ProviderFormNumber>
<ns2:Attachment id="xx4704">
<ns2:DateCreated>2022-02-10</ns2:DateCreated>
<ns2:AttachmentBasicType tc="3">File</ns2:AttachmentBasicType>
<ns2:AttachmentData64Binary>
<xop:Include href=cid:b68d5d837c4a0f3d30c410@apache.org
xmlns:xop=http://www.w3.org/2004/08/xop/include/>
</ns2:AttachmentData64Binary>
<ns2:AttachmentType tc="1">Document</ns2:AttachmentType>
<ns2:MimeTypeTC tc="17">File/PDF</ns2:MimeTypeTC>
<ns2:TransferEncodingTypeTC tc="4">Base 64</ns2:TransferEncodingTypeTC>
<ns2:AttachmentLocation tc="5">XOP Include</ns2:AttachmentLocation>
<ns2:AttachmentHashValue>abc</ns2:AttachmentHashValue>
<ns2:AttachmentHashType tc="2">2</ns2:AttachmentHashType>
</ns2:Attachment>
</ns2:FormInstance>
</ns2:OLifE>
</ns2:LifeRequest>
</ns2:Life>
</ns2:withMessg>
</ns3:process>
</soapenv:Body>

--MIME_boundary
Content-Type: application/pdf
Content-Transfer-Encoding: binary
Content-ID: <b68d5d837c4a0f3d30c410@apache.org>

...binary pdf data...

--MIME_boundary--

@dchiesa1 

[dchiesa1]

I've been away from my desk for the week, hence the reason for the delay in responding.

trying to create the XOp package with xml and PDF binary data.

You are trying to CREATE a Xop package, is that right? You want to CREATE the multipart message, from what starting pieces?

I'm sorry I'm not clear. I have re-read your initial message and I'm not clear what the INPUT is, and your desired OUTPUT.

Is the INPUT a message like this?

--MIME_boundary
Content-Type: application/soap+xml; charset=UTF-8;
type="application/soap+xml;
Content-Transfer-Encoding: binary;
Content-ID: <d19096ed-b2d9-4c72-8e87-e4f76b2534b8>

<soapenv:Body>
<ns3:process
xmlns:ns3="http://abc.com/"
xmlns:ns2="http://ACORD.org/Standards/Life/2">
<ns2:withMessg>
<ns2:Life>
<ns2:LifeRequest PrimaryObjectID="An7485">
<ns2:SupportMultipleResponsesInd tc="1">True</ns2:SupportMultipleResponsesInd>
<ns2:OLifE>
<ns2:FormInstance id="FormInstance_477364">
<ns2:FormName>Required Forms For Client Signature</ns2:FormName>
<ns2:ProviderFormNumber>PDFmanifest</ns2:ProviderFormNumber>
<ns2:Attachment id="xx4704">
<ns2:DateCreated>2022-02-10</ns2:DateCreated>
<ns2:AttachmentBasicType tc="3">File</ns2:AttachmentBasicType>
<ns2:AttachmentData64Binary>
<xop:Include href=cid:b68d5d837c4a0f3d30c410@apache.org
xmlns:xop=http://www.w3.org/2004/08/xop/include/>
</ns2:AttachmentData64Binary>
<ns2:AttachmentType tc="1">Document</ns2:AttachmentType>
<ns2:MimeTypeTC tc="17">File/PDF</ns2:MimeTypeTC>
<ns2:TransferEncodingTypeTC tc="4">Base 64</ns2:TransferEncodingTypeTC>
<ns2:AttachmentLocation tc="5">XOP Include</ns2:AttachmentLocation>
<ns2:AttachmentHashValue>abc</ns2:AttachmentHashValue>
<ns2:AttachmentHashType tc="2">2</ns2:AttachmentHashType>
</ns2:Attachment>
</ns2:FormInstance>
</ns2:OLifE>
</ns2:LifeRequest>
</ns2:Life>
</ns2:withMessg>
</ns3:process>
</soapenv:Body>

--MIME_boundary
Content-Type: application/pdf
Content-Transfer-Encoding: binary
Content-ID: <b68d5d837c4a0f3d30c410@apache.org>

...binary pdf data...

--MIME_boundary--

And if so, what is your desired OUTPUT?  What do you want to get out of that message?

Conversely, Is it that you want the message I showed above to be the OUTPUT? And if that is the case, what is the INPUT? (Where do you get the PDF data?)

I'm sorry, I'm still not clear on the problem you are trying to solve.

If a message of that type is the INPUT, and you want to get the PDF data as OUTPUT, then the callout I suggested originally, https://github.com/DinoChiesa/Apigee-Java-XOP-Handler , will solve that problem for you. Have you looked at it?  Does it do what you need?  If not, what's missing?

I am not clear on the problem you are trying to solve.

Try to explain it to me in great detail.  Maybe with an example. Lots of words. 

[TejpalSingh]

Hi @dchiesa1 

XOP request is look like this:

<soapenv:Body>
<ns3:process
xmlns:ns3="http://abc.com/"
xmlns:ns2="http://ACORD.org/Standards/Life/2">
<ns2:withMessg>
<ns2:Life>
<ns2:LifeRequest PrimaryObjectID="An7485">
<ns2:SupportMultipleResponsesInd tc="1">True</ns2:SupportMultipleResponsesInd>
<ns2:OLifE>
<ns2:FormInstance id="FormInstance_477364">
<ns2:FormName>Required Forms For Client Signature</ns2:FormName>
<ns2:ProviderFormNumber>PDFmanifest</ns2:ProviderFormNumber>
<ns2:Attachment id="xx4704">
<ns2:DateCreated>2022-02-10</ns2:DateCreated>
<ns2:AttachmentBasicType tc="3">File</ns2:AttachmentBasicType>
<ns2:AttachmentData64Binary>
<xop:Include href=cid:b68d5d837c4a0f3d30c410@apache.org
xmlns:xop=http://www.w3.org/2004/08/xop/include/>
</ns2:AttachmentData64Binary>
<ns2:AttachmentType tc="1">Document</ns2:AttachmentType>
<ns2:MimeTypeTC tc="17">File/PDF</ns2:MimeTypeTC>
<ns2:TransferEncodingTypeTC tc="4">Base 64</ns2:TransferEncodingTypeTC>
<ns2:AttachmentLocation tc="5">XOP Include</ns2:AttachmentLocation>
<ns2:AttachmentHashValue>abc</ns2:AttachmentHashValue>
<ns2:AttachmentHashType tc="2">2</ns2:AttachmentHashType>
</ns2:Attachment>
</ns2:FormInstance>
</ns2:OLifE>
</ns2:LifeRequest>
</ns2:Life>
</ns2:withMessg>
</ns3:process>
</soapenv:Body>

Output looks like something below:

<soapenv:Body>
<ns3:process
xmlns:ns3="http://abc.com/"
xmlns:ns2="http://ACORD.org/Standards/Life/2">
<ns2:withMessg>
<ns2:Life>
<ns2:LifeRequest PrimaryObjectID="An7485">
<ns2:SupportMultipleResponsesInd tc="1">True</ns2:SupportMultipleResponsesInd>
<ns2:OLifE>
<ns2:FormInstance id="FormInstance_477364">
<ns2:FormName>Required Forms For Client Signature</ns2:FormName>
<ns2:ProviderFormNumber>PDFmanifest</ns2:ProviderFormNumber>
<ns2:Attachment id="xx4704">
<ns2:DateCreated>2022-02-10</ns2:DateCreated>
<ns2:AttachmentBasicType tc="3">File</ns2:AttachmentBasicType>
<ns2:AttachmentData64Binary>binary data of pdf file (cid referenced)</ns2:AttachmentData64Binary>
<ns2:AttachmentType tc="1">Document</ns2:AttachmentType>
<ns2:MimeTypeTC tc="17">File/PDF</ns2:MimeTypeTC>
<ns2:TransferEncodingTypeTC tc="4">Base 64</ns2:TransferEncodingTypeTC>
<ns2:AttachmentLocation tc="5">XOP Include</ns2:AttachmentLocation>
<ns2:AttachmentHashValue>abc</ns2:AttachmentHashValue>
<ns2:AttachmentHashType tc="2">2</ns2:AttachmentHashType>
</ns2:Attachment>
</ns2:FormInstance>
</ns2:OLifE>
</ns2:LifeRequest>
</ns2:Life>
</ns2:withMessg>
</ns3:process>
</soapenv:Body>

So in the request, we are suppose to get a application/xop+xml Content-Type request and it will be having xop include object which has cid of attachment. 

Now in response, we need to replace the xop object with binary data of referenced pdf. Do we have any way how we can achieve this?

[dchiesa1]

Hey Tejpal

Thanks for that input. I looked into the callout and found that it was not handling whitespace properly. This could result in the embedded data to not be correct. I've updated the callout to 20220413 version. Can you try the same? Your policy configuration should be something like this:

<JavaCallout name='Java-ProcessXop-3'>
  <Properties>
    <Property name="source">message</Property>
    <Property name="action">transform_to_embedded</Property>
  </Properties>
  <ClassName>com.google.apigee.edgecallouts.XopHandler</ClassName>
  <ResourceURL>java://apigee-custom-xop-handler-20220413.jar</ResourceURL>
</JavaCallout>

I believe the output will be what you had hoped for:

<ns2:AttachmentData64Binary>binary data of pdf file (cid referenced)</ns2:AttachmentData64Binary>

Please try and let me know.

[TejpalSingh]

Hi @dchiesa1 

Thanks for Reply

But I think the above callout will only work with Multipart request.I have Request Content-Type = application/xop+xml; type="text/xml" and when I try with this, Its giving me this error - java.lang.IllegalArgumentException: Not a multipart MIME type: application/xop+xml; charset=utf-8; type="text/xml"

Request body will have only xml payload. Its doesn't have any other data (In callout its expecting multipart as request header and other two content type as well in body for xml payload and for pdf data but I don't have anything like this in my scenario. In my case I have application/xop+xml as request header and in body there will be only xml payload and no Content-Type). In this scenario, can we read the attachment based on cid reference and convert it into binary format and get the desired output?

Request Headers:
Content-Type application/xop+xml; type="text/xml"
SOAPAction processWithdrawal

Request Body:

<soapenv:Body>
<ns3:process
xmlns:ns3="http://abc.com/"
xmlns:ns2="http://ACORD.org/Standards/Life/2">
<ns2:withMessg>
<ns2:Life>
<ns2:LifeRequest PrimaryObjectID="An7485">
<ns2:SupportMultipleResponsesInd tc="1">True</ns2:SupportMultipleResponsesInd>
<ns2:OLifE>
<ns2:FormInstance id="FormInstance_477364">
<ns2:FormName>Required Forms For Client Signature</ns2:FormName>
<ns2:ProviderFormNumber>PDFmanifest</ns2:ProviderFormNumber>
<ns2:Attachment id="xx4704">
<ns2:DateCreated>2022-02-10</ns2:DateCreated>
<ns2:AttachmentBasicType tc="3">File</ns2:AttachmentBasicType>
<ns2:AttachmentData64Binary>
<xop:Include href=cid:b68d5d837c4a0f3d30c410@apache.org
xmlns:xop=http://www.w3.org/2004/08/xop/include/>
</ns2:AttachmentData64Binary>
<ns2:AttachmentType tc="1">Document</ns2:AttachmentType>
<ns2:MimeTypeTC tc="17">File/PDF</ns2:MimeTypeTC>
<ns2:TransferEncodingTypeTC tc="4">Base 64</ns2:TransferEncodingTypeTC>
<ns2:AttachmentLocation tc="5">XOP Include</ns2:AttachmentLocation>
<ns2:AttachmentHashValue>abc</ns2:AttachmentHashValue>
<ns2:AttachmentHashType tc="2">2</ns2:AttachmentHashType>
</ns2:Attachment>
</ns2:FormInstance>
</ns2:OLifE>
</ns2:LifeRequest>
</ns2:Life>
</ns2:withMessg>
</ns3:process>
</soapenv:Body>

Desired Response Body:

<soapenv:Body>
<ns3:process
xmlns:ns3="http://abc.com/"
xmlns:ns2="http://ACORD.org/Standards/Life/2">
<ns2:withMessg>
<ns2:Life>
<ns2:LifeRequest PrimaryObjectID="An7485">
<ns2:SupportMultipleResponsesInd tc="1">True</ns2:SupportMultipleResponsesInd>
<ns2:OLifE>
<ns2:FormInstance id="FormInstance_477364">
<ns2:FormName>Required Forms For Client Signature</ns2:FormName>
<ns2:ProviderFormNumber>PDFmanifest</ns2:ProviderFormNumber>
<ns2:Attachment id="xx4704">
<ns2:DateCreated>2022-02-10</ns2:DateCreated>
<ns2:AttachmentBasicType tc="3">File</ns2:AttachmentBasicType>
<ns2:AttachmentData64Binary>binary data of pdf file (cid referenced)</ns2:AttachmentData64Binary>
<ns2:AttachmentType tc="1">Document</ns2:AttachmentType>
<ns2:MimeTypeTC tc="17">File/PDF</ns2:MimeTypeTC>
<ns2:TransferEncodingTypeTC tc="4">Base 64</ns2:TransferEncodingTypeTC>
<ns2:AttachmentLocation tc="5">XOP Include</ns2:AttachmentLocation>
<ns2:AttachmentHashValue>abc</ns2:AttachmentHashValue>
<ns2:AttachmentHashType tc="2">2</ns2:AttachmentHashType>
</ns2:Attachment>
</ns2:FormInstance>
</ns2:OLifE>
</ns2:LifeRequest>
</ns2:Life>
</ns2:withMessg>
</ns3:process>
</soapenv:Body>

[dchiesa1]

Hi Tejpal

When MTOM/XOP is used to optimize a SOAP message, it is serialized into a MIME Multipart/Related message using XOP processing. The Java callout we are discussing, handles that kind of message. If you don't have that, then this callout won't help you. 

In a multipart related message, there are outer HTTP message headers, and then also headers that apply to the individual parts. The outer headers are like this:

Content-Type: Multipart/Related; boundary=1beaad94-5c5b-4965-a0dc-be74ad5ef661
             start="<11a1a188-72e5-4390-bcd3-282c9af5d835>";
             type="application/xop+xml"
(other HTTP Headers here)

The important pieces there:

• multipart/related means the message content (stuff that follows the headers) will have multiple parts, and those parts are related.
• the boundary parameter specifies the separator between the different parts 
• the start parameter specifies the content-id of the first part
• the type parameter specifies the content-type of the first part. (When the outer content-type is multipart/related, there are also additional content-type headers that apply to each of the following parts)

The inner structure is like this:

--1beaad94-5c5b-4965-a0dc-be74ad5ef661
Content-Type: application/soap+xml; charset=UTF-8
Content-Transfer-Encoding: 8bit
Content-ID: <11a1a188-72e5-4390-bcd3-282c9af5d835>

<soapenv:Body
    xmlns:soapenv='http://schemas.xmlsoap.org/soap/envelope/'>
...
<ns2:AttachmentData64Binary>
<xop:Include href='cid:b68d5d837c4a0f3d30c410@apache.org'
             xmlns:xop='http://www.w3.org/2004/08/xop/include'/>
</ns2:AttachmentData64Binary>
....
</soapenv:Body>

--1beaad94-5c5b-4965-a0dc-be74ad5ef661
Content-Type: application/pdf
Content-Transfer-Encoding: binary
Content-ID: <b68d5d837c4a0f3d30c410@apache.org>

%PDF-1.4
...
%%EOF
--1beaad94-5c5b-4965-a0dc-be74ad5ef661--

Notice there are two parts to the inner structure, and the parts are related. (Hence the name "multipart/related").  Notice each part gets its own headers! including content-type and content-id.   The first part is the XML that includes a xop:Include element. That element refers to the second part, via the href attribute, which uses the cid: URI scheme, as defined in IETF RFC 2392. Folding these two related parts together is one of the things the Xop handler callout does. It looks at the href in the xop:Include element, finds the part which is identified by that content-id, base64-encodes that part content, and replaces the xop:Include element with that data. The only way this will work is if there are two parts to begin with.

Request body will have only xml payload. Its doesn't have any other data (In callout its expecting multipart as request header and other two content type as well in body for xml payload and for pdf data but I don't have anything like this in my scenario. In my case I have application/xop+xml as request header and in body there will be only xml payload and no Content-Type).

In that case, you have an incomplete message, and obviously it is not possible to resolve the xop:Include, right? In the supported scenario, the href in the xop:Include points to content elsewhere in the message. You're telling me you have a xop:Include element, but no other content. That reference cannot be resolved. The href points to ... an unknown location.

The xop:Include element in the SOAP body example you provided is this:

<xop:Include href='cid:b68d5d837c4a0f3d30c410@apache.org'
   xmlns:xop='http://www.w3.org/2004/08/xop/include'/>

The href points to something. How do you propose to de-reference that href, if there is no second part? The href, by definition, points to A DIFFERENT PART in the multipart message. You're telling me there is just one part. Well then you cannot de-reference the xop:Include. It's not possible. The information is missing.

I think you are trying to solve a problem which is unsolvable. Or I have failed to understand what you're asking. I've done my best to try to help you, without success. We've gone round and round on the desired input and output. The confusion I had at the beginning, just trying to understand your goal, remains. I think you might need another set of eyes to look at the situation with you. Good luck!

[TejpalSingh]

Hi @dchiesa1 : Actually we are migrating the API from Datapower to Apigee and I think Datapower has the extensions or capabilities to read the attachment from those types of xml request. Is there any way we can include Datapower extensions or libraries into Apigee to make this work. I have below XSLT in Datapower which is reading attachment from cid reference and converting into base64 format. Request body (xml payload) and request headers (application/xop+xml) are same as I posted earlier.

<?xml version="1.0" encoding="utf-8"?>
<!-- This stylesheet finds the incoming attachment(s), serializes them to base64,
and embeds them in the XML infoset -->
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:dp="http://www.datapower.com/extensions"
xmlns:xop="http://www.w3.org/2004/08/xop/include"
xmlns:swa="http://ws.apache.org/axis2/mtomsample/"
xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:dpconfig="http://www.datapower.com/param/config"
xmlns:h="http://abc.com/"
extension-element-prefixes="dp"
exclude-result-prefixes="dp dpconfig">

<xsl:template match="/*[namespace-uri()='http://schemas.xmlsoap.org/soap/envelope/' and local-name()='Envelope']/*[namespace-uri()='http://schemas.xmlsoap.org/soap/envelope/' and local-name()='Body']/*[namespace-uri()='http://abc.com/' and
local-name()='attachmentRequest']/*[namespace-uri()='http://ACORD.org/Standards/Life/2' and local-name()='TXLife']/*[namespace-uri()='http://ACORD.org/Standards/Life/2' and local-name()='TXLifeRequest']/*[namespace-uri()='http://ACORD.org/Standards/Life/2' and local-name()='OLifE']/*[namespace-uri()='http://ACORD.org/Standards/Life/2' and local-name()='FormInstance']/*[namespace-uri()='http://ACORD.org/Standards/Life/2' and local-name()='Attachment']/*[namespace-uri()='http://ACORD.org/Standards/Life/2' and local-name()='AttachmentData64Binary']">
<xsl:copy>
<xsl:copy-of select="*"/>
<xsl:variable name="manifest" select="dp:variable('var://context/INPUT/attachment-manifest')"/>

<xsl:variable name="filename">
<xsl:value-of select="./*[namespace-uri()='http://www.w3.org/2004/08/xop/include' and local-name()='Include']/@href"/>
</xsl:variable>
<xsl:variable name="cid" select="concat($filename,'?Encode=base64')"/>
<xsl:variable name="elementname" select="substring-after($filename,'cid:')"/>
<xsl:variable name="base64"><dp:url-open target="{$cid}"/></xsl:variable>
<xsl:value-of select="$base64/base64" />
<xsl:message>value of base64 is <xsl:value-of select="$base64" /> </xsl:message>

</xsl:copy>
</xsl:template>

<xsl:template match="*|@*|node()">
<xsl:copy>
<xsl:apply-templates select="@*"/>
<xsl:apply-templates/>
</xsl:copy>
</xsl:template>
</xsl:stylesheet>

[dchiesa1]

Thanks for that.

I think Datapower has the extensions or capabilities to read the attachment from those types of xml request.

Yes. Examining that configuration, it appears that there is a reference to a function, dp:url-open.

<dp:url-open target="{$cid}"/>

That is a datapower function, it opens and reads from a URL.  Note: this function is not reading FROM THE XML.  It is reading from elsewhere. The information is not in the original XML. 

With that url-open function, your Datapower instance is able to de-reference the href. The href in your sample message is: cid:b68d5d837c4a0f3d30c410@apache.org. Your Datapower can convert that into a stream of bytes. As I wrote in my previous message:

The href points to something. How do you propose to de-reference that href, if there is no second part?

Your Datapower instance has access to the second part. It has the second part, somewhere. The url-open is reading something via the href that looks like cid:b68d5d837c4a0f3d30c410@apache.org . The comment at the top of the stylesheet even says that: 

<!-- This stylesheet finds the incoming attachment(s), serializes them to base64,
and embeds them in the XML infoset -->

Apigee can do that too, Apigee can "find the incoming attachment", if you provide that information to Apigee.

Normally MTOM / XOP messages are multipart/related  messages, and the Content-ID href like cid:b68d5d837c4a0f3d30c410@apache.org points to the second part in the message. The Java callout we are discussing can access the second part in the multipart message, via the href. You are telling me you don't have a second part. But your Datapower system has access to it!   Where did it come from? 

The simplest explanation I can think of, based on the XSLT and the comment at the top of it, is that Datapower has received the full multipart message. By the time this XSLT begins, the Datapower system has already parsed the multipart message and has separated the XML from the attached binary. And both of those parts are available as variables inside Datapower.  Therefore url-open can retrieve the second part via the href.   If you provide the same input data to Apigee (the full multipart message), it will work the same way. 

ps: I found a related gist. It shows looping through the "message manifest".  That manifest is how Datapower has access to the attachments.

   <xsl:template match="//swa:binaryData">
        <xsl:copy>
            <xsl:copy-of select="*"/>
            <!-- This pulls the manifest variable from the INPUT context  from this  -->
            <xsl:variable name="manifest"
                select="dp:variable('var://context/INPUT/attachment-manifest')"/>
            
            <!-- Create a variable that is the name of the attachment and loop thru getting
                all attachments. Multiple attachments can be handled thru this loop -->
            <xsl:for-each select="$manifest/*[local-name()='manifest']/
                *[local-name()='attachments']/*">
                <xsl:variable name="filename" select="./uri/text()"/>
                <xsl:variable name="cid" select="concat($filename,'?Encode=base64')"/>
                <xsl:variable name="elementname" select="substring-after($filename,'cid:')"/>
                <xsl:variable name="base64"><dp:url-open target="{$cid}"/></xsl:variable>
                <xsl:value-of select="$base64/base64" />
            </xsl:for-each>
        </xsl:copy> 
    </xsl:template>

[TejpalSingh]

Hi @dchiesa1 : We are able to convert the xop request into multipart. Now we have both xml payload and attachment information.- We have multiple xop objects or attachments. When I try to use this callout, Its giving this error.

Callout:
<JavaCallout name='Java-ProcessXop-3'>
<Properties>
<Property name="source">message</Property>
<Property name="action">transform_to_embedded</Property>
</Properties>
<ClassName>com.google.apigee.edgecallouts.XopHandler</ClassName>
<ResourceURL>java://apigee-custom-xop-handler-20220413.jar</ResourceURL>
</JavaCallout>

Error:
xop_action transform_to_embedded
xop_exception java.lang.IllegalStateException: found more than one xop:Include element in the XML document
xop_error found more than one xop:Include element in the XML document

Could you please suggest how we can handle multiple attachments with this callout?

[TejpalSingh]

Hi @dchiesa1 : Below is the complete XOP request:

Content-Type: multipart/related; type="application/xop+xml"; start="<http://tempuri.org/0/>"; start-info="text/xml"; boundary="8997b0ab-852a-406f-bc43-a8b48321efa5+id=20"

--8997b0ab-852a-406f-bc43-a8b48321efa5+id=20
Content-Type: application/xop+xml; charset=utf-8; type="text/xml"
Content-Transfer-Encoding: binary
Content-ID: <http://tempuri.org/0/>
<soapenv:Body>
<ns3:process
xmlns:ns3="http://abc.com/"
xmlns:ns2="http://ACORD.org/Standards/Life/2">
<ns2:withMessg>
<ns2:Life>
<ns2:LifeRequest PrimaryObjectID="An7485">
<ns2:SupportMultipleResponsesInd tc="1">True</ns2:SupportMultipleResponsesInd>
<ns2:OLifE>
<ns2:FormInstance id="FormInstance_477364">
<ns2:FormName>Required Forms For Client Signature</ns2:FormName>
<ns2:ProviderFormNumber>PDFmanifest</ns2:ProviderFormNumber>
<ns2:Attachment id="xx4704">
<ns2:DateCreated>2022-02-10</ns2:DateCreated>
<ns2:AttachmentBasicType tc="3">File</ns2:AttachmentBasicType>
<ns2:AttachmentData64Binary>
<xop:Include href=cid:5d31ab1f59579aad895dc7a32d310@apache.org
xmlns:xop=http://www.w3.org/2004/08/xop/include/>
</ns2:AttachmentData64Binary>
<ns2:AttachmentType tc="1">Document</ns2:AttachmentType>
<ns2:MimeTypeTC tc="17">File/PDF</ns2:MimeTypeTC>
<ns2:TransferEncodingTypeTC tc="4">Base 64</ns2:TransferEncodingTypeTC>
<ns2:AttachmentLocation tc="5">XOP Include</ns2:AttachmentLocation>
<ns2:AttachmentHashValue>abc</ns2:AttachmentHashValue>
<ns2:AttachmentHashType tc="2">2</ns2:AttachmentHashType>
</ns2:Attachment>
<ns2:Attachment id="xx4705">
<ns2:DateCreated>2022-02-10</ns2:DateCreated>
<ns2:AttachmentBasicType tc="3">File</ns2:AttachmentBasicType>
<ns2:AttachmentData64Binary>
<xop:Include href=cid:5d314343570@apache.org
xmlns:xop=http://www.w3.org/2004/08/xop/include/>
</ns2:AttachmentData64Binary>
<ns2:AttachmentType tc="1">Document</ns2:AttachmentType>
<ns2:MimeTypeTC tc="17">File/PDF</ns2:MimeTypeTC>
<ns2:TransferEncodingTypeTC tc="4">Base 64</ns2:TransferEncodingTypeTC>
<ns2:AttachmentLocation tc="5">XOP Include</ns2:AttachmentLocation>
<ns2:AttachmentHashValue>abc</ns2:AttachmentHashValue>
<ns2:AttachmentHashType tc="2">2</ns2:AttachmentHashType>
</ns2:Attachment>
</ns2:FormInstance>
</ns2:OLifE>
</ns2:LifeRequest>
</ns2:Life>
</ns2:withMessg>
</ns3:process>
</soapenv:Body>

--8997b0ab-852a-406f-bc43-a8b48321efa5+id=20
Content-Type: application/pdf
Content-Transfer-Encoding: binary
Content-ID:
<5d31ab1f59579aad895dc7a32d310@apache.org>

%PDF-1.7 %���� 1 0 obj <
</Type/Catalog/Pages 2 0 R/Lang(en-US) /StructTreeRoot 10 0 R/MarkInfo<
</Marked true>>/Metadata 20 0 R/ViewerPreferences 21 0 R>> endobj 2 0 obj <
</Type/Pages/Count 1/Kids[ 3 0 R] >> endobj 3 0 obj <undefined</Type/Page/Parent 2 0 R/Resources<undefined</Font<undefined</F1 5 0 R>>/ExtGState<undefined</GS7 7 0 R/GS8 8 0 R>>/ProcSet[/PDF/Text/ImageB/ImageC/ImageI] yt׮;��~|�7����Z�����_9}6D�q�n�������o�mm��n�}p��m.�kx��A��@��O8�I����Y��|7;q�{˥V��ݯ��5C����?�B�:�����KYL^M]�3޾վ>������pyD�Ũ���|Vs1i��5帨I�I5Eo��&N?�+�ڇ�O���"��_�XB%frS}4��Ц����C���[�{���ϒ�a�- �� endstream endobj 20 0 obj <undefined</Type/Metadata/Subtype/XML/Length 3088>> stream undefined<?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?>undefined<x:xmpmetaundefinedxmlns:x="adobe:ns:meta/" x:xmptk="3.1-701">undefined<rdf:RDFundefinedxmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">undefined<rdf:Description rdf:about=""undefinedxmlns:pdf="http://ns.adobe.com/pdf/1.3/">undefined<pdf:Producer>Microsoft® Word for Microsoft 365</pdf:Producer>undefined</rdf:Description>undefined<rdf:Description rdf:about="" x %%EOF

--8997b0ab-852a-406f-bc43-a8b48321efa5+id=20
Content-Type: application/pdf
Content-Transfer-Encoding: binary
Content-ID: undefined<5d314343570@apache.org>

%PDF-1.7 %���� 1 0 obj <undefined</Type/Catalog/Pages 2 0 R/Lang(en-US) /StructTreeRoot 10 0 R/MarkInfo<undefined</Marked true>>/Metadata 20 0 R/ViewerPreferences 21 0 R>> endobj 2 0 obj <undefined</Type/Pages/Count 1/Kids[ 3 0 R] >> endobj 3 0 obj <undefined</Type/Page/Parent 2 0 R/Resources<undefined</Font<undefined</F1 5 0 R>>/ExtGState<undefined</GS7 7 0 R/GS8 8 0 R>>/ProcSet[/PDF/Text/ImageB/ImageC/ImageI] yt׮;��~|�7����Z�����_9}6D�q�n�������o�mm��n�}p��m.�kx��A��@��O8�I����Y��|7;q�{˥V��ݯ��5C����?�B�:�����KYL^M]�3޾վ>������pyD�Ũ���|Vs1i��5帨I�I5Eo��&N?�+�ڇ�O���"��_�XB%frS}4��Ц����C���[�{���ϒ�a�- �� endstream endobj 20 0 obj <undefined</Type/Metadata/Subtype/XML/Length 3088>> stream undefined<?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?>undefined<x:xmpmetaundefinedxmlns:x="adobe:ns:meta/" x:xmptk="3.1-701">undefined<rdf:RDFundefinedxmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">undefined<rdf:Description rdf:about=""undefinedxmlns:pdf="http://ns.adobe.com/pdf/1.3/">undefined<pdf:Producer>Microsoft® Word for Microsoft 365</pdf:Producer>undefined</rdf:Description>undefined<rdf:Description rdf:about="" x %%EOF

--8997b0ab-852a-406f-bc43-a8b48321efa5+id=20--

Expected Output:

<soapenv:Body>
<ns3:process
xmlns:ns3="http://abc.com/"
xmlns:ns2="http://ACORD.org/Standards/Life/2">
<ns2:withMessg>
<ns2:Life>
<ns2:LifeRequest PrimaryObjectID="An7485">
<ns2:SupportMultipleResponsesInd tc="1">True</ns2:SupportMultipleResponsesInd>
<ns2:OLifE>
<ns2:FormInstance id="FormInstance_477364">
<ns2:FormName>Required Forms For Client Signature</ns2:FormName>
<ns2:ProviderFormNumber>PDFmanifest</ns2:ProviderFormNumber>
<ns2:Attachment id="xx4704">
<ns2:DateCreated>2022-02-10</ns2:DateCreated>
<ns2:AttachmentBasicType tc="3">File</ns2:AttachmentBasicType>
<ns2:AttachmentData64Binary>
Binary data of PDF
</ns2:AttachmentData64Binary>
<ns2:AttachmentType tc="1">Document</ns2:AttachmentType>
<ns2:MimeTypeTC tc="17">File/PDF</ns2:MimeTypeTC>
<ns2:TransferEncodingTypeTC tc="4">Base 64</ns2:TransferEncodingTypeTC>
<ns2:AttachmentLocation tc="5">XOP Include</ns2:AttachmentLocation>
<ns2:AttachmentHashValue>abc</ns2:AttachmentHashValue>
<ns2:AttachmentHashType tc="2">2</ns2:AttachmentHashType>
</ns2:Attachment>
<ns2:Attachment id="xx4705">
<ns2:DateCreated>2022-02-10</ns2:DateCreated>
<ns2:AttachmentBasicType tc="3">File</ns2:AttachmentBasicType>
<ns2:AttachmentData64Binary>
Binary data of PDF
</ns2:AttachmentData64Binary>
<ns2:AttachmentType tc="1">Document</ns2:AttachmentType>
<ns2:MimeTypeTC tc="17">File/PDF</ns2:MimeTypeTC>
<ns2:TransferEncodingTypeTC tc="4">Base 64</ns2:TransferEncodingTypeTC>
<ns2:AttachmentLocation tc="5">XOP Include</ns2:AttachmentLocation>
<ns2:AttachmentHashValue>abc</ns2:AttachmentHashValue>
<ns2:AttachmentHashType tc="2">2</ns2:AttachmentHashType>
</ns2:Attachment>
</ns2:FormInstance>
</ns2:OLifE>
</ns2:LifeRequest>
</ns2:Life>
</ns2:withMessg>
</ns3:process>
</soapenv:Body>

I tried to use the XOP handler callout but its giving error :

xop_action xop_exception xop_error 

transform_to_embedded

java.lang.IllegalStateException: found more than one xop:Include element in the XML document

found more than one xop:Include element in the XML document

It looks like that we cannot handle multiple attachments. 

I checked the callout code and in code itself we have restricted for only one XOP package. Below is the code snippet. Could you please suggest if there is any specific case or scenario for which we have restricted for one only one xop package. If we want to handle multiple attachments, then what is the possibilites, Could you please suggest

if (nodes.getLength() != 1) {
throw new IllegalStateException(
"found more than one xop:Include element in the XML document");
}

[TejpalSingh]

@dchiesa1 : Appreciate your Support.

[dchiesa1]

ok thanks for the further information. That's very helpful.

And yes, you identified the limitation in the callout, in that it handles just one attachment.  I will have time to look into this a little later today.  I'll let you know.

[dchiesa1 · Accepted solution]

ok Tejpal

I've updated the Java callout. It now handles multiple attachments with the TRANSFORM_TO_EMBEDDED action. You can get the latest from the repo, and try it. 

But before you try it with your message, one note.

The XML document in your example input, which begins like this:

<soapenv:Body>
<ns3:process
xmlns:ns3="http://abc.com/"
xmlns:ns2="http://ACORD.org/Standards/Life/2">
<ns2:withMessg>
 ...

...is not well-formed. The opening element there, soapenv:Body , refers to a namespace prefix soapenv which is not declared. Normally I would expect an Envelope element surrounding the Body. Like this:

<soapenv:Envelope xmlns:soapenv='http://schemas.xmlsoap.org/soap/envelope/'>
<soapenv:Body>
<ns3:process
xmlns:ns3="http://abc.com/"
xmlns:ns2="http://ACORD.org/Standards/Life/2">
<ns2:withMessg>
 ...

[TejpalSingh]

Thanks @dchiesa1 . Really Appreciate your Help. its working now.

[TejpalSingh]

@dchiesa1 : For this MTOM requests, some times we have really big payloads and multiple big attachments (more than 10MB) and because of that callout is failing. Is it possible to increase the Java callout execution time, So it wont give runtime execution before processing the complete request?

Sample Content-Length for big payloads- 4068826

[dchiesa1]

Hey Tejpal, I think there is a limit on Apigee payload size to 10MB or less.

So probably, changing the execution time isn't the right solution to your goal of handling payloads larger than 10mb. If you have large payloads commonly, then I would suggest a different architecture and flow path: 

1. client calls Apigee and sends a request-to-submit.  Informing Apigee that it wants to upload a large SOAP/MTOM message.
2.  Apigee proxy  generates a signed URL (via something like this), sends THAT to the client
3. the client uploads that large SOAP/MTOM payload to a GCS bucket. 
4. a separate process or function gets kicked off to process that message. A Good option is Google Cloud Functions triggered via storage finalization. (doc here) 
5. The cloud function then processes the message and does whatever is necessary

This way, the Apigee API proxy never handles the 10+mb payload. It simply generates a signed URL and allows the Google Cloud storage to ingest the larger payloads.

But you cited a payload size of around 4mb, not 10mb  If your Java callout is failing for some other reason on a payload of ~4mb, then you will want to diagnose that separately.  What's the failure?  etc etc 

[TejpalSingh]

@dchiesa1 : I debugged the issue and found the In XML payload, they are sending content-ID where special character (@) has been replaced with encoding (%40) in XOP object. Like below:

<AttachmentData64Binary>
<xop:Include href=cid:2c785f26-9d7d-4388-a66e-c31d9fda1d8a%40ACORD.org xmlns:xop=http://www.w3.org/2004/08/xop/include/>
</AttachmentData64Binary>

But in the Attachment section, they are passing content-ID without any encoding like below:
Content-ID - 2c785f26-9d7d-4388-a66e-c31d9fda1d8a@ACORD.org

So we are not able to fetch the attachment value as part 1 payload cid has encoded special character and part has special character not encoded.

So is it possible to accommodate changes in Callout where we can read encoded character or without encoded version and able to read the attachment?

[dchiesa1]

Sure thing, Tejpal

Try the newest version of the JAR (20220706).  It first tries to locate the element in the XML with the unencoded form of the content id (foobar@apache.org), and if that does not find an element, then it tries to find the element in the XML with the encoded form (foobar%40apache.org). 

See if it works for you.  

You will need to change your JavaCallout policies to reference the newer version of the jar (apigee-custom-xop-handler-20220706.jar) 

[TejpalSingh]

@dchiesa1 Thanks you for your reply. I tried with new callout, but its still giving same error: java.lang.IllegalStateException: no matching xop:include object found in the document (href='cid:5d31ab1f59579aad895dc7a32d310@apache.org')

I can see the logic for encoder, but still its not able to fetch compare the contentID. I tried with decoder as well, but same issue. Can you please help me to figure out the issue?
String urlEncodedContentId = URLEncoder.encode(contentId, StandardCharsets.UTF_8.name());
includeElement = findIncludeElement.apply(urlEncodedContentId);
if (includeElement == null) {
throw new IllegalStateException(
String.format(
"no matching xop:Include element in the XML document (href='cid:%s')",
contentId));
}


Content-Type: multipart/related; type="application/xop+xml"; start="<http://tempuri.org/0/>"; start-info="text/xml"; boundary="8997b0ab-852a-406f-bc43-a8b48321efa5+id=20"

(Please ignore the soap validation)
--8997b0ab-852a-406f-bc43-a8b48321efa5+id=20
Content-Type: application/xop+xml; charset=utf-8; type="text/xml"
Content-Transfer-Encoding: binary
Content-ID: <http://tempuri.org/0/>
<soapenv:Body>
<ns3:process
xmlns:ns3="http://abc.com/"
xmlns:ns2="http://ACORD.org/Standards/Life/2">
<ns2:withMessg>
<ns2:Life>
<ns2:LifeRequest PrimaryObjectID="An7485">
<ns2:SupportMultipleResponsesInd tc="1">True</ns2:SupportMultipleResponsesInd>
<ns2:OLifE>
<ns2:FormInstance id="FormInstance_477364">
<ns2:FormName>Required Forms For Client Signature</ns2:FormName>
<ns2:ProviderFormNumber>PDFmanifest</ns2:ProviderFormNumber>
<ns2:Attachment id="xx4704">
<ns2:DateCreated>2022-02-10</ns2:DateCreated>
<ns2:AttachmentBasicType tc="3">File</ns2:AttachmentBasicType>
<ns2:AttachmentData64Binary>
<xop:Include href=cid:5d31ab1f59579aad895dc7a32d310%40apache.org
xmlns:xop=http://www.w3.org/2004/08/xop/include/>
</ns2:AttachmentData64Binary>
<ns2:AttachmentType tc="1">Document</ns2:AttachmentType>
<ns2:MimeTypeTC tc="17">File/PDF</ns2:MimeTypeTC>
<ns2:TransferEncodingTypeTC tc="4">Base 64</ns2:TransferEncodingTypeTC>
<ns2:AttachmentLocation tc="5">XOP Include</ns2:AttachmentLocation>
<ns2:AttachmentHashValue>abc</ns2:AttachmentHashValue>
<ns2:AttachmentHashType tc="2">2</ns2:AttachmentHashType>
</ns2:Attachment>
<ns2:Attachment id="xx4705">
<ns2:DateCreated>2022-02-10</ns2:DateCreated>
<ns2:AttachmentBasicType tc="3">File</ns2:AttachmentBasicType>
<ns2:AttachmentData64Binary>
<xop:Include href=cid:5d314343570%40apache.org
xmlns:xop=http://www.w3.org/2004/08/xop/include/>
</ns2:AttachmentData64Binary>
<ns2:AttachmentType tc="1">Document</ns2:AttachmentType>
<ns2:MimeTypeTC tc="17">File/PDF</ns2:MimeTypeTC>
<ns2:TransferEncodingTypeTC tc="4">Base 64</ns2:TransferEncodingTypeTC>
<ns2:AttachmentLocation tc="5">XOP Include</ns2:AttachmentLocation>
<ns2:AttachmentHashValue>abc</ns2:AttachmentHashValue>
<ns2:AttachmentHashType tc="2">2</ns2:AttachmentHashType>
</ns2:Attachment>
</ns2:FormInstance>
</ns2:OLifE>
</ns2:LifeRequest>
</ns2:Life>
</ns2:withMessg>
</ns3:process>
</soapenv:Body>

--8997b0ab-852a-406f-bc43-a8b48321efa5+id=20
Content-Type: application/pdf
Content-Transfer-Encoding: binary
Content-ID:
<5d31ab1f59579aad895dc7a32d310@apache.org>

%PDF-1.7 %���� 1 0 obj <
</Type/Catalog/Pages 2 0 R/Lang(en-US) /StructTreeRoot 10 0 R/MarkInfo<
</Marked true>>/Metadata 20 0 R/ViewerPreferences 21 0 R>> endobj 2 0 obj <
</Type/Pages/Count 1/Kids[ 3 0 R] >> endobj 3 0 obj <undefined</Type/Page/Parent 2 0 R/Resources<undefined</Font<undefined</F1 5 0 R>>/ExtGState<undefined</GS7 7 0 R/GS8 8 0 R>>/ProcSet[/PDF/Text/ImageB/ImageC/ImageI] yt׮;��~|�7����Z�����_9}6D�q�n�������o�mm��n�}p��m.�kx��A��@��O8�I����Y��|7;q�{˥V��ݯ��5C����?�B�:�����KYL^M]�3޾վ>������pyD�Ũ���|Vs1i��5帨I�I5Eo��&N?�+�ڇ�O���"��_�XB%frS}4��Ц����C���[�{���ϒ�a�- �� endstream endobj 20 0 obj <undefined</Type/Metadata/Subtype/XML/Length 3088>> stream undefined<?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?>undefined<x:xmpmetaundefinedxmlns:x="adobe:ns:meta/" x:xmptk="3.1-701">undefined<rdf:RDFundefinedxmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">undefined<rdf:Description rdf:about=""undefinedxmlns:pdf="http://ns.adobe.com/pdf/1.3/">undefined<pdf:Producer>Microsoft® Word for Microsoft 365</pdf:Producer>undefined</rdf:Description>undefined<rdf:Description rdf:about="" x %%EOF

--8997b0ab-852a-406f-bc43-a8b48321efa5+id=20
Content-Type: application/pdf
Content-Transfer-Encoding: binary
Content-ID: undefined<5d314343570@apache.org>

%PDF-1.7 %���� 1 0 obj <undefined</Type/Catalog/Pages 2 0 R/Lang(en-US) /StructTreeRoot 10 0 R/MarkInfo<undefined</Marked true>>/Metadata 20 0 R/ViewerPreferences 21 0 R>> endobj 2 0 obj <undefined</Type/Pages/Count 1/Kids[ 3 0 R] >> endobj 3 0 obj <undefined</Type/Page/Parent 2 0 R/Resources<undefined</Font<undefined</F1 5 0 R>>/ExtGState<undefined</GS7 7 0 R/GS8 8 0 R>>/ProcSet[/PDF/Text/ImageB/ImageC/ImageI] yt׮;��~|�7����Z�����_9}6D�q�n�������o�mm��n�}p��m.�kx��A��@��O8�I����Y��|7;q�{˥V��ݯ��5C����?�B�:�����KYL^M]�3޾վ>������pyD�Ũ���|Vs1i��5帨I�I5Eo��&N?�+�ڇ�O���"��_�XB%frS}4��Ц����C���[�{���ϒ�a�- �� endstream endobj 20 0 obj <undefined</Type/Metadata/Subtype/XML/Length 3088>> stream undefined<?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?>undefined<x:xmpmetaundefinedxmlns:x="adobe:ns:meta/" x:xmptk="3.1-701">undefined<rdf:RDFundefinedxmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">undefined<rdf:Description rdf:about=""undefinedxmlns:pdf="http://ns.adobe.com/pdf/1.3/">undefined<pdf:Producer>Microsoft® Word for Microsoft 365</pdf:Producer>undefined</rdf:Description>undefined<rdf:Description rdf:about="" x %%EOF

--8997b0ab-852a-406f-bc43-a8b48321efa5+id=20--

[dchiesa1]

I understand what you wrote and what you're observing. There is a test for exactly this case in the callout. The existing test passes with the new version. If I copy the data you provided above, and use that as input to the test, the test still passes. It works for me.

There's something that confuses me about the example data you are sharing. All of the XML looks wellformed, EXCEPT for the xop:Include element. Here's an excerpt from your sample data:

<ns2:Attachment id="xx4704">
<ns2:DateCreated>2022-02-10</ns2:DateCreated>
<ns2:AttachmentBasicType tc="3">File</ns2:AttachmentBasicType>
<ns2:AttachmentData64Binary>
<xop:Include href=cid:5d31ab1f59579aad895dc7a32d310%40apache.org
xmlns:xop=http://www.w3.org/2004/08/xop/include/>
</ns2:AttachmentData64Binary>
<ns2:AttachmentType tc="1">Document</ns2:AttachmentType>
<ns2:MimeTypeTC tc="17">File/PDF</ns2:MimeTypeTC>
<ns2:TransferEncodingTypeTC tc="4">Base 64</ns2:TransferEncodingTypeTC>
<ns2:AttachmentLocation tc="5">XOP Include</ns2:AttachmentLocation>
<ns2:AttachmentHashValue>abc</ns2:AttachmentHashValue>
<ns2:AttachmentHashType tc="2">2</ns2:AttachmentHashType>
</ns2:Attachment>

Every attribute in that snippet is quoted, EXCEPT for the attributes on the xop:Include element. The ns1:Attachment element has an attribute named id , and the value is xx4704, in double quotes. The ns2:AttachmentBasicType has a tc attribute which has a quoted value. Every attribute has a quoted value. Except for the href and the xmlns:xop on the xop:Include element. Do you have any idea why that would be?

I'm sure the data is not exactly as I see it above in this forum, because attributes without quotes are not well-formed XML, and the callout would have returned a different error if it encountered this (really without quotes):

<xop:Include href=cid:5d31ab1f59579aad895dc7a32d310%40apache.org
xmlns:xop=http://www.w3.org/2004/08/xop/include />

It needs to be something like this:

<xop:Include href='cid:5d31ab1f59579aad895dc7a32d310%40apache.org'
xmlns:xop='http://www.w3.org/2004/08/xop/include' />

Either single or double quotes are ok.

I'm sure your document has quotes, somehow. But I'm wondering if the encoded @ sign (%40) is related to the lack of quotes. Are we really clear on what the document holds? Is it possible for you to ZIP the document and attach the zip file here in a response? If you can do that we can be assured that there is no issue with fidelity associated to the copy/paste into the message body here. If you do that, I could run the test on exactly the data you have on your side. 

[TejpalSingh]

@dchiesa1 : Sorry I missed the double quotes.  I trying to upload the file, but not getting any option here to upload. Also If I put the complete payload, its not taking. Below will be the snippet for xop include. If I try without encoded content-Id, its working but If I put encoded content-Id in xml, its not working. please check:

<xop:Include
xmlns:xop="http://www.w3.org/2004/08/xop/include"
href="cid:5d31ab1f59579aad895dc7a32d310%40apache.org">
</xop:Include>

[TejpalSingh]

@dchiesa1 : I tried everything, but I am niether able to paste complete payload- nor attach it. I just tried to put the xml snippet. Not sure how much it can help, but Its working where url encode is not there (both the places its use @) but with encode its not working

--8997b0ab-852a-406f-bc43-a8b48321efa5+id=20
Content-Type: application/xop+xml; charset=utf-8; type="text/xml"
Content-Transfer-Encoding: binary
Content-ID: <http://tempuri.org/0/>

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope
    xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
    <soapenv:Header>
        <ch:MessageHeader
            xmlns:ch="http://abc.com/">
            <ch:MessageName>Withdrawal</ch:MessageName>
            <ch:RouteInfo>
                <ch:FromParticipant>0057</ch:FromParticipant>
                <ch:ToParticipant>4597</ch:ToParticipant>
                <ch:Sender>EDWARD_JONES</ch:Sender>
                <ch:TransmissionDate>2022-04-07</ch:TransmissionDate>
                <ch:TransmissionTime>11:32:03.0000000-04:00</ch:TransmissionTime>
            </ch:RouteInfo>
        </ch:MessageHeader>
    </soapenv:Header>
    <soapenv:Body>
        <ns3:processWithdrawal
            xmlns:ns3="http://abc.com/"
            xmlns:ns2="http://ACORD.org/Standards/Life/2">
            <ns2:Withdrawal_Msg>
                <ns2:TXLife>
                    <ns2:TXLifeRequest PrimaryObjectID="AnnuityOrder757238">
                        <ns2:TransRefGUID>0369e970-e4c2-452c-894e-aa34aba3c428</ns2:TransRefGUID>
                        <ns2:TransType tc="105">Withdrawal</ns2:TransType>
                        <ns2:TransExeDate>2022-04-07</ns2:TransExeDate>
                        <ns2:TransExeTime>11:32:04-04:00</ns2:TransExeTime>
                        <ns2:TransMode tc="2">Original request. Sent by sender.</ns2:TransMode>
                        <ns2:PendingResponseOK tc="1">True</ns2:PendingResponseOK>
                        <ns2:SupportMultipleResponsesInd tc="1">True</ns2:SupportMultipleResponsesInd>
                        <ns2:OLifE>
                            <ns2:Holding id="AnnuityOrder757238">
                                <ns2:HoldingTypeCode tc="2">Policy</ns2:HoldingTypeCode>
                                <ns2:DistributorClientAcctNum>561651</ns2:DistributorClientAcctNum>
                                <ns2:Policy CarrierPartyID="Party_Carrier">
                                    <ns2:PolNumber>E1885415</ns2:PolNumber>
                                    <ns2:CarrierCode>79227</ns2:CarrierCode>
                                    <ns2:CusipNum>74429E164</ns2:CusipNum>
                                </ns2:Policy>
                               
                            </ns2:Holding>
                            <ns2:Party id="Party_Distributor">
                                <ns2:PartyTypeCode tc="2">Organization</ns2:PartyTypeCode>
                                
                            </ns2:Party>
                            <ns2:Party id="Party_Carrier">
                                <ns2:PartyTypeCode tc="2">Organization</ns2:PartyTypeCode>
                               
                                <ns2:Carrier>
                                    <ns2:CarrierCode>79227</ns2:CarrierCode>
                                </ns2:Carrier>
                            </ns2:Party>
                            <ns2:Party id="Party_Agent">
                                <ns2:PartyTypeCode tc="1">Person</ns2:PartyTypeCode>
                                <ns2:Person>
                                    <ns2:FirstName>HOPAN</ns2:FirstName>
                                    <ns2:LastName>DANDRIDGE</ns2:LastName>
                                </ns2:Person>
                                <ns2:Producer>
                                    <ns2:NIPRNumber>34567891</ns2:NIPRNumber>
                                </ns2:Producer>
                                <ns2:PartialIdentification>
                                    <ns2:IdentificationPart>8241</ns2:IdentificationPart>
                                    <ns2:PartialIDType tc="1">SSN</ns2:PartialIDType>
                                </ns2:PartialIdentification>
                            </ns2:Party>
                            <ns2:Party id="Party_PrimaryOwner">
                                <ns2:PartyTypeCode tc="1">Person</ns2:PartyTypeCode>
                                <ns2:Person>
                                    <ns2:FirstName>Test</ns2:FirstName>
                                    <ns2:LastName>Owner</ns2:LastName>
                                </ns2:Person>
                                
                            </ns2:Party>
                            
                            <ns2:FormInstance id="FormInstance_486844">
                                <ns2:FormName>Required Forms For Client Signature</ns2:FormName>
                                <ns2:ProviderFormNumber>PDFmanifest</ns2:ProviderFormNumber>
                                <ns2:Attachment id="xx486844">
                                    <ns2:DateCreated>2022-04-07</ns2:DateCreated>
                                    <ns2:AttachmentBasicType tc="3">File</ns2:AttachmentBasicType>
                                    <ns2:AttachmentData64Binary>
                                        <xop:Include
                                            xmlns:xop="http://www.w3.org/2004/08/xop/include" href="cid:5d31ab1f59579aad895dc7a32d31d3c859ddf4343570fc10%40apache.org">
                                        </xop:Include>
                                    </ns2:AttachmentData64Binary>
                                    <ns2:AttachmentType tc="1">Document</ns2:AttachmentType>
                                    <ns2:MimeTypeTC tc="17">File/PDF</ns2:MimeTypeTC>
                                    <ns2:TransferEncodingTypeTC tc="4">Base 64</ns2:TransferEncodingTypeTC>
                                    <ns2:AttachmentLocation tc="5">XOP Include</ns2:AttachmentLocation>
                                    <ns2:AttachmentHashValue>2352FB48E31460C789C6822926F6B728C85A70B5</ns2:AttachmentHashValue>
                                    <ns2:AttachmentHashType tc="2">2</ns2:AttachmentHashType>
                                </ns2:Attachment>
								
                            </ns2:FormInstance>
                        </ns2:OLifE>
                    </ns2:TXLifeRequest>
                </ns2:TXLife>
            </ns2:Withdrawal_Msg>
        </ns3:processWithdrawal>
    </soapenv:Body>
</soapenv:Envelope>

--8997b0ab-852a-406f-bc43-a8b48321efa5+id=20
Content-Type: application/pdf
Content-Transfer-Encoding: binary
Content-ID: <5d31ab1f59579aad895dc7a32d31d3c859ddf4343570fc10@apache.org>

%PDF-1.7 %%EOF

--8997b0ab-852a-406f-bc43-a8b48321efa5+id=20--

[dchiesa1 · Accepted solution]

Hi thanks for that ^^

I just ran more tests with that example, and still the tests pass. 

Are you certain you have the latest version of the callout?  apigee-custom-xop-handler-20220706.jar | 20182 bytes 

[TejpalSingh]

Hi @dchiesa1 : Thanks for you reply. I think I was doing something wrong. I replace all over again new callout as it is and its working fine now.

Thanks you soo much for your help as always. Really Appreciate.

[dchiesa1]

Great. Glad to hear it!