How to transform json payload in multipart/form-data request

[diogoalfarelos]

Hello

Currently i'm trying to implement a proxy where i receive multipart/form-data request with two form-datas

1\ a JSON payload

2\ a binary payload (file)

My objective is to leave the binary payload untouched and foward it normally to the backend service but i need to transform the JSON payload to a specific XML format.

I've seen some java libraries to handle binary payloads but since i don't want to change this part, is there any way i can approach this by just using the normal policies provided by Apigee Edge?

Thanks!

[dchiesa1]

In Apigee, you can use a JS callout to parse a multipart form and extract the components if all of the parts are text. Here's an answer that describes how. IT will work only with text because the JS callout will coerce your data to text data. It does not handle binary arrays or typed arrays.

If you have JSON (which is text) and some binary blob, then parsing with JS won't work well for you. The case you described:

1. parse a multi-part form
2. transform one fo the parts into a different content-type
3. assemble a new multi-part form

...would best be accomplished in Apigee with a Java callout. There is not a way (that I know of) to use builtin policies to assemble such a form.

I've previously written a Java callout to parse and create multi-part forms. Find it here: Apigee-Java-Simple-MultipartForm. You may be able to use it to perform step 1 and step 3 in that sequence.

Just looking at that callout now, it is not quite as nice as we'd want it to be.

• The Parser returns the output as a byte array
• The Creator requires the input to be a string, and allows only one part.

So that means it won't really be suitable for you, as it is today. For one thing, you'd have to encode the output of step 1 in order for step 3 to work. And for another thing, there's no way to specify the multiple distinct parts. But I'll see if I can change the callout to accommodate your requirements.

EDIT: I've updated it to do that now. Give it a try.

[diogoalfarelos]

First of all, thanks for the availability Dino.

So if i understood correctly i will have to:

1\ Java callout to MultipartFormParser. I will get a item list. In my case items[0] and items[1]

2\ transform the item (the json payload)

3\ Java callout to CreateMultipartForm.

3.1\ Assign the transformed payload to part1.

3.2\ My question now is how to assign the binary payload to part2.

I can assign directly from from context variable items[1] ? Just asking to see if i need any special attention since this is a binary payload.

i will try this.

thanks!

[dchiesa1]

how to assign the binary payload to part2.

The MultipartFormParser will assign a byte array into a context variable. BTW, You will not be able to "see" the content of this byte array in the trace window, because it's binary data, not string data. The trace window will show the value as "[B@6fd6a7dc" or something similar. That's not a bug. It's just a deficiency in the Trace feature; it cannot show byte arrays.

In the MultipartformCreator callout, you can just refer to the variable that was previously set by MultipartFormParser. This assumes there will be no change to that part of the form.

EDIT - I just tested this and encountered a problem, which I think I corrected. You will need to download again and get the latest jar, version 20210312.

[diogoalfarelos]

you already pushed the new version? i'm not seeing it.

i dont think it's related since the Parser was the part already done but when running the config for the ParseMultiform with a request with a json payload+pdf file i'm getting

mpf_itemcount=0

Since the config doesn't have anything in particular... i tried using the default 'message' and the 'request' but haven't been able to get the proper items parsed

[dchiesa1]

I think maybe I forgot to git push. The update is in the repository now.

If the parser callout is not parsing, it's because it is not finding what seems like a multipart form. It expects to find something like :

Content-Length: 16301
Content-Type: multipart/form-data; boundary=----------------------QCN1DGMIPH8GPY

----------------------QCN1DGMIPH8GPY
Content-Disposition: form-data; name="part1.json"
Content-Type: application/json


{
  "whatever" : {
    "foo" :  "bar"
  }
}


----------------------QCN1DGMIPH8GPY
Content-Disposition: form-data; name="part2.pdf"
Content-Type: application/pdf


...pdf data here...


----------------------QCN1DGMIPH8GPY--

[diogoalfarelos]

Postman was not getting the file correctly on my side. Sorry for that.

Currently i have itemCount=1. From my tests its only picking up files.

I think thats the way i described and you thought i was sending a json file and not a text field with json format

I will try to change the following code to handle my particular use case.

        if (item.isFormField()) {
          // ... ignore any fields in the form
        } else {
<br>

One final question,

Now related to the CreateMultipartForm, i should assign the result also to the 'message' variable in order to send it directly to the backend system without any more policies involved (AssignMessage)?

thanks

[dchiesa1]

Maybe

Have you checked your inbound payload? IF I were diagnosing this, I would first make sure that the inbound payload looks like I think it should look. Does it have the shape of the example I showed?

i should assign the result also to the 'message' variable in order to send it directly to the backend system without any more policies involved (AssignMessage)?

Well, yes, probably. In other words you do not need to specify a destination property.

[diogoalfarelos]

I adapted the code in the if statement i mentioned.

So now i have the file items and also the text form content

the current output i see in apigee trace is:

mpf_item_content_0 		
{ "Header": { "timeout": "30000", "eTrackingID": "FFM_teste", "application": "POSTMAN", "user": "partner@click", "process": "process" }, "DataInput": { "Item": { "ICCID": "8935103166303571495" } } }

mpf_item_content_1 		[B@57643fc9

Now i will work on the transform part of the json and assemble again with the MultiPartCreator

thanks for the help so far!

[devNikhil]

Hi @diogoalfarelos , I am facing the same issue. can you provide some suggestions or sample proxy

[dchiesa1]

Hi devNikhil

Have you seen this repo? It includes a sample proxy. Have you tried it?  If you have further questions please ask a new question by Clicking the blue box in the upper right hand corner.