Multipart (multipart/form-data) creation and parsing in Apigee

Problem statement

In case of a POST request to an API, you must encode the data that forms the body of the request.

HTML forms provide three methods of encoding –

  • application/x-www-form-urlencoded (default)
  • multipart/form-data
  • text/plain

You need to send data in ‘multipart/form-data’, in case the data being submitted has one or more file content <input type=”file”> with some other text components.

In Apigee, there is a seamless support for parsing and creating POST requests with encoding form-urlencoded and text/plain (which can include json or xml content depending upon the Content-Type header). However, when it comes to creating or parsing ‘multipart/form-data’ requests, it is often the case that you end up writing custom code.

Following are two scenarios which may arise with ‘multipart/form-data‘ requests –

  • Scenario 1: The back-end expects a ‘multipart/form-data’ request and you need to create such a request from Apigee
  • Scenario 2: Any client which sends a request to Apigee, can send a ‘multipart/form-data’ request and you need to parse the request in order to extract or manipulate individual text fields/files

Let’s discuss both these scenarios in detail below.

Before I begin, the purpose of developing this apigee-multipart-util is: Assign Message and Service callout policy don't really work well when you try and create a multipart/form-data request using Content-Type and payload. Hence, you need a stable plugin which works with minor tweaks depending on the use case.

The code and description mentioned in the scenario's below, is available here - Fresh Gravity (apigee-multipart-utils)

Scenario 1

Let’s say, the client makes an application/json type POST request to Apigee containing Base64 encoded string of the actual pdf file along with other string fields. The Apigee target expects a multipart/form-data request with the raw binary pdf file and remaining text fields.

This means that the json request from client needs to be transformed into ‘multipart/form-data’ which has one file input containing the raw pdf and other text inputs.

To handle this scenario, I’m using Mime4J library for payload creation - Mime4J

Please have a look at ‘MultipartParser.createMultipartRequest’ method –

if (action.equals(constants.CREATE_METHOD_NAME))

Every multipart request requires a boundary, which separates each component (file or text) within the body. You can create this boundary dynamically or keep a constant boundary which is what I did.

Since, the request from client was application/json, we could extract each field separately from JSON in Apigee and send it to the Java code which stores these values in String variables.

The next challenge is converting the Base64 file into a raw pdf. This is a bit tricky considering that a raw document is binary in nature, which implies once the conversion is completed, we can not deal with string.

The final output is a ‘multipart/form-data’ request, sent as a Java InputStream to Apigee which Apigee then sent to the target.

Note – If there is a requirement to create ‘multipart/form-data’ request without a raw file, you need not write a Java code. The following article can be helpful to fulfil that requirement using JavaScript.

https://community.apigee.com/questions/25630/need-to-send-a-request-to-a-service-with-contentty.html

Scenario 2

Let’s say, the client makes a ‘multipart/form-data’ request which contains a raw file along with other text components. We need to perform few transformations on the text components without hampering the binary raw file and forward it to the Apigee target.

Again I’m using Mime4J library to deal with ‘multipart/form-data’ - Mime4J

Please have a look at ‘MultipartParser.parseMultipartRequest’ method –

if (action.equals(constants.PARSE_METHOD_NAME))

Here the client is already making a ‘multipart/form-data’ request. In Java code, we can access the Apigee request object directly using Apigee jars (message-flow and expressions).

The request contains a binary raw file and hence, I made sure not to deal with String as it could damage the file. Instead, I used mime type to check if the part is text/plain and only then deal with String. In all other cases, I use Binary Streams.

Once all text manipulations are finished, I create my own multipart data payload and send that as an InputStream to the Apigee target server.

I’m also setting debug variables wherever applicable in the Java code for easy debugging.

Note – In case you are not dealing with files in your multipart request and want to extract or perform simple text manipulations, then you can refer the following article and use JavaScript instead –

https://community.apigee.com/questions/36743/how-to-extract-multipartform-data-from-post-reques.html

The Java code can be customized as per your requirement.

If you need help in sending multipart/form-data requests from POSTMAN, refer this post - https://stackoverflow.com/questions/16015548/tool-for-sending-multipart-form-data-request

Hope this article helps you in managing multipart requests in and out of Apigee.

The code has been developed by @Rampradeep Krishnamurthy and @Pritam Bhowmik with major contributions from @Aashish Pathak and @Ketaki Pandit. This project has been open-sourced by Fresh Gravity.
Comments
devmego99
Participant III

@dchiesa1  Appreciate your support

pdouglas
Participant III

@dchiesa1  your simple multipart form sample saved us! Our backend print service enabled sending PDFs, which failed with the simple Extract policy. We made a few updates to the jar to allow us to extract a text list of printers to send to the backend. 

Many thanks for this: 

DinoChiesa/Apigee-Java-Simple-MultipartForm: an Apigee custom policy implemented in Java that create...

dchiesa1
Staff

Whoo-hoo! I’m glad you found it useful!  And thank you so much for letting me know.

debjitd18
Participant I

@dchiesa1 Hi Dino,

Is there a way to assign/hardcode a Content-Disposition header when sending a request from Apigee to another REST service.

We are using a Service callout to invoke that service, and the rest of the content is being handled, but we want to avoid the users to send this one value.

A snippet from Service Callout implementation:

<Request clearPayload="true" variable="myRequest">
<IgnoreUnresolvedVariables>false</IgnoreUnresolvedVariables>
<Set>
<Headers>
<Header name="x-hmhs-keyid">{private.apiKey}</Header>
<Header name="content-type">{contenttype}</Header>
<Header name="Content-Disposition: form-data; name= appId">123</Header>
</Headers>
<Payload>{content}</Payload>
<Verb>POST</Verb>
</Set>
</Request>

I get an error when I tried this step, if you could provide inputs will be helpful.

Thanks,

Debjit

Version history
Last update:
‎08-14-2018 03:55 AM
Updated by: