Developer Portal - File Upload - multipart/formdata

[#1]

@Gitesh Koli

One of our API has file upload functionality. Here is the swagger spec parameter for the file field.

{ "name": "file", "in": "formData", "description": "Desc", "required": false, "type": "file" }

When I send the request from Smart Docs (try out), the request is sent with Content-Disposition: attachment; filename="file" instead of Content-Disposition: form-data; name="file";

Can you suggest a way to fix attachment content-disposition format?

[DChiesa]

Can you show more of your OpenAPI Spec? Specifically, What is the "consumes" value? It should be "multipart/form-data". Is it?

[#3]

yes.

"consumes": [ "multipart/form-data" ],

[#4]

@Dino

When I looked at the trace, I noticed following API call is made from smart docs and the request payload look correct.

https://apiconsole-prod.apigee.net/smartdocs/v1/sendrequest?targeturl=http%3A%2F%2F*-nonprod-dev.api...

Request Payload:

------WebKitFormBoundaryAOqAIP2jBiFYKuuP Content-Disposition: form-data; name="root-fields"

policyId=&policy=&metadata=&metadataFile=&documentName=&includeMatches=&maxContentMatchesPerRule=&skipTextExtraction=&skipWhitespaceCollapse= ------WebKitFormBoundaryAOqAIP2jBiFYKuuP Content-Disposition: form-data; name="file"; filename="ab.pdf" Content-Type: application/pdf

It doesn't appear to be an issue with smart docs. But this API call is changing the content disposition type from form data to attachment. Not sure where this is happening.

[davidmehi]

Seeing the same thing - BaskarVangili, were you able to get it to work?

[anilsr]

@BaskarVangili ,

I have fixed this issue onCloud, Can you verify & let us know.

Also, Are you sending any additional data along with file attachment using multipart/form-data ?

[#7]

Thanks Anil.

I am sending other form parameters as application/x-www-form-urlencoded along with multi-part form data.

It's still not working.

I get this error for any document.

<code>{
  "statusCode": 400,
  "message": "No document or metadata to classify.",
  "error": "CLIENT_ERROR"
}			

[#8]

@Anil Sagar @Gitesh Koli

Any update on this please?

[anilsr]

@BaskarVangili , Are you available to quickly test this fix before we push this to production ?

[#10]

yes Anil, I am available now. Can you deploy to DEV portal?

[anilsr]

@BaskarVangili , Can you please verify now & let us know how it goes.

[#12]

@Anil Sagar

I am getting different error now. we are sending few other form values along with the file. Earlier those form values were sent as application/x-www-form-urlencoded and file was sent as multipart form data. Now all the form fields are sent as multipart form data. for some reason this is breaking the target service. Any idea why was this changed?

[anilsr]

@BaskarVangili , where do you mention this 'application/x-www-form-urlencoded' in the spec only for form fields ? I don't think it's right way to do. Ideally, It should be 'formdata' right when you are sending the API which is multipart formdata combination of form fields & file ? For example, See request going in for the similar API documented using famous petstore Open API Spec here.

[#14]

I didn't mention 'application/x-www-form-urlencoded' in the spec. but thats how other fields were sent earlier and that worked fine. The only issue with previous implementation was 'Content-Disposition' was sent as 'attachment' instead of 'form-data'. But the recent change breaks the backend service and we don't have access to change the service. We have to somehow fix this from smart docs side.

[anilsr]

@BaskarVangili , It's not smartdocs proxy issue. If your target server is expecting 'application/x-www-form-urlencoded' , You need to mention same in Spec right ? By deafault, Multi Part form data disposition header for form fields is "formdata" & that's what we are doing. What if i change it to ''application/x-www-form-urlencoded'' just for your target & other targets expect it to be standard 'formdata' according to RFC spec (RFC2183) ? For example, SmartDocs famous petstore example expects it to be standard 'formdata'. So, I am afraid we can't change it just for your backend.

If your backend can't change to the standard format, I would suggest having your own smartdocs proxy to talk to your target server. Again, 'application/x-www-form-urlencoded' in Content-Disposition is not an accepted format according to RFC spec (RFC2183).

[#16]

@Anil Sagar Currently all form fields are being sent even if there is no data entered for some of the form fields. Is it possible to avoid sending the fields which do not have any value? This will help to fix our issue.

[anilsr]

@BaskarVangili , What if someone else says, I need to have that info in my api call with empty values when no value is presented ? Just playing devils advocate here 🙂

[davidmehi]

​Hi Anil, We are seeing different behavior than expected when doing multipart/form-data through smart docs. If there are other form parameters, instead of sending each form parameter as a "multi-part", it is bundling them as x-www-url-encoded into the "first part", and then doing the file as the "second part". Is this expected? The parsing logic on the backend would need to be different to support smartdocs (vs postman or another http client) I think it's the Javascript in Smartdocs that is assembling the formdata like this.

[anilsr]

@David Mehi , Yes, You are right. Not sure the reasons behind it. I am sure it's not a great way to do same. Also, We are modifying the request in smartdocs proxy for some reasons including browser compatability. Earlier, File upload was not working in smartdocs with petstore example. I have a fix on test environment. Need to validate with use case.

[davidmehi]

Thanks - do you think this is something that will "fixed" or updated in SmartDocs in a future version?

For now, we are forcing the client to base64 encode the file first and then passing it in as part of the JSON payload on a POST.

[anilsr]

@David Mehi , Fix is available on production. Smartdocs now works for multi part form data combination of form fields, files. We have verified with petstore uploadImage & works as expected.

[davidmehi]

Great! Is there a patch or anything to update in smartdocs if we already have it up and running?

[#23]

@Anil Sagar has a fix been made for on-prem deployments of the dev portal as well? We are running into this issue on prem and have been looking at options to re-write the functionality, but really would rather not if there is a working version of the javascript file that we could use.

(reposting comment here, the other place I made this comment was misplaced.)

[anilsr]

@David C. Burke , Attached is the version of same.

smartdocs-rev10-2017-10-06.zip

Hope it helps.

[Surabhi_gupta]

Hi @David Mehi/ @Anil Sagar,

Was this issue fixed? I am seeing the same behavior from Smart doc. I have an API to upload file with 4 other text fields. It's working from Postman but call from Smart doc is failing.

The cURL command on Smart doc is (not working):

curl -X POST --header "Authorization: Bearer XpKuqI3ibKK2PrMgvB3PFgkgR3Fe" -F "root-fields=CreatedBy=sanjay.patel&Description=test&DocTypeId=2&DocumentName=test" -F "FileByteStream=@SaveDealDetails_d.txt" "https://qa.jllapi.com/dealio/v1/deals/deal/documents?DealId=276570"

While in Postman is (Working):


curl -X POST \
  'http://localhost:20399/v1/deals/deal/documents?dealid=276570' \
  -H 'authorization: Basic YW1cc3VyYWJoaS5ndXB0YTp3b3JrQGpsbDIwMDY=' \
  -H 'cache-control: no-cache' \
  -H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \
  -H 'postman-token: ffe648b2-db21-514b-aed8-06d750ea4b8e' \
  -F CreatedBy=Sanjay.Patel \
  -F 'Description=test doc' \
  -F DocTypeId=2 \
  -F 'DocumentName=Executed Lease' \
  -F 'FileByteStream=@C:\Users\Surabhi.Gupta\Desktop\SaveDealDetails_d.txt'