Issue in newline characters sent in Service Callout policy

[#1]

Hi,

I have a scenario where a proxy exposed via Apigee should accept content of a CSV file uploaded from a webpage and post it to the backend. The backend service is on Windows server and can only accept newline with CR + LF. I am using a service callout to invoke the backend URL. What I observed is when the service callout happens from Apigee, the newline is always sent as only LF, even if the request received from the webpage had CR + LF. This is causing processing issues in the backend. Apigee is running on RHEL.

Content-Type header is sent as text/csv and the body has multiple lines.

Has anyone faced this issue? Is there any resolution for this?

Thanks,

Vijay.

[ozanseymen]

Hi @vijay_kannan

I don't know the answer to your question but something else came to my mind which might help you: have you enabled request streaming in your proxy?

Best practice is to enable request streaming if you are not reading the body of a request and enable response streaming if you are not reading the body of the target response. More info here: http://docs.apigee.com/api-services/content/enabling-streaming.

It would be interesting to try and enable streaming to see whether Apigee content buffering is causing the new-line change in the request pipeline.

Even if this doesn't solve the problem, proxy will gain a speed boost and hopefully another apigeek will pitch in with the real solution.

Cheers

Ozan

[#3]

Hi Ozan,

Thanks for your suggestion. I tried with streaming enabled for request (though documentation says it is mainly to be used in cases where the payload size is very big). But with streaming, the request should be sent unmodified to the backend and should ideally have worked.

But even with streaming, the behavior is the same. I am still trying different options. Will wait and see if someone has any other suggestion or update if I find a solution.

Thanks,

Vijay.

[#4]

Hi,

Wanted to provide an update on this issue. Upon further analysis and implementing some logging, I found that backend target calls as well as service callouts from Apigee is appending a newline and 2 tab characters at the beginning and end of the payload - "\n\t\t". This is happening for any type of payload JSON, XML and CSV, which is what I am testing here.

For JSON and XML, these extra characters should not cause any issue at the backend service. But in the case of CSV, it is causing the content to go wrong resulting in validation errors on the backend service side.

Request to Apigee

Content-Type: text/csv

Firstname,John\r\n

Lastname,Doe\r\n

Request posted from Apigee to Backend

Content-Type: text/csv

\n

\t\tFirstname,John\r\n

Lastname,Doe\r\n

\n

\t\t

Please let me know if anyone else is aware of or seen this issue. I am planning to open a service request with Apigee.

Thanks,

Vijay.

[#5 · Accepted solution]

Hi,

This issue is resolved. It was actually a code issue in the Assign Message policy used just before the Service Callout. While setting the payload, the variable setting was done in the next line with tabs in between. While assigning values to XML elements, care should be taken to ensure that no space or newline characters are introduced between the opening and closing tags.

Incorrect

<Payload contentType="text/csv" variablePrefix="$" variableSuffix="#">

$request.content#

</Payload>

Correct

<Payload contentType="text/csv" variablePrefix="$" variableSuffix="#">$request.content#</Payload>

Thanks everyone for the inputs and suggestions.

Thanks,

Vijay.