Content-Length gets converted to Transfer-Encoding

[#1]

Hi,

I am in the process of putting one of our existing API's behind the EMGW. For the most part it works out of the box but I am encountering issues when proxying POST requests on to an OpenRasta based backend.

Some digging revealed that OpenRasta does not handle Transfer-Encoding well, while the EMGW seems to convert all POST requests to Transfer-Encoding, even if the original request specified Content-Length

To migrate the OpenRasta based service to one of the other frameworks we currently use will take some time so I am wondering if it is possible for me to instruct EMGW to retain the Content-Length header once it has been specified on the original request.

Should the EMGW behaviour be as mentioned here: https://apigee.cloud.answerhub.com/questions/5831/how-does-api-proxy-decide-whether-to-use-transfer.... ?

David

[#2]

Hey David,

One thing to try would be the accumulate-request plugin. That should buffer a whole request body in EM and send it along. In your config the accumulate-request plugin should be put first in the sequence. If that doesn't work then a custom plugin may be needed.

I've included a sample config below.

-Matt

plugins:
  sequence:
    - accumulate-request
    <ANY OTHER PLUGINS>

[#3]

Hi @mdobson

Thanks for the suggestion. However, buffering the whole request is not sufficient as it only forwards on a transfer-encoded request containing a single chunk. It seems as if further manipulation is required to convert the request to one which specifies the Content-Length.

David

[#4]

Hey David,

It would appear that this may be a bug in EM. We're investigating, and should have a firm answer for you soon.

-Matt

[#5]

Hi @mdobson

Thanks for looking into it

[#6]

Hi @mdobson

Were you able to find out anything else regarding this?

David

[mdobsonapigee2]

Hey David, Sorry for the delay. I've got a pending PR that will allow you to collect a response body, and set a content-length header which should remedy part of the issue. Would that work for you?

-Matt

[#8]

Hi @mdobson

I am assuming you meant request body as it is the target which is unable to handle chunked transfer encoding. If that is the case, then yes, it will work for me until we've migrated away from OpenRasta.

How soon will this PR be merged? I have a very tight deadline to meet so I can possibly even build a package from the branch with the changes and test it locally once source is made available to me.

David

[#9]

Hi @mdobson

I've managed to workaround the problem by placing an Nginx proxy which forces http 1.0 between the MicroGateway and by target server

Before: WEB --> EMGW --> API

Now: WEB --> EMGW --> HTTP_1.0_PROXY --> API

My Nginx server block looks like:

server {
  listen 443 ssl http2
  server_name myproxy.server.com;


  location / {
    proxy_http_version 1.0;
    proxy_pass https://mytarget.server.com/;
    proxy_redirect https://mytarget.server.com/ /;
  }
}

David

[mdobsonapigee2]

Hey David,

Good to know you found a work around. We'll still go ahead and get things merged in on our end as soon as possible. I'll be sure to keep this thread updated when a new release is cut. Apologies the issues you've ran into on this, and I appreciate you working with us on helping to diagnose.

-Matt

[sunilkumarasp]

@mdobson Hello, Did you get this fixed by any chance? Apparently I have run into the same issue, i tried to sent the content length in ondata_request function in custom plugin, but still its the same issue.

Is there any other workaround apart from having a proxy between apigee mg and target?

Thanks

Sunil

[francois-xavier]

Hello,

We are presumably also impacted by this issue, but rather than the request of a POST, the reponse of a GET has the same un-expected change in the headers. Does the ongoing fix for POST request also applies to GET response?

Thanks

[#13]

We are having the same problem. The target backend (asp.net) cannot handle chunked request correctly for now.

From what I see in the source code there is no easy config as content-length header is always removed.

https://github.com/apigee/microgateway-core/blob/master/lib/plugins-middleware.js#L197

I would expect more from the gateway... Any clue Google ?