Sometimes getting error 502 "Unexpected EOF at target"

Not applicable

trace.ziptrace.zipHi Apigee Community,

We have installed Apigee Edge for Private Cloud Version 4.16.09.00.

In one of our proxies we noticed error response:

{"fault":{"faultstring":"Unexpected EOF at target","detail":{"errorcode":"messaging.adaptors.http.flow.UnexpectedEOFAtTarget"}}}

with status code 502 returned when sending requests to this proxy.

Most of the time developers using this proxy are receiving correct responses, this error is returned only for some of the requests send.

We thought that it might be connected to target server configuration described here: https://community.apigee.com/questions/8894/target-server-for-secure-connection-throws-502-err.html.

However performing http GET on this target server returned:

{
  "host": "foo.bar.com",
  "isEnabled": true,
  "name": "MyServerName_server",
  "port": 443,
  "sSLInfo": {
    "ciphers": [],
    "clientAuthEnabled": "false",
    "enabled": "true",
    "ignoreValidationErrors": false,
    "protocols": []
  }
}

I am attaching trace.zip with captured example error response.

Do you have any ideas what might be causing this issue? Could you help us to solve it?

4 14 22.6K
14 REPLIES 14

The message seems to indicate that the backend is returning the 502. Is this possible?

Can you correlate the requests between Apigee and the backend, and examine the ones that show 502 inside Apigee Edge, against the behavior seen at the backend?

hari_vr
Participant IV

I remember receiving this error (or a similar one, I dont remember exactly) when the certificate that we were using had issues: being that either the certificate chain that is implemented is incorrect or in the wrong order or a certificate in the chain is missing altogether.

Could you check if this might be the case at your end?

Not applicable

Thank you for the suggestions.

It seems like we have to ask someone responsible for the target server to investigate the issue on their side.

I will let you know if I was able to learn anything new about this error.

Not applicable

Hi,

I did the target server setting but getting the following error-

{ "fault": { "faultstring": "Unexpected EOF at target", "detail": { "errorcode": "messaging.adaptors.http.flow.UnexpectedEOFAtTarget" } } }

I checked that there is no backend issue, if I use direct URL I get valid response.

@Jarosław Wojtalik

did you solved this issue?

Experiencing same behaviour... 😞 any update on this?

Same issue here as well. Did you guys resolve ?

If apigee pointing to F5 Target server then this error may come.

Need to reconfigured F5 virtual server

enabled F5 automap on the virtual server

@Mukesh Nagar - would you be able to give more information on why this happens? We are seeing this sporadically. We have a 2-way TLS setup between Apigee and a F5. Currently our F5 is not set to automap and our Ops team says it's done that way for a reason.

Please refer to the playbook on UnexpectedEOFAtTarget. It provides steps to debug this issue.

HI arosław Wojtalik, have you resolved the issue? Please let me know how to resolve this issue?

Thanks in advance for your help.

danielhoenig
Participant III

We had this issue and it turned out to be how we had our F5 iRule configured. Our target server points to the F5 and then the iRule decides what server in our datacenter gets the request. The issue comes up due to how Apigee pools connections to the F5. If you have two proxies which use different servers in the data center then you will get this error. This happens because the F5 gets a request on an existing pooled connection, but the new request needs to go to a different server in the data center, which results in the F5 dropping the connection.

We resolved the issue by putting a different connection timeout in the proxies. Doing this causes Apigee to pool the connections differently, by grouping them per server in the data center.

HTTPTargetConnection Example for server 1 in the data center

    <HTTPTargetConnection>
        <LoadBalancer>
            <Server name="MainTarget"/>
        </LoadBalancer>
        <Properties>
            <Property name="keepalive.timeout.millis">50030</Property>
        </Properties>
        <Path>/server1/blah1</Path>
    </HTTPTargetConnection>

HTTPTargetConnection Example for server 2 in the data center

    <HTTPTargetConnection>
        <LoadBalancer>
            <Server name="MainTarget"/>
        </LoadBalancer>
        <Properties>
            <Property name="keepalive.timeout.millis">50029</Property>
        </Properties>
        <Path>/server2/blah2</Path>
    </HTTPTargetConnection>

Thank you for your quick response. I got your solution. The thing is we are using dynamic path as below because we are using path parameters. can we do any fix at F5 level?

<HTTPTargetConnection>
  <Properties/>
  <LoadBalancer>
  	<Server name="myTargetServer"/>
  </LoadBalancer>
  <Path>{customUrl}</Path>
</HTTPTargetConnection>

I don't know of anything you can do on the F5 other than enable F5 automap on the virtual server. I'm not an F5 guru so that's something I know nothing about.

If you haven't already...one test you can do is set the keepalive.timeout.millis to 0. This turns off Apigees connection pooling and you should not get the error. Then you know this is the issue...would you be able to put a prefix on your path? Then you can set different connection timeout's like I did.

Something like this...

<HTTPTargetConnection>
  <Properties>
        <Property name="keepalive.timeout.millis">50029</Property>
  </Properties>
  <LoadBalancer>
  	<Server name="myTargetServer"/>
  </LoadBalancer>
  <Path>/prefix1/{customUrl}</Path>
</HTTPTargetConnection>
...

<HTTPTargetConnection>
  <Properties>
        <Property name="keepalive.timeout.millis">50030</Property>
  </Properties>
  <LoadBalancer>
  	<Server name="myTargetServer"/>
  </LoadBalancer>
  <Path>/prefix2/{customUrl}</Path>
</HTTPTargetConnection>

This will require a change to your F5 to key off of the prefix values.

Although, can you set the path like that....?