Unexpected EOF Exception

I am intermittently getting the unexpected EOF exception and a 502 bad gateway error.

The request is unable to reach the target servers. What could be causing this issue ?

I have target servers created and are load balanced.

java.io.EOFException: eof unexpected
        at com.apigee.nio.channels.PatternInputChannel.doRead(PatternInputChannel.java:45) ~[nio-1.0.0.jar:na]
        at com.apigee.nio.channels.InputChannel.read(InputChannel.java:103) ~[nio-1.0.0.jar:na]
        at com.apigee.protocol.http.io.MessageReader.onRead(MessageReader.java:75) ~[http-1.0.0.jar:na]
        at com.apigee.nio.channels.DefaultNIOSupport$DefaultIOChannelHandler.onIO(NIOSupport.java:51) [nio-1.0.0.jar:na]
        at com.apigee.nio.handlers.NIOThread.run(NIOThread.java:123) [nio-1.0.0.jar:na]


Thanks

1 8 2,639
8 REPLIES 8

Looks like your backend is intermittently unexpectedly terminating the response.

Is there any cause for the connection to be terminated abruptly ? We will investigate this , but if there are any generic causes that may cause this issue, thta would help for a preliminary investigation.

The backend is not getting the request, so I suspect the request to the backend itself if cut off mid way.

We have similar symptoms in production which caused failures. How can we debug such issues?

Is there a way to trap such issues?

Dino is correct. We did tcpdump to capture whats the issue & found error stack trace(

java.io.EOFException: eof unexpected)& corresponding tcpdump timestamp which show reset packet. We reached the backend api and found there is a max rate of API calls per x seconds had been exceeded & after the limit they stop accepting the calls.

Not applicable

We have the same problem here but only happens in one MP. The second MP works fine.

What could be the problem? can I re-install the MP ?

@Jorge Middleton,

Are you sure that this issue is happening only on one MP ?

If yes, are the requests to other backend servers working on this MP correctly ?

Check if you are able to telnet to the backend server from this MP on the specific port #.

Try restarting the problematic MP and see if that makes any difference. If the problem persists, then follow the instructions in this Playbook 502 Bad Gateway.

Please refer to the Troubleshooting Playbook on 502 Bad Gateway. This Playbook provides instructions how to troubleshoot the "java.io.EOFException: eof unexpected" exception.

I found the followings:

I configured a HTTP proxy for the message processor but even though I didn't set the property conf_http_HTTPClient.use.proxy (default to FALSE), some requests were forwarded to the proxy which resulted in GatewayTimeout.

To fix the issue I had to set <Propertyname="use.proxy">false</Property> in each proxy that calls a backend system so the requests are NOT forwarded via the HTTP proxy.