[Edgemicro] Best Practices to handle unresponsive ...

cirosilsantos · 02-08-2017 10:22 AM

Hello All,

When using edgemicro gateway, what is the best practice to handle target endpoints that are unresponsive or taking a lot of time to respond?

I mean, in edgemicro implementaton, is there any strategy related to timeouts, retries or even circuit breakers to handle these situations?

Thanks in Advance

Ciro

mdobsonapigee2

Hi Ciro,

At the moment the only real tweaking that can be done with unresponsive targets is altering the timeout. You may be able to do some experimentation using plugins, but I myself haven't looked at what it would take to implement any of those strategies in a plugin.

-Matt

cirosilsantos

Hi Matthew, thanks for the answer,

I saw the newly release v2.3.3 came with some timeout handling, like can be seeing in here,

https://github.com/apigee/microgateway-core/pull/49, i haven't checked out yet, i don't know which type of configuration I can set for this timeout to work on target endpoints.

About the plugins I've been doing some investigations on timeouts, retries and circuit breakers as well, but in this case I own all the code, so I can handle requests properly. I believe this can be handled.

My question is more about the target endpoint set the api proxy. For instance, lets say in Apigee Cloud I created an Api Proxy to be used by edgemicro gateway (edgemicro_myapi), the base path is /myapi and the target endpoint is http://someexternalservice.com/api. Then using an instance of edgemicro gateway in port 8000, i make a call to http://localhost:8000/myapi......and this external service (http://someexternalservice.com/api) is current unavailable or taking a long time to respond....

My question is about these cases...because its edgemicro implementation that will handle this...so i'd like to know if is there any approach to handle it that comes out of the box with edgemicro gateway....

Thanks again

Ciro

mdobsonapigee2

At the moment aside from tweaking the timeout on client requests there isn't anything out of box to handle laggy or unresponsive targets unfortunately.

-Matt

cirosilsantos

Can you give an example with timeout configuration?

Thanks

mdobsonapigee2

Under the edgemicro section of your config file you can set the request_timeout parameter to the number of seconds you'd like edgemicro to wait for your target to respond.

cirosilsantos

I tried here with no success.

edgemicro:
  request_timeout: 10000
  port: 8000
  max_connections: 1000
  max_connections_hard: 5000
  restart_sleep: 500
  restart_max: 50
  max_times: 300
  config_change_poll_interval: 600  
  logging:
    level: error
    dir: /var/tmp
    stats_log_interval: 60
    rotate_interval: 24

I added the parameter request_timeout that you mention with 10000 (milliseconds) and also 10 (seconds) and nothing happened...

I'm using version

"current edgemicro version is 2.3.3-beta"

Do you think its something about the version i'm using or is there something wrong/missing?

Thanks

mdobsonapigee2

You'll actually want to upgrade to 2.3.3. That is when this config flag was introduced.

cirosilsantos

I updated my edgemicro installation..and set like you said:

edgemicro:
  request_timeout: 10

My version now is

current nodejs version is v4.6.0
current edgemicro version is 2.3.3

To test, i have a simple node app with a get that outputs a json..but before sending a json i added a sleep of 20 seconds just for the sake of testing....

then i start this app in localhost:3000. I have a apiproxy in apigee cloud called /ping that targets to this localhost:3000.

i was expecting once i add this request_timeout:10, once i call /ping....after 10 seconds it would give me an error, because my app is taking 20 seconds to respond....however, the call remains stuck for 20 seconds, and then respond with the json...

is there something else to setup to be able to get the timeout error?

Thanks

Ciro

mdobsonapigee2

Apologies for this issue Ciro. Here is where you can follow the progress of this issues resolution.

https://github.com/apigee/microgateway-core/pull/57

mdobsonapigee2

Confirmed. I'll look into this, and remedy in the next release.

cirosilsantos

No problem! I will follow the PR and try again once is available!

Thanks!

mdobsonapigee2

This is released please check out using npm install edgemicro@2.4.0-beta

cirosilsantos

Thanks @Matthew DobsonI will test it here!

cirosilsantos

Hey @Matthew Dobson,

I tried to test back here, but...

By default I'm behind a corporative proxy, so I ran my test back-end service (one that has a timeout set in order to simulate extra response time), then I called through EMG. So i got a proxy error and in the EMG logs:

gateway:main targetRequest +10ms cadd4fr40-0447-11e7-8ff0-e73f7d541578 GET mycompany.proxy.com 8080 http://localhost:3000/

then, i tried to call an API proxy that have an external target endpoint, which BTW did not work in previous EMG version, to see what happened, and it worked (this was a previous bug, I believe I saw in the issues page), my conclusion is that you fixed the proxy error in this beta release, although its not working for a local target endpoint. I think it should consider the NO_PROXY env var to remove proxy from localhost, it appears to me the problem is EMG trying to add proxy for localhost.

I realize that if I have a http_proxy env var, EMG will always add the proxy to the target request.

are my considerations correct? Is there any new configuration to add a no proxy list? or even to disable proxy?

Moreover, given this problem, i could not test the timeout so far.

Thanks

Ciro

mdobsonapigee2

Hey Ciro,

That's an issue with the proxy implementation. It's a newer feature. I'll file an issue, and take care of that. It should respect NO_PROXY.

-Matt

[Edgemicro] Best Practices to handle unresponsive target endpoints