Impact of HTTPTransport.io.timeout.millis on MP and threads

[Ramchanderch]

I understand apigee provides configuration of Router and MP to handle timeouts due to latencies of backend services which are over 55sec. However, it is not very clear on how these timeouts are implemented.

1. Every MP has predefined # of threads configured. Would the threads wait for the defined timeout based on "HTTPTransport.io.timeout.millis"? If so, then, wouldn't increase on timeout would block the threads which impacts the throughput of Apigee?

2. If the above said property is leveraged to handle timeouts then even router timeout should also be modified, in that scenario what is the impact on Nginx from through put perspective?

Ref: https://docs.apigee.com/api-platform/troubleshoot/runtime/504-gateway-timeout

[imesh]

>1. Would the threads wait for the defined timeout based on "HTTPTransport.io.timeout.millis"?

Yes, the NIO threads in Apigee Message Processors will wait for the given time period until they receive a response from the target server.

>If so, then, wouldn't increase on timeout would block the threads which impacts the throughput of Apigee?

Yes, if HTTPTransport.io.timeout.millis is increased and if the target servers become slow, during that time period the number of NIO threads blocked will increase.

As a result, Message Processors will have less number of threads to make API requests to target servers. Hence, it would impact the max throughput Apigee can handle during that time period assuming that the number of Message Processors available is fixed.

If available Message Processors can be horizontally scaled, the given Apigee deployment should be able to handle more throughput in such scenarios.

Moreover, the impact can be controlled by increasing the io.timeout for selected API proxies without increasing it globally for all API proxies and environments. This can be done by specifying "io.timeout.millis" property at the API proxy level within the target endpoints. Still, it might be difficult to handle the impact if the selected API proxies are receiving high volumes of traffic and target endpoints are taking considerable amount of time to respond.

>2. If the above said property is leveraged to handle timeouts then even router timeout should also be modified, in that scenario what is the impact on Nginx from throughput perspective?

Similar to Message Processors it may also impact Routers depending on the throughput, number of Routers available, resources allocated for Routers, proxy.read.timeout defined in the Routers, etc.

Nevertheless, the impact on Routers will be low compared to Message Processors as Routers can handle large volumes of requests per second (TPS) compared to Message Processors.