TCP load balancer connection timed out frequently

[ryotahashimoto]

Using TCP load balancer for several years, while the connection started timed out recently, the successful late is 10-40%, checked by either curl or nc like:

curl https://<our-domain.com>/our/app
curl: (28) Failed to connect to <our-domain.com> port 443 after 16239 ms: Couldn't connect to server
nc -z our-domain.com 443; echo $?
1

Is there any way to add more resources to reduce the timeout, or at least see the log of what is happening on the TCP load balancer?

[kolban]

Are there any errors being written to the Google Cloud Logging subsystem?  My first thought is that the back-end is down or overloaded.  I'd check where the load balancer is configured to route requests and validate that each of those is up and working.   

[ryotahashimoto]

Still quite a low utilization on the ingress-nginx-backend, will try to configure route requests.  Thank you so much for your suggestion.