f5 ELB load balancer does not load balance with ip address

sjm2000
Participant V

We are having a strange problem , which you might have encountered

Below is our quality default host having public dns also api.qa.**** .

4087-snip.png

The setup is a 5 node private cloud with 2 routers- routers being 1*.**.***.105 and 1*.**.***..100.

The f5 load balancer is load balancing between 1*.**.***.105:9001 and 1*.**.***.100:9001

Strangely enough

From intranet:

a) curl -X GET 1*.**.***.100:9001 -h internaldns works fine

b) curl -X GET internaldns:9001 works fine

c) curl -X GET 1*.**.***..100:9001 gives 404

From public Internet

We are getting 404 error code for all api's.

d) ELB is in DMZ and it will load balance between both the ip address of router.

My assumption is due to the ip address which ELB sees , its getting 404 error code.

Load balancer is f5.

Do you have any suggestions how to resolve this? I am sure this scenario is nothing new in apigee.

Regards

Sujith Mathew

Solved Solved
0 12 1,572
1 ACCEPTED SOLUTION

Not applicable

hmm...


An f5 load balancer - if im not mistaken - will use the API address and PORT (aka 10.3.244.100:9001) to route traffic - so this is sort of consistent if you cannot call that address directly.

I assume you are seeing the same behavior against BOTH servers? If you are great. If not, dont forget to check out what load balancing algo you are using. Least connections is a nasty foe sometimes.

Maybe you should re-set the host aliases and double check that you dont have any evil windows characters in the strings? in THEORY this should not impact a JSON or XML put but it might be worth checking.

If you are 16.x you might need to get some support insight into whats going on with the router.

View solution in original post

12 REPLIES 12

Are you really using ELB? I thought that was an Amazon AWS (hosted) load balancer, suitable for AWS EC2 instances? Am I wrong?

The most likely cause is what you already suggested (implicitly) by your question: somehow the hostname is being stripped between the client and the router. Is it possible that the F5 device is stripping the hostname? Can you check that?

Can you see if the F5 is receiving the call, and what it does with the inbound call?

The 404 you receive from curl -X GET 10.3.244.100:9001

I suppose that is generated by Apigee Edge. You will probably see the Apigee Edge headers in the response, no?

The 404 you receive when you invoke from the internet... is it the same 404? Does it have Apigee Edge signatures? If it DOES, then Apigee is receiving the call. If not, then the call never reaches Apigee Edge.

@dino did i confuse you? apologies .

Its a private cloud installation , no AWS, its f5 .

well.. the hostname is not used from f5 because DMZ cannot recognize internal hosts.

Yes, f5 is receiving the call.

curl -X GET 1*.**.***.100:9001 even from inside the intranet giving error , curl -X GET 1*.**.***.100:9001 -h hostname is working fine in intranet.

from the internet the apigee standard 404 page is coming for all api calls with header "404". Yes it has apigee signatures.

So --- Apigee is receiving the call. but 404

I'm stuck

if you are running 16.x your router could still be up and running and your message-processor could be in a bad state.

id get support on the line and get them to walk you through a couple of troubleshooting steps. Id outline some here but its super late already and im half asleep.

@Benjamin Goldman would really appreciate any support for sure.. just stuck here

Not applicable

hmm...


An f5 load balancer - if im not mistaken - will use the API address and PORT (aka 10.3.244.100:9001) to route traffic - so this is sort of consistent if you cannot call that address directly.

I assume you are seeing the same behavior against BOTH servers? If you are great. If not, dont forget to check out what load balancing algo you are using. Least connections is a nasty foe sometimes.

Maybe you should re-set the host aliases and double check that you dont have any evil windows characters in the strings? in THEORY this should not impact a JSON or XML put but it might be worth checking.

If you are 16.x you might need to get some support insight into whats going on with the router.

hmm... do we need to pass -h hostname from loadbalancer.. ? it seems almost stupid workaround for me

Apigee its self REQUIRES a specific host name pattern to be set to respond.

If that host name pattern is not set in the listener the traffic will be dropped by apigee.

So for instance- in one of our on-prem installations:

4079-qa-system.jpg

From my internal network I can address each individual server by internal FQDN host name, and ip address- but i need to include the port number.

Additionally, if I call my load balancer, i can call it by

api.xxxx.xxxx.com (the first line) and that host header gets passed along with the request to the back end.

I dont think we did anything special to make sure that the host headers were relayed in the f5.


I would recommend getting on the phone with apigee support about solving the direct to server problem, and work with your f5 people on the rest.

@Benjamin Goldman firstly thanks for being patient with me 🙂 .

api.xxxx.xxxx.com (the first line) -- is the f5 inside or outside DMZ .. meaning the f5 have an external IP also along with the VIP?. Reason being my f5 is inside DMZ and also contains the VIP.

api.xxxx.xxxx.com (the first line) -- does the the public domain resolve to VIP of f5 when trying from internal network ?. Reason being it's the case for me.

Trying to troubleshoot where it went wrong 🙂

Regards

Your first question does not matter: As long as your load balancer accepts requests with that host header, and forwards them on to apigee with the header intact, it doesnt matter where it is.

Your second question: in our case yes. See the above for why this matters.

Having fixed this via a workaround , it's still worth checking from your side why a well known loadbalancer like f5 does not work well with apigee.

Maybe you guys can fix in next release if issue is from apigee side . It would be helpful for other customers .

@Sujith Mathew any chance you can share what you did to fix your problem? I have been running multiple on premise instances, some with global footprints, for several years, behind F5 load balancers and I dont have any issues.

I am curious: explicitly what did you do to get your setup working?