x-forwarded-for alternative for infrastructure behind ELB

Not applicable

Hi - in the 16.01 release for private cloud it was announced that support for x-forwarded-for header will be deprecated.

Is there an article somewhere that describes how one should re-work access control policies on installations behind enterprise load balancers so that substantial investments wont be lost due to this change?

Solved Solved
1 9 1,996
1 ACCEPTED SOLUTION

adas
Participant V

@Benjamin Goldman Let me clarify what that deprecation notice really means. In the earlier versions, you could actually spoof an x-forwarded-for header and router would honour that, meaning the x-forwarded-for header would be passed on to the message-processor even though its not the actual client IP. In our cloud, we had several customers who were picking the first IP or last IP from a list of x-forwarded-for IPs. The deprecation notice was mostly targeted for those customers who have this hard coded logic of picking an IP from the list, depending on its position or index in the x-forwarded-for header.

Now what this feature actually means is that even if you send fake x-forwarded-for IP along with your request, the new Apigee router (which is based off nginx) would have the ability to drop that fake header and only pick the x-forwarded-for header for the actual connection. So if you have an ELB or some other load balancer in front of the routers, then we would accept the x-forwarded-for from the trusted IP. However if you don't have a load balancing layer and the request is sent directly to the router, it would pick the true client IP and reject any x-forwarded-for header that is sent, which is what you want from an ACL perspective. Does that answer your question. If you need any further clarification, please let me know.

Having said that it can be configured whether you want to trust the x-forwarded-for IP coming from a trusted IP. By default we consider an internal IP as trusted IP, for example in our cloud the ELB IP would be an internal IP and the router would trust it because they are in the same network (aws). If you need more information on how to configure this, I would have someone from our private cloud team clarify that for you.

View solution in original post

9 REPLIES 9

adas
Participant V

@Benjamin Goldman Let me clarify what that deprecation notice really means. In the earlier versions, you could actually spoof an x-forwarded-for header and router would honour that, meaning the x-forwarded-for header would be passed on to the message-processor even though its not the actual client IP. In our cloud, we had several customers who were picking the first IP or last IP from a list of x-forwarded-for IPs. The deprecation notice was mostly targeted for those customers who have this hard coded logic of picking an IP from the list, depending on its position or index in the x-forwarded-for header.

Now what this feature actually means is that even if you send fake x-forwarded-for IP along with your request, the new Apigee router (which is based off nginx) would have the ability to drop that fake header and only pick the x-forwarded-for header for the actual connection. So if you have an ELB or some other load balancer in front of the routers, then we would accept the x-forwarded-for from the trusted IP. However if you don't have a load balancing layer and the request is sent directly to the router, it would pick the true client IP and reject any x-forwarded-for header that is sent, which is what you want from an ACL perspective. Does that answer your question. If you need any further clarification, please let me know.

Having said that it can be configured whether you want to trust the x-forwarded-for IP coming from a trusted IP. By default we consider an internal IP as trusted IP, for example in our cloud the ELB IP would be an internal IP and the router would trust it because they are in the same network (aws). If you need more information on how to configure this, I would have someone from our private cloud team clarify that for you.

hmm - the release notes were pretty clear: x-forwarded-for is going to be deprecated. Deprecated means not supported anymore.

Id appreciate some information be posted publicly about how to set the trusted IP address/range/etc for this specific feature (hopefully this trusted range is not tied up with some other sort of trust? that would make things very complicated....)

Hey @Benjamin Goldman - When that original deprecation notice came out, the change wasn't implemented yet, so we didn't know yet what to tell users. In creating the release notes for 4.16.01, I copied/pasted release notes from all previous cloud releases that are included in this 4.16.01. It was my bad for not scrubbing that note better, after the updated feature had already been implemented. I've moved that info from the deprecated section down into the new features section and clarified it.

By default, Edge strips X-Forwarded-For. You have to set an org-level property to enable X-Forwarded-For. You'd do that architectures where you can actually trust X-Forwarded-For addresses, like @arghya das mentions.

We've got the info covered in the Access Control policy, where this is most relevant.

Apologies for the copy/paste carelessness!

Thanks guys - one of the many things im panicking about has been resolved.

Now on to the directory issues w/ the latest update...

@Benjamin Goldman can you elaborate on the directory issues? or has it been already asked as a different community post?

ha 🙂 it wouldnt be a problem in most people's environments. Not something Apigee can help me with - but just as big a blocker for me 🙂

Phew...ok 🙂

Hi @arghya das,

My consumer sends in the request via a forward proxy. I see that the source IP shows up in the X-Forwarded-For in the chain(Second position to be precise). Applied the org level ACL setting intending to use the X_FORWARDED_FOR_ALL_IP option and put in my client IP in a MatchRule block. However even now Access Control validates against the first IP in the chain.

Is there any more settings which I need to do to make this work? The ELB in front my Apigee router is currently configured to listen to TCP handshakes.

Please advise.

adas
Participant V

Since this has to do with security, we obviously didnt want to put too much information in the public domain, but I have sent a note to our private cloud team and the doc team to see if we can include this in the private cloud documentation. Thanks for bringing this up.