"Host" header doesn't get passed by httpClient

[annaku_rad]

I'm using the httpClient to make HTTP requests to a URL from within custom JavaScript code executing in a Shared-Flow that's used by API Proxy.

In the httpClient send() method I'm adding headers, but somehow the "Host" header does not get passed on to endpoint: (using it as in the example in docs)

var endpointRequest = new Request(endpointUrl, endpointOptions.method, endpointOptions.headers);
httpClient.send(endpointRequest, onComplete);

And endpointOptions.headers contains the Host- which I set explicitly:

headers["Host"] = context.getVariable('request.header.host');

When I print the endpointOptions.headers I see the expected host: 

endpointOptions headers: {.., "host":"34.95.96.135.nip.io", ..}

But in the endpoint I'm getting the host header as the endpointUrl Host. 

I saw a similar question regarding the Target- but in my case it's not the target of the Proxy, rather the target of the httpClient. This is blocking us, would like to hear how it can be relsoved, thanks

[dchiesa1]

But in the endpoint I'm getting the host header as the endpointUrl Host

I'm sorry, what does this mean? Can you clarify what the problem is?

I understand you're setting a HOST header, using the httpClient within JavaScript. And ... the request reaches the correct external endpoint. And then?

[annaku_rad]

Yes thanks, then endpoint is a server, and I see the headers it gets. If the endPoint url is: http://example.net:80/ then I see in the Host header in the endpoint: Host: example.net instead of the required 'request.header.host':

"34.95.96.135.nip.io" 

[dchiesa1]

oh I see what you're saying. You have the URL as "http://example.net" and you want to OVERRIDE that hostname with the host header.
I don't know if that actually works. To do that you may need to use the NIP.IO address in the URL. Eg "https://34.95.96.135.nip.io/foo/bar" with the Host name explicitly specified in the uRL. That should set the host header to what you want.

If that does not work, then. . . I don't know what else to suggest.

[annaku_rad]

I tried using the IP, and I still see in the Host header the IP instead of the specified Host header. I thought there may be a variable I can specify to override the Host header as suggested in this question for the Target: https://www.googlecloudcommunity.com/gc/Apigee/quot-Host-quot-header-doesn-t-get-passed-onto-the-tar...

And this one: https://www.googlecloudcommunity.com/gc/Apigee/How-can-we-retain-host-header-from-inbound-request/td...

"You could manually override the Host header value for the target by setting the variable 'target.header.host'."

Is there something similar for the httpClient external request?

[dchiesa1]

Is there something similar for the httpClient external request?

Not as far as I know

[annaku_rad]

I tried another way- to invoke the external endpoint with ServiceCallout Policy, and it also didn't work. I saw you replied in the Forum regarding this that this is a feature request- https://www.googlecloudcommunity.com/gc/Apigee/Is-there-a-way-to-set-the-host-header-on-a-service-ca... - is there an option for a feature-request for the httpClient in the JS Policy as well? And is there an estimation for resolution? Thanks

[dchiesa1]

There is an outstanding feature request, but there is no commitment that it will be implemented, let alone a schedule for implementation. 

I suggest that you create a DNS entry for the target in question and simply use the hostname you prefer to use in the URL. Rather than trying to invoke the service by IP address and specifying the HOST header separately, specify the hostname in the URL, and it will be correct. 

[annaku_rad]

Specify the hostname in the URL- this would be problematic, as I'd like to set the Host to the original one- 

context.getVariable('request.header.host');

In my case it's:

"34.95.96.135.nip.io" 

 So I guess it can cause a loop.. 🙂

[dchiesa1]

I sure don't understand what you mean. You want the API proxy to call out to... itself? 
I'm not clear. 

I'm also not clear on why you need to modify the hostname.  Just set a DNS record and invoke the target by that name.  Why is that not sufficient?  

Anyway you don't have to explain it to me.  I guess you'll work on your own solution.

[annaku_rad]

I want the API Proxy to call out an external service, but to have in the request towards it the Host header of the original request. Meaning I'd like to set the Host we got from the invoked API Proxy, in my case it's: 

"34.95.96.135.nip.io" 

I have the same use-case as in this question in the Forum regarding ServiceCallout, just with JavaScript Policy: Is there a way to set the host header on a service callout?

I am trying to use a JavaScript policy to call an API, and I want to set my own host header. No matter what I try, the edge always overwrites my host header before sending the request so the backend never gets my custom value. Is there a way to make sure my host header value makes it to the backend of a JavaScript httpClient?

[dchiesa1]

Ok , I understand the words you are writing , and I don’t understand why you would want to do what you’re describing. 

if you need to propagate the value of the original host name to the upstream server, use a different request header. Don’t use HOST.  Use “Original-Host” or anything that is not Host.  
If you absolutely need the outbound request to use the same host header, then I think you are aiming to do something that is purposefully insecure. And I don’t know how you would do that in Apigee and I don’t have any further suggestions. 

What you describe does not make sense to me. As I said I understand the words you’ve written. What I don’t understand is why you would have that requirement. 

[annaku_rad]

Regarding using a different request header- it works, but we'd like to use the same 'Host' header. I'm not sure how it's different from the feature-request of the ServiceCallout to override the 'Host' header.. https://www.googlecloudcommunity.com/gc/Apigee/Is-there-a-way-to-set-the-host-header-on-a-service-ca...

[bratac]

We have a similar situation - in Amazon, there is a Load Balancer that will read from the "host" header field, to which target host the request is meant to be routed.
See Listeners for your Application Load Balancers - Elastic Load Balancing (amazon.com)
According to Amazon, one host at a specific IP address could possibly host several different hosts with very different host names. Hence you need the "host" header to specify to which host your communication is directed towards.

[dchiesa1]

Yep, al that makes sense. 

---

@bratac wrote:

one host at a specific IP address could possibly host several different hosts with very different host names.

---

YES!  And then the service at that specific IP address could behave completely differently, depending on the hostname used to access it.  

And, the way for a client to be able to connect to those different hosts is to just specify the different hostname in the normal place, within the URL.  I understand that "overriding" the hostname with an explicit header is the feature request. What I'm saying is, there's a "normal" way to have clients specify different hosts right now: use the URL.  It does require DNS to be properly set up.   In other words,  host1.example.io and host2.whatever.com (totally different hostnames) must resolve to the same address, via a DNS lookup. But if you have that, it will just work.  

Unless I am misunderstanding something, I think there is a clean way (the right, standard way) to get the connections to work. 

The request here is to provide an extra way , aside from correctly configuring DNS, to support multiple hostnames.  And while I understand the request, I don't see why the lack of this capability needs to be an obstacle. There already is a way to do this!