How should my Virtual Host be configured?

Not applicable

I have a netscaler in front of apigee. It's listening on port 443 and forwarding all traffic to apigee's 9001 port.

How should the virtual host be set up?

A)

{
  "hostAliases" : [ "10.10.93.81:443" ],
  "interfaces" : [ ],
  "name" : "default",
  "port" : "9001"
}

B)

{
  "hostAliases" : [ "10.10.93.81:443" ],
  "interfaces" : [ ],
  "name" : "default",
  "port" : "443"
}


C)

{
  "hostAliases" : [ "10.10.93.81" ],
  "interfaces" : [ ],
  "name" : "default",
  "port" : "443"
}


I currently have it set up as (A) but I'm getting the 'Sorry, the page you are looking for is currently unavailable. Please try again later.' error.

Solved Solved
1 17 2,220
1 ACCEPTED SOLUTION

Not applicable

Ok, @Dino, @arghya das,

I was finally able to get things going consistently.

1. I had to use a variation of A) above. It turns out that Apigee thinks this is still on port 9001 even though the netscaler is on 443.

{
        "name": "netscaler.dev",
        "hostAliases": [ "10.10.93.76:9001", "apigeedev.example.com:9001" ],
        "port": "9001"
}
Then I had to blow away (delete) the /opt/nginx/conf.d directory on the edge-router machine and restart edge-router. It took a couple of tries and some patience, but it finally recreated with my updated virtual hosts and things started working.

I remember the above hint about the conf.d directory from earlier problems in this vein having to do with the edge-router not staying running. So this fix also works for that.

View solution in original post

17 REPLIES 17

Not applicable

BTW - this may be an SSL cert issue. The netscaler is listening on 443 for https calls. Does that mean I need to set up SSL info on my Virtual Host (and the associated trust/store)?

Not applicable

Ok, so new info... the netscaler is stripping https (SSL) stuff from the request (decrypting) so it should be in the clear as it gets to apigee.

adas
New Member

@Joel Schuster Option A seems right to me. Do you have any apiproxy deployed on that virtualhost ?

For example, if you had an apiproxy ''test" deployed with basepath /v1/test, you would be calling it as

curl -v https://{IP}:443/v1/test
curl -v https://{DNS}/v1/test

And provided, your virtualhost is setup as:

{
  "hostAliases" : [ "{IP}:443", "{DNS}" ],
  "interfaces" : [ ],
  "name" : "default",
  "port" : "9001"
}

In this case, netscaler terminates the SSL and sends request to the edge router on port 9001. As long as the 'host' header of the incoming request matches the hostAlias in your vhost definition, things should work. If its not working, can you paste the actual http response here.

ahhh, good stuff. One more thing @arghya das - what if there is a requirement to secure the transport from Netscaler to MP ? is there a way? I suppose we must use SSLInfo and configure the netscaler side to do the right thing. Is that right?

@Dino Netscaler strips the SSL stuff, so I don't think that's an issue in this case.

I already have a 'default' set up, I'm adding another virtual host for just the netscaler.

3444-apigee1.png

I do have a test that I use that works fine if I go directly to the default host:port.

Going to the netscaler endpoint however it obviously gets to apigee, but doesn't resolve the proxy. And I *do* have 'netscaler.dev' as one of the <VirtualHost> entries in the proxy.

Output... changed to 'example.com' to protect the guilty.

$ curl -v https://apigeedev.example.com/echo/v2
* Adding handle: conn: 0x20b2f30
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x20b2f30) send_pipe: 1, recv_pipe: 0
* About to connect() to apigeedev.example.com port 443 (#0)
*   Trying 10.10.93.76...
* Connected to apigeedev.example.com (10.10.93.76) port 443 (#0)
* successfully set certificate verify locations:
*   CAfile: C:\Program Files (x86)\Git\bin\curl-ca-bundle.crt
  CApath: none
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using RC4-MD5
* Server certificate:
*        subject: C=US; ST=California; L=El Segundo; O=example, Inc.; CN=*.example.com
*        start date: 2015-10-14 04:00:09 GMT
*        expire date: 2018-10-14 04:30:08 GMT
*        subjectAltName: apigeedev.example.com matched
*        issuer: C=US; O=Entrust, Inc.; OU=See www.entrust.net/legal-terms; OU=(c) 2012 Entrust, Inc. - for authorized use only; CN=Entrust Certification Authority - L1K
*        SSL certificate verify ok.
> GET /echo/v2 HTTP/1.1
> User-Agent: curl/7.30.0
> Host: apigeedev.example.com
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Date: Mon, 29 Aug 2016 19:02:30 GMT
< Content-Type: text/html
< Content-Length: 344
< Connection: keep-alive
< ETag: "574dd5a3-158"
* Server Apigee Router is not blacklisted
< Server: Apigee Router
<
<!DOCTYPE html>
<html>
<head>
<title>Error</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>An error occurred.</h1>
<p>Sorry, the page you are looking for is currently unavailable.<br/>
Please try again later.</p>
</body>
</html>
* Connection #0 to host apigeedev.example.com left intact


And have you verified this part:

"As long as the 'host' header of the incoming request matches the hostAlias in your vhost definition,"

In other words, netscaler must pass in a HOST header which is listed among the hostAliases in the vhost definition. Is that happening?

To help troubleshoot you may wish to create a debug session on the MP. Like this:

curl -i -u uname:pass -X POST "http://localhost:8082/v1/logsessions?session=test"

Then, run a request into netscaler for that vhost:

curl -v https://apigeedev.example.com/echo/v2

Then, disable debug on the MP:

curl -i -u uname:pass -X DELETE "http://localhost:8082/v1/logsessions/test"

This may help you troubleshoot and diagnose the netscaler->MP issues.

Just to make sure, I modified the _default_ instead of adding a new virtual host. I'm getting the exact same result.

adas
New Member

Thanks for the clarification @Joel Schuster

This error is coming from the Apigee Router. Most likely because it doesn't find a matching virtualhost in the conf, so you get a 404.

To debug this further, you would need to make a request to the router directly. Can you try the following directly on the router box:

curl -v http://{hostname}:9001/echo/v2 -H 'Host: apigeedev.example.com'

So here, we are bypassing netscaler and making a request directly to the router with the host header as if the request were being made by any other client. Can you try this and see if it works.

If this doesn't work, then we need to look at the nginx conf on the router to see what's going on.

@arghya das

Indeed, this works just as it does via the netscaler. For a direct connect it works like it should. When the host is something other I get 404.

$ curl -v http://10.10.178.16:9001/echo/v2 -H 'Host: apigeedev.example.com'
* About to connect() to 10.10.178.16 port 9001 (#0)
*   Trying 10.10.178.16...
* Adding handle: conn: 0x2093020
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x2093020) send_pipe: 1, recv_pipe: 0
* Connected to 10.10.178.16 (10.10.178.16) port 9001 (#0)
> GET /echo/v2 HTTP/1.1
> User-Agent: curl/7.30.0
> Accept: */*
> Host: apigeedev.example.com
>
< HTTP/1.1 404 Not Found
< Date: Mon, 29 Aug 2016 19:54:28 GMT
< Content-Type: text/html
< Content-Length: 344
< Connection: keep-alive
< ETag: "574dd5a3-158"
* Server Apigee Router is not blacklisted
< Server: Apigee Router
<
<!DOCTYPE html>
<html>
<head>
<title>Error</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>An error occurred.</h1>
<p>Sorry, the page you are looking for is currently unavailable.<br/>
Please try again later.</p>
</body>
</html>
* Connection #0 to host 10.10.178.16 left intact
 

3445-apigee2.png

3446-apigee3.png

Is there a log file somewhere I can tail that would give me insight into why it's making the decision it is?

adas
New Member

@Joel Schuster Thanks for your patience in digging through this issue:

Ok. That suggests that the virtual host config doesn't exist or didn't load correctly. Can you try the following:

1. Restart the edge-router
apigee-service edge-router restart

2. Go to /opt/nginx/conf.d folder. List all the files using `ls -ltr`

3. You should see something like this:
[ec2-user@ip-10-140-195-115 conf.d]$ ls -ltr
total 32
-rw-r--r-- 1 apigee apigee 1624 Aug 26 21:10 0-default.conf
-rw-r--r-- 1 apigee apigee 1062 Aug 26 21:10 0-map.conf
-rw-r--r-- 1 apigee apigee 1044 Aug 26 21:10 0-upstream-stats.conf
-rw-r--r-- 1 apigee apigee  354 Aug 26 21:10 0-upstream-pools.conf
-rw-r--r-- 1 apigee apigee 2071 Aug 26 21:10 testorg1_test_default.conf
-rw-r--r-- 1 apigee apigee 2071 Aug 26 21:13 testorg1_test_secure.conf
-rw-r--r-- 1 apigee apigee 2078 Aug 29 16:19 testorg1_test_custom.conf.bad
-rw-r--r-- 1 apigee apigee  893 Aug 29 16:19 0-edge-health.conf

In my case my or is called 'testorg1' and I have 1 environment called 'test' with 2 virtualhosts - 'default' and 'secure'. Now when you open {org}_{env}_default.conf you should see something like below:

server {
        listen 10.10.178.16:9001  ;
        listen 127.0.0.1:9001  ;


        server_name apigeedev.example.com ;
        server_tokens off;
	...
	...

This means the server is listening on that IP and port (9001) in your case and the server_name is the virtualhost alias. Do you see this ? The output of the list command might be useful for debugging further.

I understand that this may contain sensitive information that you may not want to divulge. In that case please consider opening a support ticket and one of our GSC engineers will help you out. At this point, it looks like either the nginx conf didn't get created correctly when you created/updated the virtualhost or something wrong with the conf itself. Restart of the edge-router may fix the problem too since it will try to recreate the conf at startup.

adas
New Member

For example, in my case when I do the following on the router I get a 200 OK response which means the request was routed to the runtime engine and my api call succeeded:

[ec2-user@ip-10-140-195-115 conf.d]$ curl -v http://10.140.195.115:9001/apigee/target -H 'host: 54.167.130.178:9001' -X HEAD
* About to connect() to 10.140.195.115 port 9001 (#0)
*   Trying 10.140.195.115...
* Connected to 10.140.195.115 (10.140.195.115) port 9001 (#0)
> HEAD /apigee/target HTTP/1.1
> User-Agent: curl/7.29.0
> Accept: */*
> host: 54.167.130.178:9001
>
< HTTP/1.1 200 OK
< Date: Mon, 29 Aug 2016 21:11:35 GMT
< Connection: keep-alive
< Host: 54.167.130.178:9001
< X-Forwarded-For: 10.140.195.115
< X-Forwarded-Port: 9001
< X-Forwarded-Proto: http
< User-Agent: curl/7.29.0
< Accept: */*
< X-Target-Visitor: 6
< Server: Apigee Router

@arghya das Thanks. As you see there is no server_name record beyond the default IP.

server {
        listen 10.10.178.16:9001  ;
        listen 127.0.0.1:9001  ;


        server_name 10.10.178.16 ;
        server_tokens off;
        error_log /apps/apigee/var/log/edge-router/nginx/example~development.9001_error_log error;
        access_log /apps/apigee/var/log/edge-router/nginx/example~development.9001_access_log router;
        set $client_connection "$remote_addr:$my_nginx_var_remote_port;$server_addr:$server_port";
        limit_conn perserver 5000;
        req_status server;
        client_max_body_size 0;


        location / {
                proxy_redirect off;
                proxy_http_version 1.1;
                proxy_set_header Connection $connection_header;
                proxy_set_header Host $http_host;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header X-Apigee.Host $host:9001;
                proxy_set_header X-Apigee.client.connection $client_connection;
                proxy_set_header X-Forwarded-For $my_nginx_var_xff;
                proxy_set_header X-Forwarded-Port $my_nginx_var_port;
                proxy_set_header X-Forwarded-Proto $my_nginx_var_proto;
                proxy_set_header X-Apigee.Message-ID $hostname-$pid-$connection-$connection_requests;
                proxy_pass_header Server;
                set $upstream_stats_key "$host:$server_port";
                log_by_lua_file /apps/apigee/edge-router/bin/lua_scripts/collect_upstream_stats.lua;


                proxy_next_upstream http_599;
                proxy_pass http://10.10.178.16_8998;
        }


        error_page 400 404 500 502 503 504 /50x.html;
        location /50x.html {
               root /opt/nginx/html;
               internal;


        }
}

This is despite the above configuration that I set.

I have already submitted a support ticket, so hopefully they'll get to me quickly.

Thanks for your help.

adas
New Member

After creating the virtualhost, have you tried restarting the edge router. Like I said, there may have been a glitch during the update which is why the server name didn't get picked up or the conf didn't get generated correctly. Hopefully support would sort this out.

Something occurred to me this morning. I have a feeling that there are privileges to files that are not correct. I'm going to do some research. The Apigee processes are set up in our environment to run in 'apigee' user space. But nginx config files need to be modified (by default) by root, IIRC.

Not applicable

Ok, @Dino, @arghya das,

I was finally able to get things going consistently.

1. I had to use a variation of A) above. It turns out that Apigee thinks this is still on port 9001 even though the netscaler is on 443.

{
        "name": "netscaler.dev",
        "hostAliases": [ "10.10.93.76:9001", "apigeedev.example.com:9001" ],
        "port": "9001"
}
Then I had to blow away (delete) the /opt/nginx/conf.d directory on the edge-router machine and restart edge-router. It took a couple of tries and some patience, but it finally recreated with my updated virtual hosts and things started working.

I remember the above hint about the conf.d directory from earlier problems in this vein having to do with the edge-router not staying running. So this fix also works for that.

Joel - thanks for the followup and confirmation.