This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Accept
Reject
Preferences
Log in to ask a question
Log in to ask a question

How to establish private access to Google APIs from Apigee X with VPC-Service Controls

Posted on 12-12-2023 08:43 AM
duncanchris
Staff
Reply posted on --/--/---- --:-- AM

Introduction

This article ultimately addresses how a customer who either plans to enable VPC-Service Controls on their Apigee X deployment or currently has VPC-Service Controls enabled on their Apigee X deployment can establish connectivity to invoke Google APIs using private access from their Apigee X runtime. 

For context, I recently was working with one of my customers in regards to their Apigee X deployment. The customer had followed a few of our great Apigee articles on our Google Cloud Community page that highlighted the ability to Control your Internet Egress traffic from Apigee & How to securely access Google APIs using restricted access from Apigee X. If you’d like to take a look at both of those great articles created by Daniel & Joel please take a look at them below as a precursor to this article.

With that being said the customer implemented the steps in both articles to ultimately Remove the unrestricted internet egress access that the Apigee X runtime has in place by enabling VPC-SC on their Apigee X tenant project. The customer had a hard requirement to lock down their Apigee deployment. 

The customer was also well aware that with VPC-SC enabled on Apigee that the service perimeter would only allow connectivity to the Google APIs in the restricted IP range 199.36.153.4/30. This range also includes the Apigee APIs (apigee.googleapis.com). With this setup in place the customer reached out to our team because they wanted to be able to access the Google APIs configured under the private.googleapis.com VIP as well. 

It is important to note that all public Google APIs are not accessible once VPC-SC’s have been enabled on an Apigee X tenant project. Public APIs or IPs are not accessible because the Apigee tenant no longer has internet egress enabled. VPC Service Controls supports the products listed in the Google Cloud documentation.

The documentation related to Apigee X/Hybrid and VPC-SC can be found here.

The APIs supported under the private.googleapis.com VIP can be found here.

At the end of day this was a big blocker for the customer in Production and I stepped in to come up with a workable solution to address the connectivity to that particular set of APIs. From a networking standpoint I reached out to some amazing colleagues to have joint working sessions to address this task from the customer.

There were numerous options we looked at from either utilizing a Next Generation Firewall appliance to handle the next hop to establish connectivity or by utilizing an Envoy Proxy with a Forwarding rule (with some Cloud DNS configurations) to do the trick.

We ended up deciding that a Regional load balancer with an External backend (internet NEG) should be the most feasible option to establish connectivity to the Google APIs configured under the private.googleapis.com VIP. In particular the customer wanted to access the Maps API. The Maps API will be the API service that we want to establish connectivity to for this article. 

The end architecture that this article addresses is below. 

duncanchris_0-1702395170107.png

 

High Level Steps

If you would also like to do this in your Apigee X deployment then follow the high level steps below. It’s also important to note here that the assumption is that you’ve already done steps 1 & 2 in the list below. In Step 3 you will follow those steps in our public documentation to set up the regional load balancer with an internet NEG. I will be showing some of the configurations in the commands to connect to the Maps API in particular. 

  1. Assumption here: That you already have a fully provisioned Apigee X deployment in place with VPC peering in place. Note: If your Apigee deployment is not using VPC peering then PSC must be used. 
  2. Assumption here: That you have already removed the default egress route from Apigee X by setting up a VPC Service perimeter & Enabling VPC-SC on your Apigee X tenant project. The Apigee VPC-SC steps are here.
  3. Implement this step: Configure the regional load balancer with an internet NEG to establish connectivity to the Google APIs (i.e. Restricted VIP & Private VIP). The steps to set up the regional load balancer are here.
  4. This Article will showcase how to: Test connectivity to the Google APIs from Apigee

Step 3.) Configure a regional load balancer with an internet NEG to establish connectivity to the private Google APIs

To follow this guide you will need to create an Application Load Balancer (LB) and create an internet Network Endpoint Group (NEG) in a project. For this article the LB and NEG will be created in the Apigee customer project. To complete these steps you should either be a project Owner or Editor, or you should have both of the following Compute Engine IAM roles listed in the documentation here.

duncanchris_1-1702395170073.png

To begin, ensure that you have already created the VPC network and subnet you wanted to use for the configuration. Since this is being done in my Apigee project I will just be using the network I already configured for use in my Apigee deployment, and for the subnet I will also be using the same VPC subnet I used with my Apigee deployment. In this setup you will also be creating another separate proxy-only subnet to be utilized by the regional LB. 

Also to make this implementation easier for you I would recommend setting these environment variables in Cloud Shell or in the terminal (You will need the gcloud SDK if you plan to use the terminal option) ahead of following these steps.

export LB_NETWORK=[replace with network you used with Apigee]
export SUBNET=[replace with vpc subnet you used with Apigee]
export PROXY_ONLY_SUBNET_NAME=[replace with the proxy only subnet name of your choosing]
export PROXY_ONLY_SUBNET_RANGE=[replace with the proxy subnet range of your choosing]
export REGION=[replace with the same region as your Apigee deployment]
export ROUTER_NAME=[replace with the name of your choosing]
export LB_NAT_NAME=[replace with the name of your NAT]
export LB_IP_ADDRESS=[replace with the name of your choosing]
export LB_SUBNET_NAME=[replace with vpc subnet you used with Apigee]
export INTERNET_NEG_NAME=[replace with the name of your choosing]
export HTTP_HEALTH_CHECK_NAME=[replace with the name of your choosing]
export BACKEND_SERVICE=[replace with the name of your choosing]
export URL_MAP_NAME=[replace with the name of your choosing]

Here are the steps to setup our external backend environment outside of Google Cloud:

  1. Configure a network endpoint to expose the external backend to Google Cloud. 
  2. Allow the external backend to receive traffic from Google Cloud. There are two sub steps that need to be followed here.
    1. Configure a Cloud NAT Gateway with IP addresses that are used for egress traffic from Google Cloud. The gateway will map the proxy-only subnet range to the external IP addresses.
    2. Making sure that your external backend environment is configured to allow traffic from Google Cloud to reach the external backend.

Create the VPC network and subnet (Assumption is that you’ve already created this / are using your networking from the Apigee project).

gcloud compute networks subnets create $PROXY_ONLY_SUBNET_NAME --purpose=REGIONAL_MANAGED_PROXY --role=ACTIVE --region=$REGION --network=$LB_NETWORK --range=$PROXY_ONLY_SUBNET_RANGE

Set up a Cloud NAT gateway (I followed the Dynamically allocated option but you can determine if Manual allocation is applicable to your configuration). 

  1. Create a Cloud Router:
gcloud compute routers create $ROUTER_NAME --network=$LB_NETWORK --region=$REGION
  1. Set up the Cloud NAT gateway:
gcloud compute routers nats create $LB_NAT_NAME --router=$ROUTER_NAME --endpoint-types=ENDPOINT_TYPE_MANAGED_PROXY_LB --nat-custom-subnet-ip-ranges=$PROXY_ONLY_SUBNET_NAME --auto-allocate-nat-external-ips --region=$REGION

Reserve the load balancer’s IP address

  1. Using the gcloud CLI, run the compute addresses create command:
gcloud compute addresses create $LB_IP_ADDRESS  --region=$REGION --subnet=$LB_SUBNET_NAME
  1. Use the compute addresses describe command to view the allocated IP address
gcloud compute addresses describe $LB_IP_ADDRESS --region=$REGION

Set up the internet NEG: 

  1. Create the NEG resource
gcloud compute network-endpoint-groups create $INTERNET_NEG_NAME --network-endpoint-type=INTERNET_FQDN_PORT --default-port=443 --network=$LB_NETWORK --region=$REGION
  1. Add endpoints to the NEG. If a port isn’t specified, the default port of the NEG is used. (This section is key in being able to establish which endpoint we want to configure in the NEG. Here I’m using the FQDN of the Maps API and what port I want to connect to it on).
gcloud compute network-endpoint-groups update $INTERNET_NEG_NAME --add-endpoint="fqdn=maps.googleapis.com,port=443" --region=$REGION

Create the load balancer:

  1. Optional: Create a health check. (For this configuration it should use tcp on port 80 for the Maps API)
gcloud compute health-checks create https $HTTP_HEALTH_CHECK_NAME --region=$REGION --use-serving-port
  1. Create a backend service:
gcloud compute backend-services create $BACKEND_SERVICE --load-balancing-scheme=INTERNAL_MANAGED --protocol=HTTPS --health-checks=$TCP_HEALTH_CHECK_NAME  --health-checks-region=$REGION --region=$REGION
  1. Add the internet NEG to the backend service:
gcloud compute backend-services add-backend $BACKEND_SERVICE --network-endpoint-group=$INTERNET_NEG_NAME --network-endpoint-group-region=$REGION --region=$REGION
  1. Create a URL map to route incoming requests to the backend service:
gcloud compute url-maps create $URL_MAP_NAME --default-service=$BACKEND_SERVICE --region=$REGION

For my implementation I did not perform steps 5, 6, 7 to make the Load Balancer into an HTTPS LB. My LB is an HTTP LB but the backend service is configured for HTTPS. The customer did proceed with ensuring they did steps 5, 6 and 7 which is definitely ideal for a Production environment and a real implementation.

Now we're done with the setup! Let’s test.

Step 4.) Test connectivity to the private Google APIs

First, before we test the actual connectivity through an Apigee proxy, let's re-confirm that we cannot hit the Maps API by running a few tests.

To test connectivity I decided to utilize the Timezone API configured under the Maps API portfolio. Before you can actually utilize this API you will need to generate an API Key.

  1. Follow this guide to generate an API Key in your project to use with the API call. (My recommendation is to not restrict the key)
  2. [This step is for the Apigee proxy testing] Once you have the API Key then you can create an API proxy in Apigee that acts as an interface for the Timezone API. To test the API we are going to use the curl documentation here to create a curl command that works with your Apigee environment.

Test #1. Test connectivity to the Timezone API via a curl call directly to the maps.googleapis.com endpoint. This curl is not utilizing a configured Apigee proxy and should work due to the fact that we set up a service perimeter via VPC-SC on the project (which by default the connectivity to the Google APIs in the restricted IP range 199.36.153.4/30 is established). At this point we just want to confirm that the service perimeter is working as intended. 

curl -L -X GET 'https://maps.googleapis.com/maps/api/timezone/json?location=39.6034810%2C-119.6822510&timestamp=1331161200&key=[replacewithyourapikey]'

duncanchris_2-1702395170188.png

So with this curl call we can see that it’s successfully connecting to the Timezone API while the service perimeter is enabled.

Test #2. Now let’s create an Apigee Proxy and deploy it. For the proxy configuration we are going to have the target endpoint configured in the same aspect as the curl command we used above but we will be utilizing the Apigee hostname to call the proxy in the environment the proxy is mapped to. The expectation is that the curl call through the proxy should fail. 

duncanchris_3-1702395170104.png

duncanchris_4-1702395170105.png

Now try the curl call with the Apigee hostname and ensure that you pass the API key you created above.

curl -X GET 'https://[replacewithyourApigee Hostname]/maps/api/timezone/json?location=39.6034810%2C-119.6822510&timestamp=1331161200&key=[replacewithyourapikey]'

duncanchris_5-1702395170107.png

With this we can see we're getting a 403 Forbidden. Optionally, you can also run a trace session in tandem to look at the end to end details of the API calls. 

Now that we’ve run the two tests, let’s configure the Apigee proxy in the third test to use the regional LB we created to establish connectivity via the end architecture that was shared above. 

Test #3. For this test just simply change the XML in the target endpoint postflow to point to the regional LB IP that was created above. Also, feel free to remove the SSLInfo block if applicable.

duncanchris_6-1702395170085.png

Now let’s test this proxy by running the curl again in Cloud Shell or a Terminal connected to your GCP project:

curl -X GET 'https://[replacewithyourApigee Hostname]/maps/api/timezone/json?location=39.6034810%2C-119.6822510&timestamp=1331161200&key=[replacewithyourapikey]'

Success! We are now able to connect to the Timezone API, a part of the Maps API that resides under the private.googleapis.com VIP while we have VPC-SC enabled on our Apigee X deployment.

duncanchris_7-1702395170100.png

There are further testing methods as mentioned in the articles above that may be beneficial to you after implementing this recommendation. 

Thanks a lot to @ssvaidyanathan  & @gonzalezruben  for the amazing support with the architecture and feedback during the drafting of this article!

3 0 893
Topic Labels
0 REPLIES 0
Top Solution Authors
View all