Apigee X target endpoint and Google Authentication

Hi everyone

I'd like to understand something, I'm following this article to create a PSC connection between my ApigeeX organization and my Target project which contains multiple Cloud Run services.

I successfully created the connection between the projects and when creating a new API Proxy to call a Cloud Run service it works for unauthenticated calls; nevertheless when  adding authentication I got:

The URL must be ssl enabled if Authentication is present. 

Please note the service account used for deployment has the serviceAccountUser role in the Apigee org project and run.invoker in the target project.

Per the documentation [1] when working with Southbound PCS targets I have to add this as a target:

 

  <HTTPTargetConnection>
    <URL>http://7.0.8.5:80</URL>
  </HTTPTargetConnection>

 

I have tried to use:

 

  <HTTPTargetConnection>
    <URL>https://7.0.8.5:443</URL>
  </HTTPTargetConnection>

 

But, it is not working, either.

My question is, when using this southbound PSC target how  can I use :

 

<Authentication>
    <GoogleIDToken>
      <Audience useTargetUrl="true"/>
    </GoogleIDToken>
</Authentication>

 

if my endpoint attachment IP is HTTP and not HTTPS?

I add a diagram of my current architecture

arch.png

Any guidance will be appreciated.

@miguelmendoza tagged as I  checked [2].

[1] https://cloud.google.com/apigee/docs/api-platform/troubleshoot/playbooks/network-configuration/conne...

[2] https://www.googlecloudcommunity.com/gc/Apigee/How-to-Load-Balance-Between-Cloud-Run-Targets-in-Apig...

 

Solved Solved
3 19 980
1 ACCEPTED SOLUTION

Following this thread here is how I solved it, I  created a target server and pointed to it:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<TargetEndpoint name="default">
  <Description/>
  <FaultRules/>
  <PreFlow name="PreFlow">
    <Request/>
    <Response/>
  </PreFlow>
  <PostFlow name="PostFlow">
    <Request/>
    <Response/>
  </PostFlow>
  <Flows/>
  <HTTPTargetConnection>
    <Properties/>
    <LoadBalancer>
      <Server name="my-server"/>
    </LoadBalancer>
    <Authentication>
      <HeaderName>X-Serverless-Authorization</HeaderName>
      <GoogleIDToken>
        <Audience>https://my.run.app</Audience>
      </GoogleIDToken>
    </Authentication>
  </HTTPTargetConnection>
</TargetEndpoint>



View solution in original post

19 REPLIES 19

You should use the Cloud Run *.run.app domain in the Authentication element instead of the target url. You can also dynamically set the value using a flow variable populated from environment config (kvm, property set, custom attributes, etc.).

what is the point then of using an endpoint target connection and creating and specific IP for it? Also, I have multiple Cloud Run services which is why  I'm using the URL mas </service>, if following your advice this will limit to use of one API Proxy per Cloud Run service which is not suitable for my use case.

You can do this as Ruben points out, and without DNS by setting the target URL to PSC IP and setting host via 'target.header.host' in the target flow, either the actual Cloud Run host or an ILB host.

 

<AssignMessage continueOnError="false" enabled="true" name="AM-v1">
    <AssignVariable>
        <Name>audience</Name>
        <Value>https://cloud-run-external-service-v1-ghkj3eaabq-ue.a.run.app</Value>
    </AssignVariable>
    <AssignVariable>
        <Name>target.header.host</Name>
        <Value>cloud-run-ilb.domain.net</Value>
    </AssignVariable>
</AssignMessage>

 

The target endpouint looks like:

 

    <HTTPTargetConnection>
        <URL>https://10.50.0.34</URL>
        <Authentication>
            <GoogleIDToken>
                <Audience ref="audience">https://hello-from-cloud-run-internal-ghkj3eaabq-ue.a.run.app</Audience>
                <IncludeEmail>true</IncludeEmail>
            </GoogleIDToken>
        </Authentication>
        <SSLInfo>
            <Enabled>true</Enabled>
            <IgnoreValidationErrors>true</IgnoreValidationErrors>
        </SSLInfo>
    </HTTPTargetConnection>

 

 

 

Thank you, Ruben and Kurt, for your responses, I'm going to try the aforementioned steps to fix my issue, I have a remaining question when setting the HTTP Target URL you are using https , but in this documentation [1] is mentioned "Ensure that the same port 80 is the port in the TargetEndpoint of the API proxy."  and recommends:

 

<HTTPTargetConnection>
  <URL>http://7.0.4.2:80</URL>
</HTTPTargetConnection>

 

Of course, if I use http instead of https the message "The URL must be ssl enabled if Authentication is present. "  is received, in which case,  what is the protocol used by the IP created with my target endpoint http or https? Or is it possible to use both of them to use this endpoint attachment IP?

[1] https://cloud.google.com/apigee/docs/api-platform/troubleshoot/playbooks/network-configuration/conne...

Following this thread here is how I solved it, I  created a target server and pointed to it:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<TargetEndpoint name="default">
  <Description/>
  <FaultRules/>
  <PreFlow name="PreFlow">
    <Request/>
    <Response/>
  </PreFlow>
  <PostFlow name="PostFlow">
    <Request/>
    <Response/>
  </PostFlow>
  <Flows/>
  <HTTPTargetConnection>
    <Properties/>
    <LoadBalancer>
      <Server name="my-server"/>
    </LoadBalancer>
    <Authentication>
      <HeaderName>X-Serverless-Authorization</HeaderName>
      <GoogleIDToken>
        <Audience>https://my.run.app</Audience>
      </GoogleIDToken>
    </Authentication>
  </HTTPTargetConnection>
</TargetEndpoint>



Hi @ezetina how did you configure your target server ?

I am having trouble with https. The http connection was ok but when I configure the internal load balancer frontend to accept https  and change this to https I am now getting 503.

<URL>https://7.0.24.15</URL>

I am able to verify the internal load balancer configuration on the Project side is okay by creating VM Instance and use curl.

If you are using the PSC IP, you can set that in the Target Server directly, but you will need to use the Apigee API to create the Target Server to ignore validation errors.

For example:

curl -X POST 'https://apigee.googleapis.com/v1/organizations/$ORG/environments/$ENV/targetservers' \
--header 'Content-Type: application/json' \
--data-raw '{
  "name": "cloud-run-ingore-ssl-ip",
  "host": "7.0.12.22",
  "port": 443,
  "isEnabled": true,
  "sSLInfo": {
    "enabled": true,
    "ignoreValidationErrors": true
  },
  "protocol": "HTTP"
}'

You also need to set the `target.header.host` in the target request flow to the hostname of your ILB.

<AssignMessage name="AM-host-header">
    <AssignVariable>
        <Name>target.header.host</Name>
        <Value>cloud-run-demo.example.net</Value>
    </AssignVariable>
</AssignMessage>

Then your Target Endpoint looks like:

    <HTTPTargetConnection>
        <LoadBalancer>
            <Server name="cloud-run-ingore-ssl-ip"/>
        </LoadBalancer>
        <Authentication>
            <GoogleIDToken>
                <Audience>https://hello-cloud-run-demo-x6isdfisfake-ue.a.run.app</Audience>
                <IncludeEmail>true</IncludeEmail>
            </GoogleIDToken>
        </Authentication>
    </HTTPTargetConnection>

 A couple of final notes:

  1. Peered DNS doesn't work in a non-VPC peered configuration.
  2. Load Balanced servers with HealthMonitor doesn't work in either non-VPC or VPC peered configuration, because the health check cannot access the PSC IP address.

Hi @kurtkanaskie 

Thank you for responding appreciate it.

I am using Endpoint Attachment IP. I am uncertain if the PSC IP that you mention and Endpoint Attachment IP is the same.

If they are not the same will the same config solution that you give still works ?

I am following this architecture document:

https://cloud.google.com/apigee/docs/api-platform/architecture/southbound-networking-patterns-endpoi...

But instead of instance (sample that is shown in the document) as a backend i have cloud run as a backend.

Thank you.

Yes, the PSC Endpoint Attachment created in your Org is "7.0.12.22" connected to the PSC Service Attachment in your Cloud Run project, which may be different from your Org project.

The configuration is: Apigee X --> PSC Endpoint Attachment --> PSC Service Attachment --> ILB --> Cloud Run backends.

Hi @kurtkanaskie upon trying your given configuration I am still getting 504 in Apigee X

{
    "fault": {
        "faultstring": "Gateway Timeout",
        "detail": {
            "errorcode": "messaging.adaptors.http.flow.GatewayTimeout",
            "reason": "TARGET_READ_TIMEOUT"
        }
    }
}

my targetserver configuration:

{
    "name": "auth-poc-ignore-ssl-ip",
    "host": "7.0.24.2",
    "port": 443,
    "isEnabled": true,
    "sSLInfo": {
        "enabled": true,
        "ignoreValidationErrors": true
    },
    "protocol": "HTTP"
}

 my AM configuration (actual host has been replaced):

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<AssignMessage continueOnError="false" enabled="true" name="AM-host-header">
  <DisplayName>AM-host-header</DisplayName>
  <Properties/>
  <AssignVariable>
    <Name>target.header.host</Name>
    <Value>ilbhostname.com</Value>
  </AssignVariable>
  <IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
  <AssignTo createNew="false" transport="http" type="request"/>
</AssignMessage>

Assign it to targetserver preflow (actual CF v2 link has been replaced):

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<TargetEndpoint name="default">
  <Description/>
  <FaultRules/>
  <PreFlow name="PreFlow">
    <Request>
      <Step>
        <Name>AM-host-header</Name>
      </Step>
    </Request>
    <Response/>
  </PreFlow>
  <PostFlow name="PostFlow">
    <Request/>
    <Response/>
  </PostFlow>
  <Flows/>
  <HTTPTargetConnection>
    <Properties/>
    <LoadBalancer>
    <Server name="auth-poc-ignore-ssl-ip"/>
  </LoadBalancer>
  <Authentication>
    <GoogleIDToken>
     <Audience>https://mycloudfunctionv2.cloudfunctions.net/AuthenticationPOC</Audience>
    </GoogleIDToken>
  </Authentication>
</HTTPTargetConnection>
</TargetEndpoint>

 I have also checked this document troubleshooting guide:

https://cloud.google.com/apigee/docs/api-platform/troubleshoot/playbooks/network-configuration/conne...

verified all the configuration mention are ok.

Tried calling the configured ILB on a custom created VM for testing it is working:

ktan_08_0-1711964886695.png

Also verified the ILB logs .The https call is coming when called via curl on a VM but not when called from ApigeeX.

Checking if there is still something that I can check for debugging the issue.

Hmmm, that all looks OK.

Perhaps it's firewall related?

The general steps are:

gcloud compute networks subnets create psc-subnet \
  --project=$PROJECT_ID --network default --region=us-east1 \
  --purpose=PRIVATE_SERVICE_CONNECT --range=10.100.0.0/28

gcloud compute firewall-rules create allow-psc-nat-443 \
  --project=$PROJECT_ID --network default \
  --direction=INGRESS --priority=1000 --action=ALLOW --rules=tcp:443 \
  --source-ranges=10.100.0.0/28 --target-tags=psc-demo

gcloud compute service-attachments create psc-attachment \
    --project=$PROJECT_ID --region=us-east1 \
    --producer-forwarding-rule=fr-ilb \
    --connection-preference=ACCEPT_AUTOMATIC \
    --nat-subnets=psc-subnet

kurtkanaskie_0-1711995837261.png

 

Hi @kurtkanaskie  

Just an update I have it all working thank you for your help.

Found out on the PSC Service Attachment configuration I have selected the HTTP forwarding rule that I created on the ILB instead of the HTTPS.

And after applying the configuration that you provided it is working now.

Thanks again.

Hi @ezetina ,

Can you help me in providing the exact configurations you used in your target server and what url did you use for in the audienc block. Is it your actual cloud run url?
I am also using a similar setup using psc for HTTP ILB with cloud run as my backend. And i am using url mask in my backend.

I created the  target server in this way using Terraform:

 

resource "google_apigee_endpoint_attachment" "endpoint_attachment" {
  org_id                 = "organizations/${var.project_id}"
  endpoint_attachment_id = "${var.apigee_endpoint_attachment}"
  location               = var.region
  service_attachment     = "projects/${var.project_ods_dtc_dev}/regions/${var.region}/serviceAttachments/${var.psc_name}"
}

resource "google_apigee_target_server" "target_server" {
  name        = "target-server"
  description = "ApigeeTarget Server"
  protocol    = "HTTP"
  host        = google_apigee_endpoint_attachment.endpoint_attachment.host
  port        = 80
  env_id      = module.apigee-x-core.environments.apigeex-env.id
}

 

Then I configured my target-endpoints / default.xml

 

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<TargetEndpoint name="default">
  <Description/>
  <FaultRules/>
  <PreFlow name="PreFlow">
    <Request/>
    <Response/>
  </PreFlow>
  <PostFlow name="PostFlow">
    <Request/>
    <Response/>
  </PostFlow>
  <Flows/>
  <HTTPTargetConnection>
    <Properties/>
    <LoadBalancer>
      <Server name="target-server"/>
    </LoadBalancer>
    <Authentication>
      <HeaderName>X-Serverless-Authorization</HeaderName>
      <GoogleIDToken>
        <Audience>https://myurl.run.app</Audience>
      </GoogleIDToken>
    </Authentication>
  </HTTPTargetConnection>
</TargetEndpoint>

 

Hi @ezetina ,

So the url you specified as “https://my url.run.app” will be our actual cloud run url right something like this :

https://hello-cloud-run-demo-x6isdfisfake-ue.a.run.app

However, what if I deploy a new cloud run revision? Does that mean I need to update my proxy with new url I get based on the new revision? 

 

When a new revision is deployed there is no changes in the Cloud Run URL, however if the service is deleted and then re-created the api proxy must be updated.

Hi @ezetina ,

I used the same setting like you said:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<TargetEndpoint name="default">
  <Description/>
  <FaultRules/>
  <PreFlow name="PreFlow">
    <Request/>
    <Response/>
  </PreFlow>
  <PostFlow name="PostFlow">
    <Request/>
    <Response/>
  </PostFlow>
  <Flows/>
  <HTTPTargetConnection>
    <Properties/>
    <LoadBalancer>
      <Server name="sb-target"/>
    </LoadBalancer>
    <Authentication>
      <HeaderName>X-Serverless-Authorization</HeaderName>
      <GoogleIDToken>
        <Audience>https://test-world-jyj7itorua-ew.a.run.app</Audience>
      </GoogleIDToken>
    </Authentication>
  </HTTPTargetConnection>
</TargetEndpoint>

However, I get this error: "<html><title>Error 404 (Not Found)!</title></html>".

Did you setup any additional policies like assign message etc?
Also, In my southbound ILB serverless neg, I am using url mask "/<service>".
Where do we provide the cloud run service name?
I know, we passing the entire cloud run url but still the southbound ILB serverless neg still checks for the service name right based on the URL mask "/<service>"

We found a similar  issue,  I think  this is something related to  how the URL masks works.

try to reach https://your-url/your-service/your-endpoint/

we use API ROUTE to fix this https://fastapi.tiangolo.com/reference/apirouter/?h=apiroute

router = APIRouter(prefix="/my-service")
app = FastAPI()
app.include_router(router)

Since this is not an issue for our use case, we have  followed this approach, but  I think there is a way to configure the  URL mask to avoid this extra step

I've found how to avoid using https://your-url/your-service/your-endpoint/  and use instead https://your-url/your-endpoint/ 

Try including <path> in the HTTPTargetConnection config:

<HTTPTargetConnection>
    <Path>/my-enpoint</Path>
    <Properties/>
    <LoadBalancer>
      <Server name="target-server"/>
    </LoadBalancer>
    <Authentication>
      <HeaderName>X-Serverless-Authorization</HeaderName>
      <GoogleIDToken>
        <Audience>https://run.app</Audience>
      </GoogleIDToken>
    </Authentication>
  </HTTPTargetConnection>

 https://cloud.google.com/apigee/docs/api-platform/reference/endpoint-properties-reference