VerifyJWT is giving an Invalid token error intermittently

[venkateshwargan]

Hi Team,

Please find the below steps which I am performing in Apigee to validate JWT token.

1. Decode firebase JWT token and retrieve header.kid

2. Check Firebase public certificate available in Apigee Cache(by giving incoming token header.kid as a Cache key)

3. if above key is not found in Cache then call Firebase public url(https://www.googleapis.com/robot/v1/metadata/x509/securetoken@system.gserviceaccount.com) through Service Callout Policy else go to step 5.

4. Apply Extract variable policy to retrieve public certificate based on JSON path with header.kid as a path.

5. Validate token using VerifyJWT policy using above public certificate

6. Store public certificate in Apigee Cache for 10 seconds with header.kid as a key if step Lookup failed in the above.

I am getting invalid token error in VerifyJWT policy intermittently but the same token is working in next request(within 10 seconds).

a) Why I am getting invalid token error and why it is working for next request?

b) No where mentioned in the firebase documentation that when the public certificates will change and how much duration we need to cache those keys in Apigee?

Error from debug:

{
"id": "Error",
"results": [
{
"actionResult": "DebugInfo",
"accessList": [],
"timestamp": "29-02-24 14:47:46:583",
"properties": {
"properties": [
{
"name": "error.class",
"value": "com.apigee.steps.jwt.verify.VerificationException"
},
{
"name": "state",
"value": "PROXY_REQ_FLOW"
},
{
"name": "type",
"value": "ErrorPoint"
},
{
"name": "error",
"value": "Invalid token: policy(JWT-VerifyFireBaseAuthToken)"
}
],
"property": [
{
"name": "error.class",
"value": "com.apigee.steps.jwt.verify.VerificationException"
},
{
"name": "state",
"value": "PROXY_REQ_FLOW"
},
{
"name": "type",
"value": "ErrorPoint"
},
{
"name": "error",
"value": "Invalid token: policy(JWT-VerifyFireBaseAuthToken)"
}
]
},
"headers": [],
"reasonPhrase": "",
"statusCode": "",
"uri": "",
"verb": "",
"content": "",
"ActionResult": "DebugInfo",
"uRI": ""
}
]
},

Thanks

Venkat

[dchiesa1 · Accepted solution]

If I were doing this, I would

1. convert to JWKS format for keys. Eg use this url: https://www.googleapis.com/service_accounts/v1/jwk/securetoken@system.gserviceaccount.com
2. let Apigee do the caching of the JWKS. You don't need to user ServiceCallout to call out to retrieve the keys. Let VerifyJWT do it.

It looks like this:

<VerifyJWT name='VJWT-1'>
  <Algorithm>RS256</Algorithm>
  <Source>inbound.jwt</Source>
  <IgnoreUnresolvedVariables>false</IgnoreUnresolvedVariables>
  <PublicKey>
    <JWKS uri="https://www.googleapis.com/service_accounts/v1/jwk/securetoken@system.gserviceaccount.com"/>
  </PublicKey>
  <Issuer>whatever-your-issuer-is</Issuer>
</VerifyJWT>

With the above configuration, Apigee will retrieve and cache the JWKS, and update it periodically. This can greatly simplify your proxy logic!  You would avoid DecodeJWT, LookupCache, ServiceCallout, ExtractVariables, PopulateCache, ... and maybe others.  

No where mentioned in the firebase documentation that when the public certificates will change and how much duration we need to cache those keys in Apigee?

Right. And that is not un-common among ID token issuers (Firebase, Okta, Ping, Entra ID, etc).  Often the issuer does not say, and will not say, for how long keys live.  In light of that, Ideally, the caching by the verifier would be intelligent and would

• cache the keys indefinitely
• when the required kid is not found in the cached JWKS, then re-request the JWKS

Apigee doesn't take that approach - not today anyway. Instead the VerifyJWT policy will cache the keys "for a short period", before re-requesting the keys from the endpoint. 

But this should be ok.  Most signers (like Firebase) are savvy enough to publish new keys to the JWKS endpoint, 24 hours or more prior to actually _using_ the keys to sign anything. Which means if the verifier caches keys for less than 24 hours, they'll be sure to get the keys necessary for signed tokens.  And Apigee follows that guidance - it will cache keys for less than 24 hours.  So you should be safe with this approach.

[venkateshwargan · Accepted solution]

Hi @dchiesa1 ,

Thanks for your reply and it is working.

Could you please provide any documentation link which explains VerifyJWT policy does cache JWS keys, as per below documentation link and notes, it is mentioned that keys need to be retrieved through service callout policy. Otherwise, it is difficult for me to convey my client, they may think that for each token verification we are calling Public URL.

https://cloud.google.com/apigee/docs/api-platform/reference/policies/jwt-policies-overview

Note: It's a good practice to cache the JWKS to avoid a Service Callout on every request.

Thanks

Venkat

[dchiesa1 · Accepted solution]

Thanks for the note, Venkat,

I didn't realize that page of the documentation recommended using ServiceCallout to retrieve the JWKS. I'll get that fixed.

Could you please provide any documentation link which explains VerifyJWT policy does cache JWS keys,

Here's the documentation link: https://cloud.google.com/apigee/docs/api-platform/reference/policies/verify-jwt-policy#publickeyjwks

[dchiesa1]

check the variable jwt.error for more information on the error. 

[dchiesa1 · Accepted solution]

If I were doing this, I would

1. convert to JWKS format for keys. Eg use this url: https://www.googleapis.com/service_accounts/v1/jwk/securetoken@system.gserviceaccount.com
2. let Apigee do the caching of the JWKS. You don't need to user ServiceCallout to call out to retrieve the keys. Let VerifyJWT do it.

It looks like this:

<VerifyJWT name='VJWT-1'>
  <Algorithm>RS256</Algorithm>
  <Source>inbound.jwt</Source>
  <IgnoreUnresolvedVariables>false</IgnoreUnresolvedVariables>
  <PublicKey>
    <JWKS uri="https://www.googleapis.com/service_accounts/v1/jwk/securetoken@system.gserviceaccount.com"/>
  </PublicKey>
  <Issuer>whatever-your-issuer-is</Issuer>
</VerifyJWT>

With the above configuration, Apigee will retrieve and cache the JWKS, and update it periodically. This can greatly simplify your proxy logic!  You would avoid DecodeJWT, LookupCache, ServiceCallout, ExtractVariables, PopulateCache, ... and maybe others.  

No where mentioned in the firebase documentation that when the public certificates will change and how much duration we need to cache those keys in Apigee?

Right. And that is not un-common among ID token issuers (Firebase, Okta, Ping, Entra ID, etc).  Often the issuer does not say, and will not say, for how long keys live.  In light of that, Ideally, the caching by the verifier would be intelligent and would

• cache the keys indefinitely
• when the required kid is not found in the cached JWKS, then re-request the JWKS

Apigee doesn't take that approach - not today anyway. Instead the VerifyJWT policy will cache the keys "for a short period", before re-requesting the keys from the endpoint. 

But this should be ok.  Most signers (like Firebase) are savvy enough to publish new keys to the JWKS endpoint, 24 hours or more prior to actually _using_ the keys to sign anything. Which means if the verifier caches keys for less than 24 hours, they'll be sure to get the keys necessary for signed tokens.  And Apigee follows that guidance - it will cache keys for less than 24 hours.  So you should be safe with this approach.

[venkateshwargan · Accepted solution]

Hi @dchiesa1 ,

Thanks for your reply and it is working.

Could you please provide any documentation link which explains VerifyJWT policy does cache JWS keys, as per below documentation link and notes, it is mentioned that keys need to be retrieved through service callout policy. Otherwise, it is difficult for me to convey my client, they may think that for each token verification we are calling Public URL.

https://cloud.google.com/apigee/docs/api-platform/reference/policies/jwt-policies-overview

Note: It's a good practice to cache the JWKS to avoid a Service Callout on every request.

Thanks

Venkat

[dchiesa1 · Accepted solution]

Thanks for the note, Venkat,

I didn't realize that page of the documentation recommended using ServiceCallout to retrieve the JWKS. I'll get that fixed.

Could you please provide any documentation link which explains VerifyJWT policy does cache JWS keys,

Here's the documentation link: https://cloud.google.com/apigee/docs/api-platform/reference/policies/verify-jwt-policy#publickeyjwks

[venkateshwargan · Accepted solution]

Thanks Dino, all are happy now.

[dchiesa1]

Venkat, FYI, I've updated the documentation page that you referenced, the JWT policies overview. That page no longer recommends the use of ServiceCallout and etc., for Verifying JWT through JWKS.