Error oauth.v2.ExternalTokenLengthLimitExceeded in Hybrid with External OAUTH Token

[lisabradshaw]

Working in Apigee Hybrid and getting the following error code:

steps.oauth.v2.ExternalTokenLengthLimitExceeded

We are attempting to Generate an Access Token (OAUTHV2 Client Credentials Grant Type) using an external JWT token.

Has anyone found a workaround to manage large external tokens when working with a third party?

Any help is greatly appreciated.  Thank you

[dchiesa1 · Accepted solution]

Hmm, why do you want to  store an externally-generated JWT ?

There are two kinds of tokens we can talk about: opaque tokens and JWT.  Both can be used as OAuth tokens. (Both can be passed in the Authorization header as an access token). 

I think the original intent of the GenerateAccessToken operation with an ExternalToken element was to allow Apigee to ingest an opaque access token that some external system has generated.  Later, when the app presented that token again in the context of a request-for-service, Apigee could validate the token just as it could validate an opaque token that was generated directly by Apigee. Internally, this validation involves a lookup in  persistent token store, and the retrieval of all the metadata associated to the token - the app or client id, the product, the expiry, and so on. The only way to verify an opaque token is to ask the system that generated it "is this a good token?"  It will always involve some sort of storage. The token issuer needs the list of tokens it generated, and their lifetimes, etc. The token itself is just a "lookup key". The persistent store has a limit to the size of the token it can store! That's the error you're seeing. 

If you have a JWT that was generated by an external system, that's a different story.  It's not an opaque token. It contains a whole bunch of metadata - like expiry, issuing party, subject, audience, and a bunch of other things.  A JWT is not a lookup key that points to data; a JWT IS the data. All of those claims are accessible to any system that "sees" the JWT.  The integrity of the token is independently verifiable, by any party, including by parties that did not issue the token. If we are speaking of signed JWT, the most commonly used variant, then any system that receives the JWT can verify it, and access all the data stored within it, and make decisions based on that data, without storing any part of it.  A verifying party (like Apigee I guess)  only needs access to the verifying key to verify the JWT.  In the specific case that the JWT uses an RSA algorithm, then Apigee  needs access to the public key, which often it can get by sending a GET request to a well-known JWKS endpoint. Therefore it doesn't make a ton of sense for any shared system to store JWTs. These JWT can be big... 500 bytes, 700 bytes, more.  Apigee cannot store arbitrarily large tokens. 

I don't know how to "solve" the problem you described but I can think of 2 ways to avoid it. And they'll probably be better approaches too.

1. In the API proxy that handles the request for service, Just use VerifyJWT to verify the external JWT.  You don't need to store it as an opaque OAuth token. If you want API product-level authorization, then synchronize the client id in the external JWT to the client ID handled by Apigee, and follow the VerifyJWT with a VerifyApiKey, using the clientID (maybe subject, etc) obtained from the JWT payload as the apikey. This will work, at the cost of verifying a signature for every inbound request-for-service. Could be maybe 5-8ms in wall clock time. (Verifying an opaque token is much faster)
2. Implement a token exchange.  Configure Apigee to accept an externally-generated JWT and then generate an Apigee-generated opaque token in response.  Two benefits here: (1) the client app doesn't need to send in a signed JWT with 512 or more bytes of content for each request. Instead it can send in a 48-byte opaque string.  (2) when Apigee verifies the opaque token, it's wicked fast.  The 8ms cost of signature verification is incurred only once, when Apigee issues the initial opaque token.   To implement this kind of token exchange,  you must imeplemt a token dispensing endpoint. Within that, first use VerifyJWT to verify the externally provided JWT, and then use OAuthV2/GenerateAccessToken without the ExternalToken element. The client would then receive in response the 48-byte thing. In subsequent requests for service, the client sends that opaque token.  In the API proxy that handles the request for service, invoke OAuthV2/VerifyAccessToken (very fast)   If you like you can configure the OAuthV2/GenerateAccessToken to store parts (selected claims) of the externally-generated token as custom attributes on the Apigee-generated token. And if you like, you can synchronize the expiry of the Apigee-generated token to the expiry of the externally-generated JWT.  (This feels like an obvious thing to do, but it is not necessarily required). 

So I would say, maybe reconsider why you want to store an externally-provided JWT as an opaque token.  And maybe explore other options.

[lisabradshaw]

Dino, thank you for the thorough response.  Yes our original intent is the former and that is to store an opaque token.  We will look into your second solution/recommendation to see if that works for us.

Thank you again.