Re: Websockets disconnecting when using OAuth2 Ver...

JacobHume · 02-09-2024 11:39 AM

Hi,

Running into an odd issue when we try to send messages via Websocket on a proxy which uses the OAuth2 policy (using the VerifyJWTAccessToken operation). After sending one message (sometimes a few can go through) the connection disconnects. We do not have this issue with API key verification or unauthenticated requests.

We suspected that something might be wrong with our configuration or backends, so we tried to use as simple of things as possible. We used https://github.com/DinoChiesa/devjam3-20170405/tree/master/Resources/oauth2-cc to generate tokens. For the verification step, we set up a simple proxy which just sets the secret key, extracts the token from a query parameter, and then runs OAuth2 VerifyJWTAccessToken, and for the server we used the test websocket server seen here: https://github.com/DinoChiesa/Apigee-Websocket-Example . With this stripped down example, we still will get disconnections fairly consistently when sending messages via websocket.

Thanks in advance!

EDIT: just tried with opaque tokens instead of JWT, and it worked there. We'd really like to support JWT from our API so I'm wondering if there's any configuration I need to change to get that working?

dchiesa1

@JacobHume wrote:

just tried with opaque tokens instead of JWT, and it worked there. We'd really like to support JWT from our API so I'm wondering if there's any configuration I need to change to get that working?

Fascinating! Are you saying that your websocket proxy works with OAuth2 VerifyAccessToken, and with VerifyAPIKey, but not with OAuth2 VerifyJWTAccessToken ?

Is it possible that the JWT Access Token is expired, or otherwise invalid?

@JacobHume wrote:

After sending one message (sometimes a few can go through) the connection disconnects.

This failure mode is very curious. So the VerifyJWTAccessToken succeeds, is that right? And then... after a one successful ws message, or maybe after a few messages, the connection disconnects. And at the time of the disconnect, you're not executing the VerifyJWTAccessToken, is that right? It's just a websocket, so... no policies are being executed. Am I understanding correctly?

This sounds very curious - you may need to contact Apigee support, to get this diagnosed.

Can you collect a trace / debugsession? And does it show anything beyond "disconnected" ? Any clues there?

I imagine looking in the logfile of the message processors would help, but I don't think you or I will have access to that, for Apigee X. You would need Apigee support to look there.

JacobHume

@dchiesa1 wrote:
Fascinating! Are you saying that your websocket proxy works with OAuth2 VerifyAccessToken, and with VerifyAPIKey, but not with OAuth2 VerifyJWTAccessToken ?

Yes, exactly.

@dchiesa1 wrote:
Is it possible that the JWT Access Token is expired, or otherwise invalid?

I suppose it is possible, but the same tokens work for HTTP connections.

@dchiesa1 wrote:
This failure mode is very curious. So the VerifyJWTAccessToken succeeds, is that right? And then... after a one successful ws message, or maybe after a few messages, the connection disconnects. And at the time of the disconnect, you're not executing the VerifyJWTAccessToken, is that right? It's just a websocket, so... no policies are being executed. Am I understanding correctly?
This sounds very curious - you may need to contact Apigee support, to get this diagnosed.

Yes, you have this correct. I'm not really sure how Apigee is getting involved in here once we've upgraded to Websockets, but it must be in some way, because it works if we use other auth mechanisms, just not VerifyJWTAccessToken. Most often we only send one message, but trying to send 30 and waiting for their responses has always triggered this issue.

@dchiesa1 wrote:
Can you collect a trace / debugsession? And does it show anything beyond "disconnected" ? Any clues there?

Unfortunately not much interesting there, you'd just see whatever your client will show you when the websocket is terminated without the proper websocket status code.

@dchiesa1 wrote:
I imagine looking in the logfile of the message processors would help, but I don't think you or I will have access to that, for Apigee X. You would need Apigee support to look there.

This seems like the best way forward, I agree; I'm not really sure how to get Apigee support (outside of this forum, which seems to be different). Happy to take the issue towherever makes sense.

dchiesa1

I'm not really sure how to get Apigee support (outside of this forum, which seems to be different).

You have a support contract? Your organization has the url and a signin to the Apigee support system? Or since you are using Apigee X, then you can submit a ticket via the cloud console.

Just FYI, This forum is not "official support" and is not staffed or monitored by Apigee support technicians.

Websockets disconnecting when using OAuth2 VerifyJWTAccessToken