Commas getting replaced by its encoded value %2C

[zaheen555]

Hello,

I have a requirement where I have to call a target endpoint with commas in its query string. The target url is set using 'target.url' in a js policy

For example. https:// example.com/v1/employee?prompt=100,200

This is getting replaced as https:// example.com/v1/employee?prompt=100%2C200

The backend server is not handling the encoded characters. Is there anyway this can be done in apigee?

I also tried decodeURI(targeturl) which also did not work.

Thanks

[dknezic · Accepted solution]

It's a reserved character and must be encoded so you will need to fix your backend.

https://datatracker.ietf.org/doc/html/rfc3986#section-2.2

[slatty]

Hi,

I am also having issues with this:

https://example.com/hostasset?fields=id,name

becomes

https://example.com/hostasset?fields=id%2Cname

According to the link you provided commas are used as delimiters and are in the same group as "=" and actually should NOT be encoded since it acts as a delimiter depending on the application.

"Percent- encoding a reserved character, or decoding a percent-encoded octet that corresponds to a reserved character, will change how the URI is interpreted by most applications. Thus, characters in the reserved set are protected from normalization and are therefore safe to be used by scheme-specific and producer-specific algorithms for delimiting data subcomponents within a URI."

[dchiesa1]

I guess the specifications are a bit loose, in order to allow different systems or schemes to treat different characters as delimiters. 

According to the spec, the value of every query parameter must be URI encoded.  (In JavaScript you can do this with the  builtin encodeURIComponent() function).  What we are discussing here is whether, during the encoding, a particular character, in this case ASCII 0x2C, is REQUIRED to be percent-encoded as part of that encoding.  But in any case, a sender (like Apigee) MAY percent-encode the character. More on that.

And whether or not Apigee encodes a comma as %2C, seems to me, to be mostly immaterial.  The spec states that each query param must be encoded; it follows then, that it is the responsibility of the receiving (server) side to DECODE each parameter. Whether or not one or another character within a parameter is encoded, should not matter so much.  Upon receiving "id%2Cname" as a query param value, the server should URI decode it to attain "id,name".  In a server implemented in JavaScript, you can do the decoding with decodeURIComponent(), but if you are using a node framework like express, express does the decoding automagically for you.  And at that point, after decoding, it is up to the server to decide what to do with the decoded value. 

So if your server is choking on the "id%2Cname" that Apigee is passing it, then I think the problem is with your server. It should decode properly. 

The only reasons I can think of, where this would be a problem is 1. if you have a very old server, for which you don't have the source code, in which case, I'm very sorry for your loss, but it's not a problem with Apigee.   Or 2. you are using Apigee to do something strange or impolite, in which case, please don't ask people here to help you with that. 

[slatty]

Hi Dino,

there is a third option which is my situation. My team and I are responsible for integrations with external applications so we have no control over the “backend” with this particular integration. This is the first we’ve encountered this issue and I was trying to achieve one of two things.

A. Figure out which application SHOULD be doing what. However, as you mentioned, the spec is loose so the responsibility could go either way.

B. Figure out if there is a workaround I can do in Apigee to accommodate this external application

I am attempting to contact The external application in question to see if this issue can be changed on their end as well but I’m not hopeful.

Thank you for your response.