Apigee curl from trace does not match what is actually sent

[DanS]

I am trying to connect to Jira through apigee.  During the trace I get a 405 method not allowed from the atlassian target server. So I looked for the curl of what was sent and it shows this:

curl -X POST -H 'Accept: 'application/json'' -H 'Accept-Encoding: gzip,deflate,br' -H 'Authorization: *****' -H 'Content-Type: application/json' -d '{"update":{},"fields":{"summary":"Test ticket","issuetype":{"id":"MYISSUETYPE"},"project":{"id":"MYPROJECTID"},"description":{"type":"doc","version":1,"content":[{"type":"paragraph","content":[{"text":"This is my description","type":"text"}]}]}}}' 'https://MYURLDOMAIN.atlassian.net/rest/api/3/issue'

I have redacted the private portions of the curl. 

If I copy that curl request and replace the **** with the original auth that apigee is using it works fine and creates an issue in jira.

I pointed it at the apigee mock echo endpoint and it shows extra headers in the request that are not in that curl but added by apigee on the outbound trip.  My assumption is that these additional headers are rejected by atlassian which is why it fails.

Has anyone successfully connected to jira through apigee and if so please tell me how. FWIW, I also tried doing this through a service callout and in javascript using the httpClient and nothing works. 

[dchiesa1]

AFAICT, it won't work with "just ServiceCallout" or "just httpClient" (in a JavaScript callout) or just a target, because the JIRA REST interface calls for you to use an OAuth 1.0 signature, relying on an RSA key, on every request. So you need to include some signing capability into Apigee, to produce that signature, and then attach the appropriate Authorization header for the outbound request. 

Producing the OAuth 1.0 signature that uses an RSA key is not ... rocket science, but there is some logic involved. There is no builtin policy in Apigee that generates Oauth 1.0 signatures (either RSA or HMAC). You could generate the signature this within a JavaScript callout, but I guess you'd need to rely on an external library that does RSA signatures. I guess you could also generate the signature from within a Python callout.  I don't know Python so I don't know how difficult that would be. 

If I were solving this I'd want to generate the signature within a Java callout, because it will perform better than either of those above alternatives.

[DanS]

The problem is not the auth.  The problem is headers apigee is adding.

If you look below this is what apigees echo mock service says was sent 

{"headers":{"accept":"application/json","authorization":"REDACTED","content-type":"application/json","host":"mocktarget.apigee.net","content-length":"235","x-cloud-trace-context":"363c0fe26f16d7b2b8bf74ecefcfa91a/9992175485654129216","via":"1.1 google","x-forwarded-for":"35.227.1.108, 35.227.194.212","x-forwarded-proto":"https","connection":"Keep-Alive"},"method":"POST","url":"/","body":"{\"update\":{},\"fields\":{\"summary\":\"Test ticket\",\"parent\":{\"key\":\"BP\"},\"issuetype\":{\"id\":\"10172\"},\"description\":{\"type\":\"doc\",\"version\":1,\"content\":[{\"type\":\"paragraph\",\"content\":[{\"text\":\"This is some testing text.\",\"type\":\"text\"}]}]}}}"}

 The following headers were added and they are what causes Jira to say method not allowed. Sending the curl request without these headers works fine.

"x-cloud-trace-context":"363c0fe26f16d7b2b8bf74ecefcfa91a/9992175485654129216",

"via":"1.1 google",

"x-forwarded-for":"35.227.1.108, 35.227.194.212",

"x-forwarded-proto":"https"

[dchiesa1 · Accepted solution]

I see. Well that is inconvenient, isn't it?

I can see that it would be frustrating for one or more of these additional headers to cause the connection to be rejected. We're not certain, I guess, if it is the presence of one of those specific headers, or all of them, or some subset of them, that causes the auth to fail.

Unfortunately this behavior - injecting additional headers - comes from the Google network. It's just part of what happens with outbound connections. I don't believe it can be changed. As far as I know it is not possible.

Section 3.2.1 of IETF RFC 7230 , which defines HTTP, states that

   ...recipients SHOULD ignore unrecognized header fields.
   These requirements allow HTTP's functionality to be enhanced without
   requiring prior update of deployed intermediaries.

This is really just a restatement of the Robusness Principle.  Applied to your case, we should conclude that Jira should benignly ignore the various "Extra" headers it is receiving in the request.  It should be more tolerant of these common header fields.

If the problem is specifically and only the x-forwarded-for, I think it might be possible to configure Jira to ignore that specific header, or handle it differently. Check your jira documentation. (I'm not a jira expert) I'm not sure if the same can be done with those other headers.

[DanS]

Definitely can't argue with the fact that unknown headers shouldn't break their end.  I will see if I can do anything on their end but I doubt it as we use the cloud version.  

The one thing I would say on apigee's end is that when it gives me a curl command that it says was sent it should probably be the actual curl command although I guess if that is something added at the outer edge by google probably nothing can be done about that since it doesn't know about them at the time of the trace.