ServiceCallout does not follow URL Redirects Http 302 or 301

I issue a service callout to a URL [HTTPS Endpoint] which does 3 redirects to provide the response.

If I try this URL in Postman with Follow Redirects turned on it returns a 200 OK or if I turn off the Follow redirects it comes back with a 302 and the redirect URL in the Location header.

In Apigee Edge if you issue a service callout or a javascript callout it does not come back with the content or the headers for 302 for https endpoints. I wanted to do it in Javascript where I can sequentially call the next redirect URL until I get 200 and then return the response. Seems like this is a a product limitation.

It seems to work for HTTP

0 13 1,976
13 REPLIES 13

Yes - the ServiceCallout policy does not follow 302 redirects.

If you would like to call out to a service and follow 302 redirects, you can use the JavaScript callout to do so. If you do this, Be aware of the time limit for the JS callout, by default 200ms. If you write a script to follow multiple redirects, the runtime may exceed 200ms. So you will need to adjust that. See more here.

Hi Dino. Thanks for your response. Is there a simple way to specify follow-redirects in a Javascript callout?

relatively simple?

Here is some JS that behaves this way:

var urlRe = new RegExp('(https?)://([^/]+)/?');


function resolveUrl(originalUrl, newUrl) {
  if (newUrl.startsWith("/")) {
    // relative URL
    var match = urlRe.exec(originalUrl);
    if ( ! match) { return newUrl; }
    return match[1] + '://' + match[2] + newUrl;
  }
  return newUrl;
}


function log(str) {
  var time = (new Date()).toString();
  print('[' + time.substr(11, 4) + '-' +
              time.substr(4, 3) + '-' + time.substr(8, 2) + ' ' +
              time.substr(16, 8) + '] ' + str );
}


function invokeTargetFollowRedirects(targetUrl, count, indent) {
  count = count || 0;
  indent = indent || '';
  var headers = { };
  log(indent+ "GET " + targetUrl);
  var request = new Request(targetUrl, 'GET', headers);
  var exchange = httpClient.send(request);
  exchange.waitForComplete();
  if (exchange.isSuccess()) {
    log(indent + "  ==> Success");
    var response = exchange.getResponse();
    if (response.status.code == 302 || response.status.code == 301) {
      var newLocation = response.headers['Location'] + '';
      if (newLocation != 'undefined') {
        log(indent + "  Redirect " + newLocation);
        newLocation = resolveUrl(targetUrl, newLocation);
        log(indent + "  Resolved to: " + newLocation);
        return invokeTargetFollowRedirects(newLocation, count + 1, indent + '  ');
      }
      else {
        return {error: false, redirectCount: count, status: response.status };
      }
    }
    else {
      return {error: false, redirectCount: count, status: response.status };
    }
  }
  else {
    return {error: true, redirectCount: count, status: 0 };
  }
}


var targetUrl = (new MessageTemplate(properties.target)).fill();
var response = invokeTargetFollowRedirects(targetUrl);
context.setVariable('final_response', JSON.stringify(response));

It could be simpler. A significant portion of the code there (the count, the indent, the log function) is used only for diagnostics, to show it working. If you use this in production, you will want to eliminate that stuff. and also I guess modify the function so that it returns the response of the final call. (including payload and status).

I used that code on an example target and the output was like this:

6156-invoke-target-follow-redirects.png

To use that you need a JS policy configured like this:

<Javascript name='JS-InvokeTargetAndFollow' timeLimit='3200' >
  <Properties>
    <Property name='target'>https://example.com/will-redirect</Property>
  </Properties>
  <IncludeURL>jsc://messageTemplate.js</IncludeURL>
  <ResourceURL>jsc://invokeTargetAndFollow.js</ResourceURL>
</Javascript>

The timeLimit needs to be high enough to allow for *all* the redirects.

I used that messageTemplate thing to allow the target to be specified with a variable reference, like this:

<Javascript name='JS-InvokeTargetAndFollow' timeLimit='3200' >
  <Properties>
    <Property name='target'>{my_desired_target_url}</Property>
  </Properties>
  <IncludeURL>jsc://messageTemplate.js</IncludeURL>
  <ResourceURL>jsc://invokeTargetAndFollow.js</ResourceURL>
</Javascript>

But if you don't need that, you can omit the MessageTemplate.

Attached please find a working example API Proxy.

apiproxy.zip

Thanks Dino. I will try this and update you

Hi Dino,

The first redirect works fine. The first redirect returns a 302 Found and provides a Location. The second redirect fails for some reason. When I try the URL in postman, it returns a 302 Moved Temporarily and returns a Location. The Location being returned is rather long.I am not sure if that could be an issue.

I will debug and find out.

mmmm yes, I didn't test it extensively. It could be that the resolveUrl function is naive or broken. I'll be interested to hear what you find. Check the stdout of the JS callout to get diagnostics info.

Dini, Thanks for the ServiceCallout redirect solution that you provided. It is very helpful. However I could not download the zip. if you can also update the MessageTemplate if would be great..

That's odd. I also get an error. Something went wrong between the time I posted the orginal zip and now. Try the one attached here.

apiproxy-js-follow-redirects.zip

Would I be able to use this npm module directly from edge or would I have to upload this?

var http = require('follow-redirects').http;
var https = require('follow-redirects').https;

https://github.com/olalonde/follow-redirects

I want to give it a try with nodejs.

Yes, maybe. But if you use a nodejs module, then you need to use a nodejs target, not a JS callout. They both use JavaScript but they get invoked differently within the Apigee Edge proxy.

EDIT - 2022 October 4 - nodejs targets are no longer supported in Apigee, as of several years ago.

@Dino

This is getting a bit out of hand by using JS Callout workaround kludge for solving such a frequent and simple problem. Would not it be better to implement a property that would allow ServiceCallout to make the redirect?

Yes I second that...!

Yes, I agree, Yuri. I don't know how frequent the problem is, but it seems like it might be reasonably common.

I don't have a ticket but will create one now requesting this capability for ServiceCallout. Also, independently, it seems like the httpClient within JS should be smart enough to do this on its own without any application code.

ServiceCallout: b/134933671

JS httpClient: b/134934025