Parallel calls in Apigee Hybrid Gateway

[ven]

Hi Folks,

I have a question about right way to do parallel calls in Apigee Hybrid Gateway. Would you please suggest what solution is better.

We have a requirement to make parallel calls to back end services wait for response and do some pre-processing inside Pre Proxy hook, before calling actual proxy. Service Callout cannot be used as it does not provide us a way to make multiple parallel calls and wait for the response. Java script is one way where we can do it, but Apigee documentation says that waitForComplete is anti pattern and also there is a chance that it blocks threads. Our use case is that there will be few hundreds of request every minute for which we need to run 2 to 4 parallel calls in pre proxy hook before letting the request proceed to actual proxy flow, so using the right component to ensure it does not block other requests/ or does not exhaust threads is crucial. Can you recommend a way to handle this or if using waitForComplete is ok in this scenario? We are using Apigee Hybrid 1.5.3 version

[dchiesa1 · Accepted solution]

Can you recommend a way to handle this or if using waitForComplete is ok in this scenario? We are using Apigee Hybrid 1.5.3 version

Do not use waitForComplete.  It is an anti-pattern in Apigee. ps: if you find any examples that continue to show this, please flag them for me. 

The correct approach is to use a callback for the httpClient.send method. I think this is documented. Simple example:

var prefix = 'jsvariable';
function cb(response, error) {
    if (response) {
      context.setVariable(prefix +'.response-status', response.status);
    }
    else {
      context.setVariable(prefix +'.error', 'Whoops: ' + error);
    }
}
var url = properties.targetBase + '/segment?q1=0.2&q2=whatever';
var headers = { 'ID-Index' : id, 'Other-Header' : 'other-value' };
var req = new Request(url, 'GET', headers);
httpClient.send(req, cb);

That sends a single http request, and handles the response in a callback.

Now, suppose you want to send multiple HTTP requests, and the number of HTTP requests is not known until runtime. In that case you need to invoke all of the calls, each possibly with their own callback. You can do that with JS closures.

Let's say you have an array of data items, of indeterminate length, and you want to make a set of outbound calls that execute in parallel, one for each item in the array. To do that, you could do something like this:

var start = Date.now();

function indexedOnComplete(id, numberOfCalls, response, error) {
  var prefix = 'jscall.'+ id.toFixed(0);
  context.setVariable(prefix +'.completed-at', String(Date.now() - start));
  if (response) {
    context.setVariable(prefix +'.response-status', response.status);
  }
  else {
    context.setVariable(prefix +'.error', 'Whoops: ' + error);
  }
  if (numberOfCalls - 1 == id) {
    context.setVariable('jscall.all-done', String(Date.now() - start));
  }
}

function invokeOne(id, dataItem, cb) {
  var url = properties.target + '/pathsegment?item=' + dataItem ;
  var headers = { 'ID-Index' : id };
  var req = new Request(url, 'GET', headers);
  httpClient.send(req, cb);
}

// ====================================================================

// get data items from... somewhere
const dataItems = [ ???, ???, ... ] ;

var numberOfCalls = dataItems.length;
context.setVariable('number-of-calls', numberOfCalls.toFixed(0));

dataItems.forEach(function(current, index) {
  context.setVariable('jscall.' + index.toFixed(0) + '.dataItem', current.toFixed(2));
  var cb = function(response, error) {
        return indexedOnComplete(index, numberOfCalls, response, error);
      };
  return invokeOne(index, current, cb);
});

The JS will finish after all the calls finish.

If you have long running HTTP calls, you may need to set the timeLimit attribute on the JavaScript policy XML to accommodate that:

<Javascript name='JS-Invoke-Calls' timeLimit='22000' >
  <Properties>
    <Property name='target'>https://my-target.com</Property>
  </Properties>
  <ResourceURL>jsc://invoke-calls-in-parallel.js</ResourceURL>
</Javascript>

[ven]

Thank you @dchiesa1. While I have read about call backs in multiple posts, I never tried them till date as they were always combined with WaitforComplete. I tried it now and it appears to be working for my use case. Thanks for sharing this. I will do further development based on this and let you know in case I come across any issues/limitations. It will take a week before I can come back with confirmation. I will accept this as a solution in case I dont run into any limitations.

[dchiesa1]

ok great to hear.  I'll stay tuned.