How to override default URL on Edge Micro Gateway?

Not applicable

I have one MicroGateway aware proxy defined and downloaded in to Edge MicroGateway with these defaults:

proxies:
  - max_connections: -1
    name: edgemicro_weather
    revision: '1'
    proxy_name: default
    base_path: /weather
    target_name: default
    url: 'https://weather.yahooapis.com'

Let's say for some business reason based on payload condition I want to override the target URL from https://weather.yahooapis.com to https://legacy.weather.yahooapis.com.

How do I dynamically create backend URL which overrides the default downloaded one?

1 10 1,594
10 REPLIES 10

Not applicable

I've done some digging into this myself. I found that the target URL lives in `res.proxy.parsedUrl`, and it can be manipulated prior to the request being made. This URL is used as a base, to which the request path (if any, outside of the request basepath) and request querystring are appended. The code used to build the final target URL looks like this (it lives in /gateway/lib/main.js, at least in version 1.1.1):

var target_path =
    proxy.parsedUrl.pathname +
    reqUrl.pathname.substr(proxy.basePathLength, reqUrl.pathname.length); // strip leading base_path
if (reqUrl.search) target_path += reqUrl.search;
target_path = target_path.replace(double_slash_regex, '/');

var treqOptions = {
    hostname: proxy.parsedUrl.hostname,
    port: proxy.parsedUrl.port,
    path: target_path,
    method: req.method,
    headers: target_headers, // pass through the modified headers
    agent: proxy.agent
};

So, in order to manipulate the target url, you might do something like this from within a plugin (keep in mind that this needs to run in the request flow):

res.proxy.parsedUrl.hostname = "dynamic-hostname.com";
res.proxy.parsedUrl.pathname = "/dynamic/path";
reqUrl.pathname = res.proxy.base_path + '/more/dynamic/path'; // It is important that `proxy.base_path` remains at the beginning of `reqUrl.pathname` 
reqUrl.search = '?dynamic=querystring';

Keep in mind that, as far as I can tell, none of this is explicitly documented. It could be subject to change in future versions, and there may even be more straightforward ways of achieving this.

In fact, I really hope someone can correct me and tell me a better way of achieving this, because the method I outlined above is very ugly. Why should you need to manipulate both the request and response objects just to have full dynamic control of the target URL? Very ugly indeed.

Agreed @epackwood ugly... also seems that you you may want to abstract away some of this in-case the core API changes. I'd hate to take this approach with a bunch of APIs then have the 'res.proxy.parsedUrl' go away and starts breaking stuff.

@epackwood Thanks! However, the suggested solution appears to work only with onrequest event in the request flow. How can we get it to work with a onend_request event?

I'm not sure. I have run into this sort of issue with `onend_request` as well. I found that at this point, it seems to be too late to alter the target request. Since the documentation suggests that this is the point when all data has been received from the client, it would definitely be advantageous to be able to alter the request there. E.g. by taking a property from a JSON payload, and moving it into a header, or determining aspects of the target URL based on POSTed form data.

prabhat
Participant V

What you have suggested is indeed a custom plugin. Since you have it figured out, how about you pushing your entire plugin code somewhere ( on github may be) so that others can benefit.

I think pretty much everything would work but Analytics that you would see on Edge might refer to the request which Edgemicro received and will not show dynamically added paths.

Speaking of GitHub, I think we can look into how to share this with the larger community. But if we show you ours... we'd want to see yours... Is there a GitHub repo for Edge Micro we can monitor, to create issues and possibly contribute to?

Yes, that's a good point and is a work in progress. BTW, Edgemicro being a node.js based impl, you already have access to all of its source including plugins. 🙂

Right. But would be awesome to be able to fork, create issues and create pull requests.

+1 and totally agreed.

@prabhat Good point about the analytics. I'm also curious, is this a use-case that's not intended for the microgateway? I'm always a little hesitant to rely on undocumented implementation details like this, as (as @Kristopher Kleva mentioned) you never know when they might change out from under you.