How do I run multiple Microgateway processes on the same VM?

The use case is that I have a single VM and I need to run multiple Microgateway processes on that VM for redundancy. Yes, I know we could create additional VMs, but at this time that is not possible.

If we only run a single instance of Microgateway, then if that process crashes or if we have to restart it due to a configuration change, then all requests to the gateway will fail. How do I run multiple Microgateway processes on a single VM?

This post will describe some of the available options.

0 5 556
5 REPLIES 5

Former Community Member
Not applicable

The edgemicro start parameter (p) is the number of processes to start, defaults to # of cores. Internally, it uses the node cluster module. The master process should restart a child process if it dies.

Thanks @Srinandan Sridhar. The only issue with this approach is that if I have to restart the MG then all the child processes are terminated. So if I only have one MG parent process running on a single VM, then all API requests to that gateway will fail during the restart or reconfiguration.

I will start with the cluster option. You have the ability to start Microgateway in cluster mode and an example of the start command is shown below.

./edgemicro start -o {org} -e {env} -k {key} -s {secret} --cluster --processes 2

http://docs.apigee.com/microgateway/latest/edge-microgateway-cli#startcommand

If you start Microgateway in cluster mode then it will create child processes. In the example below I started it with 2 processes. So you can see that the if you run the ps command and grep for edgemicro you will see that there are 3 processes running - the parent and two children. If you run it in cluster mode then this will not solve the problem of configuration restarts. So if you restart Microgateway then the child processes will also be destroyed and your API gateway will be down.

ps -ef | grep edgemicro

501 19809 69258   0 11:42AM ttys000    0:00.00 grep edgemicro

  501 19777 69569   0 11:39AM ttys001    0:00.64 node /usr/local/bin/edgemicro start -o org -e test -k key -s secret --cluster --processes 2

  501 19778 19777   0 11:39AM ttys001    0:00.61 /usr/local/bin/node /usr/local/bin/edgemicro start -o org -e test -k key -s secret --cluster --processes 2

  501 19779 19777   0 11:39AM ttys001    0:00.59 /usr/local/bin/node /usr/local/bin/edgemicro start -o org -e test -k key -s secret --cluster --processes 2

Option 1:

Run Microgateway within a Docker container and use Kubernetes or Mesos Marathon to manage the container instances. There is a lot to be said about this option and it should be discussed in a separate post.

Option 2:

Start Microgateway on two separate ports on the same VM. How? When you configure Microgateway it creates a .edgemicro folder in the user's home directory and a default.yaml file. If you open that file you will see a port option with the default value set to 8000 (see partial export below).

edgemicro:
  port: 8000
  max_connections: 1000
  max_connections_hard: 5000
  restart_sleep: 500
  restart_max: 50
  max_times: 300
  logging:
    level: error
    dir: /var/tmp
    stats_log_interval: 60
    rotate_interval: 24
  plugins:
    sequence:
      - oauth

Start the gateway with the following command from the [microgateway install folder]/cli directory.

./edgemicro start -o org -e test -k key -s secret

A partial output from the start command is shown below and notice that it starts on port 8000.

installed plugin from oauth

3ae8a050-58ce-11e6-bed1-514db3957584 edge micro listening on port 8000

edge micro started

edgemicro started successfully.

When you start Microgateway it creates a new cached config file org-env-config.yaml in the ~/.edgemicro folder.

Open the file and change the port to 8001, then save it.

Open a new terminal window and issue the start command again. You should see the following output.

installed plugin from oauth

3ae8a050-58ce-11e6-bed1-514db3957584 edge micro listening on port 8001

edge micro started

edgemicro started successfully.

Open another terminal window and run the following linux command:

netstat -an | grep LISTEN

You should see 8000 and 8001 listening.

tcp46      0      0  *.8001                 *.*                    LISTEN     

tcp46      0      0  *.8000                 *.*                    LISTEN

I admit this is a clunky process to change the config file, but at this point I don't see any other available alternatives. With this approach you now have the option to stop one process and restart it and still have another process running to service API requests.

Lastly, if you have two Microgateway processes running on the same VM, then how do I load balance it? You could use Nginx running on the same VM so all requests will hit Nginx first and it would send the requests round robin to each process. View this blog post to see how to configure Nginx and Microgateway. This deployment model is one of several; if you review the overview page and then search for "With load balancer", you will see the architecture digram below.

3283-screen-shot-2016-08-02-at-103109-pm.png

What we need is a --port option added to the command line which will override the port in the config file. This way we don't have to change the org-env-config.yaml file.

Any takers?

@Srinandan Sridhar what do you think?

still need to figure out a good process management system for this...

a dev lab w/ people running all of these commands could make sense - but w/o the ability to strictly start from a config file w/ some process management tool this becomes a bit of a disaster to productionize.

@Benjamin Goldman

I agree. I'm working on a POC with Chef/Vagrant to provision VMs, install Microgateway with its dependencies and start it. I'll will create a new post with that info when it is complete.