'spawn E2BIG' error with microgateway

[mohanchippada]

I'm getting bellow error with microgateway deployed on openshift. Please suggest.

Log Location: [ /opt/apigee/logs/edgemicro.log ]
cd /opt/apigee && edgemicro start -o myorg -e test -k xxxx -s yyy -r 8000 -d /opt/apigee/plugins &
current nodejs version is v8.12.0
current edgemicro version is 2.5.28
info: products download from https://myorg-test.apigee.net/edgemicro-auth/products returned 200 OK 
info: jwt_public_key download from https://myorg-test.apigee.net/edgemicro-auth/publicKey returned 200 OK 
info: config download from https://edgemicroservices.apigee.net/edgemicro/bootstrap/organization/myorg/environment/test returned 200 OK 
internal/child_process.js:313
    throw errnoException(err, 'spawn');
    ^


Error: spawn E2BIG
    at ChildProcess.spawn (internal/child_process.js:313:11)
    at exports.spawn (child_process.js:503:9)
    at exports.fork (child_process.js:104:10)
    at createWorkerProcess (internal/cluster/master.js:131:10)
    at EventEmitter.cluster.fork (internal/cluster/master.js:167:25)
    at fork (/usr/local/lib/node_modules/edgemicro/cli/lib/reload-cluster.js:70:21)
    at EventEmitter.ReloadCluster.self.run (/usr/local/lib/node_modules/edgemicro/cli/lib/reload-cluster.js:204:7)
    at edgeconfig.get (/usr/local/lib/node_modules/edgemicro/cli/lib/gateway.js:140:19)
    at /usr/local/lib/node_modules/edgemicro/node_modules/microgateway-config/lib/network.js:173:13
    at /usr/local/lib/node_modules/edgemicro/node_modules/microgateway-config/lib/network.js:402:9
    at /usr/local/lib/node_modules/edgemicro/node_modules/async/lib/async.js:726:13
    at /usr/local/lib/node_modules/edgemicro/node_modules/async/lib/async.js:52:16
    at done (/usr/local/lib/node_modules/edgemicro/node_modules/async/lib/async.js:246:17)
    at /usr/local/lib/node_modules/edgemicro/node_modules/async/lib/async.js:44:16
    at /usr/local/lib/node_modules/edgemicro/node_modules/async/lib/async.js:723:17
    at /usr/local/lib/node_modules/edgemicro/node_modules/async/lib/async.js:167:37
fs.js:675
  return binding.read(fd, buffer, offset, length, position);
                 ^


Error: EFAULT: bad address in system call argument, read
    at Object.fs.readSync (fs.js:675:18)
    at tryReadSync (fs.js:540:20)
    at Object.fs.readFileSync (fs.js:575:19)
    at Object.Module._extensions..js (module.js:663:20)
    at Module.load (module.js:566:32)
    at tryModuleLoad (module.js:506:12)
    at Function.Module._load (module.js:498:3)
    at Module.require (module.js:597:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/usr/local/lib/node_modules/edgemicro/node_modules/microgateway-config/lib/io.js:7:11)
fs.js:675
  return binding.read(fd, buffer, offset, length, position);
                 ^


Error: EFAULT: bad address in system call argument, read
    at Object.fs.readSync (fs.js:675:18)
    at tryReadSync (fs.js:540:20)
    at Object.fs.readFileSync (fs.js:575:19)
    at Object.Module._extensions..json (module.js:670:20)
    at Module.load (module.js:566:32)
    at tryModuleLoad (module.js:506:12)
    at Function.Module._load (module.js:498:3)
    at Module.require (module.js:597:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/usr/local/lib/node_modules/edgemicro/node_modules/psl/index.js:14:19)
using pluginDir
using plugin dir /opt/apigee/plugins

Thanks,

Mohan Chippada

[mohanchippada]

I found the culprit is ARG_MAX setting and is set to a very low limit.

$ getconf ARG_MAX
131072

[mohanchippada]

When restarted the pod on openshift cluster and got a different result in the logs, this time the main process was started, see the log below. Conclusion: E2BIG error doesn't seem to be systematic.

Log Location: [ /opt/apigee/logs/edgemicro.log ]
cd /opt/apigee && edgemicro start -o myorg -e test -k xxxx -s yyyy -r 8000 -d /opt/apigee/plugins &
current nodejs version is v8.12.0
current edgemicro version is 2.5.28
info: jwt_public_key download from https://myorg-test.apigee.net/edgemicro-auth/publicKey returned 200 OK 
info: products download from https://myorg-test.apigee.net/edgemicro-auth/products returned 200 OK 
info: config download from https://edgemicroservices.apigee.net/edgemicro/bootstrap/organization/myorg/environment/test returned 200 OK 
PROCESS PID : 18
TypeError: respawnIntervalManager.getIntervalForNextSpawn is not a function
    at replaceWorker (/usr/local/lib/node_modules/edgemicro/cli/lib/reload-cluster.js:99:43)
    at EventEmitter.replaceAndTerminateWorker (/usr/local/lib/node_modules/edgemicro/cli/lib/reload-cluster.js:123:5)
    at emitOne (events.js:116:13)
    at EventEmitter.emit (events.js:211:7)
    at emit (/usr/local/lib/node_modules/edgemicro/cli/lib/reload-cluster.js:59:15)
    at EventEmitter.emitWorkerDisconnect (/usr/local/lib/node_modules/edgemicro/cli/lib/reload-cluster.js:171:5)
    at emitOne (events.js:116:13)
    at EventEmitter.emit (events.js:211:7)
    at ChildProcess.worker.process.once (internal/cluster/master.js:213:13)
    at Object.onceWrapper (events.js:313:30)
    at emitNone (events.js:106:13)
    at ChildProcess.emit (events.js:208:7)
    at finish (internal/child_process.js:747:14)
    at _combinedTickCallback (internal/process/next_tick.js:132:7)
    at process._tickCallback (internal/process/next_tick.js:181:9)
Removing the socket file as part of cleanup
fs.js:675
  return binding.read(fd, buffer, offset, length, position);
                 ^


Error: EFAULT: bad address in system call argument, read
    at Object.fs.readSync (fs.js:675:18)
    at tryReadSync (fs.js:540:20)
    at Object.fs.readFileSync (fs.js:575:19)
    at Object.Module._extensions..js (module.js:663:20)
    at Module.load (module.js:566:32)
    at tryModuleLoad (module.js:506:12)
    at Function.Module._load (module.js:498:3)
    at Module.require (module.js:597:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/usr/local/lib/node_modules/edgemicro/node_modules/microgateway-config/lib/io.js:7:11)
fs.js:675
  return binding.read(fd, buffer, offset, length, position);
                 ^


Error: EFAULT: bad address in system call argument, read
    at Object.fs.readSync (fs.js:675:18)
    at tryReadSync (fs.js:540:20)
    at Object.fs.readFileSync (fs.js:575:19)
    at Object.Module._extensions..js (module.js:663:20)
    at Module.load (module.js:566:32)
    at tryModuleLoad (module.js:506:12)
    at Function.Module._load (module.js:498:3)
    at Module.require (module.js:597:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/usr/local/lib/node_modules/edgemicro/node_modules/microgateway-config/lib/io.js:7:11)
using pluginDir
using plugin dir /opt/apigee/plugins
using pluginDir
using pluginDir
using plugin dir /opt/apigee/plugins
using plugin dir /opt/apigee/plugins
using pluginDir
using plugin dir /opt/apigee/plugins
using pluginDir
using plugin dir /opt/apigee/plugins

Thanks,

Mohan Chipada

[etienned]

Hi @mohan.chippada

How did you get that resolved? I am currently struggling with the same thing.

[mohanchippada · Accepted solution]

Root cause is resources needed for microgateway is not specified in the openshift deployment config file and default ones imposed by the PaaS don't fit our needs.

Fix is to set the resources needed for the proper functioning of the container as the default ones imposed by the PaaS don't fit our needs.

resources:
  requests:
    memory: 500Mi
    cpu: 300m
  limits:
    memory: 1Gi
    cpu: 600m

Regards,

Mohan Chippada