I'm hoping someone here has seen or can otherwise help me with this problem. When we try to start a version of the microgateway we get no functionality due to the problem "no such file or directory, access '/home/node/edgemicro.sock'" (the stack trace is below).
My understanding is that this directory is really only used in clustered container setups so I'm first wondering if we even need this (now) and if there's a way to not require it.
My understanding is that this directory is really only used in clustered container setups so I'm first wondering if we even need this (now) and if there's a way to not require it.
Note that I don't know if the message about not having access to apigee-access is a root cause, nor the messages about unset environment variables EDGEMICRO_LOCAL and EDGEMICRO_LOCAL_PROXY nor how or where they should be set.
Things I've tried:
1) Reinstallations of the docker image and
2) Restarting Docker (then restarting container)
3) Creating the 'node' user on the host and chown-ing the directory to that user
4) Starting container as root and another user in the docker group
4a) In all cases above the directory edgemicro.sock is not created
5) Creating an external docker volume mapping (outside->inside) /home/user/edgemicro.sock -> /home/node/edgemicro.sock The directory edgemicro.sock is always created but the app in the contaner fails to start and the 'docker logs' command for the container shows messages like:
Edgemicro seems to be already running.
If the server is not running, it might because of incorrect shutdown of the prevous start.
Try removing /home/node/edgemicro.sock and start again
6) Reinstallations of the docker image and container re-running 🙂
7) Finding and grokking as much salient documentation as seems related...and this forum
Here are some environment facts:
Microgateway version: latest
Docker version 1.13.1, build 07f3374/1.13.1
CentOS Linux release 7.5.1804 (Core), selinux running in Permissive mode
Note that I know just enough Docker to be dangerous and am pretty new to the microgateway too so I have no ego on this matter.
======================== Output from 'docker logs <microgateway container name>' command: ===================
Tue, 05 Feb 2019 15:32:34 GMT apigee Operating without access to apigee-access 2019-02-05T15:32:34.406Z start EDGEMICRO_LOCAL: undefined 2019-02-05T15:32:34.407Z start EDGEMICRO_LOCAL_PROXY: undefined 2019-02-05T15:32:34.408Z microgateway Error: ENOENT: no such file or directory, access '/home/node/edgemicro.sock' at Object.fs.accessSync (fs.js:301:11) at Gateway.start (/home/node/edgemicro-oneapi/node_modules/edgemicro/cli/lib/gateway.js:28:12) at Command.commander.command.option.option.option.option.option.option.option.option.option.option.option.option.option.description.action (/home/node/edgemicro-oneapi/node_modules/edgemicro/cli/cmd.js:243:21) at Command.listener (/home/node/edgemicro-oneapi/node_modules/commander/index.js:315:8) at emitTwo (events.js:126:13) at Command.emit (events.js:214:7) at Command.parseArgs (/home/node/edgemicro-oneapi/node_modules/commander/index.js:654:12) at Command.parse (/home/node/edgemicro-oneapi/node_modules/commander/index.js:474:21) at setup (/home/node/edgemicro-oneapi/node_modules/edgemicro/cli/cmd.js:564:15) at Object.<anonymous> (/home/node/edgemicro-oneapi/node_modules/edgemicro/cli/edgemicro:49:21) at Module._compile (module.js:653:30) at Object.Module._extensions..js (module.js:664:10) at Module.load (module.js:566:32) at tryModuleLoad (module.js:506:12) at Function.Module._load (module.js:498:3) at Function.Module.runMain (module.js:694:10)
Bump this because for a bit more attention
What version of MG/Docker container are you trying with? Are you using the one shipped by Apigee or using your own?
This is repackaged container from a third party. However, I'm pretty certain that they haven't modified Apigee's implementation to the point where it would affect the edgemicro.sock implementation. Thus I posted the question here (and also soon with the third party) with the thought that the solution might be provided by, and at the same time help, the community.
Can you please try with the apigee build
docker pull gcr.io/apigee-microgateway/edgemicro:latest
Thanks for the feedback @srinandans . So, the above error message turned out to be a red herring.
Actually, the mgw was taking a long time to attempt connection and timeout from the Apigee website for bootstrapping, etc. I didn't see this doing a "docker logs" on the mgw because it was taking a while to generate the timeouts. Instead I fell for the misleading error message above which is thrown almost immediately after (re)start.
The timeouts were cured by 1) specifying DNS servers in docker host /etc/docker/daemon.json ( "dns": ["my.dns.ip.1", "my.dns.ip.2", "8.8.8.8" ] ) and rebooting the host because it had messed up the iptables routing...which is a little bit disconcerting.
Hi,
I've just tried this one modifying the daemon.json from .docker also the Docker Desktop Network Settings, enabling the fixed DNS but the issue still there while I try to run my docker image with edgemicro start.
Any other clues for this issue ?
Make sure that /home/node exists and is writeable by the ID running edgemicro.
edgemicro.sock is one of , if not *the* first file edgemicro needs to write, so a missing write permission would cause your error.
Definitely do NOT mount edgemicro.sock as a directory. Its a special FIFO socket that should remain local to your container.
Hi Eric,
Thanks for your prompt reply. I have been checking the container while running and it just stops with a simple edgemicro init. And sadly, yes as you mentioned, its related to permissions:
... current nodejs version is v8.15.1 current edgemicro version is 2.5.30 apigee Operating without access to apigee-access +0ms failed to init configpath file Error: EACCES: permission denied, open '/home/node/.edgemicro/default.yaml' ...
I am trying to run it with Docker Desktop on a Windows Enterprise 10 computer (using Hyper-V). I have been reading and following thousands of post about this but none of them useful.
Can you please aid me and add suggestions or advices (not possible to switch to a Unix operative system) to allow a docker edgemicro image to write on the shared unit drive ?
Any help will be fully appreciated.
PD: I can run a touch file using sh on the container after failing, but when it comes to run an edgemicro init inside it, says the same to me:
failed to init configpath file Error: EACCES: permission denied, open '/home/node/.edgemicro/default.yaml'
In a Windows Container you should use Windows folder structures, I have built my own container for a Windows Environment(Windows server 2016), it is working, but still not production ready. It uses Windowservercore image, which is a bit big, so still looking at changing to nanoserver.
Hi Etienne,
Thanks for the comment and idea, but the situation here is that I am "consuming" this image and the container running as I do not "own" the code I can not modify it. I am still checking for a solution there...
Hi Alan,
I ran into the same issue, if you are trying to use any of the images available on-line in a Windows Container it will always fail. If you can't use Hyper-V to switch to a Linux container then you will have to rebuild. Mine is working nicely.
Hi Etienne,
Thanks for the feedback, actually I am using Hyper-V with Linux Containers on Windows 10 Enterprise. I can pull the image from the registry with its dependencies and so on (node:alpine, etc), but each time I try to run it fails at "edgemicro start OR init" because it can not write inside the container living on the VM, it seems that writing permissions from the shared C drive are missed
Sorry for not understanding. So if you want to run this container in a system that is not granting you with Super User privileges then it will fail (Like for example: Openshift). If it is Docker running on your own machine and the default registry location is not allowing you to write `then change the MACHINE_STORAGE_PATH of your Docker installation to where you can write, to see your installation directory type
docker info
and look for the Docker Root Dir (This is where all containers will be created)
Hi Etienne,
Once again thanks for your assistance I really appreciate it. So I tried with the environment variable (on env.list file), but same issue even shared the host directory on the network and changing its permissions. These are those values, and yes my user is Super User on this Windows machine of my own (I created the test directory without special characters for testing purposes)
PS C:\Users\test> docker info Containers: 1 Running: 0 Paused: 0 Stopped: 1 Images: 6 Server Version: 18.09.3 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true Logging Driver: json-file Cgroup Driver: cgroupfs Plugins: Volume: local Network: bridge host macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: runc Default Runtime: runc Init Binary: docker-init containerd version: e6b3f5632f50dbc4e9cb6288d911bf4f5e95b18e runc version: 6635b4f0c6af3810594d2770f662f34ddc15b40d init version: fec3683 Security Options: seccomp Profile: default Kernel Version: 4.14.104-boot2docker Operating System: Boot2Docker 18.09.3 (TCL 8.2.1) OSType: linux Architecture: x86_64 CPUs: 1 Total Memory: 913.3MiB Name: default ID: 2X22:E6JC:R4PX:XM37:U3R4:R5WQ:DVBB:RYJ6:VDUF:76AX:5HD2:FZCN Docker Root Dir: /mnt/sda1/var/lib/docker Debug Mode (client): false Debug Mode (server): false Registry: https://index.docker.io/v1/ Labels: provider=hyperv Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false Product License: Community Engin
The --env-file is:
MACHINE_STORAGE_PATH=/c/Users/test COMPOSE_CONVERT_WINDOWS_PATHS=1 DEBUG=* EDGEMICRO_ORG=internal EDGEMICRO_ENV=development EDGEMICRO_KEY=xxx EDGEMICRO_SECRET=xxx
So I assume you are running this on a Hyper-V Manager VM or VirtualBox VM. Can you create any container, like when running the following:
docker run hello-world
I know there is an issue with Windows10 and Hyper-V with file-sharing and volume mounting. Is there any reason why you can't install Docker directly on your laptop?
Morgen Etienne,
Hope your day started well. Thanks again, so in reply to this one, yes. I run this with Hyper-V using the MobyLinuxVM (Internal: DockerNAT), the Virtual Switch (External: Ethernet card) and a "default" docker machine correctly running (ACTIVE: *) on this Windows 10 lap.
I can run all those samples perfectly, but each time comes to write to my local shared disk (C) it always says permission denied (all /home/node directories are created ok)
c:\test>docker run --privileged --env-file env.list --rm -p 8000:8080 -it -v //c/test/:/home/node/.edgemicro -w /home/node --name mgw-all registry.oneapi-project.net/microgateway/microgateway:latest edgemicro init current nodejs version is v8.15.1 current edgemicro version is 2.5.30 apigee Operating without access to apigee-access +0ms failed to init configpath file Error: EACCES: permission denied, open '/home/node/.edgemicro/default.yaml' config initialized to /home/node/.edgemicro/default.yaml
i.e. hello-world: OK
c:\test>docker run hello-world Hello from Docker! This message shows that your installation appears to be working correctly. To generate this message, Docker took the following steps: 1. The Docker client contacted the Docker daemon. 2. The Docker daemon pulled the "hello-world" image from the Docker Hub. (amd64) 3. The Docker daemon created a new container from that image which runs the executable that produces the output you are currently reading. 4. The Docker daemon streamed that output to the Docker client, which sent it to your terminal. To try something more ambitious, you can run an Ubuntu container with: $ docker run -it ubuntu bash Share images, automate workflows, and more with a free Docker ID: https://hub.docker.com/ For more examples and ideas, visit: https://docs.docker.com/get-started/
i.e. docker.sock: OK
c:\test>docker run --rm -v //var/run:/data alpine ls //data acpid.pid acpid.socket crond.pid docker docker.pid docker.sock haveged.pid ntpd.pid sshd.pid udhcpc.eth0.pid utmp vmblock-fuse
I will be more than happy if there is a hint that can pop-out this one, I am still looking for other ways to solve this (I will be double checking again my Hyper-V, File-Sharing and Volume Mounting).
Tried with the current registry from ApiGee, same issue:
c:\test>docker run --privileged --env-file env.list --rm -p 8000:8080 -it -v //c/test/:/home/node/.edgemicro -w /home/node --name mgw-all gcr.io/apigee-microgateway/edgemicro:latest edgemicro init Log Location: [ /opt/apigee/logs/edgemicro.log ] cd /opt/apigee && edgemicro start -o internal -e development -k xxx -s xxx -r 8000 -d /opt/apigee/plugins & current nodejs version is v8.12.0 current edgemicro version is 2.5.30 Fri, 08 Mar 2019 11:59:45 GMT apigee Operating without access to apigee-access 2019-03-08T11:59:45.831Z start EDGEMICRO_LOCAL: undefined 2019-03-08T11:59:45.831Z start EDGEMICRO_LOCAL_PROXY: undefined Fri, 08 Mar 2019 11:59:45 GMT portastic:test Trying to test port 8000 2019-03-08T11:59:45.837Z microgateway Error: ENOENT: no such file or directory, access '/opt/apigee/edgemicro.sock' at Object.fs.accessSync (fs.js:301:11)
[ For the record ] Went back to the basics and the ApiGee Edge Microgateway ran on Windows 10 + docker + all the other s---tuff. But now another issue, the .yaml file was wrongly generated:
docker run --rm -P -p 8000:8000 -d --name edgemicro -v /var/tmp:/opt/apigee/logs -e EDGEMICRO_PROCESSES=1 -e EDGEMICRO_ORG=int -e EDGEMICRO_ENV=dev -e EDGEMICRO_KEY=xxx -e EDGEMICRO_SECRET=xxx -e EDGEMICRO_CONFIG=$EDGEMICRO_CONFIG -e SERVICE_NAME=edgemicro --security-opt=no-new-privileges --cap-drop=ALL gcr.io/apigee-microgateway/edgemicro:latest
Container running, but after digging into the logs I found the next:
~/logs $ more edgemicro.log base64: truncated base64 input cd /opt/apigee && edgemicro start -o int -e dev -k xxx -s xxx -p 1 -d /opt/apigee/plugins & current nodejs version is v8.12.0 current edgemicro version is 2.5.30 error reading config from /opt/apigee/.edgemicro/int-dev-config.yaml { YAMLException: the stream contains non-printable characters at line 2, column 1: ^ at generateError (/usr/local/lib/node_modules/edgemicro/node_modules/js-yaml/lib/js-yaml/loader.js:165:10) at throwError (/usr/local/lib/node_modules/edgemicro/node_modules/js-yaml/lib/js-yaml/loader.js:171:9) at captureSegment (/usr/local/lib/node_modules/edgemicro/node_modules/js-yaml/lib/js-yaml/loader.js:259:7) at readPlainScalar (/usr/local/lib/node_modules/edgemicro/node_modules/js-yaml/lib/js-yaml/loader.js:506:3) at composeNode (/usr/local/lib/node_modules/edgemicro/node_modules/js-yaml/lib/js-yaml/loader.js:1348:20) at readBlockMapping (/usr/local/lib/node_modules/edgemicro/node_modules/js-yaml/lib/js-yaml/loader.js:1009:16) at composeNode (/usr/local/lib/node_modules/edgemicro/node_modules/js-yaml/lib/js-yaml/loader.js:1332:12) at readDocument (/usr/local/lib/node_modules/edgemicro/node_modules/js-yaml/lib/js-yaml/loader.js:1492:3) at loadDocuments (/usr/local/lib/node_modules/edgemicro/node_modules/js-yaml/lib/js-yaml/loader.js:1548:5) at load (/usr/local/lib/node_modules/edgemicro/node_modules/js-yaml/lib/js-yaml/loader.js:1569:19) at Object.safeLoad (/usr/local/lib/node_modules/edgemicro/node_modules/js-yaml/lib/js-yaml/loader.js:1591:10) at IO.loadSync (/usr/local/lib/node_modules/edgemicro/node_modules/microgateway-config/lib/io.js:87:20) at Loader.get (/usr/local/lib/node_modules/edgemicro/node_modules/microgateway-config/lib/network.js:95:35) at Object.get (/usr/local/lib/node_modules/edgemicro/node_modules/microgateway-config/index.js:16:30) at Gateway.start (/usr/local/lib/node_modules/edgemicro/cli/lib/gateway.js:60:16) at Command.commander.command.option.option.option.option.option.option.option.option.option.option.option.option.option.description.action (/usr/local/lib/node_modules/edgemicro/cli/cmd.js:243:21) name: 'YAMLException', reason: 'the stream contains non-printable characters', mark: Mark { name: null, buffer: '\u00101�0��8#�\n\u0000', position: 10, line: 1, column: 0 }, message: 'the stream contains non-printable characters at line 2, column 1:\n \n ^' } /usr/local/lib/node_modules/edgemicro/node_modules/microgateway-config/lib/io.js:91 throw new Error(err); ^ Error: YAMLException: the stream contains non-printable characters at line 2, column 1: ^ at IO.loadSync (/usr/local/lib/node_modules/edgemicro/node_modules/microgateway-config/lib/io.js:91:11) at Loader.get (/usr/local/lib/node_modules/edgemicro/node_modules/microgateway-config/lib/network.js:95:35) at Object.get (/usr/local/lib/node_modules/edgemicro/node_modules/microgateway-config/index.js:16:30) at Gateway.start (/usr/local/lib/node_modules/edgemicro/cli/lib/gateway.js:60:16) at Command.commander.command.option.option.option.option.option.option.option.option.option.option.option.option.option.description.action (/usr/local/lib/node_modules/edgemicro/cli/cmd.js:243:21) at Command.listener (/usr/local/lib/node_modules/edgemicro/node_modules/commander/index.js:315:8) at emitTwo (events.js:126:13) at Command.emit (events.js:214:7) at Command.parseArgs (/usr/local/lib/node_modules/edgemicro/node_modules/commander/index.js:654:12) at Command.parse (/usr/local/lib/node_modules/edgemicro/node_modules/commander/index.js:474:21) at setup (/usr/local/lib/node_modules/edgemicro/cli/cmd.js:564:15) at Object.<anonymous> (/usr/local/lib/node_modules/edgemicro/cli/edgemicro:49:21) at Module._compile (module.js:653:30) at Object.Module._extensions..js (module.js:664:10) at Module.load (module.js:566:32) at tryModuleLoad (module.js:506:12)
Any ideas/help ?, thanks again in advance
Hi Alan,
When you say going back to basics with Docker + Windows10, are you no longer running on Hyper-V Manager.
The best is to start isolating your issues
So just run the basics, start with this, thus you are removing volume mounting and security options. This should work, then you can start adding the additional parameters.
docker run -P -p 8000:8000 -d --name edgemicro -e EDGEMICRO_DOCKER=1 -e EDGEMICRO_ORG=xxx -e EDGEMICRO_ENV=xxx -e EDGEMICRO_KEY=xxx -e EDGEMICRO_SECRET=xxx -e EDGEMICRO_CONFIG=xxx -e SERVICE_NAME=edgemicro gcr.io/apigee-microgateway/edgemicro:2.5.30
How did you generate your config-file: in Windows10 you can use the following
certutil -f -encode xxx-xxx-config.yaml encodeconf.txt
or to test your conf file in Windows10:
echo %EDGEMICRO_CONFIG% >> %crt2.tmp certutil -decode -f crt2.tmp C:\Users\ContainerAdministrator\.edgemicro\%EDGEMICRO_ORG%-%EDGEMICRO_ENV%-config.yaml
Just have a look that it then with notepad++ to see if it is valid.
Hi Etienne,
Hope your week started well and everything is going good over there.
Once again thanks for your support, so I tried your last command and it runs smoothly, so the default.yaml is created with default values, but when I check the content of the xxx-xxx-config.yaml is totally empty. So I think the problem is more related to the local mounting point on the Windows system (my laptop) and the target mounting point (the one on the running container).
I know that is a mess and quite frustrating to run docker on Windows with Hyper-V and make containers run there.
But, do you have any clue on this one ?. Because, I do not find my yaml config because the mounting point is not working for me:
docker run ... -v //c/myWindowsPath:/home/node/.edgemicro ... gcr.io/apigee-microgateway/edgemicro:latest
Hi Etienne,
Hope your week is going really great.
Thanks again for your support following up this thread so finally I came with a solution, quite "amiss" since the beginning :c but I am happy it is working now.
[Problem]: Permissions, Accesses, Source Mounting Issues
[Solution]: Overall Windows + Docker Settings. I add a link to that thread here: https://github.com/docker/docker.github.io/issues/2979
Thanks again and wish you all a great week and weekend in advance.
Greets from Stuttgart,
Alan
User | Count |
---|---|
1 | |
1 | |
1 | |
1 | |
1 |