Edge Private Cloud 16.09 Install Issue with apigee-nginx rpm

[ccovney]

Hi everyone,

During installation of Apigee Edge Private Cloud version 4.16.09 (i.e. the latest version), I'm running into a frustrating error in the Router/Message-proc (RMP) installation. **Specifically, with the apigee-ngnix package.** Before I get into the details, here is some basic info about the machine on which the installation is taking place:

# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.2 (Maipo)

# uname -a
Linux *** 3.10.0-327.18.2.el7.x86_64 *** Fri Apr 8 05:09:53 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux

http://docs.apigee.com/private-cloud/latest/installation-overview

The command to issue the RMP install is the following, and prior to running this command, I have followed the official apigee docs (link above) and everything up to this point goes smoothly.

/opt/apigee/apigee-setup/bin/setup.sh -p rmp -f /tmp/myConfig.txt

And the subsequent output (containing the error in question) is as follows:

apigee-service: edge-router: edge-router is running

Checking for router on 127.0.0.1 port 8081 ............. OK

Checking for router uuid 649f58ba-bb61-469e-b169-ceb955961242

Checking if router is up .............................................................

router is not up.

Error: setup.sh: /opt/apigee/apigee-service/bin/apigee-service exited with unexpected status 1

When I dig into the logs, I see the following errors-

/opt/apigee/var/log/edge-router/edge-router.log:

java.io.IOException: Cannot run program "/opt/nginx/scripts/apigee-nginx": error=2, No such file or directory

at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)

Very bizarre that the apigee-ngnix file is not being created by the installation process (as it should).

/opt/apigee/var/log/edge-router/logs/system.log:

2017-01-05 13:11:09,012 main-EventThread ERROR o.a.c.ConnectionState - ConnectionState.checkState() : Authentication failed

...

2017-01-06 11:18:55,249 qtp371397455-42 ERROR REST - CustomJAXRSInvoker.performInvocation() : CustomJAXRSInvoker.performInvocation : Method com.apigee.registration.ServersAPI.isServerCompletelyUp threw an exception.

...

2017-01-06 11:18:55,253 qtp371397455-42 ERROR REST - ExceptionMapper.toResponse() : Error occurred : Service not up yet.

com.apigee.registration.ServersAPI.isServerCompletelyUp(ServersAPI.java:106)

sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

2017-01-06 11:18:55,255 qtp371397455-42 ERROR REST - ExceptionMapper.toResponse() : Returning error response : ErrorResponse{errorCode = null, errorMessage = Service not up yet.}

...

2017-01-06 11:18:55,576 main ERROR LOAD-BALANCER - LoadBalancingManagementServiceImpl.start() : Could not Initialize adaptor nginx

...

2017-01-06 11:18:55,603 main ERROR Router-Service - RouterServiceImpl.start() : Router not started because, Load Balancing could not be initialized

2017-01-06 11:18:55,614 main ERROR KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Got a life cycle exception while starting service [RouterService, Nginx Initialization failed] : {}

com.apigee.errors.http.server.ServiceFailureException: Nginx Initialization failed

at com.apigee.proxy.service.RouterServiceImpl.start(RouterServiceImpl.java:65) ~[message-router-proxy-1.0.0.jar:na]

I'm quite baffled how this could be happening. I don't see any other errors in the various log files. Any ideas of what could be awry here? Perhaps where to look for other clues and/or how to proceed in the installation?

I've tried removing all the Apigee data (including nginx files/folders) and redoing the install, but the error repeatedly occurs.

I've also tried opening up the permission settings beyond what should be necessary in the folder structure around nginx (just to be safe), but to no avail.

Note that the installation for all the other components in the Apigee Edge install (e.g. management server, analytics, datastore, etc) works without even the slightest issue! Thanks in advance for any and all help or insight!

Best,

Chris

[cmbrown]

are you running this as apigee user or as root....there is a similar post related to this as a known piece we need to fix in the router install script. The nginx is looking for root permissions versus those of apigee user.

[ccovney]

Hi @Christin

Thanks for checking this out. I am running the entire install as root user. However, the permissions of the /opt directory is all owned by apigee (including /opt/nginx, etcetera).

Should I forcibly change the permissions of all the nginx files/folders to root instead of apigee? Will the apigee user still be able to manipulate the necessary files/folders in order for the RMP to run and process HTTP traffic?

Also I noticed that even after I completely uninstall apigee, the below files still persists:

# find . -name "*nginx*"
./run/systemd/generator.late/apigee-nginx.service
./run/systemd/generator.late/nginx.service

Should I remove any of these before beginning another install attempt?

Thanks!

[ccovney]

Strangely, those steps did not seem to be apply in that the were no files whatsoever in the conf.d folder. To summarize the issue:

1. Apigee is not creating any conf files within the conf.d

2. Apigee claims the /opt/nginx/scripts/apigee-nginx file is not there, when it is absolutely there. I confirmed this even while the install is in progress from another terminal.

I've tried the install with both apigee permissions and root permissions in the nginx folder, but neither work. Is there any other info on this error that might be applicable to a workaround?

Best,

Chris

[ccovney]

Hi @Christin

I've tried changing the ownership of the entire /opt/nginx directory to root:root, but note that there is no clear way to prevent the apigee install process from setting it back to ownership apigee:apigee. The only way I was able to actually change the ownership so that it would affect the install was to repeatedly issue chown -R root:root /opt/nginx while the install was happening. However, this also did not change the outcome of the install.

If ownership of /opt/nginx is the root cause here, perhaps there is a fix to the install scripts themselves that I can try? Haha I may have no choice but to simply go rogue and tamper with your install scripts, in the interest of time.

Thanks again for all your help so far!

Best,

Chris

[ccovney · Accepted solution]

Directly before installing the router and message processor component, run the below commands, which are a viable workaround for this issue:

mkdir /opt/apigee/etc/edge-router.d
echo RUN_USER=root >> /opt/apigee/etc/edge-router.d/RUN_USER.sh
chown -R apigee:apigee /opt/apigee/etc/edge-router.d

Best,

Chris