edge-management-server installation failed

[han-zhang]

We are install APi services 4.16.09 on Redhat linux.

I ran below commands to setup everything

/opt/apigee/apigee-setup/bin/setup.sh -p sa -f /opt/apigeeconfig/orgConfig

However we got error

apigee-service: edge-management-server: edge-management-server is running Checking for management-server on 127.0.0.1 port 8080 ... OK Checking for management-server uuid 9d75ccef-3571-4f58-935b-a50fef061252 Checking if management-server is up ............................................................. management-server is not up. Error: setup.sh: /opt/apigee/apigee-service/bin/apigee-service exited with unexpected status 1

Below is from log:

2016-09-20 14:16:57,854 main INFO KERNEL - MicroKernel.initInternals() : MicroKernel.start() : MicroKernel start initiated... 2016-09-20 14:16:57,858 main INFO KERNEL - MicroKernel.logJavaAndOSProperties() : Java/OS Properties - {java.ext.dirs=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.102-1.b14.el7_2.x86_64/jre/lib/ext:/usr/java/packages/lib/ext, java.runtime.name=OpenJDK Runtime Environment, java.runtime.version=1.8.0_102-b14, java.specification.version=1.8, java.vendor=Oracle Corporation, java.security.nssprovider.enabled=false, java.class.version=52.0, java.vendor.url=http://java.oracle.com/, java.specification.name=Java Platform API Specification, java.security.auth.login.config=/opt/apigee/edge-management-server-4.16.09-0.0.670/conf/jaas.config, java.awt.printerjob=sun.print.PSPrinterJob, os.arch=amd64, java.vm.version=25.102-b14, os.name=Linux, java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.102-1.b14.el7_2.x86_64/jre, os.version=3.10.0-327.28.3.el7.x86_64, java.endorsed.dirs=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.102-1.b14.el7_2.x86_64/jre/lib/endorsed, java.version=1.8.0_102, java.awt.graphicsenv=sun.awt.X11GraphicsEnvironment, java.vm.specification.version=1.8, java.vendor.url.bug=http://bugreport.sun.com/bugreport/, java.vm.specification.name=Java Virtual Machine Specification, java.specification.vendor=Oracle Corporation, java.vm.vendor=Oracle Corporation, java.vm.info=mixed mode, java.security.properties=../conf/jvmsecurity.properties, java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib, java.vm.name=OpenJDK 64-Bit Server VM, java.vm.specification.vendor=Oracle Corporation, java.io.tmpdir=/var/tmp} 2016-09-20 14:16:57,897 main INFO KERNEL.CONFIG - KernelConfiguration.loadKernelProperties() : KernelConfiguration.loadKernelProperties() : MicroKernel properties : {checkmarx.scan.validator.enabled=true, debugsession.timeout=120, profile=management-server, java.security.properties=../conf/jvmsecurity.properties, jsse.enableSNIExtension=false, jute.maxbuffer=10485750, java.io.tmpdir=/var/tmp, has.monetization=${microkernel_hasMonetization}, qpid.session.command_limit=524288, bundle.validation.schema.enabled=true, com.warrenstrange.googleauth.rng.algorithm=SHA1PRNG, default.response.format=json, rbac.number.of.tries.for.email.search=1, jdk.tls.allowUnsafeServerCertChange=true, astyanax.datastores=taurus-datastore,kms-datastore,dc-datastore,cache-datastore,counter-datastore,keyvaluemap-datastore,application-datastore,audit-datastore,apimodel-datastore,auth-datastore,scheduler-datastore,edgenotification-datastore, javax.xml.xpath.XPathFactory=http://java.sun.com/jaxp/xpath/dom=org.apache.xpath.jaxp.XPathFactoryImpl, migration.mode.status=false, bean.retry.enabled=true, useG1GC=false, javax.xml.transform.TransformerFactory=net.sf.saxon.TransformerFactoryImpl, useStringDeduplication=false, qpid.session.byte_limit=8388608, lib.dir.name=lib/infra/services,lib/gateway/services,lib/gateway/steps,lib/analytics/services, sun.net.maxDatagramSockets=2048, pod=central, com.warrenstrange.googleauth.rng.algorithmProvider=SUN, validation.targetendpoint.connection.check.enabled=true, casssandra.maxConnectTimeInMillis=-1, setCodeCache=false, json.request.schema.validation.enabled=true, license.file.location=/opt/apigee/customer/conf/license.txt, log.level=INFO, virtual.host.restricted.ports=8081, validation.entity.name.enabled=true, shutdownServicesOnStop=true, region=default, python.verbose=error, bundle.validation.enabled=false, authenticated.user.header.name=X-Apigee-Current-User, javacallout.policy.validate=false, java.security.nssprovider.enabled=false} 2016-09-20 14:16:58,595 main ERROR KERNEL - MicroKernel.scanDeployments() : MicroKernel.scanDeployments() : Some modules and services were present in the profile but not in the class path 2016-09-20 14:16:58,596 main ERROR KERNEL - MicroKernel.scanDeployments() : Missing modules: [gateway-metrics] 2016-09-20 14:16:58,596 main ERROR KERNEL - MicroKernel.scanDeployments() : Missing services: [CumulativeMetricsService] 2016-09-20 14:16:58,596 main ERROR KERNEL - MicroKernel.scanDeployments() : Not exiting this time but continuing anyway. 2016-09-20 14:16:58,778 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.init() : ServiceDeployer.init() : Initializing service : MachineKeyService that implements InitializeAwareService 2016-09-20 14:16:58,782 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.init() : ServiceDeployer.init() : Successfully initialized the service : MachineKeyService 2016-09-20 14:16:58,799 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.init() : ServiceDeployer.init() : Initializing service : WebService that implements InitializeAwareService 2016-09-20 14:16:58,822 main INFO c.a.r.f.c.Container - Container.getConnectors() : Container.getConnectors() - Serving http on port - 8080 2016-09-20 14:16:58,953 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.init() : ServiceDeployer.init() : Successfully initialized the service : WebService 2016-09-20 14:16:58,953 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.init() : ServiceDeployer.init() : Initializing service : CommunicationService that implements InitializeAwareService 2016-09-20 14:16:58,962 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.init() : ServiceDeployer.init() : Successfully initialized the service : CommunicationService 2016-09-20 14:16:58,966 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.init() : ServiceDeployer.init() : Initializing service : EventService that implements InitializeAwareService 2016-09-20 14:16:58,972 main INFO FILE_LOGGER - BufferedWriter.start() : Starting logging thread 2016-09-20 14:16:58,976 main INFO FILE_LOGGER - SizeBasedFileRotator.<init>() : Creating sizeBased rotator with maxSize 104857600 2016-09-20 14:16:58,983 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.init() : ServiceDeployer.init() : Successfully initialized the service : EventService 2016-09-20 14:16:58,983 Thread-1 INFO FILE_LOGGER - BufferedWriter.run() : Signalling threadInit completion 2016-09-20 14:16:58,989 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.init() : ServiceDeployer.init() : Initializing service : ZooKeeperService that implements InitializeAwareService 2016-09-20 14:16:59,073 main INFO o.a.c.f.i.CuratorFrameworkImpl - CuratorFrameworkImpl.start() : Starting 2016-09-20 14:16:59,113 main INFO ZOOKEEPER - ZooKeeperServiceImpl.start() : Use binocular pathwatcher service: false 2016-09-20 14:16:59,113 main INFO ZOOKEEPER - ZooKeeperServiceImpl.start() : Use curator based pathwatcher service: false 2016-09-20 14:16:59,114 main INFO c.a.e.ValidationService - ValidationService.register() : Registered validator for component [zookeeper] 2016-09-20 14:16:59,115 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.init() : ServiceDeployer.init() : Successfully initialized the service : ZooKeeperService 2016-09-20 14:16:59,117 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.init() : ServiceDeployer.init() : Initializing service : RepositoryService that implements InitializeAwareService 2016-09-20 14:16:59,122 main-EventThread ERROR o.a.c.ConnectionState - ConnectionState.checkState() : Authentication failed 2016-09-20 14:16:59,138 main INFO m.p.c.m.ConfigurableConsistencyLevel - ConfigurableConsistencyLevel.setDefaultReadConsistencyLevel() : Default read ConsistencyLevel set to: LOCAL_QUORUM. 2016-09-20 14:16:59,138 main INFO m.p.c.m.ConfigurableConsistencyLevel - ConfigurableConsistencyLevel.setDefaultWriteConsistencyLevel() : Default write ConsistencyLevel set to: LOCAL_QUORUM. 2016-09-20 14:16:59,160 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.init() : ServiceDeployer.init() : Successfully initialized the service : RepositoryService 2016-09-20 14:16:59,160 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.init() : ServiceDeployer.init() : Initializing service : ServerRegistrationService that implements InitializeAwareService 2016-09-20 14:16:59,193 main-EventThread INFO o.a.c.f.s.ConnectionStateManager - ConnectionStateManager.postState() : State change: CONNECTED 2016-09-20 14:16:59,202 main INFO REGISTRATION - ServerRegistrationServiceImpl.initResolutionScopes() : server types [scheduler-datastore, audit-datastore, mint-rdbms-datastore, kms-datastore, dc-datastore, application-datastore, postgres-server, counter-datastore, user-settings-datastore, analytics-datastore, router, endpoint, consumer-server, aries-processor, ldap, message-processor, edgenotification-datastore, dw-server, ingest-server, qpid-server, aries-datastore, cache-datastore, zookeeper, apimodel-datastore, taurus-datastore, keyvaluemap-datastore, notification-datastore, mint-datastore, ax-hadoop-ingester, management-server, reportcrud-datastore, auth-datastore] 2016-09-20 14:16:59,208 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.init() : ServiceDeployer.init() : Successfully initialized the service : ServerRegistrationService 2016-09-20 14:16:59,219 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.init() : ServiceDeployer.init() : Initializing service : ServerBindService that implements InitializeAwareService 2016-09-20 14:16:59,221 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.init() : ServiceDeployer.init() : Successfully initialized the service : ServerBindService 2016-09-20 14:16:59,245 main INFO K.DATASTORE.NOSQL - EntityCache.<clinit>() : Default KMS entity cache TTL set to 3 2016-09-20 14:16:59,497 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : MachineKeyService 2016-09-20 14:16:59,498 main INFO FILE_LOGGER - BufferedWriter.start() : Starting logging thread 2016-09-20 14:16:59,498 main INFO FILE_LOGGER - SizeBasedFileRotator.<init>() : Creating sizeBased rotator with maxSize 134217728 2016-09-20 14:16:59,501 Thread-2 INFO FILE_LOGGER - BufferedWriter.run() : Signalling threadInit completion 2016-09-20 14:16:59,501 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : MonitoringService 2016-09-20 14:16:59,504 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : WebService 2016-09-20 14:16:59,505 WebServer INFO o.e.jetty.server.Server - Server.doStart() : jetty-8.0.4.v20111024 2016-09-20 14:16:59,604 WebServer INFO o.e.j.w.StandardDescriptorProcessor - StandardDescriptorProcessor.visitServlet() : NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet 2016-09-20 14:16:59,634 WebServer INFO o.e.j.s.h.ContextHandler - ContextHandler.callContextInitialized() : started o.e.j.w.WebAppContext{/,file:/opt/apigee/edge-management-server-4.16.09-0.0.670/webapps/api/},/opt/apigee/edge-management-server-4.16.09-0.0.670/webapps/api 2016-09-20 14:16:59,635 WebServer INFO o.e.j.s.h.ContextHandler - ContextHandler.callContextInitialized() : started o.e.j.w.WebAppContext{/,file:/opt/apigee/edge-management-server-4.16.09-0.0.670/webapps/api/},/opt/apigee/edge-management-server-4.16.09-0.0.670/webapps/api 2016-09-20 14:16:59,638 main WARN SERVICES.COMMUNICATION - MBeanManager.getRMIConnectorProperties() : RMI host is not provided. Fetching host address from network registry 2016-09-20 14:16:59,660 main WARN RMI - RMIServer.createSocketFactories() : SSL is disabled for 0.0.0.0:1099 2016-09-20 14:16:59,834 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : CommunicationService 2016-09-20 14:16:59,845 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : DebugService 2016-09-20 14:16:59,845 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : EventService 2016-09-20 14:16:59,852 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : AlertService 2016-09-20 14:16:59,853 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : LogService 2016-09-20 14:17:00,132 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : OrganizationService 2016-09-20 14:17:00,133 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : SystemInfoService 2016-09-20 14:17:00,133 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : ZooKeeperService 2016-09-20 14:17:00,137 main INFO ZOOKEEPER - ZooKeeperServiceImpl.registerPathWatcher() : attaching watcher on path /featureflag/{featureName}/global/on/{time}, async is false 2016-09-20 14:17:00,171 main INFO ZOOKEEPER - ZooKeeperServiceImpl.registerPathWatcher() : attaching watcher on path /featureflag/{featureName}/global/off/{time}, async is false 2016-09-20 14:17:00,172 main INFO ZOOKEEPER - ZooKeeperServiceImpl.registerPathWatcher() : attaching watcher on path /featureflag/{featureName}/{subType}/{name}/on/{time}, async is false 2016-09-20 14:17:00,173 main INFO ZOOKEEPER - ZooKeeperServiceImpl.registerPathWatcher() : attaching watcher on path /featureflag/{featureName}/{subType}/{name}/off/{time}, async is false 2016-09-20 14:17:00,174 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : FeatureFlagService 2016-09-20 14:17:00,248 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : LicensingService 2016-09-20 14:17:00,248 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : RepositoryService 2016-09-20 14:17:00,259 WebServer INFO o.e.j.s.AbstractConnector - AbstractConnector.doStart() : Started SelectChannelConnector@0.0.0.0:8080 STARTING 2016-09-20 14:17:00,332 main WARN REGISTRATION - CommunicationInfoBuilder.loadIPAddresses() : CommunicationInfoBuilder.loadIPAddresses : File /opt/apigee/ec2.properties is missing. Will use localhost as default for all the hosts 2016-09-20 14:17:00,497 main INFO REGISTRATION - ServerRegistrationServiceImpl.storeServerRegistrationPath() : Registering the server uuid 9d75ccef-3571-4f58-935b-a50fef061252 with region default and pod central information 2016-09-20 14:17:00,520 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : ServerRegistrationService 2016-09-20 14:17:00,564 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : MetricsService 2016-09-20 14:17:00,565 main INFO APIMODEL.COMMON - ApiModelRepositoryServiceImpl.start() : Starting API Modelling service... 2016-09-20 14:17:00,593 main INFO APIMODEL.COMMON - DataStoreFactory.init() : In DataStoreFactory.init() properties : {authurl=https://api.enterprise.apigee.com/v1/users/{user}/authenticate, cassandra.defaults.thrift.socketTimeoutInMillis=10000, records.limit.per.apicall=500, authschemes.v2.migration.enabled=true, authschemes.v1.disabled=false, datastoretype=apimodel-datastore, testapi.proxy=https://apiconsole-prod.apigee.net/smartdocs/v1/sendrequest, read.consistencylevel=LOCAL_QUORUM, write.consistencylevel=LOCAL_QUORUM, keyspace=apimodel, endpoint=https://api.enterprise.apigee.com, authschemes.v2.enabled=true} 2016-09-20 14:17:00,595 main INFO m.p.c.m.ConfigurableConsistencyLevel - ConfigurableConsistencyLevel.setDefaultReadConsistencyLevel() : Default read ConsistencyLevel set to: LOCAL_QUORUM. 2016-09-20 14:17:00,595 main INFO m.p.c.m.ConfigurableConsistencyLevel - ConfigurableConsistencyLevel.setDefaultWriteConsistencyLevel() : Default write ConsistencyLevel set to: ONE. 2016-09-20 14:17:00,598 main INFO ZOOKEEPER - ZooKeeperServiceImpl.registerPathWatcher() : attaching watcher on path /regions/default/pods/central/types/apimodel-datastore/{uuid}, async is true 2016-09-20 14:17:00,608 main INFO APIMODEL.COMMON - ApiModelDataStore.init() : ApiModelDataStore.init(servers) : Initializing call for cassandra audit datastore with server beans 2016-09-20 14:17:00,608 main INFO APIMODEL.COMMON - ApiModelDataStore.initializeDataStoreClient() : ApiModelDataStore.initializeDataStoreClient() : Initializing data stores 2016-09-20 14:17:00,608 main INFO APIMODEL.COMMON - ApiModelDataStore.initializeDataStoreClient() : ApiModelDataStore.initializeDataStoreClient() : No datastores configured 2016-09-20 14:17:00,608 main INFO APIMODEL.COMMON - ApiModelDataStore.init() : ApiModelDataStore.init() : Initializing cassandra api model datastore 2016-09-20 14:17:00,608 main INFO APIMODEL.COMMON - DataStoreFactory.init() : DataStoreFactory.init() : DataStoreFactory initialized 2016-09-20 14:17:00,618 main INFO APIMODEL.COMMON - ApiModelRepositoryServiceImpl.start() : Started API Modelling service successfully 2016-09-20 14:17:00,618 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : ApiModelRepositoryService 2016-09-20 14:17:00,621 main INFO m.p.c.m.ConfigurableConsistencyLevel - ConfigurableConsistencyLevel.setDefaultReadConsistencyLevel() : Default read ConsistencyLevel set to: ONE. 2016-09-20 14:17:00,621 main INFO m.p.c.m.ConfigurableConsistencyLevel - ConfigurableConsistencyLevel.setDefaultWriteConsistencyLevel() : Default write ConsistencyLevel set to: ONE. 2016-09-20 14:17:00,630 main INFO ZOOKEEPER - ZooKeeperServiceImpl.registerPathWatcher() : attaching watcher on path /regions/default/pods/central/types/audit-datastore/{uuid}, async is true 2016-09-20 14:17:00,636 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : AuditService 2016-09-20 14:17:00,639 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : AuditQueryService 2016-09-20 14:17:00,654 main INFO ZOOKEEPER - ZooKeeperServiceImpl.registerPathWatcher() : attaching watcher on path /regions/default/pods/central/types/scheduler-datastore/{uuid}, async is true 2016-09-20 14:17:00,659 main INFO m.p.c.m.ConfigurableConsistencyLevel - ConfigurableConsistencyLevel.setDefaultReadConsistencyLevel() : Default read ConsistencyLevel set to: LOCAL_QUORUM. 2016-09-20 14:17:00,659 main INFO m.p.c.m.ConfigurableConsistencyLevel - ConfigurableConsistencyLevel.setDefaultWriteConsistencyLevel() : Default write ConsistencyLevel set to: LOCAL_QUORUM. 2016-09-20 14:17:00,724 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : SchedulerService 2016-09-20 14:17:00,726 main WARN KERNEL.DEPLOYMENT - ServiceDeployer.loadPropertiesFromResources() : ServiceDeployer.getProperties : Resource file notification.properties not found in classpath 2016-09-20 14:17:00,732 main INFO ZOOKEEPER - ZooKeeperServiceImpl.registerPathWatcher() : attaching watcher on path /regions/default/pods/central/types/edgenotification-datastore/{uuid}, async is true 2016-09-20 14:17:00,736 main INFO m.p.c.m.ConfigurableConsistencyLevel - ConfigurableConsistencyLevel.setDefaultReadConsistencyLevel() : Default read ConsistencyLevel set to: LOCAL_QUORUM. 2016-09-20 14:17:00,736 main INFO m.p.c.m.ConfigurableConsistencyLevel - ConfigurableConsistencyLevel.setDefaultWriteConsistencyLevel() : Default write ConsistencyLevel set to: LOCAL_QUORUM. 2016-09-20 14:17:00,737 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : EdgeNotificationService 2016-09-20 14:17:00,782 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : ServerBindService 2016-09-20 14:17:00,784 main INFO CpsLibrary - CpsPropertiesUtil.<clinit>() : Cps enabled flag is false 2016-09-20 14:17:00,785 main INFO CPS_MANAGEMENT - CpsManagementServiceImpl.start() : Starting CpsManagementService, Properties:{kms.cache.enabled=false, kms.read.nonlocalconsistencylevel=QUORUM, kms.maxactive_cassandra_connections=8, kvm.write.consistencylevel=LOCAL_QUORUM, kvm.read.nonlocalconsistencylevel=QUORUM, kvm.localdc.enabled=true, kms.localdc.enabled=true, kvm.maxactive_cassandra_connections=8, kvm.cache.enabled=false, cassandra.hipaa-ring01.dml_password=NULL, kms.cassandra.defaults.socketReadTimeoutInMillis=10000, kvm.cassandra.defaults.connectTimeoutInMillis=10000, kms.write.consistencylevel=LOCAL_QUORUM, kms.read.localconsistencylevel=LOCAL_QUORUM, kvm.perses.enabled=true, cassandra.hipaa-ring01.ddl_password=NULL, kvm.read.localconsistencylevel=LOCAL_QUORUM, cassandra.hipaa-ring01.dml_user=NULL, kms.app_names_fetch_limit_for_developers_listing=100, kvm.cassandra.defaults.socketReadTimeoutInMillis=10000, cassandra.hipaa-ring01.ddl_user=NULL, kms.cassandra.defaults.connectTimeoutInMillis=10000, kms.perses.enabled=true} 2016-09-20 14:17:00,790 main WARN i.a.c.r.i.ZookeeperClientDetails - ZookeeperClientDetails.<init>() : Zookeeper client is connecting to a single zk host :localhost:2181,high availability is compromised 2016-09-20 14:17:00,792 main INFO o.a.c.f.i.CuratorFrameworkImpl - CuratorFrameworkImpl.start() : Starting 2016-09-20 14:17:00,801 main INFO i.a.t.r.RepositoryFactory - RepositoryFactory.get() : RepositoryFactory: [class io.apigee.tenant.repository.impl.zk.TenantRepositoryImpl] 2016-09-20 14:17:00,805 main INFO CpsLibrary - AbstractCredentialsStore.getCredentialsMap() : Value is empty for property:cassandra.hipaa-ring01.ddl_password in properties 2016-09-20 14:17:00,805 main INFO CpsLibrary - AbstractCredentialsStore.getCredentialsMap() : Value is empty for property:cassandra.hipaa-ring01.ddl_user in properties 2016-09-20 14:17:00,817 main INFO CpsLibrary - AbstractCredentialsStore.getCredentialsMap() : Value is empty for property:cassandra.hipaa-ring01.dml_password in properties 2016-09-20 14:17:00,817 main INFO CpsLibrary - AbstractCredentialsStore.getCredentialsMap() : Value is empty for property:cassandra.hipaa-ring01.dml_user in properties 2016-09-20 14:17:00,817 main INFO i.a.t.r.RepositoryFactory - RepositoryFactory.get() : RepositoryFactory: [class io.apigee.tenant.repository.impl.zk.TenantRepositoryImpl] 2016-09-20 14:17:00,817 main-EventThread ERROR o.a.c.ConnectionState - ConnectionState.checkState() : Authentication failed 2016-09-20 14:17:00,821 main INFO ZOOKEEPER.LIBRARY - CuratorBasedPathWatcherService.initWatchingInfra() : Enable Curator In Async mode: true 2016-09-20 14:17:00,825 main-EventThread INFO o.a.c.f.s.ConnectionStateManager - ConnectionStateManager.postState() : State change: CONNECTED 2016-09-20 14:17:00,840 main INFO ZOOKEEPER.LIBRARY - CuratorBasedPathWatcherService.initWatchingInfra() : Enable Curator In Async mode: true 2016-09-20 14:17:00,841 main INFO ZOOKEEPER.LIBRARY - CuratorBasedPathWatcherService.initWatchingInfra() : Enable Curator In Async mode: true 2016-09-20 14:17:00,849 Apigee-Main-1 INFO CPS_MANAGEMENT - CpsManagementServiceImpl$3.run() : Initializing KvmServiceFactoryFacade 2016-09-20 14:17:00,862 Apigee-Main-1 INFO CpsLibrary - CpsServiceFactoryFacade.getCpsServiceFactoryFacade() : ConnectionPool configuration options for serviceType[KVM] and namespace[kvm.] is {LOCAL_CONSISTENCY_LEVEL=LOCAL_QUORUM, MAX_CONNECTIONS_PER_HOST=8, SOCKET_READ_TIMEOUT_MILLIS=10000, SOCKET_CONNECTION_TIMEOUT_MILLIS=10000, NON_LOCAL_CONSISTENCY_LEVEL=QUORUM, LOCAL_DC=default} 2016-09-20 14:17:00,863 Apigee-Main-1 INFO CpsLibrary - CpsServiceFactoryFacade.getCpsServiceFactoryFacade() : ServiceFactory options for serviceType[KVM] and namespace[kvm.] is {ENABLE_PERSES=true, ENABLE_LOCAL_CACHE=false} 2016-09-20 14:17:00,863 Apigee-Main-1 INFO CpsLibrary - AbstractCredentialsStore.getCredentialsMap() : Value is empty for property:cassandra.hipaa-ring01.dml_password in properties 2016-09-20 14:17:00,863 Apigee-Main-1 INFO CpsLibrary - AbstractCredentialsStore.getCredentialsMap() : Value is empty for property:cassandra.hipaa-ring01.dml_user in properties 2016-09-20 14:17:00,869 Apigee-Main-1 INFO CPS_MANAGEMENT - CpsManagementServiceImpl.getAllRings() : No rings configured; skipping initialization of kvmServiceFactory 2016-09-20 14:17:00,869 Apigee-Main-2 INFO CPS_MANAGEMENT - CpsManagementServiceImpl$2.run() : Initializing KmsServiceFactoryFacade 2016-09-20 14:17:00,869 Apigee-Main-2 INFO CpsLibrary - CpsServiceFactoryFacade.getCpsServiceFactoryFacade() : ConnectionPool configuration options for serviceType[KMS] and namespace[kms.] is {LOCAL_CONSISTENCY_LEVEL=LOCAL_QUORUM, MAX_CONNECTIONS_PER_HOST=8, SOCKET_READ_TIMEOUT_MILLIS=10000, SOCKET_CONNECTION_TIMEOUT_MILLIS=10000, NON_LOCAL_CONSISTENCY_LEVEL=QUORUM, LOCAL_DC=default} 2016-09-20 14:17:00,869 Apigee-Main-2 INFO CpsLibrary - CpsServiceFactoryFacade.getCpsServiceFactoryFacade() : ServiceFactory options for serviceType[KMS] and namespace[kms.] is {ENABLE_PERSES=true, ENABLE_LOCAL_CACHE=false} 2016-09-20 14:17:00,869 Apigee-Main-2 INFO CpsLibrary - AbstractCredentialsStore.getCredentialsMap() : Value is empty for property:cassandra.hipaa-ring01.dml_password in properties 2016-09-20 14:17:00,869 Apigee-Main-2 INFO CpsLibrary - AbstractCredentialsStore.getCredentialsMap() : Value is empty for property:cassandra.hipaa-ring01.dml_user in properties 2016-09-20 14:17:00,871 Apigee-Main-2 INFO CPS_MANAGEMENT - CpsManagementServiceImpl.getAllRings() : No rings configured; skipping initialization of kvmServiceFactory 2016-09-20 14:17:00,871 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : CpsManagementService 2016-09-20 14:17:00,872 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : ServerBindObserverService 2016-09-20 14:17:00,874 main INFO CLUSTER - ClusterServiceImpl.start() : Registering to join a cluster with servers of type management-server 2016-09-20 14:17:00,876 main INFO ZOOKEEPER - ZooKeeperServiceImpl.registerPathWatcher() : attaching watcher on path /regions/default/pods/central/types/management-server/{uuid}, async is true 2016-09-20 14:17:00,962 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : ClusterService 2016-09-20 14:17:00,979 main INFO SERVICES.CACHE - EHCacheFlush.init() : Initialize EHCacheFlush clear expired elements every 300 seconds 2016-09-20 14:17:00,980 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : CacheService 2016-09-20 14:17:01,249 Timer-0 INFO SERVICE.LICENSE - ScheduledLicenseValidator.run() : License found and is valid !!! 2016-09-20 14:17:01,286 qtp992743048-43 ERROR REST - CustomJAXRSInvoker.performInvocation() : CustomJAXRSInvoker.performInvocation : Method com.apigee.registration.RegistrationManagementAPI.isServerCompletelyUp threw an exception. 2016-09-20 14:17:01,288 qtp992743048-43 ERROR REST - ExceptionMapper.toResponse() : Error occurred : Service not up yet. com.apigee.registration.ServersAPI.isServerCompletelyUp(ServersAPI.java:106) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

2016-09-20 14:17:01,289 qtp992743048-43 ERROR REST - ExceptionMapper.toResponse() : Returning error response : ErrorResponse{errorCode = null, errorMessage = Service not up yet.} 2016-09-20 14:17:01,297 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : AccessControl 2016-09-20 14:17:01,303 main ERROR - CredentialUtil.decrypt() : cannot decrypt - REPLACE_ME_IN_ETC_APIGEE_TOKEN_DEFAULT_PROPERTIES invalid block size; Input length must be multiple of 16 when decrypting with padded cipher 2016-09-20 14:17:01,322 main WARN n.s.e.c.CacheConfiguration - CacheConfiguration.warnMaxEntriesLocalHeap() : Cache: auth_cache has a maxElementsInMemory of 0. This might lead to performance degradation or OutOfMemoryError at Terracotta client.From Ehcache 2.0 onwards this has been changed to mean a store with no capacity limit. Set it to 1 if you want no elements cached in memory 2016-09-20 14:17:01,323 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : AuthenticationService 2016-09-20 14:17:01,325 main ERROR - CredentialUtil.decrypt() : cannot decrypt - REPLACE_ME_IN_ETC_APIGEE_TOKEN_DEFAULT_PROPERTIES invalid block size; Input length must be multiple of 16 when decrypting with padded cipher 2016-09-20 14:17:01,325 main INFO m.p.c.m.ConfigurableConsistencyLevel - ConfigurableConsistencyLevel.setDefaultReadConsistencyLevel() : Default read ConsistencyLevel set to: ONE. 2016-09-20 14:17:01,325 main INFO m.p.c.m.ConfigurableConsistencyLevel - ConfigurableConsistencyLevel.setDefaultWriteConsistencyLevel() : Default write ConsistencyLevel set to: ONE. 2016-09-20 14:17:01,327 main INFO ZOOKEEPER - ZooKeeperServiceImpl.registerPathWatcher() : attaching watcher on path /regions/default/pods/central/types/auth-datastore/{uuid}, async is true 2016-09-20 14:17:01,332 main WARN n.s.e.c.CacheConfiguration - CacheConfiguration.warnMaxEntriesLocalHeap() : Cache: creds_cache has a maxElementsInMemory of 0. This might lead to performance degradation or OutOfMemoryError at Terracotta client.From Ehcache 2.0 onwards this has been changed to mean a store with no capacity limit. Set it to 1 if you want no elements cached in memory 2016-09-20 14:17:01,335 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : UserService 2016-09-20 14:17:01,336 main ERROR - CredentialUtil.decrypt() : cannot decrypt - REPLACE_ME_IN_ETC_APIGEE_TOKEN_DEFAULT_PROPERTIES invalid block size; Input length must be multiple of 16 when decrypting with padded cipher 2016-09-20 14:17:01,337 main INFO m.p.c.m.ConfigurableConsistencyLevel - ConfigurableConsistencyLevel.setDefaultReadConsistencyLevel() : Default read ConsistencyLevel set to: ONE. 2016-09-20 14:17:01,337 main INFO m.p.c.m.ConfigurableConsistencyLevel - ConfigurableConsistencyLevel.setDefaultWriteConsistencyLevel() : Default write ConsistencyLevel set to: ONE. 2016-09-20 14:17:01,340 main WARN n.s.e.c.CacheConfiguration - CacheConfiguration.warnMaxEntriesLocalHeap() : Cache: PermissionsCache has a maxElementsInMemory of 0. This might lead to performance degradation or OutOfMemoryError at Terracotta client.From Ehcache 2.0 onwards this has been changed to mean a store with no capacity limit. Set it to 1 if you want no elements cached in memory 2016-09-20 14:17:01,340 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : ResourceService 2016-09-20 14:17:01,341 main INFO c.a.l.m.c.SourceCategoryFactory - SourceCategoryFactory.init() : init called with props: {log.category.nodejs.displayName=nodejs, log.category.nodejs.name=REPLACE_ME_IN_ETC_APIGEE_TOKEN_DEFAULT_PROPERTIES, http.pool.max.connections=50, log.categories=nodejs, connection.timeout=90000, socket.timeout=90000, http.pool.max.per.route=50} 2016-09-20 14:17:01,342 main INFO c.a.l.m.c.SourceCategoryFactory - SourceCategoryFactory.addSourceCategory() : SourceCategory added with name: REPLACE_ME_IN_ETC_APIGEE_TOKEN_DEFAULT_PROPERTIES displayName: nodejs 2016-09-20 14:17:01,564 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : LoggingService 2016-09-20 14:17:01,654 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : ServerMetricsService 2016-09-20 14:17:01,656 main ERROR - CredentialUtil.decrypt() : cannot decrypt - REPLACE_ME_IN_ETC_APIGEE_TOKEN_DEFAULT_PROPERTIES invalid block size; Input length must be multiple of 16 when decrypting with padded cipher 2016-09-20 14:17:01,688 main ERROR KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Got a life cycle exception while starting service [SecurityService, LDAP connection failed] : {} com.apigee.rest.framework.security.ConnectionException: LDAP connection failed at com.apigee.rbac.datastore.ldap.LdapContextFactory.logAndThrowAuthnFailedException(LdapContextFactory.java:145) ~[rbac-1.0.0.jar:na] at com.apigee.rbac.datastore.ldap.LdapContextFactory.createDirCtx(LdapContextFactory.java:91) ~[rbac-1.0.0.jar:na] at com.apigee.rbac.datastore.ldap.LdapContextFactory.createLdapContext(LdapContextFactory.java:55) ~[rbac-1.0.0.jar:na] at com.apigee.rbac.datastore.LdapDataStore.createAndGetDirContext(LdapDataStore.java:412) ~[rbac-1.0.0.jar:na] at com.apigee.rbac.datastore.LdapDataStore.initializeNameSpace(LdapDataStore.java:449) ~[rbac-1.0.0.jar:na] at com.apigee.rbac.impl.UserServiceImpl.initialize(UserServiceImpl.java:98) ~[rbac-1.0.0.jar:na] at com.apigee.security.usermanagement.UserManager.<init>(UserManager.java:77) ~[security-1.0.0.jar:na] at com.apigee.security.SecurityManager.<init>(SecurityManager.java:78) ~[security-1.0.0.jar:na] at com.apigee.security.SecurityServiceImpl.start(SecurityServiceImpl.java:77) ~[security-1.0.0.jar:na] at com.apigee.kernel.service.deployment.ServiceDeployer.startService(ServiceDeployer.java:168) [microkernel-1.0.0.jar:na] at com.apigee.kernel.service.deployment.ServiceDeployer.deploy(ServiceDeployer.java:71) [microkernel-1.0.0.jar:na] at com.apigee.kernel.service.deployment.ServiceDeployer.deployDependantServices(ServiceDeployer.java:356) [microkernel-1.0.0.jar:na] at com.apigee.kernel.service.deployment.ServiceDeployer.deploy(ServiceDeployer.java:77) [microkernel-1.0.0.jar:na] at com.apigee.kernel.service.deployment.ServiceDeployer.deployDependantServices(ServiceDeployer.java:356) [microkernel-1.0.0.jar:na] at com.apigee.kernel.service.deployment.ServiceDeployer.deploy(ServiceDeployer.java:77) [microkernel-1.0.0.jar:na] at com.apigee.kernel.MicroKernel.deployAll(MicroKernel.java:178) [microkernel-1.0.0.jar:na] at com.apigee.kernel.MicroKernel.start(MicroKernel.java:139) [microkernel-1.0.0.jar:na] at com.apigee.kernel.MicroKernel.start(MicroKernel.java:135) [microkernel-1.0.0.jar:na] at com.apigee.kernel.MicroKernel.main(MicroKernel.java:84) [microkernel-1.0.0.jar:na] Caused by: javax.naming.CommunicationException: ldap:10389 at com.sun.jndi.ldap.Connection.<init>(Connection.java:216) ~[na:1.8.0_102] at com.sun.jndi.ldap.LdapClient.<init>(LdapClient.java:137) ~[na:1.8.0_102] at com.sun.jndi.ldap.LdapClientFactory.createPooledConnection(LdapClientFactory.java:64) ~[na:1.8.0_102] at com.sun.jndi.ldap.pool.Connections.<init>(Connections.java:115) ~[na:1.8.0_102] at com.sun.jndi.ldap.pool.Pool.getPooledConnection(Pool.java:132) ~[na:1.8.0_102] at com.sun.jndi.ldap.LdapPoolManager.getLdapClient(LdapPoolManager.java:329) ~[na:1.8.0_102] at com.sun.jndi.ldap.LdapClient.getInstance(LdapClient.java:1606) ~[na:1.8.0_102] at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2746) ~[na:1.8.0_102] at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:319) ~[na:1.8.0_102] at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:192) ~[na:1.8.0_102] at com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:210) ~[na:1.8.0_102] at com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:153) ~[na:1.8.0_102] at com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:83) ~[na:1.8.0_102] at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684) ~[na:1.8.0_102] at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:313) ~[na:1.8.0_102] at javax.naming.InitialContext.init(InitialContext.java:244) ~[na:1.8.0_102] at javax.naming.InitialContext.<init>(InitialContext.java:216) ~[na:1.8.0_102] at javax.naming.directory.InitialDirContext.<init>(InitialDirContext.java:101) ~[na:1.8.0_102] at com.apigee.rbac.datastore.ldap.LdapContextFactory.connect(LdapContextFactory.java:113) ~[rbac-1.0.0.jar:na] at com.apigee.rbac.datastore.ldap.LdapContextFactory.createDirCtx(LdapContextFactory.java:73) ~[rbac-1.0.0.jar:na] ... 17 common frames omitted Caused by: java.net.UnknownHostException: ldap at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184) ~[na:1.8.0_102] at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_102] at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_102] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_102] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_102] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_102] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_102] at com.sun.jndi.ldap.Connection.createSocket(Connection.java:350) ~[na:1.8.0_102] at com.sun.jndi.ldap.Connection.<init>(Connection.java:203) ~[na:1.8.0_102] ... 36 common frames omitted 2016-09-20 14:17:01,690 main ERROR KERNEL - MicroKernel.deployAll() : MicroKernel.deployAll() : Error in deploying the deployment : ZooKeeperService com.apigee.rest.framework.security.ConnectionException: LDAP connection failed at com.apigee.rbac.datastore.ldap.LdapContextFactory.logAndThrowAuthnFailedException(LdapContextFactory.java:145) ~[rbac-1.0.0.jar:na] at com.apigee.rbac.datastore.ldap.LdapContextFactory.createDirCtx(LdapContextFactory.java:91) ~[rbac-1.0.0.jar:na] at com.apigee.rbac.datastore.ldap.LdapContextFactory.createLdapContext(LdapContextFactory.java:55) ~[rbac-1.0.0.jar:na] at com.apigee.rbac.datastore.LdapDataStore.createAndGetDirContext(LdapDataStore.java:412) ~[rbac-1.0.0.jar:na] at com.apigee.rbac.datastore.LdapDataStore.initializeNameSpace(LdapDataStore.java:449) ~[rbac-1.0.0.jar:na] at com.apigee.rbac.impl.UserServiceImpl.initialize(UserServiceImpl.java:98) ~[rbac-1.0.0.jar:na] at com.apigee.security.usermanagement.UserManager.<init>(UserManager.java:77) ~[security-1.0.0.jar:na] at com.apigee.security.SecurityManager.<init>(SecurityManager.java:78) ~[security-1.0.0.jar:na] at com.apigee.security.SecurityServiceImpl.start(SecurityServiceImpl.java:77) ~[security-1.0.0.jar:na] at com.apigee.kernel.service.deployment.ServiceDeployer.startService(ServiceDeployer.java:168) ~[microkernel-1.0.0.jar:na] at com.apigee.kernel.service.deployment.ServiceDeployer.deploy(ServiceDeployer.java:71) ~[microkernel-1.0.0.jar:na] at com.apigee.kernel.service.deployment.ServiceDeployer.deployDependantServices(ServiceDeployer.java:356) ~[microkernel-1.0.0.jar:na] at com.apigee.kernel.service.deployment.ServiceDeployer.deploy(ServiceDeployer.java:77) ~[microkernel-1.0.0.jar:na] at com.apigee.kernel.service.deployment.ServiceDeployer.deployDependantServices(ServiceDeployer.java:356) ~[microkernel-1.0.0.jar:na] at com.apigee.kernel.service.deployment.ServiceDeployer.deploy(ServiceDeployer.java:77) ~[microkernel-1.0.0.jar:na] at com.apigee.kernel.MicroKernel.deployAll(MicroKernel.java:178) [microkernel-1.0.0.jar:na] at com.apigee.kernel.MicroKernel.start(MicroKernel.java:139) [microkernel-1.0.0.jar:na] at com.apigee.kernel.MicroKernel.start(MicroKernel.java:135) [microkernel-1.0.0.jar:na] at com.apigee.kernel.MicroKernel.main(MicroKernel.java:84) [microkernel-1.0.0.jar:na] Caused by: javax.naming.CommunicationException: ldap:10389 at com.sun.jndi.ldap.Connection.<init>(Connection.java:216) ~[na:1.8.0_102] at com.sun.jndi.ldap.LdapClient.<init>(LdapClient.java:137) ~[na:1.8.0_102] at com.sun.jndi.ldap.LdapClientFactory.createPooledConnection(LdapClientFactory.java:64) ~[na:1.8.0_102] at com.sun.jndi.ldap.pool.Connections.<init>(Connections.java:115) ~[na:1.8.0_102] at com.sun.jndi.ldap.pool.Pool.getPooledConnection(Pool.java:132) ~[na:1.8.0_102] at com.sun.jndi.ldap.LdapPoolManager.getLdapClient(LdapPoolManager.java:329) ~[na:1.8.0_102] at com.sun.jndi.ldap.LdapClient.getInstance(LdapClient.java:1606) ~[na:1.8.0_102] at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2746) ~[na:1.8.0_102] at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:319) ~[na:1.8.0_102] at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:192) ~[na:1.8.0_102] at com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:210) ~[na:1.8.0_102] at com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:153) ~[na:1.8.0_102] at com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:83) ~[na:1.8.0_102] at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684) ~[na:1.8.0_102] at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:313) ~[na:1.8.0_102] at javax.naming.InitialContext.init(InitialContext.java:244) ~[na:1.8.0_102] at javax.naming.InitialContext.<init>(InitialContext.java:216) ~[na:1.8.0_102] at javax.naming.directory.InitialDirContext.<init>(InitialDirContext.java:101) ~[na:1.8.0_102] at com.apigee.rbac.datastore.ldap.LdapContextFactory.connect(LdapContextFactory.java:113) ~[rbac-1.0.0.jar:na] at com.apigee.rbac.datastore.ldap.LdapContextFactory.createDirCtx(LdapContextFactory.java:73) ~[rbac-1.0.0.jar:na] ... 17 common frames omitted Caused by: java.net.UnknownHostException: ldap at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184) ~[na:1.8.0_102] at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_102] at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_102] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_102] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_102] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_102] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_102] at com.sun.jndi.ldap.Connection.createSocket(Connection.java:350) ~[na:1.8.0_102] at com.sun.jndi.ldap.Connection.<init>(Connection.java:203) ~[na:1.8.0_102] ... 36 common frames omitted

[han-zhang · Accepted solution]

Self resolved

1. Using real IP in config file

2. Given access for root user to run exec in /tmp folder