Look Up Caches NullPointerException exception when fetching entry
Hi Guys,
We are facing one issue where our Populate Cache Policies are working fine but Lookup Caches are not working properly after upgrading to apigee version 4.19.01
We did MP restart and also tried killing apigee process but the issue still persists. Can someone know what is the exact reason for these errors and how it can be resolved.
We are getting below error in MP Logs:
2019-02-06 15:42:41,198 org:XXX env:test api:oauth_nl rev:30 messageid:elba-3559-858-3 policy:Verify-Api-Key Apigee-Main-74 ERROR SERVICES.CACHE - DistributedCacheManagerImpl.getEntry() : DistributedCacheManagerImpl.getEntry() - exception when fetching entry api_products@@@XXXXX@@@nl-api-billing-product - ignoring it.... java.lang.NullPointerException: null at com.apigee.cache.CacheMemoryLimiter.get(CacheMemoryLimiter.java:167) at com.apigee.cache.CacheManagerImpl.updateMemoryLimiter(CacheManagerImpl.java:151) at com.apigee.cache.CacheManagerImpl.getEntryFromCache(CacheManagerImpl.java:274) at com.apigee.cache.CacheManagerImpl.getEntry(CacheManagerImpl.java:267) at com.apigee.cache.distributed.DistributedCacheManagerImpl.getEntry(DistributedCacheManagerImpl.java:305) at com.apigee.keymanagement.util.CacheUtil.getFromCache(CacheUtil.java:206) at com.apigee.keymanagement.dao.nosql.impl.BaseDaoImpl.fetchEntityFromDataStore(BaseDaoImpl.java:355) at com.apigee.keymanagement.dao.nosql.impl.BaseDaoImpl.getByKey(BaseDaoImpl.java:196) at com.apigee.keymanagement.dao.nosql.impl.BaseDaoImpl.getByKey(BaseDaoImpl.java:192) at com.apigee.keymanagement.dao.nosql.impl.ApiProductDaoImpl.getByName(ApiProductDaoImpl.java:167) at com.apigee.keymanagement.dao.nosql.impl.ApiProductDaoImpl.getByName(ApiProductDaoImpl.java:29) at com.apigee.keymanagement.persistence.adapters.LegacyRuntimeAdapter.getApiProductByName(LegacyRuntimeAdapter.java:673) at com.apigee.keymanagement.dao.nosql.util.ApiProductUtil.getApiProductByNameAndOrg(ApiProductUtil.java:40) at com.apigee.keymanagement.dao.nosql.util.ApiProductUtil.populateApiProductAttributes(ApiProductUtil.java:120) at com.apigee.keymanagement.util.ConsumerKeyUtil.getConsumerDetailsForApiKey(ConsumerKeyUtil.java:62) at com.apigee.oauth.v2.persistence.OAuth2RuntimeServiceImpl.getConsumerForApiKey(OAuth2RuntimeServiceImpl.java:134) at com.apigee.oauth.v2.connectors.LocalOAuthServiceConnector.getClientAttributesForApiKey(LocalOAuthServiceConnector.java:202) at com.apigee.oauth.v2.OAuthServiceImpl.getClientAttributesForApiKey(OAuthServiceImpl.java:544) at com.apigee.steps.oauth.v2.VerifyApiKeyStepExecution.validateKeyAndPopulateFlowVariables(VerifyApiKeyStepExecution.java:37) at com.apigee.steps.oauth.v2.VerifyApiKeyStepExecutionBase.execute(VerifyApiKeyStepExecutionBase.java:83) at com.apigee.messaging.runtime.steps.StepExecution.execute(StepExecution.java:156) at com.apigee.flow.execution.AbstractAsyncExecutionStrategy$AsyncExecutionTask.call(AbstractAsyncExecutionStrategy.java:74) at com.apigee.flow.execution.AbstractAsyncExecutionStrategy$AsyncExecutionTask.call(AbstractAsyncExecutionStrategy.java:45) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2019-02-06 15:42:41,200 org:XXXX env:test api:oauth_nl rev:30 messageid:elba-3559-858-3 policy:Verify-Api-Key Apigee-Main-74 WARN SERVICES.CACHE - CacheManagerImpl.addEntry() : Exception when adding cache entry api_products@@@XXXX@@@nl-api-billing-product - ignoring it.... java.lang.NullPointerException: null 2019-02-06 15:42:41,200 org:XXXX env:test api:oauth_nl rev:30 messageid:elba-3559-858-3 policy:Verify-Api-Key Apigee-Main-74 ERROR SERVICES.CACHE - DistributedCacheManagerImpl.getEntry() : DistributedCacheManagerImpl.getEntry() - exception when fetching entry organizations@@@XXXX - ignoring it.... java.lang.NullPointerException: null at com.apigee.cache.CacheMemoryLimiter.get(CacheMemoryLimiter.java:167) at com.apigee.cache.CacheManagerImpl.updateMemoryLimiter(CacheManagerImpl.java:151) at com.apigee.cache.CacheManagerImpl.getEntryFromCache(CacheManagerImpl.java:274) at com.apigee.cache.CacheManagerImpl.getEntry(CacheManagerImpl.java:267) at com.apigee.cache.distributed.DistributedCacheManagerImpl.getEntry(DistributedCacheManagerImpl.java:305) at com.apigee.keymanagement.util.CacheUtil.getFromCache(CacheUtil.java:206) at com.apigee.keymanagement.dao.nosql.impl.BaseDaoImpl.fetchEntityFromDataStore(BaseDaoImpl.java:355) at com.apigee.keymanagement.dao.nosql.impl.BaseDaoImpl.getByKey(BaseDaoImpl.java:196) at com.apigee.keymanagement.dao.nosql.impl.BaseDaoImpl.getByKey(BaseDaoImpl.java:192) "system.log" 411947L, 40801531C
It seems to me, Apigee Edge shouldn't log NPEs. Finding an NPE in the log seems to indicate a bug. Let me look into it.
ref: b/124049692
Hi Saransh
Can you give me some additional context around this log message? You mentioned LookupCache policy. But the stacktrace you provided corresponds to the VerifyApiKey policy, not LookupCache.
The behavior you're reporting here is that LookupCache is not behaving properly. Do you have a small api proxy that reproduces this behavior? A simple test case.
Also, we need to look into the VerifyApiKey problem - the one that leads to the NPE in the log.
