Hi everyone,
We have a Apigee hybrid multi region deployment (One GCP Project/Apigee Org connected to two runtimes with a cassandra ring connecting them).
We are seeing the following error in stackdriver logs when we deployed our second region to the Apigee Organization in GCP. The proxies are getting deployed in both regions and I can invoke them in each region. But the trace functionality does not seem to work when we invoke the API proxies against the second region.
{"level":"SEVERE","thread":"Apigee-Timer-4","mdc":{},"className":"com.apigee.hybrid.runtime.signals.trace.sync.pubsub.PubSubContext","method":"createSubscription","severity":"SEVERE","message":"Upstream subscription creation request failed due to error with status code : 429 reason : Too Many Requests response : {\n \"error\": {\n \"code\": 429,\n \"message\": \"Quota exhausted for subscriptions per project\",\n \"status\": \"RESOURCE_EXHAUSTED\",\n \"details\": [\n {\n \"@type\": \"type.googleapis.com/google.rpc.RequestInfo\",\n \"requestId\": \"1320799346334806466\"\n }\n ]\n }\n}\n","formattedDate":"2023-12-14T15:58:33.142Z","logger":"CONFIG-CHANGE"}
{"level":"SEVERE","thread":"Apigee-Timer-4","mdc":{},"className":"com.apigee.hybrid.runtime.signals.context.SignalsContextImpl","method":"initializePubSubContext","severity":"SEVERE","message":"ControlPlaneCommunicationFailure exception occurred. Backing off signal polling","formattedDate":"2023-12-14T15:58:33.143Z","logger":"CONFIG-CHANGE","exceptionStackTrace":"com.apigee.hybrid.runtime.contract.sync.ControlPlaneCommunicationFailure{ code \u003d runtime.contract.sync.SubscriptionRequestError, message \u003d Signals subscription request with control plane failed, error msg: Too Many Requests, associated contexts \u003d []}\n"}
Appreciate any pointers to identify what is causing this issue.
Many Thanks
LinkedIn
Twitter
