Error in deployment for environment test. The revision is deployed and traffic can flow, but flow may be impaired. Call timed out

[#1]

Hi, I am new to Apigee Edge. I have just created the account and I am trying to deploy one of the two default API proxies weatherapi. I keep getting the error in the subject of this post. I have tried to deploy my own very simple API proxy but I get the same message. Note: right after creating the account a few days ago everything worked fine. It does not work since Friday I think. I am evaluating Apigee against Layer7 Thanks

[#2 · Accepted solution]

This should be resolved, it was related to a infra glitch. Please let us know if you still see this issue. We are actively looking at it right now. We are monitoring if any of the the orgs are facing this issue.

Thank you.

Sincere apologies for the inconvenience caused

[anilsr]

Dear @Nico ,

Welcome to Apigee Community .

Inconvenience is regretted. We have earlier seen this issue and our Engineering Team is working on same. Actually, Runtime traffic is not affected because of above error, you can still call your API and it should work as expected. Above message comes up in Edge UI if there is a time out or congestion issue with any of the routers in the free pod. It's a temporary issue and you should able to work with API Proxy as expected. I will let Engineering team provide detailed root cause for same.

Cheers,

Anil Sagar

[#4 · Accepted solution]

This should be resolved, it was related to a infra glitch. Please let us know if you still see this issue. We are actively looking at it right now. We are monitoring if any of the the orgs are facing this issue.

Thank you.

Sincere apologies for the inconvenience caused

[#5]

@sriki77

I am trying to get my first api proxy running by following this doc - http://apigee.com/docs/api-services/tutorials/add-...

I am pretty sure I have followed the steps properly, but I get this error when deploy to prod or test environment.

Error in deployment for environment test.

The revision is deployed, but traffic cannot flow. Received an unknown event with description SYNC Application /organizations/veeseekay/apiproxies/weather/revisions/1/; and Call timed out

When I try hitting the url, i get a different error

http://veeseekay-test.apigee.net/v1/weather

{"fault":{"faultstring":"Internal server error Received unknown request for Organization:veeseekay;Environment:test;APIProxy:weather;Revision:1","detail":{"errorcode":"messaging.adaptors.http.ServerError"}}}

[#6]

Has this issue returned? I can't get anything deployed this morning. Thanks.

[sebastian_kraje]

Still have this problem on my 3 independent instances (4.15.07.03) from time to time, but yesterday had it 3 times during 8 hours work. I'm preparing to migrate to 4.16, hope it will fix it.

[#8]

Hi Team ,

Our team recently craeted 4 Apigee E learning users and when they try to deploy simple created API proxy we are also getting same issue but for some of users it is working fine

Error in deployment for environment test.
The revision is deployed and traffic can flow, but flow may be impaired. Call timed out; either server is down or server is not reachable

Please could you help to resolve this issue ASAP, our POC is stopped due to this issue.

[sebastian_kraje]

I had this problem with 4.15.07.03 version, after upgrade 4.16.01 problem disappeared.

[sebastian_kraje · Accepted solution]

Have similar from some time, restart of management-server and message-processor fix problem for ~300h

My version is 4.15.07.03 (Private Cloud) , each apigee component have on different host,

(on v4.15.04.03 problem exist too)

Error in UI during deploy:

Error in deployment for environment prod.

The revision is deployed, but traffic cannot flow. Call timed out; either server is down or server is not reachable.

/apigee4/var/log/apigee/management-server/logs/system.log:

2016-03-30 02:58:38,587 org:merck env:prod qtp1237989929-96102 INFO DISTRIBUTION - RemoteServicesDeploymentHandler.deploy() : RemoteServicesDeploymentHandler.deploy : Deployment request : DeployEvent{organization='merck', application='up-test', applicationRevision='1', deploymentSpec=basepath=/;env=prod;, deploymentID=null}

2016-03-30 02:58:48,645 org:merck env:prod qtp1237989929-96102 ERROR DISTRIBUTION - RemoteServicesDeploymentHandler.deployToServers() : RemoteServicesDeploymentHandler.deployToServers : Deployment exception for server with uuid 5de18053-0512-4230-9e32-add42c75bfd6 : cause = Call timed out

com.apigee.rpc.RPCException: Call timed out

at com.apigee.rpc.impl.AbstractCallerImpl.handleTimeout(AbstractCallerImpl.java:64) ~[rpc-1.0.0.jar:na]

at com.apigee.rpc.impl.RPCMachineImpl$OutgoingCall.handleTimeout(RPCMachineImpl.java:483) ~[rpc-1.0.0.jar:na]

at com.apigee.rpc.impl.RPCMachineImpl$OutgoingCall.access$000(RPCMachineImpl.java:402) ~[rpc-1.0.0.jar:na]

at com.apigee.rpc.impl.RPCMachineImpl$OutgoingCall$1.run(RPCMachineImpl.java:437) ~[rpc-1.0.0.jar:na]

at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:532) ~[netty-all-4.0.0.CR1.jar:na]

at io.netty.util.HashedWheelTimer$Worker.notifyExpiredTimeouts(HashedWheelTimer.java:430) ~[netty-all-4.0.0.CR1.jar:na]

at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:371) ~[netty-all-4.0.0.CR1.jar:na]

at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_80]

2016-03-30 02:58:48,797 org:merck env:prod qtp1237989929-96102 INFO DISTRIBUTION - RemoteServicesDeploymentHandler.deployToMPsAndRouters() : RemoteServicesDeploymentHandler.deployToMPsAndRouters() - MPs deployment success - deploying to routers

On message-processor:

$ check.sh

Check Processes:

----------------

message-processor (pid 11250) is running...

Check Services:

---------------

Message Processor:

UUID=5de18053-0512-4230-9e32-add42c75bfd6

==> 200

$ curl localhost:8082/v1/servers/self

{

"buildInfo" : {

"buildNumber" : "jenkins-4G_Release_Sanity-1194 ",

"buildTimestamp" : "1449109653537",

"rPMName" : "apigee-rpm-1.0.0.1194.8f421a2.1512030212-1507_01",

"releaseID" : "1507_01",

"sCMBranch" : "origin/OPDK_1507",

"sCMRevision" : "8f421a25aa6c8563a2641d0d5054face9787b627"

},

"externalHostName" : "localhost",

"externalIP" : "xx.xx.xx.xx",

"internalHostName" : "localhost",

"internalIP" : "xx.xx.xx.xx",

"isUp" : true,

"pod" : "gateway",

"reachable" : true,

"region" : "dc-1",

"tags" : {

"property" : [ {

"name" : "jmx.rmi.port",

"value" : "1101"

}, {

"name" : "http.port",

"value" : "8998"

}, {

"name" : "http.management.port",

"value" : "8082"

}, {

"name" : "started.at",

"value" : "1458121818267"

}, {

"name" : "http.ssl.flag",

"value" : "true"

}, {

"name" : "rpc.port",

"value" : "4528"

}, {

"name" : "Profile",

"value" : "MessageProcessor"

}, {

"name" : "up.time",

"value" : "333 hour(s) 56 minute(s) 14 second(s)"

}, {

"name" : "startup.time",

"value" : "2 minute(s) 25 second(s)"

} ]

},

"type" : [ "message-processor" ],

"uUID" : "5de18053-0512-4230-9e32-add42c75bfd6"

}

On management-server

$ check.sh

Check Processes:

----------------

management-server (pid 14471) is running...

Check Services:

---------------

Management Server:

UUID=3c6f5716-d359-4663-953c-c7b85a93cce3

==> 200

Check Registrations:

--------------------

Enter Apigee Edge admin password:

(...)

Next check pod gateway.

--> curl -X GET http://localhost:8080/v1/servers?pod=gateway [...]

[ {

"internalIP" : "xxx.xxx.xxx.xxx",

"isUp" : true,

"pod" : "gateway",

"reachable" : true,

"region" : "dc-1",

"tags" : {

"property" : [ ]

},

"type" : [ "dc-datastore", "cache-datastore", "keyvaluemap-datastore", "kms-datastore", "counter-datastore" ],

"uUID" : "c6d1c0d2-ffa5-4ffe-b950-9b06b75e41eb"

}, {

"externalHostName" : "localhost",

"externalIP" : "xxx.xxx.xxx.xxx",

"internalHostName" : "localhost",

"internalIP" : "xxx.xxx.xxx.xxx",

"isUp" : true,

"pod" : "gateway",

"reachable" : true,

"region" : "dc-1",

"tags" : {

"property" : [ {

"name" : "started.at",

"value" : "1458121713116"

}, {

"name" : "startup.time",

"value" : "14 second(s)"

}, {

"name" : "Profile",

"value" : "Router-Netty"

}, {

"name" : "rpc.port",

"value" : "4527"

}, {

"name" : "http.management.port",

"value" : "8081"

}, {

"name" : "jmx.rmi.port",

"value" : "1100"

}, {

"name" : "up.time",

"value" : "333 hour(s) 54 minute(s) 50 second(s)"

} ]

},

"type" : [ "router" ],

"uUID" : "547ac4d4-8557-42dc-9409-8912833dad0a"

}, {

"internalIP" : "xxx.xxx.xxx.xxx",

"isUp" : true,

"pod" : "gateway",

"reachable" : true,

"region" : "dc-1",

"tags" : {

"property" : [ ]

},

"type" : [ "dc-datastore", "cache-datastore", "keyvaluemap-datastore", "kms-datastore", "counter-datastore" ],

"uUID" : "a55c0c41-e337-4963-ab2e-ce5ac24c05b6"

}, {

"internalIP" : "xxx.xxx.xxx.xxx",

"isUp" : true,

"pod" : "gateway",

"reachable" : true,

"region" : "dc-1",

"tags" : {

"property" : [ ]

},

"type" : [ "dc-datastore", "cache-datastore", "keyvaluemap-datastore", "kms-datastore", "counter-datastore" ],

"uUID" : "3b6939c9-2551-4537-a8ec-291a0c5ed434"

}, {

"externalHostName" : "localhost",

"externalIP" : "xxx.xxx.xxx.xxx",

"internalHostName" : "localhost",

"internalIP" : "xxx.xxx.xxx.xxx",

"isUp" : true,

"pod" : "gateway",

"reachable" : true,

"region" : "dc-1",

"tags" : {

"property" : [ {

"name" : "jmx.rmi.port",

"value" : "1101"

}, {

"name" : "http.port",

"value" : "8998"

}, {

"name" : "http.management.port",

"value" : "8082"

}, {

"name" : "started.at",

"value" : "1458121818267"

}, {

"name" : "http.ssl.flag",

"value" : "true"

}, {

"name" : "rpc.port",

"value" : "4528"

}, {

"name" : "Profile",

"value" : "MessageProcessor"

}, {

"name" : "startup.time",

"value" : "2 minute(s) 25 second(s)"

}, {

"name" : "up.time",

"value" : "333 hour(s) 53 minute(s) 5 second(s)"

} ]

},

"type" : [ "message-processor" ],

"uUID" : "5de18053-0512-4230-9e32-add42c75bfd6"

} ]

==> 200

(...)

[#11]

Thanks for your posting. We were having this issue in our OPDK environment (4.15.07.03). Recycling the management server and mp resolved the issue.

[#12]

Hi Sebastian, This is not our apigee environment, this is E-Learning apigee instance and we can't upgrade or restart this environment, I think Apigee team only can do this and the url we use to login E-Learning apigee is https://login.apigee.com/login.

How can I report this issue to Apigee E-leaning maintenance Team

[sebastian_kraje]

In this case you should raise ticket on Apigee Support