Proxy Deployment Error

[#1]

I've been trying to deploy a very simple proxy for a week now into our production environment. I have deployed other proxies successfully, but this one always fails. Can anyone make any sense out of the following message? Also, now I can't undeploy/redeploy.

Error in deployment for environment prod. The revision is deployed and traffic can flow, but flow may be impaired. com.apigee.zookeeper.ZooKeeperException{ code = zookeeper.ErrorSettingData, message = Error while setting data for path : /organizations/Shutterfly/environments/prod/apiproxies/shipping-prod/revisions/3/statuses/78ebd00d-2663-4596-a2de-27df6c47e93a/status, associated contexts = []}; and com.apigee.kernel.exceptions.spi.UncheckedException{ code = application.bootstrap.FailedToConfigure, message = Configuration failed, associated contexts = []}

status:

{ "aPIProxy": "shipping-prod", "environment": [ { "configuration": { "basePath": "/", "steps": [] }, "name": "prod", "server": [ { "status": "undeployed", "type": [ "message-processor" ], "uUID": "7dbfb2ad-8857-43c7-b292-275e538083c7" }, { "status": "undeployed", "type": [ "message-processor" ], "uUID": "930f2499-3d18-4fe5-8de6-4eafde694ae3" }, { "status": "undeployed", "type": [ "message-processor" ], "uUID": "b1ba53df-9976-41ef-bf4f-08be157938a2" }, { "status": "undeployed", "type": [ "message-processor" ], "uUID": "c4ed1436-b7c5-4fc0-b632-28df876d29c6" }, { "status": "undeployed", "type": [ "message-processor" ], "uUID": "9572536e-d8fb-4c9d-ad62-8be983f63296" }, { "status": "undeployed", "type": [ "message-processor" ], "uUID": "1d310304-6b9b-4162-848b-ea6eb88d7895" }, { "status": "undeployed", "type": [ "router" ], "uUID": "3322d9ce-66cd-4c84-9e1f-236518062ed4" }, { "status": "undeployed", "type": [ "router" ], "uUID": "8ca2c2bf-982c-4ae1-8b8a-6a9003b1f041" }, { "status": "undeployed", "type": [ "router" ], "uUID": "2f63ab90-221d-4054-a96a-66c3c79bdb64" }, { "error": "com.apigee.zookeeper.ZooKeeperException{ code = zookeeper.ErrorSettingData, message = Error while setting data for path : /organizations/Shutterfly/environments/prod/apiproxies/shipping-prod/revisions/3/statuses/78ebd00d-2663-4596-a2de-27df6c47e93a/status, associated contexts = []}", "status": "error", "type": [ "router" ], "uUID": "78ebd00d-2663-4596-a2de-27df6c47e93a" }, { "status": "undeployed", "type": [ "router" ], "uUID": "a1802e7d-8c23-47e7-8abb-e32cb97c13f1" }, { "status": "undeployed", "type": [ "router" ], "uUID": "4a4b2e88-c486-4381-8f40-ae627e197205" }, { "error": "com.apigee.kernel.exceptions.spi.UncheckedException{ code = application.bootstrap.FailedToConfigure, message = Configuration failed, associated contexts = []}", "status": "error", "type": [ "router" ], "uUID": "fe55b507-9399-40bf-9b2d-bd99ff088a25" } ], "state": "error" } ], "name": "3", "organization": "Shutterfly" }

[williamssean]

Hello @Jonathan Baney

Can you share the proxy? I can try to deploy it to my org.

[#3]

isalive-rev1-2015-06-16.zip A super simple proxy with a single AssignMessage policy and no target. This is failing

[#4]

From the zookeeper log:

2016-06-01 09:35:26,452 [myid:3] - INFO [ProcessThread(sid:3 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x25429fcdd150000 type:create cxid:0x81c6 zxid:0x140000179e txntype:-1 reqpath:n/a Error Path:/organizations/Shutterfly/environments/prod/apiproxies/IsAlive-prod/revisions/2/statuses/78ebd00d-2663-4596-a2de-27df6c47e93a Error:KeeperErrorCode = NodeExists for /organizations/Shutterfly/environments/prod/apiproxies/IsAlive-prod/revisions/2/statuses/78ebd00d-2663-4596-a2de-27df6c47e93a

[williamssean]

I was able to deploy the proxy to my org, but with a few changes.

1) I had to remove 2 of the virtual hosts, since they are not defined in my org. It doesn't seem like you are having any issues with the virtual host.

2) I changed the AssignMessage policy as shown below.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<AssignMessage async="false" continueOnError="false" enabled="true" name="AssignMessageOK">
    <DisplayName>AssignMessage.OK</DisplayName>
    <FaultRules/>
    <Properties/>
    <Set>
        <Payload content-type="text/plain">OK</Payload>
    </Set>
    <IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
    <AssignTo createNew="false" transport="http" type="response"/>
</AssignMessage>

I was able to deploy it and I received the OK response.

[#6]

Thanks for trying this out, but the issue isn't the proxy, but appears to be centered around zookeeper which we are currently working with apigee support to address.

[#7]

Hi Jonathan,

At the moment I have the same issue with my Zookeepers. I can see the same errors.

Did you find any resolution around this problem?

Thanks,

Rez

[#8]

I guess to a few bad upgrades, we had some cruft in the system (message processors that no longer existed, stuff like that). So when I would deploy, I'd get the failure. We did a clean up of all of the registered routers and message processors and that seemed to really help. In addition, we increased the rpc timeout value from 10 seconds (default) to 30 seconds and that seems to also to have helped tremendously.

We had one proxy in particular where the target call can take up to 5 - 6 seconds (sometimes a bit longer) and that was causing the rpc tmeout problem, so we'd always have one message processor that would fail, waiting for traffic to drain and all that. That's the best answer I can give at this time.