revision is deployed, but traffic cannot flow

Not applicable

I am working on a shared flow that a couple of my API proxies use. All was working OK till around revision 10 or 11 when I started getting an inexplicable error when trying to deploy a new version:

The revision is deployed, but traffic cannot flow. com.apigee.kernel.exceptions.spi.UncheckedException{ code = application.bootstrap.FailedToConfigure, message = Configuration failed, associated contexts = []}

In the new revision, I was trying to add a LookupCache operation to the flow. The associated PopulateCache was done and deployed OK.

When I got the error at first, I was on a sketchy WIFI, so I retried on a better connection. No change. Then, I thought it was something that would be "eventually consistent," so I tried waiting. That didn't help. Then, I searched and found this thread, and realized I was in trouble 😞

I tried the troubleshooting tips that @wwitman referred me to. Those didn't help. Specifically, I tried to force undeploy the shared flow and redeploy it. That didn't work. I checked the Apigee status page, but there didn't seem to be any indication of an outage. I then tried to create a new revision from the Project menu and deploy that; didn't work. I tried to make a new shared flow, downloaded the current revision (11) from the shared flow and uploaded that to the new shared flow. Got the same error. I then tried to upload revision 9, which I downloaded a couple days ago, to this new shared flow; that also produced the error above when I tried to deploy it. I could not see the history of the shared flow like the troubleshooting tips mention because that's not supported on shared flows, but I did delete the LookupCache operation (which is what I was working on before) and tried to deploy that; same error. I created a no-target API proxy and a shared flow, each that did nothing, and I could deploy those.

The username I'm using is the same as for this forum, the org is travis-eval, the name of the shared flow I'm having trouble with is called Introspection, the environment I'm trying to deploy to is prod.

It would be most appreciated to get some help unjamming this. Writing this policy has taken days and I need to use this in a POC we're conducting for a customer, so any help here would be most appreciated.

TIA!

Solved Solved
1 12 4,443
1 ACCEPTED SOLUTION

@Travis Spencer,

I figured out the issue in your API Proxy.

The condition has to be completely in a single line or if it goes to multiple lines then the condition i.e., either "and", "or" and the following expression has to be in the same line.

Here I will show the failing condition first:

<Condition>(introspectionResponse.header.content-Type =| "application/json") and
            introspectionPayload.jwt is null
</Condition>

In this example, the condition "and" and the expression "introspectionPayload.jwt is null" are in separate lines. So, I think this is not accepted and causes the failure.

I found the working conditions to be as follows:

Option #1

Have all the conditions in a single line as shown below:

<Condition>(introspectionResponse.header.content-Type =| "application/jwt") and introspectionPayload.jwt is null</Condition>

Option #2

If there are multiple conditions, then ensure that condition i.e., "and" in this example and expression in the same line

<Condition>(introspectionResponse.header.content-Type =| "application/json")
            and introspectionPayload.jwt is null
</Condition>

I have actually gone ahead and fixed the conditions in your SharedFlow "Introspection" and saved it as revision #13 and also deployed it in prod environment of your org successfully.

In summary, if you have multiple conditions it's better to use braces around each of the conditions and also have the condition and expression in the same line.

Please verify and let me know.

View solution in original post

12 REPLIES 12

@Travis Spencer,

Are you on Private Cloud or Public Cloud ?

If you are on Private Cloud, please check the Message Processor's system.log to see if there are any error messages/exceptions. Update the error message here, I will look further and get back to you.

If you are on Public Cloud, please let me know the org and API Proxy names via email. I will investigate further and get back to you.

Thanks,

Amar

public cloud, @AMAR DEVEGOWDA. Org name is travis-eval, shared flow is Introspection, environment is prod.

@Travis Spencer,

Thanks for sharing the details. I will look and get back to you.

@Travis Spencer,

  1. I checked out Message Processor logs and found that the deployment was failing because of the following exception:
    2017-11-23 09:11:04,498  Apigee-Main-6 ERROR MESSAGING.RUNTIME - AbstractConfigurator.loadXMLConfigurations() : Unable to Load default for path /organizations/travis-eval/apiproxies/Introspection/revisions/12/sharedflows/default
    2017-11-23 09:11:04,499  Apigee-Main-6 ERROR MESSAGING.RUNTIME - Application.sync() :  sync error for Introspection and revision 12
    2017-11-23 09:11:04,499  Apigee-Main-6 ERROR MESSAGING.RUNTIME - Application.sync() :  Actual Error
    com.apigee.expressions.parser.ParseException: Both the operands for EQUALS expression should be data expressions
            at com.apigee.expressions.parser.ExpressionParser.buildExpressionTree(ExpressionParser.java:337) ~[expressions-1.0.0.jar:na]
            at com.apigee.expressions.parser.ExpressionParser.parse(ExpressionParser.java:24) ~[expressions-1.0.0.jar:na]
            at com.apigee.expressions.parser.ExpressionParser.parseLogicExpression(ExpressionParser.java:28) ~[expressions-1.0.0.jar:na]
            at com.apigee.messaging.runtime.Step.getExpression(Step.java:67) ~[message-processor-1.0.0.jar:na]
            at com.apigee.messaging.runtime.Step.handleAdd(Step.java:58) ~[message-processor-1.0.0.jar:na]
            at com.apigee.messaging.runtime.SharedFlowRuntime.addStep(SharedFlowRuntime.java:81) ~[message-processor-1.0.0.jar:na]
            at com.apigee.messaging.runtime.SharedFlowRuntime.addSteps(SharedFlowRuntime.java:74) ~[message-processor-1.0.0.jar:na]
            at com.apigee.messaging.runtime.SharedFlowRuntime.handleAdd(SharedFlowRuntime.java:60) ~[message-processor-1.0.0.jar:na]
            at com.apigee.entities.AbstractConfigurator.handleUpdate(AbstractConfigurator.java:130) ~[config-entities-1.0.0.jar:na]
            at com.apigee.messaging.runtime.Application.handleUpdate(Application.java:241) ~[message-processor-1.0.0.jar:na]
            at com.apigee.entities.AbstractConfigurator.loadXMLConfigurations(AbstractConfigurator.java:214) ~[config-entities-1.0.0.jar:na]
            at com.apigee.messaging.runtime.Application.sync(Application.java:599) ~[message-processor-1.0.0.jar:na]
            at com.apigee.messaging.runtime.Application.sync(Application.java:559) ~[message-processor-1.0.0.jar:na]
            at com.apigee.messaging.runtime.ApplicationManager.handleUpdate(ApplicationManager.java:119) [message-processor-1.0.0.jar:na]
    	...<snipped>
    	
  2. This basically gave me a clue that some Condition flow must be having an issue where the operands are not matching.
  3. So I checked the ProxyEndpoint and found that there were multiple condition flows.
  4. Reviewed every condition flow and noticed that there were two condition flows which did not look right for me:
    <Condition>introspectionResponse.header.content-Type =| "application/jwt" and introspectionPayload.jwt is null</Condition>
    <Condition>introspectionResponse.header.content-Type =| "application/json" and introspectionPayload.jwt is null</Condition>
  5. I modified these two condition flows to have braces for the first condition as shown below:
    <Condition>(introspectionResponse.header.content-Type =| "application/jwt")   and introspectionPayload.jwt is null</Condition>
    <Condition>(introspectionResponse.header.content-Type =| "application/json")  and introspectionPayload.jwt is null</Condition>
  6. With this change, I was able to deploy the SharedFlow in my org successfully.

Can you please make the same change in your org, undeploy and deploy the SharedFlow ?

Thanks much, @AMAR DEVEGOWDA, for the reply!

I tried your suggestion, but it didn't work for me. Did you do something else as well? More specifically, I changed the above conditions and tried to deploy the policy but got the same error. I then added parenthesis around all expressions in all conditions and tried to deploy the policy -- same error. I then tried to do a force unload and deploy -- same error. I then made a new revision of the policy and tried to deploy that -- same error.

Am I missing something else in my policy that needs to be changed?

@Travis Spencer,

I figured out the issue in your API Proxy.

The condition has to be completely in a single line or if it goes to multiple lines then the condition i.e., either "and", "or" and the following expression has to be in the same line.

Here I will show the failing condition first:

<Condition>(introspectionResponse.header.content-Type =| "application/json") and
            introspectionPayload.jwt is null
</Condition>

In this example, the condition "and" and the expression "introspectionPayload.jwt is null" are in separate lines. So, I think this is not accepted and causes the failure.

I found the working conditions to be as follows:

Option #1

Have all the conditions in a single line as shown below:

<Condition>(introspectionResponse.header.content-Type =| "application/jwt") and introspectionPayload.jwt is null</Condition>

Option #2

If there are multiple conditions, then ensure that condition i.e., "and" in this example and expression in the same line

<Condition>(introspectionResponse.header.content-Type =| "application/json")
            and introspectionPayload.jwt is null
</Condition>

I have actually gone ahead and fixed the conditions in your SharedFlow "Introspection" and saved it as revision #13 and also deployed it in prod environment of your org successfully.

In summary, if you have multiple conditions it's better to use braces around each of the conditions and also have the condition and expression in the same line.

Please verify and let me know.

It's deployed now! Thanks, @AMAR DEVEGOWDA, for all the help!

And now it's back! These conditions are so incredibly brittle and when things go wrong, there's NO feedback about what is invalid. I'm writing normal conditions that any programmer would expect to work and I see nothing in the docs about special cases where things that I would expect to work actually don't. This is very frustrating!

Hi Amar,

I am facing similar deployment error for login-app proxy available in https://github.com/apigee/api-platform-samples/tree/master/sample-proxies/oauth-advanced bundle. I made all the required config changes and also verified that i did not leave any two lines conditions as you have mentioned in above answer. i could not figure out the issue. I do not face issue with other three proxies. Appreciate quick response from you or @anilsagar

Please find attached bundle.

The management api call for deployment says that the deployment on message processor failed.

{
    "environment": "test",
    "name": "login-app",
    "organization": "madhuwarriors-eval",
    "revision": [
        {
            "configuration": {
                "basePath": "/",
                "configVersion": "SHA-512:d3986819474ba2260cbe11fd8354c4eab48d3e15c5e17ce7b6484ede154e174c610fd426343c8d6394d15ce1fbe735b85dfaa193fb48d4f5dcf35ae5b1bfc7c7:ap-northeast-1",
                "steps": []
            },
            "name": "2",
            "server": [
                {
                    "errorCode": "scripts.node.FeatureNotEnabled",
                    "pod": {
                        "name": "rgce1mp001-7",
                        "region": "us-central1"
                    },
                    "status": "error",
                    "type": [
                        "message-processor"
                    ],
                    "uUID": "177cbcac-f138-4260-be04-2123990e3ad2"
                },
                {
                    "errorCode": "scripts.node.FeatureNotEnabled",
                    "pod": {
                        "name": "rgce1mp001-7",
                        "region": "us-central1"
                    },
                    "status": "error",
                    "type": [
                        "message-processor"
                    ],
                    "uUID": "cb63b7d6-d403-4ca1-ad36-716bb53acc2f"
                },
                {
                    "errorCode": "scripts.node.FeatureNotEnabled",
                    "pod": {
                        "name": "rgce1mp001-7",
                        "region": "us-central1"
                    },
                    "status": "error",
                    "type": [
                        "message-processor"
                    ],
                    "uUID": "7ff0616e-f8c7-4932-97d7-49e52c2ed34d"
                },
                {
                    "errorCode": "scripts.node.FeatureNotEnabled",
                    "pod": {
                        "name": "rgce1mp001-7",
                        "region": "us-central1"
                    },
                    "status": "error",
                    "type": [
                        "message-processor"
                    ],
                    "uUID": "b5409e53-79f9-4e6f-8191-959f9a52d9f4"
                },
                {
                    "pod": {
                        "name": "rgce1rt001-1",
                        "region": "us-central1"
                    },
                    "status": "deployed",
                    "type": [
                        "router"
                    ],
                    "uUID": "f3103d52-2765-4a0a-90fa-2b2af774c076"
                },
                {
                    "pod": {
                        "name": "rgce1rt001-1",
                        "region": "us-central1"
                    },
                    "status": "deployed",
                    "type": [
                        "router"
                    ],
                    "uUID": "03b8c66e-efc0-4b9c-a2ab-85a8bfa4c22c"
                },
                {
                    "pod": {
                        "name": "rgce1rt001-1",
                        "region": "us-central1"
                    },
                    "status": "deployed",
                    "type": [
                        "router"
                    ],
                    "uUID": "af8eb7dc-f661-4cf6-9196-55e6a1524c49"
                },
                {
                    "pod": {
                        "name": "rgce1rt001-1",
                        "region": "us-central1"
                    },
                    "status": "deployed",
                    "type": [
                        "router"
                    ],
                    "uUID": "2be7c14e-61cc-47c2-a83f-f3c38070151e"
                }
            ],
            "state": "error"
        }
    ]
}


It would be helpful if Apigee provides some information about the part of code that causes issues so that it would be easy for developers to identify issues and fix on their own most of the time. For getting the server logs, we need to reach out to support team.

It would be helpful if ...

It would be helpful if you could post new questions with the new question button. Not as comments on answers to old questions.

Hi Dino,

Please see above link for my new question. Can you help me with this issue asap?