Apigee X Set Integration Request policy throwing error on JSON

[mglenney]

I am trying to setup a POC of an Apigee X endpoint that acts as a webhook sending data into a GCP Pub/Sub topic via an integration.  I have my Integration Request Policy set as follows:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<SetIntegrationRequest continueOnError="false" enabled="true" name="set-integration-request">
    <DisplayName>Set Integration Request</DisplayName>
    <ProjectId>dev-pudgy-penguin</ProjectId>
    <IntegrationName>github-webhook-pubsub2</IntegrationName>
    <IntegrationRegion>us-east1</IntegrationRegion>
    <ApiTrigger>api_trigger/github-webhook-pubsubs_API_1</ApiTrigger>>
    <Parameters>
        <Parameter name="request_body" type="json" ref="request.content"/>
    </Parameters>
</SetIntegrationRequest>

I'm getting the following error on that policy when looking at the debug:

curl -X POST https://34.160.151.177.nip.io/v1/github-webhook2 -H "Content-Type: application/json" -d @sample_github.json
{"fault":{"faultstring":"Unresolved variable : sha","detail":{"errorcode":"entities.UnresolvedVariable"}}}

The `request.content` is a JSON being passed in (from Github webhook) that has things like this in it:

"blobs_url": "https://git.company.com/api/v3/repos/mike-glenney/fluffy-dollop/git/blobs{/sha}"

I "think" this is what is causing the error above. I'm assuming Apigee is seeing the {} and trying to extract that as a variable.  The JSON is valid according to my linters and I'm setting type="json".

Thanks,

MG

[ssvaidyanathan]

@mglenney 

I just tried with a simple Integration (with Email task instead of Pub/Sub) and it worked as expected.

I created a variable called request_body in my Integration. While creating the variable, I provided the following as the sample payload

{
	"foo": "bar",
	"blobs_url": "https://git.company.com/api/v3/repos/mike-glenney/fluffy-dollop/git/blobs{/sha}"
}

And then used the same variable as the Email body to be sent

On the Apigee proxy side, I created a simple API proxy with Integration target and in the SetIntegrationRequest policy, I have

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<SetIntegrationRequest continueOnError="false" enabled="true" name="set-integration-request">
    <DisplayName>Set Integration Request</DisplayName>
    <ProjectId>dummy</ProjectId>
    <IntegrationName>SSV_Email</IntegrationName>
    <IntegrationRegion>us-west1</IntegrationRegion>
    <ApiTrigger>api_trigger/SSV_Email_API_1</ApiTrigger>
    <Parameters>
        <Parameter name="request_body" type="json" ref="request.content"/>
    </Parameters>
</SetIntegrationRequest>

When I ran the following cURL, it worked

curl -X POST \
  https://api-dev.bap-amer-west-demo1.net/testemail \
  -H 'content-type: application/json' \
  -d '{
	"foo": "bar",
	"blobs_url": "https://git.company.com/api/v3/repos/mike-glenney/fluffy-dollop/git/blobs{/sha}"
}'

I got the following response

{
    "executionId": "e16da911-bd7d-4960-98aa-b1a0bcf27a26",
    "eventParameters": {
        "parameters": [
            {
                "key": "request_body",
                "value": {
                    "jsonValue": "{\n  \"foo\": \"bar\",\n  \"blobs_url\": \"https://git.company.com/api/v3/repos/mike-glenney/fluffy-dollop/git/blobs{/sha}\"\n}"
                },
                "dataType": "JSON_VALUE"
            }
        ]
    },
    "outputParameters": {
        "request_body": {
            "foo": "bar",
            "blobs_url": "https://git.company.com/api/v3/repos/mike-glenney/fluffy-dollop/git/blobs{/sha}"
        }
    }
}

Not sure whats wrong on your setup? Do you have any other policy in your proxy apart from the SetIntegrationRequest? If you have any policy like Extract Variables, try setting the IgnoreUnresolvedVariables property to true

<IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>

[mglenney]

I'm not sure.  Maybe the error message is a red herring.  When I use the JSON you used, it's fine.  But with the full webhook content from Github I get the error.  That {/sha} shows up on 7 different lines in the JSON.  The content is 7K.  Perhaps that's the issue

[ssvaidyanathan]

@mglenney - I am able to reproduce this issue. Let me get back to you on this

[ssvaidyanathan]

@mglenney - I don't think its the payload size, I did some more research. I found the sample payload from the GitHub docs. The issue is with any string having {} without the "/" for example

"statuses_url": "https://api.github.com/repos/octo-org/octo-repo/statuses/{sha}"

The following works

"statuses_url": "https://api.github.com/repos/octo-org/octo-repo/statuses{/sha}"

and 

"statuses_url": "https://api.github.com/repos/octo-org/octo-repo/statuses/{/sha}"

but

"statuses_url": "https://api.github.com/repos/octo-org/octo-repo/statuses/{sha}"

does not. I have opened a bug with the team. Will provide an update once its resolved. Thanks for bringing this up

[tanyakhera]

@mglenney this issue has been fixed, a payload having characters enclosed within `{` and `}` should work without issue now. This issue occurred previously because the Set Integration Request policy would treat strings enclosed within `{` and `}` as a flow variable and try to resolve it. This behaviour has been changed.

Note: Now, the Set Integration Request policy specifically, treats strings enclosed within `#$` and `$#` in the payload as flow variables.