I need help to understand how to filter the fields my in response data, using Javascript policies.

Not applicable

Hi, at first, thank you for your attention.

// ( FYI, not really the nerve of the problem 🙂

// I am currently using a custom API Proxy, interacting with my Apigee BaaS thanks to built in Node.JS // & usergrid SDK.

My proxy just GET a few data about a list of events that are stocked in my BaaS.

So if I make this request: (GET) https://7v7v-test.apigee.net/demo/evenements , this is the actual response :

	[
    {
        "uuid": "9e804250-7d0a-11e7-8e06-12c1e35638f0",
        "type": "evenement",
        "name": "test3",
        "created": 1502287022044,
        "modified": 1502287022044,
        "metadata": {
            "path": "/evenements/9e804250-7d0a-11e7-8e06-12c1e35638f0",
            "size": 322
        }
    },
    {
        "uuid": "930c484b-7d0a-11e7-8e06-12c1e35638f0",
        "type": "evenement",
        "name": "test2",
        "created": 1502287002829,
        "modified": 1502287002829,
        "metadata": {
            "path": "/evenements/930c484b-7d0a-11e7-8e06-12c1e35638f0",
            "size": 322
        }
    },
    {
        "uuid": "8c812c5f-7d0a-11e7-9ea2-122e0737977d",
        "type": "evenement",
        "name": "test",
        "created": 1502286991851,
        "modified": 1502286991851,
        "metadata": {
            "path": "/evenements/8c812c5f-7d0a-11e7-9ea2-122e0737977d",
            "size": 321
        }
    }
]

OK that's great. Now I would like to filter some data in each object of the response, example: "uuid", "type", "created", "modified", "metadata.size".

I am actually using these Javascript fields filtering scripts & policies from this source: (an Apigee Developer):

applyFieldFilter.js (GitHub)

fieldFiltering.js (GitHub)

__________________________________________________________________________________

So I uploaded these scripts on my Apigee proxy, linked a custom policy to it, following the guidelines of the scripts commentaries.

Once these filtering scripts & polices applied, here comes the problem.

Here I'll show you how my Filtering policy looks like for the following request:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Javascript timeLimit="800" name="fieldfiltering">
    <Properties>
        <Property name="fields">uuid</Property>
        <Property name="action">exclude</Property>
    </Properties>
    <IncludeURL>jsc://fieldFiltering.js</IncludeURL>
    <ResourceURL>jsc://applyFieldFilter.js</ResourceURL>
</Javascript>

For the moment I only try to filter the "uuid".

Now when I make my request (GET) https://7v7v-test.apigee.net/demo/evenements, the response looks like this:

{
    "0": {
        "uuid": "9e804250-7d0a-11e7-8e06-12c1e35638f0",
        "type": "evenement",
        "name": "test3",
        "created": 1502287022044,
        "modified": 1502287022044,
        "metadata": {
            "path": "/evenements/9e804250-7d0a-11e7-8e06-12c1e35638f0",
            "size": 322
        }
    },
    "1": {
        "uuid": "930c484b-7d0a-11e7-8e06-12c1e35638f0",
        "type": "evenement",
        "name": "test2",
        "created": 1502287002829,
        "modified": 1502287002829,
        "metadata": {
            "path": "/evenements/930c484b-7d0a-11e7-8e06-12c1e35638f0",
            "size": 322
        }
    },
    "2": {
        "uuid": "8c812c5f-7d0a-11e7-9ea2-122e0737977d",
        "type": "evenement",
        "name": "test",
        "created": 1502286991851,
        "modified": 1502286991851,
        "metadata": {
            "path": "/evenements/8c812c5f-7d0a-11e7-9ea2-122e0737977d",
            "size": 321
        }
    },
    "diagnostics": {
        "productName": null,
        "appName": null,
        "proxyName": "test_update_evenements",
        "filterFields": [
            "uuid"
        ],
        "filterAction": "exclude"
    }
}
The problem is that the scripts adds a generic ID to each object, and therefore the parameter "uuid" of my policy is not taken in account:
    <Properties>
        <Property name="fields">uuid</Property>
        <Property name="action">exclude</Property>
    </Properties>

Now I can get it to delete the "uuid" of the object 0 and 1 if I pass as parameters to my policy:

    <Properties>
        <Property name="fields">0.uuid,1.uuid</Property>
        <Property name="action">exclude</Property>
    </Properties>

Here see in the response 0.uuid & 1.uuid got filtered:

{
    "0": {
        "type": "evenement",
        "name": "test3",
        "created": 1502287022044,
        "modified": 1502287022044,
        "metadata": {
            "path": "/evenements/9e804250-7d0a-11e7-8e06-12c1e35638f0",
            "size": 322
        }
    },
    "1": {
        "type": "evenement",
        "name": "test2",
        "created": 1502287002829,
        "modified": 1502287002829,
        "metadata": {
            "path": "/evenements/930c484b-7d0a-11e7-8e06-12c1e35638f0",
            "size": 322
        }
    },
    "2": {
        "uuid": "8c812c5f-7d0a-11e7-9ea2-122e0737977d",
        "type": "evenement",
        "name": "test",
        "created": 1502286991851,
        "modified": 1502286991851,
        "metadata": {
            "path": "/evenements/8c812c5f-7d0a-11e7-9ea2-122e0737977d",
            "size": 321
        }
    },
    "diagnostics": {
        "productName": null,
        "appName": null,
        "proxyName": "test_update_evenements",
        "filterFields": [
            "0.uuid",
            "1.uuid"
        ],
        "filterAction": "exclude"
    }
}

As you may have guessed it, I do not want to refer individually to each entity in the parameters.

I simply don't know how to refer to these "0", "1", "2"... objects, with only one parameter that will apply indefinitely to each object.

OVERALL I would prefer to NOT have these numeric fields before my objects.

Part of the problem is that it transforms my array into 1 object, and that is not how I want my API response to be like.

Another solution may be to not create these "0", "1", "2"... objects ? So I could directly refer to the fields to filter ?

The answer is maybe very basic but I lack of knowledge in programming ! 😄

Thank you for not being afraid of my long post !


I tried to detail & explain my problem as much as possible, if you have any question or tip please feel to comment.

@Anil Sagar @Dino

Solved Solved
2 5 1,645
1 ACCEPTED SOLUTION

Hi @HABIBI AGDAM Maxime

I'm not afraid of your long post! Thanks for all the detail. Very clear.

I understand that you want to eliminate fields like uuid, type, created, modified, and so on. I understand that you are seeing the array transformed into a hash with properties like "0", "1", and so on.

The field filtering code that you are using - code that I wrote - did not consider that the input might be an array. It assumes the input is a hash, and generates a hash as output. Those hash properties are the result.

Let me look at the JavaScript to see if I can make it a little smarter.

Later....

OK, Can you try the attached files?

Just replace them with what you obtained form github.

modfied-filter-logic.zip

When I try this with your input, I get this result:

[
  {
    "type": "evenement",
    "name": "test3",
    "created": 1502287022044,
    "modified": 1502287022044,
    "metadata": {
      "path": "/evenements/9e804250-7d0a-11e7-8e06-12c1e35638f0",
      "size": 322
    }
  },
  {
    "type": "evenement",
    "name": "test2",
    "created": 1502287002829,
    "modified": 1502287002829,
    "metadata": {
      "path": "/evenements/930c484b-7d0a-11e7-8e06-12c1e35638f0",
      "size": 322
    }
  },
  {
    "type": "evenement",
    "name": "test",
    "created": 1502286991851,
    "modified": 1502286991851,
    "metadata": {
      "path": "/evenements/8c812c5f-7d0a-11e7-9ea2-122e0737977d",
      "size": 321
    }
  }
]

...which I think is what you want.

Also: I have updated the github repo with these changes. if you just "git pull", you will get these updates.

View solution in original post

5 REPLIES 5

Hi @HABIBI AGDAM Maxime

I'm not afraid of your long post! Thanks for all the detail. Very clear.

I understand that you want to eliminate fields like uuid, type, created, modified, and so on. I understand that you are seeing the array transformed into a hash with properties like "0", "1", and so on.

The field filtering code that you are using - code that I wrote - did not consider that the input might be an array. It assumes the input is a hash, and generates a hash as output. Those hash properties are the result.

Let me look at the JavaScript to see if I can make it a little smarter.

Later....

OK, Can you try the attached files?

Just replace them with what you obtained form github.

modfied-filter-logic.zip

When I try this with your input, I get this result:

[
  {
    "type": "evenement",
    "name": "test3",
    "created": 1502287022044,
    "modified": 1502287022044,
    "metadata": {
      "path": "/evenements/9e804250-7d0a-11e7-8e06-12c1e35638f0",
      "size": 322
    }
  },
  {
    "type": "evenement",
    "name": "test2",
    "created": 1502287002829,
    "modified": 1502287002829,
    "metadata": {
      "path": "/evenements/930c484b-7d0a-11e7-8e06-12c1e35638f0",
      "size": 322
    }
  },
  {
    "type": "evenement",
    "name": "test",
    "created": 1502286991851,
    "modified": 1502286991851,
    "metadata": {
      "path": "/evenements/8c812c5f-7d0a-11e7-9ea2-122e0737977d",
      "size": 321
    }
  }
]

...which I think is what you want.

Also: I have updated the github repo with these changes. if you just "git pull", you will get these updates.

Thank you for you attention Dino. I am in vacations at the moment and will try out your updated solution & post my answer when I will be back.
Maxime

Right on - thanks for the note. I look forward to hearing back from you, if you get a chance to try it out!

Wowowow, thank you Dino ! It works like a charm !

I can now apply awesome field filtering policies on my API responses !

Just to let you know, the "applyFieldFilter.js" contained in your .zip package did not contain the necessary modifications in the code commentaries about "source & destination" so I had to check it up on the GitHub repo to make it work, but once that done, it did work perfectly.

Link to the GitHub File

Thank you for your time&work,
Maxime

Hi @HABIBI AGDAM Maxime - did my answer satisfy?