How to find all the proxies using a particular shared flow (find dependent Item list)

joydeep02780
Participant II

Dear All,

Is there a way to find a particular item is used by what other items? for example if a shared flow is used by many proxies, how to find all those proxies. So that if I want to modify the flow i can make sure there is no impact to the proxies using this shared flow.

Similarly it can be any other Apigee item, like any policy etc.

Thanks in advance!

3 21 2,409
21 REPLIES 21

Not applicable

@Joydeep Paul,

As a principle, Apigee platform provides management APIs to retrieve information for different entities. Information retrieved via management APIs can be parsed to fulfill required objective.

For example, for 'shared flow', there is no OOTB utility available. However, as I said earlier, there are management APIs available to retrieve this information and perform string operations accordingly.

For example, 'Shared Flow' can be invoked by two mechanism: Flow Hook and Flow callout policy.

1. To find list of shared flows from Flow Hook, management APIs can be used.

2. For second mechanism i.e. FlowCallout, there is another management API to find list of policies for a specific revision of a specific API Proxy. This management API will provides list of policies by name. So by standardizing policy name, one can get 'FlowCallout' policy. Now, there is another management API to retrieve details of this specific policy.

Cheers,
Rajesh Doda

@Rajesh Doda,

Thank you very much! I will try using management API and update incase any issue.

@Joydeep Paul

I realize this is an old question, but I just came across @Dino-at-Google's EdgeTools repo which should help.

For example:

$ node ./findPoliciesByName.js -o $ORG -n -R FC-Log
Apigee Edge Policy finder tool, version: 20170822-0958
Node.js v10.15.0


[2019-Feb-25 14:48:17] start
[2019-Feb-25 14:48:18] api: helloworld-v1 []
[2019-Feb-25 14:48:19] api: pingstatus-v1-sharedflows ["apis/pingstatus-v1-sharedflows/revisions/1/policies/FC-Log"]
[2019-Feb-25 14:48:19] api: pingstatus-v1 []
[2019-Feb-25 14:48:24] sharedflow: PreProxy []
[2019-Feb-25 14:48:24] sharedflow: Log []
[2019-Feb-25 14:48:25] sharedflow: PreTarget []
[2019-Feb-25 14:48:25] sharedflow: GetLogValues []
[2019-Feb-25 14:48:26] sharedflow: ProxyDefaultFaultRule ["sharedflows/ProxyDefaultFaultRule/revisions/1/policies/FC-Log"]
[2019-Feb-25 14:48:26] sharedflow: PostTarget []
[2019-Feb-25 14:48:26] sharedflow: PostProxy ["sharedflows/PostProxy/revisions/1/policies/FC-Log"]
[2019-Feb-25 14:48:27] sharedflow: ProxyFaultRules []
[2019-Feb-25 14:48:27] occurrences within proxies: ["apis/pingstatus-v1-sharedflows/revisions/1/policies/FC-Log"]
[2019-Feb-25 14:48:27] occurrences within sharedflows: ["sharedflows/ProxyDefaultFaultRule/revisions/1/policies/FC-Log","sharedflows/PostProxy/revisions/1/policies/FC-Log"]

This tool may also be useful: apigee-edge-scanner which will allow you to list all the FlowCallouts used by any proxy.

For example (note the use of FlowCalloutBean):

$ node ./scanProxies.js --notoken -n --org $ORG --policytype FlowCalloutBean

Apigee Edge proxy scanner tool, version: 20190211-1411
Node.js v10.15.0


[
  {
    "name": "pingstatus-v1-sharedflows",
    "revision": "1",
    "policies": [
      "FC-GetLogValues",
      "FC-Log",
      "FC-ProxyErrorHandling",
      "FC-VerbPathNotFound"
    ],
    "scan": "policy type name 'FlowCalloutBean'"
  }
]


Hope that helps

This would be a nice feature to have in Apigee in my opinion.

For others, doing a source code search say in Bitbucket or Github would show you which proxies use a particular shared flow also (assuming it's the environment that you are concerned with). Cheers.

I've created a pull request for findSharedFlowAccess in EdgeTools.

https://github.com/DinoChiesa/EdgeTools/pull/3

$ node ./findSharedFlowAccess.js -h
Apigee Edge SharedFlow check tool, version: 2020-05-26:15:57:00
Node.js v12.16.2

[2020-May-26 17:09:35] start
Usage: node findSharedFlowAccess.js

  -M, --mgmtserver=ARG   the base path, including optional port, of the Edge mgmt server. Defaults to https://api.enterprise.apigee.com . 
  -u, --username=ARG     org user with permissions to read Edge configuration.
  -p, --password=ARG     password for the org user.
  -n, --netrc            retrieve the username + password from the .netrc file. In lieu of -u/-p
  -o, --org=ARG          the Edge organization.
  -Z, --ssoZone=ARG      specify the SSO zone to use when authenticating.
  -C, --passcode=ARG     specify the passcode to use when authenticating.
  -J, --keyfile=ARG      the keyfile for a service account, for use with apigee.googleapis.com.
  -T, --notoken          do not try to obtain an oauth token.
  -N, --forcenew         force obtain a new oauth token.
  -v, --verbose          
  -h, --help             
  -s, --sharedflow=ARG   Optional. SharedFlow name to find.
  -d, --deployed         Optional. Search only deployed proxies.
  -e, --environment=ARG  Optional. Search only deployed proxies in environment, -d must be specified

Thanks for answering! The more you can help developers the better. I was looking for something a while ago so it's too late for me but thanks anyway. 🙂

Good morning,

I'm trying to use this useful tool.

I have created the .netrc file on my home directory. However, I am not reaching to this file:

oliveras@ES-EMBP1051 apigee-edge-scanner-master % node ./scanProxies.js -o schwarz-work -n -v  --policytype GenerateJWT

Apigee Edge proxy scanner tool, version: 20190211-1411

Node.js v10.16.0

 

The specified host (api.enterprise.apigee.com) is not present in the .netrc file.

Usage:

  node scanProxies.js [OPTION]

I also tried to use -M -u -p command but then this error is shown:

[2022-Feb-08 11:30:20] found no stashed token.

error: Error: missing username or password

    at maybeGetNewToken (/Users/oliveras/Desktop/LIDL/Github/apigee-edge-scanner-master/node_modules/apigee-edge-js/lib/edge.js:78:15)

    at Edge.<anonymous> (/Users/oliveras/Desktop/LIDL/Github/apigee-edge-scanner-master/node_modules/apigee-edge-js/lib/edge.js:173:16)

    at Promise (/Users/oliveras/Desktop/LIDL/Github/apigee-edge-scanner-master/node_modules/apigee-edge-js/lib/promiseWrap.js:36:10)

    at new Promise (<anonymous>)

    at Edge.connect (/Users/oliveras/Desktop/LIDL/Github/apigee-edge-scanner-master/node_modules/apigee-edge-js/lib/promiseWrap.js:16:12)

    at Object.<anonymous> (/Users/oliveras/Desktop/LIDL/Github/apigee-edge-scanner-master/scanProxies.js:106:12)

    at Module._compile (internal/modules/cjs/loader.js:776:30)

    at Object.Module._extensions..js (internal/modules/cjs/loader.js:787:10)

    at Module.load (internal/modules/cjs/loader.js:653:32)

    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)

Could you please help me? What I am doing wrong?

Good morning,

I'm trying to use this useful tool.

I have created the .netrc file on my home directory. However, I am not reaching to this file:

oliveras@ES-EMBP1051 apigee-edge-scanner-master % node ./scanProxies.js -o schwarz-work -n -v  --policytype GenerateJWT

Apigee Edge proxy scanner tool, version: 20190211-1411

Node.js v10.16.0

The specified host (api.enterprise.apigee.com) is not present in the .netrc file.

 

I also tried to use -M -u -p command but then this error is shown:

[2022-Feb-08 11:30:20] found no stashed token.

error: Error: missing username or password

 

Could you please help me? What I am doing wrong?

"The specified host (api.enterprise.apigee.com) is not present in the .netrc file."

means the hostname is not present in your .netrc file. The .netrc file is just a way to provide credentials for the given host.

The following applies to Apigee Edge.

You can optionally tell the scanProxies.js tool to look in the .netrc file for credentials to use to authenticate to Apigee Edge. But if you want to use it, there should be a stanza like this in the file:

 

machine api.enterprise.apigee.com
  login myusername@email.com
  password VerySecret!

 

As an alternative, you can specify the username on the command line when you invoke the tool, like this:

 

node ./scanProxies.js -v -o schwarz-work -u myuser@email.com --policytype GenerateJWT

 

..and in that case the tool should prompt you for the password. If you like you can avoid the prompt, and specify the password also on the command line, like this:

 

node ./scanProxies.js -v -o schwarz-work -u myuser@email.com -p VerySecret   --policytype GenerateJWT

 

If you have multi-factor OAuth, or SAML, or otherwise cannot use basic auth to authenticate to Apigee, then ... you will need to use a different approach to logging in. You can obtain a passcode from login.apigee.com/passcode, and specify it on the command line with the -C option:

 

node ./scanProxies.js -v -o schwarz-work -u myuser@email.com -C PASSCODEHERE  --policytype GenerateJWT

 

Before trying any of that, you should get the latest version of the scanProxies tool from the apigee-edge-scanner repo. (git pull)

The following applies to Apigee X or hybrid

If you are connecting to Apigee X or hybrid, then you need to authenticate differently. It won't work to use basic authentication. You can use the gcloud command to obtain an access token, then specify the token on the command line, though, like this:

 

TOKEN=$(gcloud auth print-access-token)
node ./scanProxies.js -v --apigeex -o schwarz-work --token $TOKEN --policytype GenerateJWT

 

EDIT: I've just updated this scanner tool so that it works with Apigee X or hybrid. Get the latest version.

Thank you very much for your fast reply.

I can't use Basic Authentication and I am using SAML approach.

Also I downloaded the last version.

Both cases is giving me "unable to get local issuer certificate".

javioliveras_0-1644506481465.png

and the second one:

javioliveras_1-1644506610761.png

In this second case, I see scanProxies.js tries to do a rest call to /apis. This rest call in postman is working properly:

javioliveras_2-1644506715484.png

Do you know where is the error?

Thanks in advance,

 

You didn't show me the command line you used for scanProxies.js

It seems that there is some problem on your machine, verifying the TLS certificate that is presented by schwarz.login.apigee.com . I am unsure if you have configured a key with a distinct CA for that domain.  If you have, then you need to configure nodejs to trust the certificate for that custom signing CA.   

OR, your workstation has an out of date certificate store.

This article may help solve that.

https://stackoverflow.com/questions/29283040/how-to-add-custom-certificate-authority-ca-to-nodejs

It seems that your postman is configured to trust the signing CA, while nodejs is not configured to trust those things.

Hi @dchiesa1,

Finally it is almost working. I follow the link you provided me and after this command:

javioliveras_0-1644579706254.png

it is reaching.to Apigee. I use the following command:

node ./scanProxies.js -v --apigeex -o schwarz-work --token $TOKEN --policytype GenerateJWT

After some api calls with 200 OK, the process returns the following error:

javioliveras_1-1644579854329.png

 

I suppose this is a bug in PolicyType.js, right?

 

Hi again @dchiesa1 ,

I updated policyType.js as below:

javioliveras_0-1644583422164.png

 

And now it seems is searching properly. I executed it and I'm still waiting. A lot of proxies in this organization, jajaj

The scanner has finished with 0 founds:

javioliveras_1-1644584771215.png

 

I executed this command:

apigee-edge-scanner-main % node ./scanProxies.js -v --apigeex -o schwarz-work --token $TOKEN --policytype BS-basicauth-v1_FC-BasicAuth0

 

 

And it is used in some proxies. For example this one (last line)

javioliveras_3-1644584895324.png

 

Am I using the command properly? It is searching by policy name? I want to find all proxies using an specific SharedFlow.

Thanks for the tip on the bug! Nice catch.

Yes, I think you are not using the scanner properly

The policytype argument accepts a policy type like "AssignMessage" or "VerifyJWT" or "FlowCallout" .

It looks like you passed "BS-basicauth-v1_FC-BasicAuth0" for the policytype. That is not the name of a policy type in Apigee.

I think that might be a name of a policy that you might use in some of your API proxies.

If you would like to scan proxies for a policy that has a name that matches a specific pattern.... the version of the tool you tried does not do that. But, good news! I just updated the scanner tool to find matches on policy names.

Perform a "git pull" to get the latest version of the tool.

Then use it like this:

 

 node ./scanProxies.js -v --apigeex --token $TOKEN -o $ORG  --policyname AM-\.*

 

The new --policyname argument accepts a regular expression. So you could specify "FC-.\*" and it would find policies with names like "FC-GetToken" or "FC-SF1" or "FC-1" . If you follow a naming convention and use "FC" as a name prefix for all FlowCallout policies, then you would find policies that are FlowCallouts.

If you want to scan for a particular policy name, then just specify that full name.

This is different than checking for policy TYPE. a policy TYPE is things like AssignMessage and FlowCallout. A policy name is whatever you want it to be.

Good morning @dchiesa1,

It is working like a charm!

Thank you for your support.

what is the format for using the token in .netrc file ?

The .netrc file holds username/password combinations for particular endpoints. You can find information about the format here.

It;s probably easiest to supply a token on the command line. to do that, use the --token argument as I showed in my example above. Obtain the token in the normal way. (If Apigee X, then obtain it with gcloud auth)

Looking to do the same but having a hard time finding relevant API references in https://apidocs.apigee.com/apis for this.

Can someone share the API call sequence please?

Thanks!

"do the same" means what exactly? What specifically are you attempting to do?  Maybe you could post a New Question? 

I want to find all proxies that are using a particular sharedflow. Still have not found a way to do this via mgmt api and guess we will have to resort to some sort of search to be able to get this relationship.

You're correct that you have to do a "brute force" search in order to find those things. 

A script that does this is here: https://github.com/DinoChiesa/apigee-edge-js-examples/blob/main/findFlowCallouts.js

Basic logic: 

get a list of all the proxies. for each proxy:

get a list of all the revisions.  For each revision, or maybe only for the last one:

export the proxy bundle to a temporary directory

unzip the bundle

examine all the policies, for each FlowCallout policy:

see if the target SharedFlow is the one you're after.