What is the best way to delete unused/inactive revisions of API Proxies?

[#1]

Hi,

In my organization few proxies are having more than 100 revisions in which only 5 are active. I am looking for best options to delete 95 revisions in a proxy. can some one please suggest.

thanks,

Satish

[anilsr]

Dear @Davana ,

• Write a simple script that calls Management API , Delete API Proxy revision using simple tool like curl & shell script.
• Use tools like Apigee Grunt Plugin Tool which supports delete a Api Proxy revision.

Cheers,

Anil Sagar

[#3]

Thanks for quick response Anil.

how to get a active revisions out of 100 revisions? I am getting revisions list but I am not able to get active revisions out of that(eg: 5 out of 100). Please share the api cmd if there are one.

I can script and delete unused revisions if I can get active/used revision in a proxy?

thanks,

Davana

[anilsr]

Dear @Davana , What do you mean by active revision ? Deployed revision ?

[#5]

Hi,

I have 5 environments and each time I deploy a proxy, a new revision is generated and assigned to respective environment. so far I have deployed more than 150 times to support different versions of proxy on 5 environments.

At any point only 5 revisions are used. So, I would like to clean old revisions. In this case i would like to delete at least unused revisions which we no longer use.

I am looking for an api cmd to identify revisions which are assigned to environment. So that I can delete rest of them.

I would like to delete 153-156,158-161 revisions from enclosed image. Please take a look and suggest me.

Thanks in Advance,

Davana

[#6]

oops forgot the attachment.

[dchiesa1 · Accepted solution]

Here's a new answer to an old question.

To delete multiple older revisions of an API proxy, you can use the nodejs script cleanOldRevisions.js.

Usage:
  node cleanOldRevisions.js [OPTION]

Options:
  -M, --mgmtserver=ARG the base path, including optional port, of the Edge mgmt server. Defaults to https://api.enterprise.apigee.com .
  -u, --username=ARG   org user with permissions to read Edge configuration.
  -p, --password=ARG   password for the org user.
  -n, --netrc          retrieve the username + password from the .netrc file. In lieu of -u/-p
  -o, --org=ARG        the Edge organization.
  -v, --verbose
  -h, --help
  -R, --regexp=ARG     Optional. Limit the culling to proxies with names matching this regexp.
  -K, --numToKeep=ARG  Required. Max number of revisions of each proxy to retain.

If you want to keep the most recent 5 revisions of *all* API Proxies in an organization, then you would use this:

./cleanOldRevisions.js -o $ORG -v -K 5 

If you want to keep 3 revisions of Proxies matching a specific regexp, you can do this:

./cleanOldRevisions.js -o $ORG -v -K 3 -R "^Catalog.*" 

[apigee-4]

Hi Dino,

Great script, thanks!

Any change that it will get the same functionality for Shared Flows?

Or should I create a Github feature request for that?

Gijs

[dchiesa1]

I'll extend the script to handle sharedflows as well.

[dchiesa1]

ok done. Try out version 0.3.9. (or just git pull from master)

[apigee-4]

Wow, that is fast. I will give it a go soon! Thanks

[apigee-4]

Works like a charm.

Tested with a concrete regex (^xxx$), on both Sharedflowonly and a regular API proxy.

Many Thanks Dino

[pbagora]

Hi Sorry to comment on such an old thread, i am having a little trouble with this script. It works perfectly with proxies but not deleting old revisions of sharedflow. When i run script it does recognise there are 230 sharedflows but not deleting old revisions. Any tips on what i could try ? I even tried the -S option.

TIA

[dchiesa1]

Jack, maybe there's a bug! Can you tell me more about your situation?

What specific options did you pass to the script? (omit credetials)

Are you using Apigee SaaS or opdk or ...hybrid?

After you get back to me, I'll look at it more closely and test... I'll get back to you.

[pbagora]

Thanks for quick reply Dino...i am running it on opdk 4.19.01,

Running the following command (tried both with and without -S) -

node cleanOldRevisions.js -M <redacted> -n -T -o <redacted> -S -v -K 20

It works perfectly for proxies, but doesn't for SF. Here's the start of the output command

[2020-Jun-03 13:39:59] start [2020-Jun-03 13:39:59] connect: {"orgname":"ittest","loginBaseUrl":"https://login.apigee.com","user":"****","mgmtServer":"****"}

[2020-Jun-03 13:39:59] GET http://****:8080/v1/o/****/ [2020-Jun-03 13:40:00] status: 200

[2020-Jun-03 13:40:00] GET http://****:8080/v1/o/****/apis [2020-Jun-03 13:40:00] status: 200

[2020-Jun-03 13:40:00] found 230 sharedflows

end of output - summary deleted: []

[dchiesa1]

Thanks.

If your script tells you there are 230 SharedFlows, then ... it should check each one of them individually. You would expect to see 230 queries, one for each of the SF's, to see how many revisions each one has.

When you say "end of output" .. was that actually in the output? Was there any intervening output between "found 230 sharedflows" and "summary deleted: []" ?

There should be a timestamp preceding the "summary deleted" line. What was the time?

Normally the script can request... oh, maybe 10 SF's per second or so. So I'd expect it to run in 23 seconds, maybe more if it is deleting a bunch of SF revisions.

You used a -K of 20. That option tells the script how many revisions to keep. It's possible that You don't have any sharedflows with more than 20 revisions. In that case the script would not delete anything, and you would expect to see "summary deleted: []" after it analyzes all 230 sharedflows. Is that possibly what happened?

One implication of this: if you run the same command twice, the first time it might delete a few revisions, the second time it will analyze all the sharedflows and result in "summary deleted: []" .

If that commentary doesn't solve the mystery, I want to check a few other things. There should be a signature line at the top of the output. Like this:

Apigee Edge Proxy / Sharedflow revision cleaner tool, version: 20190925-1705
Node.js v12.14.1
<br>

Do you have that? What does yours say? If you are on a back-version of the tool, please get the latest (git pull from the apigee-edge-js-examples repo) .

Next I'd like to see the output of the debug:

NODE_DEBUG=request node cleanOldRevisions.js -M <redacted> -n -T -o <redacted> -S -v -K 20<br>

That should show you every request and response. which might give us some more insight.

[pbagora]

capture.pngThis is what mine shows

Apigee Edge Proxy / Sharedflow revision cleaner tool, version: 20190925-1705 Node.js v12.14.1 [2020-Jun-03 21:32:16] start

One thing that just clicked to me is that i have only 10 sharedflows .. 230 is the proxies i have .. but somehow script is looking as proxies and taking that as sharedflow count (i was thinking ealier that 230 might be coming from # of revisions but that's not the case)

It's not an issue with using -K 20 as an option as i have a shared flow that has 120+ revisions and i am checking that one if older revisions get deleted.

Even when i run the script with -S option it checks revisions of ALL the proxies, finds nothing that has 20+ revisions and the exits... but it does NOT do that for sharedflow, its like the GET revisions call is not being called for SharedFlow.

I just ran it again and changed the -K option to 10 and same thing - goes through all the proxy revisions even though with -S but not for sharedflows. Feel like i might be missing something here as an option perhaps ?

The run below started at 21:32:16 EST and ended at 21:32:29. Around 700 lines of output wiht GET call, status 200 and then a list of revisions for those proxies.. but no call for SF.

node cleanOldRevisions.js -M http://<hostname>:8080 -n -T -o test -S -v -K 10

Apigee Edge Proxy / Sharedflow revision cleaner tool, version: 20190925-1705 Node.js v12.14.1 [2020-Jun-03 21:32:16] start [2020-Jun-03 21:32:16] connect: {"orgname":"test","loginBaseUrl":"https://login.apigee.com","user":"****","mgmtServer":"http://<hostname>:8080"}

[2020-Jun-03 21:32:16] GET http://<hostname>:8080/v1/o/test/

[2020-Jun-03 21:32:16] status: 200

[2020-Jun-03 21:32:16] GET http://<hostname>:8080/v1/o/test/apis

[2020-Jun-03 21:32:16] status: 200

[2020-Jun-03 21:32:16] found 230 sharedflows

[2020-Jun-03 21:32:16] GET http://<hostname>:8080/v1/o/test/apis/proxy1/revisions

[2020-Jun-03 21:32:17] status: 200

[2020-Jun-03 21:32:17] revisions proxy1: ["4","5","6","7","8"]

[2020-Jun-03 21:32:17] GET http://<hostname>:8080/v1/o/test/apis/proxy2/revisions

[2020-Jun-03 21:32:17] status: 200

[2020-Jun-03 21:32:17] revisions proxy2: ["31","32","33","34","35","36"]

and so on for all the proxies.

If i use the -S option, are proxy revisions supposed to be called ?

Here's the first part of the output with debug, after this it just repeats calling each proxy and not SF

NODE_DEBUG=request node cleanOldRevisions.js -M http://<hostname>:8080 -n -T -o test -S -K 10

Apigee Edge Proxy / Sharedflow revision cleaner tool, version: 20190925-1705 Node.js v12.14.1

[2020-Jun-03 21:54:26] start

REQUEST { headers: { accept: 'application/json', authorization: 'Basic ******' }, uri: 'http://<hostname>:8080/v1/o/test/', callback: [Function], method: 'GET' }

REQUEST make request http://<hostname>:8080/v1/o/test/

REQUEST onRequestResponse http://<hostname>:8080/v1/o/test/ 200 { connection: 'close', date: 'Thu, 04 Jun 2020 01:54:27 GMT, Thu, 04 Jun 2020 01:54:27 GMT', 'content-type': 'application/json', 'x-apigee.user': '******', 'x-apigee.organization': 'test', vary: 'Accept-Encoding, User-Agent', 'content-length': '463' }

REQUEST reading response's body

REQUEST finish init function http://<hostname>:8080/v1/o/test/

REQUEST response end http://<hostname>:8080/v1/o/test/ 200 { connection: 'close', date: 'Thu, 04 Jun 2020 01:54:27 GMT, Thu, 04 Jun 2020 01:54:27 GMT', 'content-type': 'application/json', 'x-apigee.user': '******', 'x-apigee.organization': 'test', vary: 'Accept-Encoding, User-Agent', 'content-length': '463' }

REQUEST end event http://<hostname>:8080/v1/o/test/

REQUEST has body http://<hostname>:8080/v1/o/test/ 463

REQUEST emitting complete http://<hostname>:8080/v1/o/test/

REQUEST { headers: { accept: 'application/json', authorization: 'Basic ******' }, url: 'http://<hostname>:8080/v1/o/test/apis', callback: [Function], method: 'GET' }

REQUEST make request http://<hostname>:8080/v1/o/test/apis

REQUEST onRequestResponse http://<hostname>:8080/v1/o/test/apis 200 { connection: 'close', date: 'Thu, 04 Jun 2020 01:54:27 GMT, Thu, 04 Jun 2020 01:54:27 GMT', 'content-type': 'application/json', 'x-apigee.user': '******', 'x-apigee.organization': 'test', vary: 'Accept-Encoding, User-Agent', 'content-length': '5101' }

REQUEST reading response's body

REQUEST finish init function http://<hostname>:8080/v1/o/test/apis

REQUEST response end http://<hostname>:8080/v1/o/test/apis 200 { connection: 'close', date: 'Thu, 04 Jun 2020 01:54:27 GMT, Thu, 04 Jun 2020 01:54:27 GMT', 'content-type': 'application/json', 'x-apigee.user': '******', 'x-apigee.organization': 'test', vary: 'Accept-Encoding, User-Agent', 'content-length': '5101' } REQUEST end event http://<hostname>:8080/v1/o/test/apis REQUEST has body http://<hostname>:8080/v1/o/test/apis 5101

REQUEST emitting complete http://<hostname>:8080/v1/o/test/apis

[dchiesa1]

ok thank you , helpful.

What I see is

• you have the latest version of the script
• you have a recent version of node

let me look further.

....aha! there was a bug in the script.

I've updated it.

The corrected version is "20200604-0830"

Use "git pull" to get the latest, please, and retry.

Sorry about the bug.

[ayushopposite]

Hi All

I have created a JAVA based solution to Delete Unused Revisions for a Particular Proxy.

deleterevisions-rev4-2019-07-09.zip

Unzip this, It will contain an API proxy , Deploy it on the APIGEE environment and Start hitting the request with basic authorization in the request.

curl -X POST \

http://{Apigee Base URL}/removeUnused/v1/revisions \

-H 'Accept: application/json' \

-H 'Authorization: Basic **********' \

-H 'Cache-Control: no-cache' \

-H 'Connection: keep-alive' \

-H 'Content-Type: application/json' \

-H 'Host: apigee host' \

-H 'Postman-Token: 1efc39a7-85ad-46f5-8b89-9f39e305a6ca,f57245d2-dd9d-4e47-ac90-096293d428c8' \

-H 'User-Agent: PostmanRuntime/7.15.0' \

-H 'accept-encoding: gzip, deflate' \

-H 'cache-control: no-cache' \

-H 'content-length: 120' \

-d '{

"org":"Org Name",

"proxy":"Proxy Name",

"url":"Mgmt Server URL",

"revisionsToBekept":"Number of Revisions which you want to keep"

}'

Now it will give a JSON response like this and having four scenarios. The main element would be the proxy name with having elements as

1. DeletedRevisions: Array of deleted revisions
2. NotDeletedrevisions: Array of revisions which can’t be deleted as they are deployed
3. Message: When there are no revisions which can be deleted.

Scenarios:

1. When there are no revisions to be deleted: {"OAuth":[{"DeletedRevisions":[]},{"NotDeletedrevisions":[]},{"message":"not enough revision to be deleted"}]}

• It will give only the message with both the array as blank.

1. When there are revisions to be deleted but not deployed to any environment: {"OAuth":[{"DeletedRevisions":["177","178","179","180","181","182","183","184"]},{"NotDeletedrevisions":[]},{"message":""}]}

1. When there are Revisions which can be deleted and Revisions which are Deployed to the Environment: {"OAuth":[{"DeletedRevisions":["185","186","187","188","190","191","192","193","194","195","196","197","198","199","200","201","202","203","204"]},{"NotDeletedrevisions":["189"]},{"message":""}]}

1. When there is only one Revision which is deployed and can’t be deleted: {"OAuth":[{"DeletedRevisions":[]},{"NotDeletedrevisions":["205"]},{"message":""}]}

Please post here for more questions.