How to compare proxy versions across different organization?

[Arnaduga]

Hello

I'm working on Apigee X, with 2 different organizations: nonprod and prod.

But, because these organizations are completely split, proxies deployment can only be done thanks to export/import (manually, API, sackmesser or CICD or ...).

This has a consequence: as lifecycle is different and proxy revision number are not the same as well: my proxy will be in revision 123 in nonprod, but only in revision 6 in production (123 versions to develop it, 6 deployments).

I was trying to imagine a way to take a snapshot of my environments and deployments, and I was thinking of calculating the HASH (md5sum) of the bundle, by hoping it to be the same for both but ... unfortunately, no, there aren't.

$ # nonprod/rev3 has been exported and imported into prod/rev1

$ curl -H "Authorization: Bearer $AUTH" https://apigee.googleapis.com/v1/organizations/nonprod/apis/myProxy/revisions/3?format=bundle -o nonprod-rev3.zip

$ curl -H "Authorization: Bearer $AUTH" https://apigee.googleapis.com/v1/organizations/prod/apis/myProxy/revisions/1?format=bundle -o prod-rev1.zip

$ md5sum nonprod-rev3.zip
d17332a5f8071b80fafbdf9f745e2b44  nonprod-rev3.zip

$ md5sum prod-rev1.zip
d5fbffe675e0eb006cf9b8487e738451  prod-rev1.zip

Both have different hash, because the "apiproxy.xml" file (inside the archive) is different: it contains the revision number and dates:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<APIProxy revision="3" name="myProxy">
  <DisplayName/>
  <Description/>
  <CreatedAt>1656400595916</CreatedAt>
  <LastModifiedAt>1656401723156</LastModifiedAt>

... and

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<APIProxy revision="1" name="mYproxy">
  <DisplayName/>
  <Description/>
  <CreatedAt>1659368013424</CreatedAt>
  <LastModifiedAt>1659368013424</LastModifiedAt>

I've tried (not elegant, I know) to remove the apiproxy.xml file from archive, but ... not working as well.

TL;DR: is there a way to compare REVISIONS from two environments to be sure of the proxy version?

The other option I can image is: storing a resource file INSIDE the proxy (or a properties's one) with the exact proxy version (semver for intsance). But, it may suffer from a lack of update, that is why I was trying to imagine something different...

Thanks!

[dchiesa1 · Accepted solution]

TL;DR: is there a way to compare REVISIONS from two environments to be sure of the proxy version?

The other option I can image is: storing a resource file INSIDE the proxy (or a properties's one) with the exact proxy version (semver for intsance). But, it may suffer from a lack of update, that is why I was trying to imagine something different...

There's no capability within Apigee to diff or compare multiple revisions across environments.  A good system for doing that sort of thing is a source code repository, like github or some other git-based system.  If you store your proxy configurations in a source code repository, then , you can use the built-in diff facility in the source code repository to get a comparison. 

If you do not store your proxy configurations in a source code repository, then , you could perform a diff in the way you had originally imagined, by simply excluding the top level API Proxy .xml file, the one that includes the revision number and the time stamp. So, export everything from each proxy, then unzip the respective bundles, delete those two files from the unzipped directories, then perform a recursive diff on the resulting apiproxy directory. 

[Arnaduga]

That's what I was afraid of 🙂

I'm using a git repo, but for the moment, I am storing only the complete bundle (the ZIP file), not individual files. Not really efficient, anyway.

Thank you for your reply, Dino!

[dchiesa1]

I thought of something else :

you don’t need to unzip the file. A zip includes a CRC32 of each file in the zip. So really all you need to do is read the central directory records within the two zips and compare the crc’s of each file entry , EXCEPT for the top level entry. 

I am not sure if nodejs has a tool that reads zip records. I know that .NET has such a thing. 

[Arnaduga]

I finally found a bash way, based on what you suggested.

The unzip command can also display each and every file CRC-32 (I guess from the central directory records you mentioned).

So, by combining unzip command, awk, tail, etc... to ignore the file that would change (file /apiproxy/*.xml) I am now able to successfully achieve my comparison!

# Check if org 2 deployment is the latest available
$ bundle-compare.sh org2-rev3.zip org1-rev55.zip
5c5 < 539B c77d7f1e apiproxy/resources/jsc/process.js --- > 842B da54a3c5 apiproxy/resources/jsc/process.js

$ echo $?
1

# Check if org 2 deployment is same as org 1 / 40
$ bundle-compare.sh org2-rev3.zip org1-rev40.zip
$
$ echo $?
0

Script can be improved (output, for instance), for sure, but it makes the first step of the job.

Script source is here, if it could help someone else: https://github.com/arnaduga/apigee-tooling/tree/master/bundle-compare

Thank you for your help!

Arnaduga

[Arnaduga]

... and my initial needs was "snapshot", so I also developed a script to extract "signature" of deployments:

$ cat config.jon
{
  "instances": [
    {
      "organization": "orgName1",
      "environments": ["env1", "env2"]
    },
    {
      "organization": "orgName2",
      "environments": ["env3"]
    }
  ]
}

$ ./deployments-status.sh -c config.json 1> results.txt 2> logs.txt
$

$ head -n 10 logs.txt
[INFO] MAIN - Config file input: config.json
[INFO] PREREQ - Checking dependencies
[INFO] PARSE - Getting authentication gcloud token
[INFO] AUTH - Generating a new token
[INFO] ORG - Checking organization orgName1
[INFO] ENV - Checking environment env1
[INFO] ENVDEPL - Get deployments on organization/orgName1/environments/env2
[INFO] HASH - Getting BUNDLE for orgName1 // Proxy1 // 2
[INFO] HASH - Getting BUNDLE for orgName1 // Proxy2 // 27
[INFO] HASH - Getting BUNDLE for orgName1 // Proxy3 // 12

$ cat results.txt
orgName1             env1     Proxy1                         2    2022-08-01T17:30:10  6df4d50a41a5d20bc4faad8a6f09aa8f
orgName1             env1     Proxy2                         27   2022-08-01T17:35:35  2c98f873d2dd2968f367b2a9f0fc7fd5
orgName1             env1     Proxy3                         12   2022-08-01T17:38:44  8ea4a3e98928dc9fe56d7293dbb35b66
orgName1             env2     Proxy2                         25   2022-07-25T17:31:32  f6e4a89f41c5faecd687eb29aca37569
orgName1             env1     Proxy3                         7    2022-07-15T14:44:21  6ff5905c684b2a67bff77c1b8d7f17ff
orgName2             env3     Proxy2                         2    2022-07-01T07:13:50  f6e4a89f41c5faecd687eb29aca37569

$ # to SORT based on SIGNATURE
$ cat results.txt | sort -k6
orgName1             env1     Proxy2                         27   2022-08-01T17:35:35  2c98f873d2dd2968f367b2a9f0fc7fd5
orgName1             env1     Proxy1                         2    2022-08-01T17:30:10  6df4d50a41a5d20bc4faad8a6f09aa8f
orgName1             env1     Proxy3                         7    2022-07-15T14:44:21  6ff5905c684b2a67bff77c1b8d7f17ff
orgName1             env1     Proxy3                         12   2022-08-01T17:38:44  8ea4a3e98928dc9fe56d7293dbb35b66
orgName1             env2     Proxy2                         25   2022-07-25T17:31:32  f6e4a89f41c5faecd687eb29aca37569
orgName2             env3     Proxy2                         2    2022-07-01T07:13:50  f6e4a89f41c5faecd687eb29aca37569

Also available on the same repo, here: https://github.com/arnaduga/apigee-tooling/tree/master/deployments-status

😁Enjoy!

Arnaduga

[dchiesa1]

Outstanding.  Nice work. Thanks for sharing it.