Apige Hybrid 1.5 Service Account key rotation-Automation

[aramkrishna6]

Hi,

As per docs lists following information about Service Account key rotation

Best practices for using and managing service accounts  |  IAM Documentation  |  Google Cloud

Understanding service accounts  |  IAM Documentation  |  Google Cloud

About service accounts  |  Apigee X  |  Google Cloud

Understanding service accounts  |  IAM Documentation  |  Google Cloud

Last link indicates about deleting or recreating Service account

I am looking for Service Account rotation automation procedures & not sure, if we have any reference document  to automate overall process of Service account rotation and can rotate before service account expires , we are using GCP IAM with GKE Anthos on-prem     versione 1.8.1 & Apigee Hybrid Runtime versione 1.5.3 from dev to prod.

There are multiple service account, which are created during the Apigee Hybrid installation for various purposes, and something which can still work when we migrate to latest Apigee Hybrid version ?

2) Is migration from Apigee 1.5.3 to Apigee 1.6.7 (Latest version) sould be possible or requires any hops ?

@dino 

[dchiesa1 · Accepted solution]

Here is what I use for that purpose.

export SA_DIR=/Users/me/dev/hybrid-apigee/hybrid-files/service-accounts
export PROJECT=my-hybrid-proj1
export ORG=$PROJECT

# list keys and expiry:
/usr/local/bin/bash ./bin/ahr-sa-ctl sa-keys-list all

# report on keys and days til expiry
/usr/local/bin/bash ./bin/ahr-sa-ctl sa-keys-report

# rotate keys
/usr/local/bin/bash ./bin/ahr-sa-ctl sa-keys-rotate

# report on keys again
/usr/local/bin/bash ./bin/ahr-sa-ctl sa-keys-report

# remove unnneded (expired) keys
/usr/local/bin/bash ./bin/ahr-sa-ctl sa-keys-delete all  --filter "EXPIRES_AT<2022-03-31T23:59"

More detail on the usage here.

This tool is courtesy of my colleague, @yuriyl .

[aramkrishna6]

Thanks @dino  for details.

How do we avoid outage risk, when following above process and what other additional factors need to be considered to avoid outage and automation for lower or prod.  environments. Any other detailed document on automation to avoid outage risk is available for reference ?

[aramkrishna6]

@dino   and @yuriyl . Let me know, 

Listed steps in article indicated about automation across all steps from reporting to rotation to delete, hence such steps can be  automated without downtime using Jenkins/CloudBuild pipelines or Corn jobs. with periodic run.  

For eg:  (ahr-sa-ctl sa-keys-rotate) The rotate command uses rolling update commands to restart specific components with zero-downtime (as per document), hence read the output of each command using the shell or equivalent and based on same have to apply the rotation may be before 30 days of expiry with automation. (Time frame 30 days is just an example, may differ for each organization).

All such steps to be followed in lower env/sandbox, before automating without downtime for prod, let me know, if we have any detailed reference on same for similar automation or if I missed anything ?