{ Community }
  • Academy
  • Docs
  • Developers
  • Resources
    • Community Articles
    • Apigee on GitHub
    • Code Samples
    • Videos & eBooks
    • Accelerator Methodology
  • Support
  • Ask a Question
  • Spaces
    • Product Announcements
    • General
    • Edge/API Management
    • Developer Portal (Drupal-based)
    • Developer Portal (Integrated)
    • API Design
    • APIM on Istio
    • Extensions
    • Business of APIs
    • Academy/Certification
    • Adapter for Envoy
    • Analytics
    • Events
    • Hybrid
    • Integration (AWS, PCF, Etc.)
    • Microgateway
    • Monetization
    • Private Cloud Deployment
    • 日本語コミュニティ
    • Insights
    • IoT Apigee Link
    • BaaS/Usergrid
    • BaaS Transition/Migration
    • Apigee-127
    • New Customers
    • Topics
    • Questions
    • Articles
    • Ideas
    • Leaderboard
    • Badges
  • Log in
  • Sign up

Get answers, ideas, and support from the Apigee Community

  • Home /
  • Private Cloud Deployment /
avatar image
1
Question by mchalmers · May 17, 2016 at 02:28 PM · 2k Views automationtoolsscript

What tools are recommended to automate Cloud Ops? Any scripts / tools / best practices?

Comment
Add comment
10 |5000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by Apigeeks only
  • Viewable by the original poster
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Close

2 Answers

  • Sort: 
avatar image
5

Answer by Geir Sjurseth · May 17, 2016 at 10:12 PM

There are loads and loads of tools and different categories of course and range from prescriptive frameworks to home-grown scripting based on shell.

In the general OS-style automation category you have things like chef/puppet, ansible and salt. I'm personally partial to ansible, but they all have their strengths and weaknesses.

All of these are very often in cloud and devops... Then there are tools like terraform which are very ansible-like but are decidedly cloud operations centric. There's also tools like cloudformation, but I have to be honest I haven't touched it.

We do have internal projects ongoing at apigee that use terraform and of course a fair deal running ansible as well....

-----

There's a whole lot more to be said about generic devops with tools like jenkins, travis-ci, bamboo, etc... But those have decidedly less to do with the whole cloud topic.

-----

My own recommendation?

Terraform for the IaaS pieces: provision all the machines for apigee

Ansible for the per machine pieces: start/stop/install/setup apigee bits

Tmux sessions for same-time shell-access and even sending commands to all machines at once.... This last one is a bit archaic, but I've grown quite accustomed to it and it makes for rapidly being able to apply ad-hoc commands to a huge number of machines all at once and in parallel.

/geir

Comment
Add comment Show 5 · Link
10 |5000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by Apigeeks only
  • Viewable by the original poster
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image Benjamin Goldman · Jun 30, 2016 at 06:26 PM 0
Link

This is more or less exactly what we do to test/deploy/manage apigee in our cloud/iaas environments.

You will find that terraform is really easy to use once you get past the hard parts, and that ansible is a significantly simpler way of managing configs and installs than some of the other options (puppet or chef for example)

that said - prior to 16.01 i do manage config files using puppet (for its easy access to augeas libraries behind the scenes... really simplifies things and means I dont have to deal with templates, etc) - but I can trigger those jobs from ansible (or even from terraform) if I want.

One thing I have been preaching at my organization is to NOT REINVENT THE WHEEL (sorry Apigee guys!) - and rely on the Apigee scripts and install packages to do their jobs. Dont try to re-engineer them as that is clearly waste and will lead to you having to re-engineer every time Apigee changes something.

avatar image Benjamin Goldman · Jun 30, 2016 at 06:30 PM 0
Link

FYI - one really cool thing you can do in ansible is set the number of times a command can be run in parallel against a group of servers.

For things like Datastore installs - i set this to 1. For other things that arnt so tempermental (like base installs of RMP boxes, etc) i dont set it, and let ANSIBLE run things in parallel - which it is good at.

I would be intersted @Geir Sjurseth in what use cases you need Tmux for... especially since I just had some staff turnover and am taking time to re-engineer some of our processes before 16.01 upgrades...

avatar image Geir Sjurseth ♦ Benjamin Goldman · Jul 01, 2016 at 06:10 AM 0
Link
@Benjamin Goldman

I tend to setup a bastion host for whatever environment i'm administering and then create a tmux session for all the boxes in that setup (sometimes multipe). Then I can use tmux to send keystrokes to multiple windows. It's pretty much identical to sending keys to tabs inside of iterm.... Except that it can live on between machines and ssh-sessions and is fully customizable.

This is an example where I'm pulling out all the hosts from my /etc/hosts file, excluding the kafka hosts and then naming each session according to the name.... Then I use that same "session-name" to send commands to it. In this case below I'm telling it to ssh-login to all the machines in question.

for a in $(fgrep vf /etc/hosts | egrep -v 'kafka' | awk '{print $2}')
do
  tmux new-window -n $a
  tmux send-keys -t $a "ssh -i key.pem  centos@${a}" C-m
done

Then I can create bash functions that use those same named windows to send arbitrary commands to lists of named machines....

Also, since it's tmux I can detach and then login again from anywhere and regain my tmux session and continue editing it.... Super useful if you want to start tailing all message-processors at once while piping to a custom grep and then let it run... I can then login hours later and pull it up.

Not the same use case at all for things like ansible and such where you're more scripting specific and regularly recurring commands, but great when you want to control and history of the actual sessions managing the environments.

This may have been confusing, so if you have any more questions please feel free to ping me.

Thanks!

/geir

avatar image Benjamin Goldman · Jun 30, 2016 at 07:03 PM 1
Link

A really important thing to keep in mind: you will find that a lot of automation might fly in the face of your corporate security policies.

I know that I have had to do a lot of work to argue for changes in ours to allow me to do some simple things w/o having to re-package the apigee installers (this goes for 14.xx, 15.xx and now it is even more important in 16.xx)

Understanding this will save you trouble later.

It is really easy for me to find these issues because im IN the secured enterprise environment. Apigee has not been very good at understanding just how locked down some corporate infrastructure (even cloud) truely is.

avatar image Vicky cmd · May 09, 2018 at 07:29 AM 0
Link

Hi, Your post is good. Its so useful for me about Devops.Thank you for your post. Devops Training in Chennai

avatar image
0

Answer by arghya das · Jun 30, 2016 at 06:56 PM

Internally in Apigee for testing our private cloud installation and upgrade scripts, we wrote a bunch of automation jobs that use ansible. With 1601 and onwards that works really well with the private cloud model. We are also planning to use "terraform" to launch the instances and then have ansible scripts to perform the rest of the installation and upgrade process. The ansible scripts directly call the various apigee interfaces like apigee-service, adminapi etc.

Comment
Add comment · Link
10 |5000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by Apigeeks only
  • Viewable by the original poster
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Follow this Question

Answers Answers and Comments

26 People are following this question.

avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image

Related Questions

Edge private cloud connectivity checks 1 Answer

Do we have any initiatives for installing and configurating APIGEE using automation like Chef or Puppet 2 Answers

Getting list of all R-MPs with their IPs, UUIDs 2 Answers

Is there a capacity planning tool available for On Prem installations? 1 Answer

Documentation about Apigee utilities ( >= 4.16.01) 0 Answers

  • Products
    • Edge - APIs
    • Insights - Big Data
    • Plans
  • Developers
    • Overview
    • Documentation
  • Resources
    • Overview
    • Blog
    • Apigee Institute
    • Academy
    • Documentation
  • Company
    • Overview
    • Press
    • Customers
    • Partners
    • Team
    • Events
    • Careers
    • Contact Us
  • Support
    • Support Overview
    • Documentation
    • Status
    • Edge Support Portal
    • Privacy Policy
    • Terms & Conditions
© 2021 Apigee Corp. All rights reserved. - Apigee Community Terms of Use - Powered by AnswerHub
  • Anonymous
  • Sign in
  • Create
  • Ask a question
  • Create an article
  • Post an idea
  • Spaces
  • Product Announcements
  • General
  • Edge/API Management
  • Developer Portal (Drupal-based)
  • Developer Portal (Integrated)
  • API Design
  • APIM on Istio
  • Extensions
  • Business of APIs
  • Academy/Certification
  • Adapter for Envoy
  • Analytics
  • Events
  • Hybrid
  • Integration (AWS, PCF, Etc.)
  • Microgateway
  • Monetization
  • Private Cloud Deployment
  • 日本語コミュニティ
  • Insights
  • IoT Apigee Link
  • BaaS/Usergrid
  • BaaS Transition/Migration
  • Apigee-127
  • New Customers
  • Explore
  • Topics
  • Questions
  • Articles
  • Ideas
  • Badges