What tools are recommended to automate Cloud Ops? Any scripts / tools / best practices?

mchalmers
Participant I
 
1 11 3,353
11 REPLIES 11

Not applicable

There are loads and loads of tools and different categories of course and range from prescriptive frameworks to home-grown scripting based on shell.

In the general OS-style automation category you have things like chef/puppet, ansible and salt. I'm personally partial to ansible, but they all have their strengths and weaknesses.

All of these are very often in cloud and devops... Then there are tools like terraform which are very ansible-like but are decidedly cloud operations centric. There's also tools like cloudformation, but I have to be honest I haven't touched it.

We do have internal projects ongoing at apigee that use terraform and of course a fair deal running ansible as well....

-----

There's a whole lot more to be said about generic devops with tools like jenkins, travis-ci, bamboo, etc... But those have decidedly less to do with the whole cloud topic.

-----

My own recommendation?

Terraform for the IaaS pieces: provision all the machines for apigee

Ansible for the per machine pieces: start/stop/install/setup apigee bits

Tmux sessions for same-time shell-access and even sending commands to all machines at once.... This last one is a bit archaic, but I've grown quite accustomed to it and it makes for rapidly being able to apply ad-hoc commands to a huge number of machines all at once and in parallel.

/geir

This is more or less exactly what we do to test/deploy/manage apigee in our cloud/iaas environments.

You will find that terraform is really easy to use once you get past the hard parts, and that ansible is a significantly simpler way of managing configs and installs than some of the other options (puppet or chef for example)

that said - prior to 16.01 i do manage config files using puppet (for its easy access to augeas libraries behind the scenes... really simplifies things and means I dont have to deal with templates, etc) - but I can trigger those jobs from ansible (or even from terraform) if I want.

One thing I have been preaching at my organization is to NOT REINVENT THE WHEEL (sorry Apigee guys!) - and rely on the Apigee scripts and install packages to do their jobs. Dont try to re-engineer them as that is clearly waste and will lead to you having to re-engineer every time Apigee changes something.

FYI - one really cool thing you can do in ansible is set the number of times a command can be run in parallel against a group of servers.

For things like Datastore installs - i set this to 1. For other things that arnt so tempermental (like base installs of RMP boxes, etc) i dont set it, and let ANSIBLE run things in parallel - which it is good at.

I would be intersted @Geir Sjurseth in what use cases you need Tmux for... especially since I just had some staff turnover and am taking time to re-engineer some of our processes before 16.01 upgrades...

@Benjamin Goldman

I tend to setup a bastion host for whatever environment i'm administering and then create a tmux session for all the boxes in that setup (sometimes multipe). Then I can use tmux to send keystrokes to multiple windows. It's pretty much identical to sending keys to tabs inside of iterm.... Except that it can live on between machines and ssh-sessions and is fully customizable.

This is an example where I'm pulling out all the hosts from my /etc/hosts file, excluding the kafka hosts and then naming each session according to the name.... Then I use that same "session-name" to send commands to it. In this case below I'm telling it to ssh-login to all the machines in question.

for a in $(fgrep vf /etc/hosts | egrep -v 'kafka' | awk '{print $2}')
do
  tmux new-window -n $a
  tmux send-keys -t $a "ssh -i key.pem  centos@${a}" C-m
done

Then I can create bash functions that use those same named windows to send arbitrary commands to lists of named machines....

Also, since it's tmux I can detach and then login again from anywhere and regain my tmux session and continue editing it.... Super useful if you want to start tailing all message-processors at once while piping to a custom grep and then let it run... I can then login hours later and pull it up.

Not the same use case at all for things like ansible and such where you're more scripting specific and regularly recurring commands, but great when you want to control and history of the actual sessions managing the environments.

This may have been confusing, so if you have any more questions please feel free to ping me.

Thanks!

/geir

A really important thing to keep in mind: you will find that a lot of automation might fly in the face of your corporate security policies.

I know that I have had to do a lot of work to argue for changes in ours to allow me to do some simple things w/o having to re-package the apigee installers (this goes for 14.xx, 15.xx and now it is even more important in 16.xx)

Understanding this will save you trouble later.

It is really easy for me to find these issues because im IN the secured enterprise environment. Apigee has not been very good at understanding just how locked down some corporate infrastructure (even cloud) truely is.

Hi, Your post is good. Its so useful for me about Devops.Thank you for your post. Devops Training in Chennai

adas
Participant V

Internally in Apigee for testing our private cloud installation and upgrade scripts, we wrote a bunch of automation jobs that use ansible. With 1601 and onwards that works really well with the private cloud model. We are also planning to use "terraform" to launch the instances and then have ansible scripts to perform the rest of the installation and upgrade process. The ansible scripts directly call the various apigee interfaces like apigee-service, adminapi etc.

Not applicable

Matt,

There are defiantly many alternatives and depending on the scope. Cloud Ops could mean many different things ranging from Cloud infrastructure (AWS instances or similar), network, security, and applications hosted on IaaS providers.

Focusing on Apigee Private Cloud automation. There are many examples of customers that have successfully automated Private Cloud installs (fresh install and capacity addition) using many of the popular automation tools on the market as well as homegrown tools.

What are the common tools used?

In no particular order, we often see customers using the following tools to automate Private Cloud installations:

What are the best practices for automating Private Cloud installation?

  • Learn how to install using Private Cloud installation process first (manual installation).
  • Don’t focus on automation. Focus on documenting and testing runbooks and once you are happy with them, proceed automating.
  • Decide if automating Private Cloud install is a true requirement for your operating model and scope. If you only need to install 2 or 3 planets, you may be better off doing manual install. Private Cloud installers already automate a significant part of the automation process.
  • Autoscaling. Private Cloud provides all the necessary building block to facilitate the horizontal scalability of components. Given the right automation, autoscaling is possible for Router and Message Processors (key components used on runtime traffic).

Are the tools above all I need for automation?

Well, it depends on scope, but in general terms no, fully automating operational activities for Private Cloud may require more.

Deployment and configuration management automation will require different tool set. Often the tools needed for deployment and configuration management are already present a customer’s environments since they are often already in use as part of customer’s SDLC.

What tools are suitable for deployment and configuration management automation?

The following are common tools used:

apigee-adminapi

Private Cloud 16.05 introduced new utility called apigee-adminapi. apigee-adminapi

is a wrapper to the Edge Management API, allowing you to execute a wide range of Management API operations from the command line.

apigee-adminapi opens the door to new automation possibilities and it can be easily combined with the tools mentioned above.

Out of the box, apigee-adminapi is installed in the Management Servers under the following path:

/opt/apigee/apigee-adminapi/bin/apigee-adminapi.sh

Not applicable

devops is very booming now in every feild

factocert ISO Certification in Saudi Arabia

Not applicable

We are the company of online service providers for computer and electronics types of equipment. If you need any kind of help you with promoting your business online and computer repair service. If you face issue Norton error 8504, on Norton antivirus. Then you can contact Norton live person helpline number to get help.

Not applicable

There's really a lot of them. I tried a bunch and I stopped on Terraform. It does everything I need it to do, so I'm happy.

I would be a lot happier if my wife would allow me to play https://xn--lck0a4d411qemf.jp/%e3%82%aa%e3%83%b3%e3%83%a9%e3%82%a4%e3%83%b3%e3%82%ab%e3%82%b8%e3%83%... a little bit more. Just a little bit. She's a little dramatic, my wife. Says that I don't spend enough time at home as it is.