What is DevOps?

I realize this topic has probably been beaten to death, but I had to put together a presentation for a group of my peers and I thought it might work as a blog post.  Plus, adding it here helps me internalize my thoughts about a topic.  I hope some of it is a useful distillation of the information out there on this huge topic.  If you find it interesting, I highly recommend checking out one of the books I list below.

A Definition:

DevOps is the practice of operations and development engineers participating together in the entire service lifecycle, from design through the development process to production support.  It’s also characterized by operations using many of the same techniques as developers.

Think automated infrastructure provisioning.  You’ll frequently hear the phrase “infrastructure as code.”  What that means is that provisioning activities are driven by a recipe that can be treated like a program.  For example, the application Puppet has a concept called “manifests” which are used to create an application and also to determine if the running machines comply to that specification.

The Three Ways:

In “The Phoenix Project,” Gene Kim talks about the Three Ways, methods used to continuously improve IT operations.  These have been taken from manufacturing theories used in many organizations today.  (credit for the images goes to Gene Kim on his https://itrevolution.com/ website)

The First Way

Picture1

The First Way emphasizes the performance of the entire system.  It also encourages IT to look at Operations as a Customer of Development.  It consists of Dev creating services which are transitioned to Operations and then consumed by the Business.

The Second Way

Picture2.png

The Second Way is all about feedback loops.  There should be continuous feedback about the results of the product delivered to Operations by Development.  This enables continuous improvement to be built-in.

The Third Way

Picture3.png

The Third Way is about the culture of the organization.  It’s about creating a culture that fosters two things: continual experimentation,  and understanding that repetition and practice is the prerequisite to mastery.  IT can be very resistant to change.  Also failures can result in finger pointing and this can create an “us versus them” environment.  I think this way is probably the hardest to implement, because it can require a real mind shift in the people of the organization.

Common DevOps Practices

Let’s talk about some of the more common practices organizations use to implement a DevOps culture.

Version Control

This is key to the concept I mentioned above around “infrastructure as code.”  You need to have some way to control the configuration of your systems and the best way to do this is some type of version control system.  Many companies are using Git and Github for this, although you might also see systems like svn and cvs.  This is also where products like Puppet and Chef come in, as they provide a way to consume these “recipes” when building and maintaining systems.

Automated Testing

Instrumental in implementing the Second Way, some type of automated testing should be built into an environment so that continual improvements can be realized.  Also, this will help minimize issues creeping into Production.  Some examples of testing frameworks include Pester and Cucumber.  These are both examples of software that is designed to provide BDD, or Behavior-Driven Development.  A good read about what BDD is and why it can help improve your processes and app development is here.  You can also find a good intro into testing methodologies here.

Virtualization

This is almost an obvious one, but the advent of virtualization enabled the implementation of DevOps throughout organizations.  It made it much simpler to deploy systems automatically and based on a configuration described by code.  Systems like containers and Docker have taken this to the next level by abstracting even further from the underlying hardware.  New tools like NSX and network virtualization extend this promise of “infrastructure as code” by allowing Ops to control not only the systems, but also the networks that connect them.

More Reading

Here are some good resources if you want to delve more into the world of DevOps and help improve your environment.

A Rubrik Python Primer

Capture

One of my co-workers over at virtuallysober.com recently posted about using Rubrik’s REST API with PowerShell.  As I’ve been working on my Python-fu, I thought I’d piggyback (or steal…) on his idea and do a similar thing with Python.  First, I’ll distill some of the things I’ve learned about consuming RESTful APIs with Python.  Then, I’ll dive into some of the things you might do with our APIs.

RESTful API Primer

I won’t spend a ton of time on this, as there are a lot of good references out there on what a REST API entails.  The first place to start, like most things, is with Wikipedia.  That might be a bit dry, so a less pedantic place to learn about it might be here.  The basics are that you can communicate with a web service using very straightforward commands like GET, PATCH, POST, etc… The big concept is that those commands are stateless, with the command containing all of the information or state to perform the action.  Also, the API will specify something called an endpoint, which is basically a URL that can accept these RESTful commands.  Rubrik makes it nice to determine what those might be by publishing the documentation on the cluster itself:

https://<rubrik_ip_address>/docs/v1/

Talking to a Webserver in Python

curl

First of all, we need a way to talk to the Rubrik.  You can do this in a couple different ways in Python.  The first is the “curl” command which would look something like this:

curl -k -u admin:pass -X GET 'https://<rubrik_ip_address>/api/v1/vmware/vm'

Let’s parse the above command.  We’ve used the “k” flag to bypass an alert about self-signed certs.  Also, we specified the username and password after the “u” flag.  The next thing is we tell the server what HTTP method we’ll use; here we used a GET command.  Then, the actual endpoint is used.  In this case, we’re asking the cluster for a list of all the VMs, which will be returned as an array of key-value pairs.

However, a big problem with this method is that we need to put our password in plaintext in our code.  What if we wanted to create a “token” instead that could be used in other commands.  We need first to get an authorization code from the Rubrik in order to validate our access to the system.  How do we do that?  By hitting another endpoint, of course!

curl -k -u admin:pass -X POST "https://<rubrik_ip_address>/api/v1/session"

The response will be an array containing the session ID, the token, and the User ID.  Then, the token can be extracted from the array and then used in subsequent commands to the system like so:

curl -k -H 'Authorization: Bearer $token_id' -X GET 'https://<rubrik_ip_address>/api/v1/vmware/vm'

The requests library

Curl is one way to access your system, but probably not the most useful.   A better method when you want to use it programmatically is the excellent Requests library in Python.  This is a library that allows your program/script to pass HTTP requests natively and use the data that returns.  The documentation for requests is very good and you can find it here.

Let’s go through a basic example of how you might connect to Rubrik similarly to the above example.  First, we need to import the requests module, then we will create an object that contains the VMs.

import requests
r = requests.get('<rubrik_ip_address>/api/v1/vmware/vm', verify = False, auth =('admin','pass'))

We use the ‘verify = False’ because the system is using a self-signed certificate.  Again, this has the problem of putting the password in the code in plaintext.  We could get around that by encoding the password with the base64 module then passing it into each command.  However, it’s much more useful to authenticate the session and use the token in each of the proceeding commands.

import requests
session = requests.post('<rubrik_ip_address>/api/v1/session', verify = False, auth =('admin','pass'))
session_token = session.json()
authorization = 'Bearer ' + session_token['token']
vm_list = requests.get('<rubrik_ip_address>/api/v1/vmware/vm', verify = False, headers = {'Content-Type': 'application/json', 'Authorization': authorization})
vm_list_json = vm_list.json()

You’ll notice we take the results of the initial POST command and contain them in the ‘token’ object.  Once we’ve done that, we can access values from with that object by referencing the key, in this case our key is ‘token’.  Once we’ve stored the results of our command in the ‘vm_list’ object, we can retrieve information from it by using the same method we retrieved our key – calling keys that are contained within the JSON file.

Learning More

Now, if you’ve read any of my previous posts, you know I’m a relative novice to the world of Python programming.  So, this represents the very basics of connecting to your Rubrik (or any RESTful system, for that matter).  I recommend going into your system and exploring both the documentation and also our explorer, which is based on the Swagger framework.

In future posts, I’ll go into how you might actually use this information in your day-to-day operations and scripts.

 

Also, if you’d like to learn more about our API and how you might use PowerShell with it, check out my colleague Joshua Stenhouse’s blog at https://virtuallysober.com/2017/05/08/introduction-to-rubrik-rest-apis-using-powershell-swagger/.

Learning Python, Part 2

So, I’ve decided to switch gears in my Python journey and take a more academic approach.  To that end, I enrolled in the excellent edX program and I’m taking MIT’s 6.00.1x Introduction to Computer Science and Programming Using Python (https://courses.edx.org/courses/course-v1:MITx+6.00.1x_11+1T2017/info).  We’re on week 2 and it’s been a great way to learn programming and eventually Python.  They take a more computer science approach, teaching you how to think like  a programmer rather than just typing in stuff.  As you can imagine, being an MIT course, it’s pretty challenging already.  The first problem set used a lot of logic and required us to think about what was actually going on in the program.

I’d highly recommend edX as a way to enhance your professional knowledge, or just pursue something you’ve always liked and want to know more about it. You can take the courses for free, or pay for a verified certificate.  I’m considering the certificate, as I think an MIT course could be a good thing to put on my resume!

Learning Python

So, I’ve decided to start working on my Python skills.  I’ve always been a code borrrower, using others’ scripts and modifying them to fit my own needs. I imagine this describes a lot of you out there.   It’s always easier to edit other work than create something on your own.  Like anything in the Linux world, there are a lot of resources out there, both free and paid.  I found a good subreddit about it here: https://www.reddit.com/r/learnpython/

Based on that, I’ve decided to go through the course at Learn Python the Hard Way.  I like his approach of repetitive lessons.  Also, he stresses using the command line, rather than an IDE.  The initial exercises are really basic, but I’ve been slogging through them and I’ll update here with my progress.