Cloudberry Backup review

Yes, I’ve decided to write another review.  It was never my plan when I started this blog to be a review site, but I just felt this product was too good and too few people had heard of it.  Cloudberry Backup is a personal backup product for protecting your systems that can utilize almost all of the public cloud providers out there as a storage target.  One look at the cloud storage selection screen below will show you it’s a dizzying array of options.

Picture1.png

The interface is nice and clean too.  Let’s walk through how you would setup backup to Amazon Glacier, which is a very cost effective method for storing data that you don’t need to retrieve frequently (or quickly), which makes it perfect for personal backup.

I realize it’s ironic me reviewing a backup product, considering I work at a backup company.  However, Cloudberry targets the individual and small business segments, which is much different than our product.  Another great use case for this might be to backup a home lab, as well, something I imagine many of my readers have.

Configuring Cloudberry Backup

First, you need to create an account with Amazon.  If you haven’t done that yet, Cloudberry placed a helpful link in the dialog box that will take you to AWS account creation.  Then, once you’ve done that, you name the storage account and enter in the Access Key and Secret key from your AWS account.  After inputting the Access Key and Secret Key, you’ll be able to select the Vault name from the drop down box.  That is where data for this account will be stored.

Picture2

Next you’ll create a backup plan to store data in the account you created.  A really nice feature of the product is that it stores your backup plans with the data so if you ever have to restore, you’ll have everything you need.  Also, it can synchronize with data already stored in the cloud, which was very helpful for me recently when I had to rebuild my computer and re-install Cloudberry.  I didn’t have to send all the data again to the archive.  Here is what my backup plans look like.  Obviously, you’ll create your own to match the data you have.

Picture3

First you’ll select the type of backup. Cloudberry gives you the option to backup locally first then from there to the cloud and calls this a Hybrid Backup.  We’ll be doing the Cloud Backup.

Capture.PNG

Then, you pick a backup storage account (or create one):

Capture.PNG

Name the backup and leave the box checked to put the backup plan config with the backups.

Capture

We’ll be doing a regular backup.  Cloudberry has the option of combining a lot of small files into what it calls an Archive Backup but that won’t be needed here.  It can be useful if you have a lot of small files because it reduces the number of requests to the cloud, which is something providers charge you for.  Also, you can select several options around how VSS (Volume Shadow Copy Service) treats your files.  This is a service in Windows that allows open files to be backed up, when they would normally be locked by the program that opened them.  For a Documents folder, I recommend turning on VSS and using the System VSS provider, as it will ensure docs you’ve opened get protected.

Capture.PNG

Select your backup source:

Capture

Next, are a bunch of Advanced Options that you may or may not need.  Cloudberry gives you some fairly sophisticated options to skip files, or only backup files of a given type (like PDF and DOC files).  The full version offers Compression and Encryption.  However, there is a very good free edition for home use which doesn’t have those options.  It does provide all the data protection functionality you might need though.

After going through the Advanced filters and the Compression/Encryption options, you can determine a retention policy for your data.  This allows you to keep versions of your data, if needed.  For most home use, you’ll probably want to keep just the latest version of your files.

Then, the backup can be scheduled, as you’d expect.  The product also has a Real-Time feature, which constantly monitors a given backup set and copies them to the cloud storage.  This might be useful for a Documents or Projects folder, where you wouldn’t want to lose data that you worked on in a given day.  In backup terms, your RPO (Recovery Point Objective) would be effectively zero.

Your last option is to determine how you want to be alerted if backups fail.  The system can email using Cloudberry’s service, or you can specify your own server if you happen to have one (Gmail provides SMTP service to it’s users, for example).  The last screen before you create the plan is a summary screen to validate your settings.

Capture.PNG

Cost Analysis

So, this is where a product like Cloudberry really shines.  I was previously paying a service over $50/year for a limited (300GB) amount of storage.  Also, I was completely locked into their service and at the whim of them changing or going out of business.  Years ago, I used a product that did that and I had to start my backups all over.  For a while, I was in a scary, unprotected state where if my system had taken a dirt nap I would have had no way to recover.

Anyways, I’m currently spending around $1 a month keeping my data in Glacier.  Considering that the product is either free, or $30 for the paid version, the savings adds up pretty much after the first year.  As you can imagine, YMMV but I expect the flexibility and savings will really appeal to my readers (who I imagine are a bit more on the technical side).

Summary

Cloudberry Backup Desktop Edition is a great product for backing up your home computers.  It offers amazing flexibility over where you want to send your data, including all the major cloud players.  If you need a product to backup your personal stuff (and we all should!), I highly recommend it.

Disclaimer: Cloudberry Labs provided me a free license to the paid edition as a vExpert.  However, I can honestly say I would continue using the product even without that consideration.

What is DevOps?

I realize this topic has probably been beaten to death, but I had to put together a presentation for a group of my peers and I thought it might work as a blog post.  Plus, adding it here helps me internalize my thoughts about a topic.  I hope some of it is a useful distillation of the information out there on this huge topic.  If you find it interesting, I highly recommend checking out one of the books I list below.

A Definition:

DevOps is the practice of operations and development engineers participating together in the entire service lifecycle, from design through the development process to production support.  It’s also characterized by operations using many of the same techniques as developers.

Think automated infrastructure provisioning.  You’ll frequently hear the phrase “infrastructure as code.”  What that means is that provisioning activities are driven by a recipe that can be treated like a program.  For example, the application Puppet has a concept called “manifests” which are used to create an application and also to determine if the running machines comply to that specification.

The Three Ways:

In “The Phoenix Project,” Gene Kim talks about the Three Ways, methods used to continuously improve IT operations.  These have been taken from manufacturing theories used in many organizations today.  (credit for the images goes to Gene Kim on his https://itrevolution.com/ website)

The First Way

Picture1

The First Way emphasizes the performance of the entire system.  It also encourages IT to look at Operations as a Customer of Development.  It consists of Dev creating services which are transitioned to Operations and then consumed by the Business.

The Second Way

Picture2.png

The Second Way is all about feedback loops.  There should be continuous feedback about the results of the product delivered to Operations by Development.  This enables continuous improvement to be built-in.

The Third Way

Picture3.png

The Third Way is about the culture of the organization.  It’s about creating a culture that fosters two things: continual experimentation,  and understanding that repetition and practice is the prerequisite to mastery.  IT can be very resistant to change.  Also failures can result in finger pointing and this can create an “us versus them” environment.  I think this way is probably the hardest to implement, because it can require a real mind shift in the people of the organization.

Common DevOps Practices

Let’s talk about some of the more common practices organizations use to implement a DevOps culture.

Version Control

This is key to the concept I mentioned above around “infrastructure as code.”  You need to have some way to control the configuration of your systems and the best way to do this is some type of version control system.  Many companies are using Git and Github for this, although you might also see systems like svn and cvs.  This is also where products like Puppet and Chef come in, as they provide a way to consume these “recipes” when building and maintaining systems.

Automated Testing

Instrumental in implementing the Second Way, some type of automated testing should be built into an environment so that continual improvements can be realized.  Also, this will help minimize issues creeping into Production.  Some examples of testing frameworks include Pester and Cucumber.  These are both examples of software that is designed to provide BDD, or Behavior-Driven Development.  A good read about what BDD is and why it can help improve your processes and app development is here.  You can also find a good intro into testing methodologies here.

Virtualization

This is almost an obvious one, but the advent of virtualization enabled the implementation of DevOps throughout organizations.  It made it much simpler to deploy systems automatically and based on a configuration described by code.  Systems like containers and Docker have taken this to the next level by abstracting even further from the underlying hardware.  New tools like NSX and network virtualization extend this promise of “infrastructure as code” by allowing Ops to control not only the systems, but also the networks that connect them.

More Reading

Here are some good resources if you want to delve more into the world of DevOps and help improve your environment.

A Rubrik Python Primer

Capture

One of my co-workers over at virtuallysober.com recently posted about using Rubrik’s REST API with PowerShell.  As I’ve been working on my Python-fu, I thought I’d piggyback (or steal…) on his idea and do a similar thing with Python.  First, I’ll distill some of the things I’ve learned about consuming RESTful APIs with Python.  Then, I’ll dive into some of the things you might do with our APIs.

RESTful API Primer

I won’t spend a ton of time on this, as there are a lot of good references out there on what a REST API entails.  The first place to start, like most things, is with Wikipedia.  That might be a bit dry, so a less pedantic place to learn about it might be here.  The basics are that you can communicate with a web service using very straightforward commands like GET, PATCH, POST, etc… The big concept is that those commands are stateless, with the command containing all of the information or state to perform the action.  Also, the API will specify something called an endpoint, which is basically a URL that can accept these RESTful commands.  Rubrik makes it nice to determine what those might be by publishing the documentation on the cluster itself:

https://<rubrik_ip_address>/docs/v1/

Talking to a Webserver in Python

curl

First of all, we need a way to talk to the Rubrik.  You can do this in a couple different ways in Python.  The first is the “curl” command which would look something like this:

curl -k -u admin:pass -X GET 'https://<rubrik_ip_address>/api/v1/vmware/vm'

Let’s parse the above command.  We’ve used the “k” flag to bypass an alert about self-signed certs.  Also, we specified the username and password after the “u” flag.  The next thing is we tell the server what HTTP method we’ll use; here we used a GET command.  Then, the actual endpoint is used.  In this case, we’re asking the cluster for a list of all the VMs, which will be returned as an array of key-value pairs.

However, a big problem with this method is that we need to put our password in plaintext in our code.  What if we wanted to create a “token” instead that could be used in other commands.  We need first to get an authorization code from the Rubrik in order to validate our access to the system.  How do we do that?  By hitting another endpoint, of course!

curl -k -u admin:pass -X POST "https://<rubrik_ip_address>/api/v1/session"

The response will be an array containing the session ID, the token, and the User ID.  Then, the token can be extracted from the array and then used in subsequent commands to the system like so:

curl -k -H 'Authorization: Bearer $token_id' -X GET 'https://<rubrik_ip_address>/api/v1/vmware/vm'

The requests library

Curl is one way to access your system, but probably not the most useful.   A better method when you want to use it programmatically is the excellent Requests library in Python.  This is a library that allows your program/script to pass HTTP requests natively and use the data that returns.  The documentation for requests is very good and you can find it here.

Let’s go through a basic example of how you might connect to Rubrik similarly to the above example.  First, we need to import the requests module, then we will create an object that contains the VMs.

import requests
r = requests.get('<rubrik_ip_address>/api/v1/vmware/vm', verify = False, auth =('admin','pass'))

We use the ‘verify = False’ because the system is using a self-signed certificate.  Again, this has the problem of putting the password in the code in plaintext.  We could get around that by encoding the password with the base64 module then passing it into each command.  However, it’s much more useful to authenticate the session and use the token in each of the proceeding commands.

import requests
session = requests.post('<rubrik_ip_address>/api/v1/session', verify = False, auth =('admin','pass'))
session_token = session.json()
authorization = 'Bearer ' + session_token['token']
vm_list = requests.get('<rubrik_ip_address>/api/v1/vmware/vm', verify = False, headers = {'Content-Type': 'application/json', 'Authorization': authorization})
vm_list_json = vm_list.json()

You’ll notice we take the results of the initial POST command and contain them in the ‘token’ object.  Once we’ve done that, we can access values from with that object by referencing the key, in this case our key is ‘token’.  Once we’ve stored the results of our command in the ‘vm_list’ object, we can retrieve information from it by using the same method we retrieved our key – calling keys that are contained within the JSON file.

Learning More

Now, if you’ve read any of my previous posts, you know I’m a relative novice to the world of Python programming.  So, this represents the very basics of connecting to your Rubrik (or any RESTful system, for that matter).  I recommend going into your system and exploring both the documentation and also our explorer, which is based on the Swagger framework.

In future posts, I’ll go into how you might actually use this information in your day-to-day operations and scripts.

 

Also, if you’d like to learn more about our API and how you might use PowerShell with it, check out my colleague Joshua Stenhouse’s blog at https://virtuallysober.com/2017/05/08/introduction-to-rubrik-rest-apis-using-powershell-swagger/.

Ravello Systems Review

Recently, I was given access to an account at Ravello Systems (full disclosure: this is a free account given to vExperts) and I thought I’d write about my experience. For those of you that don’t know, it’s a front end for deploying workloads in AWS and was bought by Oracle in 2015.

I’ve had an account with them for a while, but really never needed to utilize it due to having some pretty sweet home lab gear provided by my previous job. However, with me going over to Rubrik in March, I obviously had to return that stuff and I’m not sure if I’m going to purchase new gear on my own. It’s just getting so you don’t need a homelab for a lot of things anymore, and tools like Ravello make that possible.

Well, on to the review. The interface is really nice in that it looks like a standard blueprint and has a lot in common with a Visio or Lucidchart drawing. You add components to the design and on the right pane you can configure their settings.

ravello1.PNG

The account comes with various pre-configured VMs, like the one shown above which can be used to install ESXi.  I’m building a vSphere farm in that example.  You do have to provide your own software images and licenses, but they can be uploaded easily.  Once that’s done, you simply connect the ISO to the VM and install ESXi normally.  You can also do some cool things with their import tool, like pulling in running VM images from your existing vSphere environments,  sort of like a V2V converter.

The networking options are fairly robust, as well.  You can configure DHCP or static addresses, as well as control which NICs have external access.

Finally, the entire platform has a REST API available, if you want to automate the provisioning or management of your environments here.  This could be really powerful, as it extends the functionality to any scripts or automation tools you might have.

ravello2.PNG

For a potential homelab / SMB lab use, I think this could be really powerful.  It reduces or eliminates the need to buy gear that will eventually become obsolete (or get taken back by your previous employer!).

A New Adventure

So, yesterday was my first day at Rubrik and I couldn’t be more excited.  I had a great 4 1/2 years at IVOXY in the VAR (value-added reseller, for those of you wondering) space, but figured it was time to go back to an IT vendor.  I couldn’t be more happy so far with the company and the team I’m working with.

If you’re not familiar, Rubrik was founded to look at backup in a different way.  We don’t require difficult installation, or complex policies, or even a full-time backup admin.  With an appliance model, and simple configuration, we can be up and protecting an environment in under an hour.  Let’s see any of the traditional software on hardware options do that!

Anyways, if you want more information about Rubrik and what we do, you can find really good info on our website. Cheers!

Don’t backup, go forward!

Deploying NSX 6.3 in a vSphere 6.5 Environment (Part 2)

Last time, we went through the initial setup steps to get NSX 6.3 deployed in your vSphere 6.5 environment.  Today, we’ll finish all of the initial configuration and get your environment to a place where you can start deploying the tasty bits of NSX, like distributed firewalling.

When you first login with your user, you may see the following error.  This is because you need to explicitly give rights to the NSX installation and NSX Manager.  It is initially only given to the login you used to install the service, generally the administrator@vsphere.local user.

nsx6-3_deploy_pic19

To fix it, login as the user you used to install and go to the Networking and Security section and select NSX Managers:

NSX6.3_deploy_pic20.png

Once there, select the NSX Manager on the Navigator tab and under Manage select Users.  Add the user you’d like to have access (yourself at least!) to the system and give yourself the appropriate rights.  For my lab, I’ve given myself the Enterprise Administrator role which is the NSX god role.  Then you can log back into your Web Client and you will see the NSX Manager listed and you can continue with configuring NSX.

Select installation under Networking & Security.  Now we’re going to deploy the NSX Controller nodes.  In a production environment, three controllers are deployed for each NSX instance.  As the boys from Monty Python said, “the number shall be three, two is not enough and four shall be right out.”  I’m paraphrasing, obviously, but the gist is that there are exactly 3 NSX Controller nodes in an NSX implementation.  I’m only going to deploy one due to resource limitations, but you can do that in a lab environment.

Click on the green plus symbol to add a controller and fill in the dialog box:

nsx6-3_deploy_pic12

IF you haven’t created any IP Pools yet, you’ll need to do that to continue deploying the controller.

nsx6-3_deploy_pic14

Click on Installation under Networking & Security and then Host Preparation.  Once there, select the cluster you want to install NSX on and under Actions click Install.  As you can see, I’m following VMware’s recommended practice of having a resource and a management cluster.   If you do this in production, you may also have an Edge cluster to hold any Edge devices you deploy.

This is where we install the ESXi VIBs and then complete the configuration of the VXLAN transport network.  If you have problems, as I did, you can follow the steps in this KB: https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2075600

I ended up trying to manually install the VIB on each of my hosts using this KB about a problem with vSphere Update Manager: https://kb.vmware.com/selfservice/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=2053782.  It still failed, but after patching my ESXi hosts to the latest version using the great instructions Vladan has here I was able to install the VIBs from the GUI.

NSX6.3_deploy_pic13.png

Once that’s finished, we’ll move on to the creation of the VTEP (VXLAN tunneling endpoint).  This will create a portgroup on the distributed virtual switch that I already created.  Ultimately, it creates a new vmkernel port on each host in that portgroup that the system uses as the VTEP.

You’ll need a larger than standard MTU for this to work correctly, as the VXLAN encapsulation adds bytes to the end of the frame.  The minimum is 1550, although the recommended value is 1600 bytes.  Keep in mind, the underlying network must support the increased value.  When the system asks for IP addresses of the VTEP, I recommend using an IP Pool like we did for the controller(s).

Click on Not Configured under the VXLAN column, choose the VLAN and IP addressing scheme and click Finish:

nsx6-3_deploy_pic16

When it’s done, you should see the following:

NSX6.3_deploy_pic17.png

At this point, NSX is configured and operating on your cluster.  You can see the portgroup created for the VXLAN traffic by going to Networking and selecting the portgroup on your distributed virtual switch:

NSX6.3_deploy_pic18.png

Next time, I’ll dive deeper into some of the cooler features of NSX 6.3 and what things you might deploy initially to justify the money you’ll have to spend for the licensing!

Protecting Children with Technology

I’m sure a lot of you have seen the recent testimony by Ashton Kutcher in front of the Senate concerning child exploitation.  If you haven’t, you can see it here: https://www.youtube.com/watch?v=HsgAq72bAoU

It really moved me and got me thinking about what a wonderful use of technology his Thorn project is.  I’ve donated some money and also offered up my services around data center infrastructure.  They’re looking for all different types of technologists, but especially coders who can help disrupt the online activities of these predators.  If you have mad skills in these areas, I highly recommend checking them out and maybe donating some of your time and knowledge to an amazing project.

Here is the link to their site: https://www.wearethorn.org