How to set up a standalone Swift installation in a VM and test it

In this tutorial, we will be speaking about a “Swiftbox”. This is nothing more than our terminology for an Openstack installation that only needs and uses Swift and Keystone. The use and setup of this Swiftbox will be explained in this article.

The reason why someone might want a stripped-down OpenStack installation with only Swift and Keystone running, is that it allows easy testing of Swift services. A Swiftbox can be used to try understanding how object storage works. Also, having an independent object storage, is a good perk: it allows testing or running various different projects, with only one Swiftbox to be configured for everything.

A use case for this standalone Swift installation is to reproduce an isolated and potentially local environment to test applications that need or want to use Swift as their object storage backend. This can prove useful to experiment with the technology as well as to debug or exercise existing applications.

For the simplified nature of the Swift installation here described (everything runs inside a VM and not over a cluster of nodes!), this procedure should not be considered to run Swift for anything else than a stubby backend for a testing environment.

The main steps to set up a Swiftbox are:

  • Creating and configuring a VM (we will use Vagrant and run a Ubuntu Server 14.04 box)
  • Configuring Devstack
  • Configuring a Keystone endpoint for Swift
  • Testing Swift
  • Some troubleshooting

Continue reading

Getting Started with OpenShift and OpenStack

In Mobile Cloud Networking (MCN) we rely heavily on OpenStack, OpenShift and of course Automation. So that developers can get working fast with their own local infrastructure, we’ve spent time setting up an automated workflow, using Vagrant and puppet to setup both OpenStack and OpenShift. If you want to experiment with both OpenStack and OpenShift locally, simply clone this project:

$ git clone https://github.com/dizz/os-ops.git

Once it has been cloned you’ll need to initialise the submodules:

$ git submodule init
$ git submodule update

After that just you can begin the setup of OpenStack and OpenShift. You’ll need an installation of VirtualBox and Vagrant.

OpenStack

  • run in controller/worker mode:
      $ vagrant up os_ctl
      $ vagrant up os_cmp
    

There’s some gotchas, so look at the known issues in the README, specific to OpenStack. Otherwise, open your web browser at: http://10.10.10.51.

OpenShift

You’ve two OpenShift options:

  • run all-in-one:
      $ cd os-ops
      $ vagrant up ops_aio
    
  • run in controller/worker mode:
      $ cd os-ops
      $ vagrant up ops_ctl
      $ vagrant up ops_node
    

Once done open your web browser at: https://10.10.10.53/console/applications. There more info in the README.

In the next post we’ll look at getting OpenShift running on OpenStack, quickly and fast using two approaches, direct with puppet and using Heat orchestration.

Automated OpenStack High Availability installation now available

The ICCLab developed a new High Availability solution for OpenStack which relies on DRBD and Pacemaker. OpenStack services are installed on top of a redundant 2 node MySQL database. The 2 node MySQL database stores its data tables on a DRBD device which is distributed on the 2 nodes. OpenStack can be reached via a virtual IP address. This makes the user feel that he is dealing with only one OpenStack node. All OpenStack services are monitored by the Pacemaker tool. When a service fails, Pacemaker will restart it on either node.

Fig. 1: Architecture of OpenStack HA.

Fig. 1: Architecture of OpenStack HA.

The 2 node OpenStack solution can be installed automatically using Vagrant and Puppet. The automated OpenStack HA installation is available on a Github repository.

Automated Vagrant installation of MySQL HA using DRBD, Corosync and Pacemaker

Fig. 1: Redundant MySQL Server nodes using Pacemaker, Corosync and DRBD.

Fig. 1: Redundant MySQL Server nodes using Pacemaker, Corosync and DRBD.

If automation is required, Vagrant and Puppet seem to be the most adequate tools to implement it. What about automatic installation of High Availability database servers? As part of  our Cloud Dependability efforts, the ICCLab works on automatic installation of High Availability systems. One such HA system is a MySQL Server – combined with DRBD, Corosync and Pacemaker.

In this system the server-logic of the MySQL Server runs locally on different virtual machine nodes, while all database files are stored on a clustered DRBD-device which is distributed on all the nodes. The DRBD resource is used by Corosync which acts as resource layer for Pacemaker. If one of the nodes fails, Pacemaker automagically restarts the MySQL server on another node and synchronizes the data on the DRBD device. This combined DRBD and Pacemaker approach is best practice in the IT industry.

At ICCLab we have developed an automatic installation script which creates 2 virtual machines and configures MySQL, DRBD, Corosync and Pacemaker on both machines. The automated installation script can be downloaded from Github.

OpenStack Grizzly installation for the lazy

As kindof advertisement for the new OpenStack Grizzly release we have created an automated single-node OpenStack Grizzly installation which uses Vagrant and Puppet. The automated installation can be downloaded from Github using the following URL: https://github.com/kobe6661/vagrant_grizzly_install.git

Please feel free to install it on your machine and test the new release.

DRBD-Test environment for Vagrant available

There is always room to test different HA technologies in a simulated VM environment. At ICCLab we have created such a DRBD test environment for PostgreSQL databases. This environment is now available on Github.

The test environment installation uses Vagrant as tool to install VMs, Virtualbox as VM runtime environment and Puppet as VM configurator. It includes a Vagrant installation script (usually called a “Vagrantfile”) which sets up two virtual machines which run a clustered highly available PostgreSQL database.

In order to use the environment, you have to download it and then run the Vagrant installation script. The Vagrant installation script of the test environment essentially does the following things:

  • It creates two virtual machines with 1 GB RAM, one 80 GB harddrive and an extra 5 GB harddrive (which is used as DRBD device).
  • It creates an SSH tunnel between the two VM nodes which is used for DRBD synchronization.
  • It installs, configures and runs the DRBD device on both machines.
  • It installs, configures and runs Corosync and Pacemaker on both machines.
  • It creates a distributed PostgreSQL  database which runs on the DRBD device and which is managed by the Corosync/Pacemaker software.

This environment can easily be installed and then be used for testing of the DRBD technology. It can be downloaded from the following Github repository:

https://github.com/kobe6661/dependability_test_fw.git

Installation instructions can be found here.

Vagrant, Devstack and the ICCLab

What?

So what is vagrant? In the words of its creator it allows you to:

“Create and configure lightweight, reproducible, and portable development environments.”

Vagrant is a ruby framework that automates a lot of the boring, painful setup a developer needs to do to work with services. In the case of the ICCLab those services are generally OpenStack services. We use vagrant to create consistent reproducible setups of our testbed on local development machines.

Why?

In the ICCLab we operate two testbeds, one that is stable and operates an OpenStack environment that does not change often. The other is a research testbed that is used to investigate the latest features of OpenStack, evaluate our own modifications or experiments upon OpenStack (e.g. Hadoop, CloudFoundry etc.). In order for code modifications to be placed on to the research test bed it must first prove that it is worthy. To prove itself it must be shown that it can run locally on a laptop/desktop and can be installed and configured automatically. The great advantage of this is that vagrant supports the same configuration framework, puppet, as is used on the test beds. Essentially what vagrant allows us to do is model our infrastructure but locally before deploying changes to metal.

How?

So the best way to get started with vagrant is by example. In this example, we’ll show you how to create a vagrant project to create an OpenStack devstack environment.

Install it!

To install vagrant, make sure you have virtualbox already installed. Then simply install it. On a mac it’s easiest to use the bundled installer but otherwise just execute gem install vagrant. Once installed execute vagrant help so see what you can do. You should see something like this:

[gist id=5309928]

The most common commands you’ll use are up, halt, reload and ssh

Play with it!

The example we will bring you through is setting up a devstack environment. To see all the code check out the github project here.

The first thing you need to do when creating a new vagrant project is to create a directory to host all your files. Once done you’ll need to execute:

[gist id=5310061]

Once done you should find a Vagrantfile created in your directory. This contains a basic template of how your vagrant project. For the purposes of this example we’ll use the following content:

[gist id=5310052]

What is important to note in this devstack_config.vm.box. This tells vagrant what ‘box’ it will use. A box is simply a VM image with a particular initial configuration (see here for more details). Boxes can also be created with veewee. You can also install other boxes from vagrantbox.es.

The next most important piece in this is the devstack_config.vm.provision block. This details how your software will be installed. In this example we are using puppet (in local mode) to install devstack. In the code block we specify where to find additional modules and where to find the vagrant specific manifests. Most importantly we note that the main “entry point” manifest is (devstack_puppet.manifest_file variable).

In our example, site.pp encodes the following steps to create our devstack VM:

  1. Install git
  2. Check out the devstack repository
  3. Customise the devstack installation by setting up the devstack localrc file
  4. Run devstack by executing stack.sh

You can see the contents of this manifest here.

If you’ve got this far then with the vagrant project cloned from github all you’ll have to do to get your devstack VM up and running is:

vagrant up

 

Easy eh?

Wrap up

The latest vagrant will add support for provisioning on the cloud (Amazon, OpenStack, Rackspace) and is also independent of hypervisor choice including support (paid) for VMware fusion.

How to Test your OpenStack Deployment?

Like us in the ICCLab, you have likely spent lots of time researching the best means to deploy OpenStack and you’ve decided upon a particular method (at the ICCLab we use foreman and puppet). You’ve implemented OpenStack with your chosen deployment plan and technologies and you now have an operational OpenStack cluster. The question you now have to ask is:

“How do I test that all functionality is operating correctly?”

You could certainly take the time to write a suite of tests using the various OpenStack python clients and maintain those. However there is an OpenStack project already available that can save you a lot of time. OpenStack Tempest is a project and suite that comprises of a set of integration tests. Tempest is used to validate the OpenStack code base through it’s integration with Jenkins (continuous integration server). Tempests calls against OpenStack service API endpoints and uses the python unittest2 and nosetest frameworks at its core.

If you wish to experiment with Tempest locally, try it out with devstack. Devstack automatically configures Tempest for use with it. To ease things, simply use vagrant-devstack (README here) do the following:

  1. Install VirtualBox
  2. Install vagrant
  3. git clone https://github.com/dizz/vagrant-devstack.git
  4. vagrant up
  5. vagrant ssh
  6. cd /opt/stack/tempest
  7. ./run_tests.sh

You will now see quite an amount of tests being run against your devstack installation. It will take time! If you wish to integrate Tempest with your Jenkins CI server see information on devstack gate. There is also a Tempest Jenkins plugin. Finally, if you wish to run Tempest against a “real” installation of OpenStack you will need to configure the Tempest configuration file (etc/tempest.conf) and change the relevant information (more here).