by Josef Spillner
An academic entity – more concretely, a research laboratory – resembles a stateful function: It receives input and generates output based on both the input and on previously generated knowledge and results. The input is typically a mix of fancy ideas, industry necessities, as well as funding and equipment. The output encompasses publications, software and other re-usable artefacts.
In the Service Prototyping Lab, we rely on access to well-maintained cloud environments as one form of input to come up with and test relevant concepts and methods on how to bring applications and services online on top of programmable platforms and infrastructure (i.e., PaaS and IaaS). This Samichlaus post reports on our findings after having used several such environments in parallel over several months.
Joyent recently open sourced the IaaS Platform SmartDataCenter and the Object Storage Manta, the software they use for their own service offerings. So, what’s all the buzz about? Why should you be excited? Why is it even worth talking (or in this case, writing) about SDC when we have OpenStack? In this blog post I will cover some of the fundamentals of SDC and why it’s worth a second look.
by Josef Spillner
The relocation of the ICCLab hardware and the integration of 9 additional nodes is now complete. The whole movement was done within one day thanks to the support of Pietro, Philipp and Michael – Thanks Guys! Now our lab runs 15 compute nodes, 1 controller node and 1 NAS. We will segment this infrastructure to build a development environment including 10 nodes where we can develop and test our work on OpenStack and a production environment including 5 nodes for production purposes. As the next step we are will to redeploy OpenStack by means of automation tools Puppet and Foreman as was presented at the EGI Technical Forum. Let’s see how fast we can deploy 15 nodes from scratch! We’ll be studying, timing and evaluating it!
It doesn’t make sense to continually download the same operating system packages when you can cache them along side your Foreman installation. Assuming you use a debian based OS, in order to do this and have a cached copy of all packages you use within your infrastructure simply install apt-cacher or apt-cacher-ng. Our preference is for apt-cacher-ng but we’ll show you how to install both.
# Installing apt-cacher-ng
You shouldn’t have to adjust the configuration of apt-cacher-ng for the basic functionality it offers, however if you need to adjust settings in `/etc/apt-cacher-ng/acng.conf`.
# Installing apt-cacher
Then set the contents of `apt-cacher-conf` to:
***Note:*** you might want to change the hostname of the ubuntu mirror and interface (eth1 is specific to previous articles on Foreman)
Reports are created every 24 hours, which can be accessed at `http://$FOREMAN:3241/report`. To force the creation run:
# Configuring Foreman
Once you have setup apt-cacher you can then create a new Foreman “Installation Media Source”. Simply supply a sensible name and importantly set the URL of that installation media source to `http://$FOREMAN:3241/ubuntu`, where $FOREMAN is either the IP or FQDN of your Foreman host.
Here in the ICCLab, the framework of choice is OpenStack, which enjoys significant industry and academic support and is reaching good levels of maturity. The lab will support pre-production usage scenarios on top of OpenStack services as well as experimental research on OpenStack technology and extensions. Currently the actively deployed OpenStack services are the OpenStack compute service (including keystone, glance and nova), and Swift, an object storage service.
The lab is equipped with COTS computing units, each running on 8×2.4 Ghz Cores, 64GB RAM and 4×1TB local storage per unit. To store templates and other data there is a 12TB NFS or iSCSI Storage which is connected to the switch with a 10Gbit Ethernet interface. The Computing Units are connected to a 1Gbit network for data and another 1Gbit net for control traffic.
At the heart of the CCLab is the Management Server, which provides an easy way to stage different setups for different OpenStack instances (productive, experimental, etc.). The Management Server provides a DHCP, PXE and NFS Server and some pre-configured processes which allow a bare metal computing unit to be provisioned automatically, using Foreman, and then have preassigned roles installed, using a combination of Foreman and Puppet. This provides a great deal of flexibility and support for different usage scenarios.
Puppet is an infrastructure automation system for the efficient management of large scale infrastructures. Using a declarative approach puppet can manage infrastructure lifecycles from provisioning, configuration, update and compliance management. All of these management capabilities are managed logically in a centralised fashion, however the system itself can be implemented in a distributed manner. A key motivation in using puppet is that all system configuration is codified using puppet’s declarative language. This enables the sharing of “infrastructure as code” not only through out an organisation but outside of an organisation by following open source models. Puppet is the automation framework that enables [the ICCLab research infrastructure](http://www.cloudcomp.ch/research/foundation/projects/the-init-cloud-lab/).
[OpenStack](http://www.openstack.org) is a cloud operating system that controls large pools of compute, storage, and networking resources throughout a datacenter, all managed through a dashboard that gives administrators control while empowering their users to provision resources through a web interface. OpenStack is the infrastructure management framework that is currently powering [the ICCLab research infrastructure](http://www.cloudcomp.ch/research/foundation/projects/the-init-cloud-lab/).