In this blog post we will describe the necessary steps to get an installation of OpenShift Origin v3 up and running on OpenStack. OpenShift Origin v3 offers a ton of features over its predecessor we covered over a year ago. Most notably is the support for Docker containers and the usage of Kubernetes.

Getting ready

Before we start the installation, a few prerequisites must be met.

This tutorial relies on the official Ansible installer, therefore you need to install Ansible on your system of choice. Instructions for the installation for various platforms can be found here.

As we are going to deploy OpenShift Origin V3 on OpenStack, you need to have access to an OpenStack cloud that features the orchestrator Heat and uses Neutron for networking and have appropriate quotas set on your tenant. Depending on the size of your OpenShift deployment, more or less is required, but take these guidelines as a recommended minimum:

• Compute: 5*m1.large or equivalent flavor
• Networking: 5 Floating IPs, 1 network/subnet/router, 3 security groups

Your OpenStack installation requires a CentOS 7.1 image available to your user. If it is not available, you can get the latest one here.

Last but not least, you need access to a domain or subdomain and DNS server where you can create A records and wildcard A records.

Starting the installation

Start the installation by cloning the installer from git:

git clone https://github.com/openshift/openshift-ansible.git
cd openshift-ansible

Next, make sure you have your OpenStack credentials in your shells environment. This might look something like this:

source openstack-openrc.sh

Now we start the actual installation via the installer:

bin/cluster create -t origin \
-n 3 \
-o image_name=CentOS-7-Generic-Cloud \
-o external_net=external-net \
-o floating_ip_pool=external-net \
-o master_flavor=m1.large \
-o node_flavor=m1.large \
-o infra_flavor=m1.large \
openstack \
opsv3

Make sure to adopt the parameters to your OpenStack environment. Here is a brief explanation of the parameters:

bin/cluster create -t origin: deploy the free "Origin" version
-n: number of compute nodes to provision
-o image_name: name of the CentOS image to use
-o external_net: your external network
-o floating_ip_pool: network for floating ips
-o master_flavor/node_flavor/infra_flavor: flavors for the roles
openstack: use the openstack provider
opsv3: name for the Heat Stack created

 

The installation will take a while, consider having a coffee 😉

Post-Install

Once the installation is done, a few modifications are necessary to finish the setup. First,  ssh to your master node. To find its IP address, either use the OpenStack Dashboard Horizon or the CLI:

# heat output-show ${STACK_ID} master_floating_ips

 

On the master node, edit /etc/origin/master/master-config.yaml with the editor of your choice. Replace the following values with your your domain:

* masterPublicURL
* assetPublicURL
* publicURL
* corsAllowedOrigins
* subdomain

Note that some values have multiple occurrences in the file. Below are example values for our domain openshift3.cloudcomplab.ch:

assetConfig:
  masterPublicURL: https://master.openshift3.cloudcomplab.ch:8443
  publicURL: https://master.openshift3.cloudcomplab.ch:8443/console/
oauthConfig:
  assetPublicURL: https://master.openshift3.cloudcomplab.ch:8443/console/
corsAllowedOrigins:
 - master.openshift3.cloudcomplab.ch
 - openshift3.cloudcomplab.ch
 - 160.85.4.68
routingConfig:
 subdomain: "apps.openshift3.cloudcomplab.ch"

Next, we are going to replace all occurrences of opsv3-master-0.novalocal to our masters domain name in a couple of files:

/etc/origin/master/admin.kubeconfig
/etc/origin/master/openshift-master.kubeconfig
/etc/origin/master/openshift-registry.kubeconfig
/etc/origin/master/openshift-router.kubeconfig

Please note that the novalocal name might be slightly different in your installation depending on the name you specified by the installer.

Create the Docker Registry

Next, we create a local docker registry. This is easily accomplished with a single command as seen below. The oadm command is used for the administration features of OpenShift 3.

oadm registry --config=/etc/origin/master/admin.kubeconfig \
--credentials=/etc/origin/master/openshift-registry.kubeconfig

 

Create the Service Router

Creating the service router is a little more work. We will work with the oc command, which handles the basic user interactions with OpenShift 3 on your command line. First we create a new service account:

echo '{"kind":"ServiceAccount","apiVersion":"v1","metadata":{"name":"router"}}' | oc create -f -

Next, we add the created service account to the privileged users:

oc edit scc privileged

This opens a file in your $EDITOR. Add the following line under users:

users:
- system:serviceaccount:default:router

Now we are ready to create the service router:

oadm router service-router --replicas=4 --credentials=/etc/origin/master/openshift-router.kubeconfig --service-account=router

Note that we specified 4 replicas. This matches the amount of compute-nodes we specified in the installer plus the infrastructure node that also serves as a compute node. This way, a service router is deployed on each node. In the past we experienced problems with the service router dying and getting rescheduled to a different node. This way, OpenShift will keep a service router on every node at any time.

To verify that each node has a service router, use something along these lines:

# oc get pods | grep service-router-1 | while read pod trash ; do oc describe pods ${pod} | grep 'Node:\|^Name:' ; done
Name: service-router-1-2lzm2
Node: 192.168.113.5/192.168.113.5
Name: service-router-1-hx767
Node: 192.168.113.6/192.168.113.6
Name: service-router-1-ra2um
Node: 192.168.113.7/192.168.113.7
Name: service-router-1-z6qys
Node: 192.168.113.3/192.168.113.3

DNS setup

Lastly, we create the necessary DNS records. This will differ depending on your DNS setup. In our case, we use CloudFlare as our DNS service, below is an image of the relevant DNS records created:

 

As you can see, there is an A record for the master as well as a wildcard A record for the subdomain that points to all our compute nodes.

Take OpenShift for a spin!

Now your initial OpenShift setup is complete. As further steps, you might want to configure authentication or try out one of the examples.