[Note: This blog post was originally published on the XiFi blog here.]
One of the main jobs performed by the Infrastructures in XiFi is to manage quotas: the resources available are not infinite and consequently resource management is necessary. In Openstack this is done through quotas. Here we discuss how we work with them in Openstack.
The main modus operandi of the Infrastructures is via command line – the attractive browser based tools developed within XiFi are primarily for the end users: the Infrastuctures typically spend their time in the trenches working with command line tools. Hence, quota management is done via command line tools.
Quotas naturally arise in many areas within Openstack; indeed, each of the major components has its own quotas and quota management. For example, Nova has its own quotas, Neutron has its own quotas, Cinder has its own quotas etc. Further, as each of these projects has evolved separately, the quota management mechanisms have slightly different syntax and worse, there is some overlap between the quotas in some cases which can lead to confusion: for example, Nova quotas contain floating IPs (for the older Nova network solution) as does Neutron.
In XiFi, the main resources we deal with are instances, cores, RAM and floating IPs. Also, most of the nodes have Neutron deployed. In this context, then, the quota management can be done simply using the following Nova and Neutron commands:
nova quota-update --instances <INSTANCES> --cores <CORES> --ram <RAM> --floating-ips <FIPs> <PROJECTID> neutron quota-update --floatingip <FIPs> --tenant-id <PROJECTID>
Being dynamic administrators who strongly embrace the DevOps approach, we decided to write some basic scripts to support modification of quotas (this was emphatically not because we could not remember the syntax of the above commands!).
This proved less trivial than we thought as the documentation for the python libraries for quota management in Openstack is limited at best. Here we highlight some of the points relating to using quotas programmatically in Openstack.
Quotas are managed through the Nova and Neutron clients. Consequently, these must be initialized using the standard approaches.
from neutronclient.neutron import client as neutronc
from novaclient import client as novac
neutron = neutronc.Client('2.0',
auth_url=os.environ['OS_AUTH_URL'],
tenant_name=os.environ['OS_TENANT_NAME'],
username=os.environ['OS_USERNAME'],
password=os.environ['OS_PASSWORD'],
region_name=os.environ['OS_REGION_NAME'])
nova = novac.Client('2', auth_url=os.environ['OS_AUTH_URL'],
tenant_id=os.environ['OS_TENANT_ID'],
username=os.environ['OS_USERNAME'],
api_key=os.environ['OS_PASSWORD'],
region_name=os.environ['OS_REGION_NAME'])
Nova quotas can be obtained for a specific project as follows (they are returned as a QuerySet object) – note that this assumes administrator access in the openrc:
nova_qs = nova.quotas.get(project_id) #print nova_qs
Neutron is slightly different:
neutron_qs = neutron.show_quota(project_id) #print neutron_qs
and the return value is a dict which contains one entry called ‘quota’ which itself is a dict containing the quota elements.
Printing this is straightforward in python – one thing to note is that the QuerySet object for Nova supports a to_dict transformation, so printing the quota info can be done as follows:
nova_qs = nova.quotas.get(project_id).to_dict()
print 'Nova quota for project '+project_id
for k,v in nova_qs.iteritems():
print ' '+str(k)+': '+str(v)
Nova and Neutron have slightly different mechanisms to support update of quotas. In Nova, named parameters are passed to the update() function, such that the names match exactly the names in the quota. In Neutron, a dict is passed similar to the dict which is returned from the show_quotas() function. Here is how it looks:
nova_qs = nova.quotas.update(project_id,
instances=q_info['instances'], cores=q_info['cores'],
floating_ips=q_info['public_ips'])
#print 'neutron - floatingips()'
neutron_qs = neutron.update_quota(project_id, {'quota'
{'floatingip': q_info['public_ips'] }})
We put these basic building blocks into a script which we use to modify the quotas on our systems. More on pastebin if you’re interested.