Tag: mongodb

Testing PyMongo applications with MockupDB

In one of our projects, we needed to test some mongo based backend functionality: we wrote a small application which comprised of a mongo backend and a python app which communicated with the backend via pymongo. We like the flexibility of mongo in a rapid prototyping context and did not want to go with a full fledged ORM model for this app. Here we describe how we used MockupDB to perform some unit testing on this app. Continue reading

Advanced Queries to Ceilometer with a mongo backend

We are changing our ceilometer backend from mysql to mongodb in one of our experimental Openstack installations. The reason for this change is that mongo seems to deliver better ceilometer performance than mysql; further ceilometer data structures are a more natural fit with mongo (and indeed, this is largely where they came from). This can be seen in a typical record below which is clearly hierarchical and contains so-called embedded documents (in mongo terminology):

"_id" : ObjectId("53bbe7ea926fc4597b42aafc"),
"counter_name" : "instance",
"resource_metadata" : {
    "status" : "active",
    "display_name" : "Test",
    "name" : "instance-00000001",
    "image" : {
        "id" : "bdfaab74-6542-4cbb-94f1-5306662208a7",
        "name" : "cirros-0.3.2-x86_64-uec"
    "host" : "7c261f3a33c099538d448be797e5ce0c0d8cf8bf9f75dd59ce04df86",

Mongo natively provides support for queries of data structured in this fashion. More specifically, mongo enables data at different levels of the hierarchy to be queried – something which is difficult in SQL.

In python, this can be done simply as follows:

query = [{'field': 'timestamp', 'op': 'gt', 'value': date},
{'field': 'metadata.status', 'op': 'eq', 'value': 'active'}]

sample_list = ceilometer.samples.list(meter_name='instance', q=query)

The interesting point, which we did not understand clearly until now, is that ceilometer with a mongo backend supports exactly this type of query. Thus, the following command line query can obtain all instances that were active for a certain period of time:

ceilometer sample-list -m instance -q “timestamp>date; metadata.status=’active’” 

Then ceilometer will return all the samples of instances active in this time range.

| Resource ID                          | Name      | Type     | Timestamp           |
| 7535b9f6-01a6-410e-980d-338031e7a2c4 | instance  | instance | 2014-07-09T09:30:05 |
| 7535b9f6-01a6-410e-980d-338031e7a2c4 | instance  | instance | 2014-07-09T09:20:05 |
| 7535b9f6-01a6-410e-980d-338031e7a2c4 | instance  | instance | 2014-07-09T09:10:04 |

Querying ceilometer with a mysql database in this fashion results in an error.