{"id":10489,"date":"2016-08-18T15:48:27","date_gmt":"2016-08-18T13:48:27","guid":{"rendered":"https:\/\/blog.zhaw.ch\/icclab\/?p=10489"},"modified":"2016-08-18T15:50:50","modified_gmt":"2016-08-18T13:50:50","slug":"testing-pymongo-applications-with-mockupdb","status":"publish","type":"post","link":"https:\/\/blog.zhaw.ch\/icclab\/testing-pymongo-applications-with-mockupdb\/","title":{"rendered":"Testing PyMongo applications with MockupDB"},"content":{"rendered":"

In one of our projects, we needed to test some mongo based backend functionality: we wrote a small application which comprised of a mongo backend and a python app which communicated with the backend via pymongo. We like the flexibility of mongo in a rapid prototyping context and did not want to go with a full fledged ORM model for this app. Here we describe how we used MockupDB to perform some unit testing on this app.<\/p>\n

PyMongo is the official<\/a> Python driver for connecting Mongo databases: it has a full set of features for managing documents and also a large set of tools and frameworks<\/a> that can be used in conjunction with PyMongo. Despite this rich ecosystem, there is little documentation on best practices for testing PyMongo applications – mongobox<\/a> is provided \u00a0as a framework tool for testing; its model is one in which a MongoDB instance is created with a temporary directory to store database files for testing – this solution can be heavy for some types of testing, unit testing in particular.<\/p>\n

Just recently when looking for approaches to testing our web-based application we found MockupDB<\/a>, a wire protocol<\/a> server for MongoDB. MockupDB fits pretty much our testing requirements – we needed to test mostly CRUD operations in the REST API interface of our application, which connects to a mongo database and reads or modifies the content of documents before serving them. But MockupDB goes beyond that, allowing testing pattern-matching, cursor failure, request failure and much more<\/a>.<\/p>\n

Here is an example of a test used as starting point for our application (it uses the standard python testing conventions).<\/p>\n

\r\n# test_database.py\r\n\r\nfrom mockupdb import go, MockupDB\r\nimport unittest\r\nImport datetime\r\n\r\n# Our own database class using PyMongo\r\nfrom database import MongoDatabase\r\n\r\nDOCUMENT = {\"_id\": \"123ab-456cd-789ef\", 'name': \"abc123\",\r\n\r\n\"flavor_id\": 1, \"image_id\": 2, \"job_duration\": 120}\r\n\r\nclass MockupDBDatabaseTest(unittest.TestCase):\r\n\r\n    def setUp(self):\r\n        self.server = MockupDB(auto_ismaster={\"maxWireVersion\": 3})\r\n        self.server.run()\r\n\r\n        # Replace pymongo connection url to use MockupDB\r\n        self.database = MongoDatabase(self.server.uri)\r\n\r\n    def tearDown(self):\r\n        self.server.stop()\r\n\r\n    def test_list_documents(self):\r\n        document = DOCUMENT\r\n\r\n        # adding datetime object to document\r\n        document['request_time'] = datetime.datetime.now()\r\n\r\n        # The database list method converts any datetime object returned\r\n        # from the database to string. See below.\r\n        document_query = go(self.database.list, {})\r\n\r\n        request = self.server.receives()\r\n\r\n        # returns to pymongo a list containing only 1 the document\r\n        request.reply([DOCUMENT])\r\n\r\n        self.assertIsInstance(document_query()[0]['request_time'], str)\r\n        self.assertEqual(\r\n            document['request_time'].strftime(\"%Y-%m-%dT%H:%M:%S\"),\r\n            document_query()[0]['request_time']\r\n        )\r\n\r\n# database.py\r\n\r\nclass MongoDatabase(object):\r\n    def __init__(self, uri):\r\n        self.client = MongoClient(uri)\r\n        self.db = self.client.db\r\n        self.collection = self.db.collection\r\n\r\n    def list(self, query):\r\n        data_list = list()\r\n        for data in self.collection.find(query):\r\n            data['id'] = str(data.pop('_id'))\r\n            data = self._convert_datetime_to_str(data)\r\n            data_list.append(data)\r\n        return data_list\r\n\r\n    def _convert_datetime_to_str(self, data):\r\n        for key in data.keys():\r\n            if isinstance(data[key], datetime.datetime):\r\n                data[key] = data[key].strftime('%Y-%m-%dT%H:%M:%S')\r\n        return data\r\n<\/code><\/pre>\n
MockupDB was the first solution we came across when looking for alternatives to unit testing pymongo methods, subsequently we also found a few other packages mocking MongoDB – Mongomock<\/a> and Ming<\/a> (with its mongo in memory module), although we have not had a chance to look into these as yet.<\/p>\n
Generally, MockupDB is a powerful yet flexible package which can be used for both simple and more complex testing cases providing simple functions with a very small learning curve. So far we have only used it for simple unit tests but we are also keen to try out more complex scenarios with e.g. multiple MongoDB configurations where there are network failures.<\/p>\n
If you are interested in more information about MockupDB and testing examples you can check these links:<\/p>\n