My Adventures in Coding

January 2, 2011

Creating a REST API in Python using Bottle and MongoDB

Filed under: MongoDB,Python,REST — Brian @ 10:15 pm
Tags: , , , , , ,

I have been using Bottle and MongoDB for a REST API project for almost a year now. I frequently get asked the question “Bottle, what is Bottle, I have never heard of it” and “Ok, I have read the Bottle tutorial, it is simple, but how do I use Bottle with MongoDB?”. So I decided to write up a simple “Getting Started” tutorial for my friends curious about these technologies.

Prerequisites: Python 2.7 or higher

Setup MongoDB

MongoDB is a document store, a schema less database. What makes MongoDB unique (and why we decided to use it) is that MongoDB is a hybrid document store. What this means is that MongoDB offers the freedom of storing data in a schema less fashion, while still providing a flexible query syntax that will make anyone familiar with SQL feel comfortable.

You will need to download the latest version of MongoDB from the downloads page. Let’s download MongoDB and get it up and running.

First, setup the data folder. By default MongoDB stores all data files in the folder /data/db, so let’s create the folder:

sudo mkdir -m 777 -p /data/db

Download MongoDB:

wget http://fastdl.mongodb.org/osx/mongodb-osx-x86_64-1.6.5.tgz
tar -zxvf mongodb-osx-x86_64-1.6.5.tgz

Now let’s start MongoDB:

cd mongodb-osx-x86_64-1.6.5/bin
./mongod

NOTE: We have not created a database, or even a collection (you can think of a collection as a schema free table). When a document is being saved, MongoDB will lazily create the collection if it does not exist, and if the database does not exist, it will also create the database. It’s just that simple!

If you would like a more detailed introduction to using MongoDB, the query syntax, etc, please refer to my previous post: Getting Started with MongoDB

Setup Bottle

Bottle is a very simple Python web framework that is contained in a single Python file (bottle.py). We have been using Bottle for creating REST APIs and have been very impressed with just how easy it is to use.

Install the Bottle package into your Python site-packages folder:

pip install bottle

Setup PyMongo

PyMongo is a library for interacting with MongoDB from a Python application. This package bridges the gap between Bottle and MongoDB.

Install the PyMongo package into your Python site-packages folder:

pip install pymongo

Putting it all together: A Simple REST API

Now, on to the fun stuff! Let’s write a simple Python REST API, with a GET and a PUT, using Bottle, that allows us to save and retrieve documents to and from MongoDB using PyMongo.

Create a new file:

touch myrestapi.py

Cut and paste the following code:

import json
import bottle
from bottle import route, run, request, abort
from pymongo import Connection

connection = Connection('localhost', 27017)
db = connection.mydatabase

@route('/documents', method='PUT')
def put_document():
	data = request.body.readline()
	if not data:
		abort(400, 'No data received')
	entity = json.loads(data)
	if not entity.has_key('_id'):
		abort(400, 'No _id specified')
	try:
		db['documents'].save(entity)
	except ValidationError as ve:
		abort(400, str(ve))
	
@route('/documents/:id', method='GET')
def get_document(id):
	entity = db['documents'].find_one({'_id':id})
	if not entity:
		abort(404, 'No document with id %s' % id)
	return entity

run(host='localhost', port=8080)

Start the application:

python myrestapi.py

Using our new REST API

Now let’s try saving and retrieving some documents. If you do not have a preferred REST client handy and are using the Google Chrome browser, you should try the extension Simple REST Client.

Save a document
  • Open up your favorite REST client and point it at the following URL:
  • Now select PUT and enter the following data:
    • {"_id": "doc1", "name": "Test Document 1"}

NOTE: The primary key of a document in a MongoDB collection is always the field “_id”. If you specify an “_id” in your document it will be used, if you do not, MongoDB will generate a unique id.

Retrieve a document
  • In your REST client enter the URL for retrieving the document we just saved:
  • Now select GET and you should see the same document returned:
    • {"_id": "doc1", "name": "Test Document 1"}

That is it! You now have a basic, working, REST API written in Python using Bottle and MongoDB!

About these ads

21 Comments »

  1. How can I mock REST services from external websites in Python?…

    As a path of least resistance – is there anything stopping you from creating/reading/deleting dummy resources on these REST services themselves? It’s by far the most straightforward way to do it. If you have a 1,000 call budget on your account, using …

    Trackback by Quora — April 20, 2011 @ 10:29 pm | Reply

  2. How can I mock REST services from external websites in Python?…

    First of all, are you really sure you need to mock these services out? If I remember your previous question details correctly, you’re using GData, Flickr, and others. Can you just use test/dummy data on most of these accounts? Request are very cheap, …

    Trackback by Quora — April 22, 2011 @ 12:27 pm | Reply

  3. Thanks for your post, it was very helpful. I am having the hardest time getting this to work from a web page using jquery. Here is what I’m calling.

    $(document).ready(function() {
    $(“#clickme”).click(function(evt) {
    evt.preventDefault();
    alert(“clicked”);
    $.get(“http://localhost:8080/document/doc1″, function(data) { alert(data); }, “json”);
    });
    });

    I never get the callback, frustrating! Even if i switch the url to http://www.google.com and type to html I get nothing back.

    Comment by Lee — May 27, 2011 @ 7:33 am | Reply

    • Use the Chrome or Firefox Javascript console. I’d bet you’re getting a security error because there’s no CORS headers supplied.

      Comment by Adam Baxter (@voltagex) — September 2, 2012 @ 9:30 pm | Reply

  4. Dear Brian excellent post. One question, how can you secure this Api. Any suggestions.

    Comment by Kambos — June 15, 2011 @ 7:03 am | Reply

    • Hi, I am glad you found the post useful! The Python REST API I helped build at work is strictly an internal use only API. Our systems team just has the server the API is running on locked down so only certain applications can talk to it. Nothing too extreme. I have not yet had the opportunity to work on a project designing and securing a public facing API , so unfortunately I can’t suggest any preferred approaches.

      Comment by Brian — June 15, 2011 @ 11:20 pm | Reply

  5. it’s work! thank you for your kindly help.
    that’s a easy way to build a RESTful Web API!!

    Comment by amos — August 22, 2011 @ 1:39 pm | Reply

    • That’s great! I am glad the post was helpful!

      Comment by Brian — August 22, 2011 @ 1:56 pm | Reply

  6. Which is the part that requires python 2.7? Just trying to figure out how to replicate this on OpenShift

    Comment by Steven Citron-Pousty — July 9, 2012 @ 4:20 pm | Reply

    • I believe Bottle and MongoDB will both work on Python 2.6. At the time I wrote the article, I was using 2.7, so I never had a chance to test it on 2.6.

      Comment by Brian — July 9, 2012 @ 8:15 pm | Reply

  7. Thank you for the post. I am a newby python developer and am trying to use bottle inside eclipse pydev. The “import bottle” statement works fine, but I run into trouble with the line “from bottle import route, run, request, abort” . No matter where I place the object “route” on that line, it is not recognized. It treats that object as if it cannot be found inside the “bottle” module. Searching the bottle module, I can find the other objects on that line, but I cannot find “route”. If I try to do the same task inside an interactive python shell, this problem goes away. So I am clearly missing something here (due to my newness to Python). Incidently, the bottle guys want the install to be done with easy_install, and you say to use pip. Could I be self-inflicting pain if I used both?

    Comment by Scott — September 4, 2012 @ 12:37 pm | Reply

    • I had to change the code to this in order to work, as I was getting errors trying to resolve “route” as an object in pydev and was getting bad magic numbers from Formular (redid my installations with easy_install), and then added the bottle “egg” folder directly to the sys path. Any ideas why route is not recognized, but run, request, abort, and response are? Code is:

      import json
      import bottle

      from bottle import Bottle, run, request, abort, response
      from pymongo import Connection

      app = Bottle()

      connection = Connection(‘localhost’, 27017)
      db = connection.mydatabase

      @app.route(‘/documents’, method=’PUT’)
      def put_document():
      data = request.body().readline()
      if not data:
      abort(400, ‘No data received’)
      entity = json.loads(data)
      if not entity.has_key(‘_id’):
      abort(400, ‘No _id specified’)
      try:
      db['documents'].save(entity)
      except TypeError as ve:
      abort(400, str(ve))

      @app.route(‘/documents/:id’, method=’GET’)
      def get_document(id):
      entity = db['documents'].find_one({‘_id’:id})
      if not entity:
      abort(404, ‘No document with id %s’ % id)
      return entity

      run(host=’localhost’, port=8080)

      which does work (inside Pydev + mac)

      Comment by Scott — September 4, 2012 @ 3:16 pm | Reply

      • Note that I had to get rid of ValidationError due to the magic number problem and instead catch the mongo specific error. Maybe I had the wrong class of ValidationError?

        Comment by Scott — September 4, 2012 @ 3:17 pm

  8. ValidationError was not recognized, but I was able to solve that problem by installing “formular”. Is that the source of that class?

    Comment by Scott — September 4, 2012 @ 12:38 pm | Reply

  9. When I try using the PUT, I get:

    AttributeError: StringIO instance has no __call__ method

    which is thrown by:

    data = request.body().readline()

    How can this issue be solved? Sorry, I am a beginner in Python.

    Comment by Michael Colburn — December 11, 2012 @ 11:18 am | Reply

  10. Thanks! Awesome tutorial.

    Comment by brillosuds — January 7, 2013 @ 8:22 am | Reply

  11. Republicou isso em Weekend Nerde comentado:
    A simple example how to implement a REST API with bottle.py

    Comment by Eduardo — August 12, 2013 @ 5:38 am | Reply

  12. Thanks, Brian it is a very usefull tutorial. Did you have examples using web2py, instead bootle to publish?

    Comment by Dino — October 26, 2013 @ 3:00 pm | Reply

  13. bottle is quite awesome. im sure flask is also. have you load tested yr ‘bongodb’ on aws or another scalable service.

    Comment by adrian — November 26, 2013 @ 1:11 pm | Reply

    • Hi, thanks for the comment. Our bottle/mongo application has been in production now for about two years and has worked very well. We put a great deal of load on the system and have not really had any issues. Our system pulls data in from outside sources, saves it, then our customers can pull data from us whenever they want. So for our setup we run a cluster of three nodes, a read/write master with two fail over slaves. This Master node is used by our systems that write all the new data being pulled in. Then our customers that pull the aggregated data from us on the other side talk to read only slave nodes. At the moment we have five of these read only slaves for handling all of our read traffic. Overall, this has worked well, as our customer base increases, we gradually add more read only nodes. Overall the setup has worked well for us, very simple, very fast, and so far Bottle has not shown any problems handling the load we throw at it, which is fairly high.

      I haven’t tried flask.pocoo.org before, but I will definitely try it out!

      Comment by Brian — December 2, 2013 @ 11:25 am | Reply

      • ive been using azure tables with bottle. would like to have async capabilities on bottle. have you used pagination (limit and continuation) on mongo.

        Comment by adrian — December 16, 2013 @ 1:30 am


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

The Rubric Theme. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: