Requests and Routing
Part 1, Chapter 3
In the second chapter of the course, we'll build the most important parts of the framework:
- The request handlers (think Django views)
- Routing -- both simple (like
/books/) and parameterized (like/books/{id}/)
Now, before I start doing something new, I like to think about the end result. In this case, at the end of the day, we want to be able to use this framework in production and thus we want our framework to be served by a fast, lightweight, production-ready application server. I have been using Gunicorn in all of my projects the past few years, and I am very satisfied with the results. So, we'll go with Gunicorn here as well.
Gunicorn is a WSGI HTTP Server, so it expects a specific entrypoint to our application, just like you learned in the last chapter.
To recap, to be WSGI-compatible, you need a callable object (a function or a class) that expects two parameters (environ and start_response) and returns an iterable response.
With those details out of the way, let's get started with our awesome framework.
Requests
First, think of a name for your framework and create a folder with that name.
We'll be using the name
bumbothroughout this course. You can use that same name or come up with something new.
$ mkdir bumbo
Change directories to that folder. Create a virtual environment and activate it:
$ cd bumbo
$ python3.9 -m venv venv
$ source venv/bin/activate
Now, create a file named app.py where we'll store our entrypoint for Gunicorn:
(venv)$ touch app.py
Inside app.py, add the following WSGI-compatible dummy function to see if it works with Gunicorn:
# app.py
def app(environ, start_response):
response_body = b"Hello, World!"
status = "200 OK"
start_response(status, headers=[])
return iter([response_body])
It's nearly the same as the function you created in the previous chapter but much simpler. Instead of listing all the environment variables, it simply says "Hello, World!".
Now, to test, first install Gunicorn:
(venv)$ pip install gunicorn
Then, run the code:
(venv)$ gunicorn app:app
In the above command, the first app (left of the colon) is the file which you created and the second app (right of the colon) is the name of the function you just wrote. If all is good, you will see the following output in your console:
[2021-02-05 08:43:36 -0600] [81282] [INFO] Starting gunicorn 20.0.4
[2021-02-05 08:43:36 -0600] [81282] [INFO] Listening at: http://127.0.0.1:8000 (81282)
[2021-02-05 08:43:36 -0600] [81282] [INFO] Using worker: sync
[2021-02-05 08:43:36 -0600] [81284] [INFO] Booting worker with pid: 81284
If you see this, open your browser and go to http://localhost:8000. You should see our good old friend: the Hello, World! message. Awesome! You'll build the rest of the framework off of this function.
Now, you will need quite a few helper methods in the future, so let's turn this function into a class since those helper methods are much easier to write inside a class:
Create an api.py file:
(venv)$ touch api.py
Inside this file, create the following API class:
# api.py
class API:
def __call__(self, environ, start_response):
response_body = b"Hello, World!"
status = "200 OK"
start_response(status, headers=[])
return iter([response_body])
We'll look at what this class does together in a bit.
But first, delete everything inside app.py and write the following:
# app.py
from api import API
app = API()
Restart (or re-run) Gunicorn and check the result in the browser. It should be the same as before since we converted our function named app to a class called API and overrode its __call__ method.
It's worth noting that the __call__ method is called when you call the instances of this class:
app = API()
app() # this is where __call__ could be called
So, why are we only creating an instance of the API but not calling it in app.py? Because the calling of the instance of the API class is the responsibility of the web server (e.g., Gunicorn). In other words, it will be called when you run gunicorn app:app.
Now that you created the class, let's make the code look a bit more elegant because all those bytes (i.e., b"Hello World") and the start_response function seem a bit confusing, right?
Thankfully, there is a cool package called WebOb that provides classes for HTTP requests and responses by wrapping the WSGI request environment and response status, headers, and body. By using this package, we can pass the environ and start_response to the classes, which are provided by this package, and not have to deal with them ourselves.
Before we continue, take a few minutes to look over the WebOb documentation. Make sure you understand how the Request and Response classes work. Briefly review the rest of the API as well.
Let's refactor this code!
Refactoring
First, install WebOb:
(venv)$ pip install webob
Import the Request and Response classes at the beginning of the api.py file:
# api.py
from webob import Request, Response
...
And now you can use them inside the __call__ method:
# api.py
from webob import Request, Response
class API:
def __call__(self, environ, start_response):
request = Request(environ)
response = Response()
response.text = "Hello, World!"
return response(environ, start_response)
Looks much better!
Restart Gunicorn and you should see the same result as before. The best part here is that the code above requires little explanation since it's mostly self-explanatory. You created a request and a response and then returned that response. Awesome!
Did you notice that we are not yet using the request? Let's change that. We can use it to get the user agent info from it. At the same time, we can refactor the response creation into its own method called handle_request. You will see why this is better later.
# api.py
from webob import Request, Response
class API:
def __call__(self, environ, start_response):
request = Request(environ)
response = self.handle_request(request)
return response(environ, start_response)
def handle_request(self, request):
user_agent = request.environ.get("HTTP_USER_AGENT", "No User Agent Found")
response = Response()
response.text = f"Hello, my friend with this user agent: {user_agent}"
return response
Restart Gunicorn and you should see the new message in the browser.
For example:
Hello, my friend with this user agent:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:85.0) Gecko/20100101 Firefox/85.0
Did you see it? Cool. Let's move on.
Routing
At this point, your framework handles all the requests in the same way: Whatever request it receives, it simply returns the same response which is created in the handle_request method. Ultimately, it needs to be dynamic though. That is, it needs to serve the request coming from /home/ differently than the one coming from /about/.
Simple
To that end, inside app.py, create two methods that will handle those two requests:
# app.py
from api import API
app = API()
def home(request, response):
response.text = "Hello from the HOME page"
def about(request, response):
response.text = "Hello from the ABOUT page"
Now, you need to somehow associate the two methods with the above mentioned paths -- /home/ and /about/. I like the Flask way of managing this with decorators, which looks like this:
# app.py
from api import API
app = API()
@app.route("/home")
def home(request, response):
response.text = "Hello from the HOME page"
@app.route("/about")
def about(request, response):
response.text = "Hello from the ABOUT page"
Make the above changes to app.py.
So, what do you think? Look good? Let's create and implement the decorators!
As you can see, the route method above is a decorator that accepts a path and wraps the methods. Implement it in the API class like so:
# api.py
from webob import Request, Response
class API:
def __init__(self):
self.routes = {}
def __call__(self, environ, start_response):
...
def route(self, path):
def wrapper(handler):
self.routes[path] = handler
return handler
return wrapper
def handle_request(self, request):
...
What's happening?
First, In the __init__ method, you defined a dict called self.routes where the framework will store paths as keys and handlers as values.
Values of that dict will look something like this:
{
"/home": <function home at 0x1100a70c8>,
"/about": <function about at 0x1101a80c3>
}
Then, in the route method, you took a path as an argument and in the wrapper method you added this path in the self.routes dictionary as a key and the handler as a value.
At this point, you have all the pieces of the puzzle. You have the handlers and the paths associated with them. Now, when a request comes in, you need to check its path, find an appropriate handler, call that handler, and return an appropriate response.
Refactor the handle_request method to do just that:
# api.py
from webob import Request, Response
class API:
...
def handle_request(self, request):
response = Response()
for path, handler in self.routes.items():
if path == request.path:
handler(request, response)
return response
Here, you iterated over self.routes and compared paths with the path of the request. If there is a match, you then called the handler associated with that path.
You should now have:
# api.py
from webob import Request, Response
class API:
def __init__(self):
self.routes = {}
def __call__(self, environ, start_response):
request = Request(environ)
response = self.handle_request(request)
return response(environ, start_response)
def route(self, path):
def wrapper(handler):
self.routes[path] = handler
return handler
return wrapper
def handle_request(self, request):
response = Response()
for path, handler in self.routes.items():
if path == request.path:
handler(request, response)
return response
Restart Gunicorn again and try those paths in the browser:
You should see the corresponding messages. Pretty cool, right?
However, what happens if the path is not found?
Try http://localhost:8000/nonexistent/. You should see a big, ugly error message: Internal Server Error. If you turn to your console, you should see the issue:
TypeError: 'NoneType' object is not callable
You should see the same TypeError for the request for the favicon:
[2021-02-05 12:57:43 -0600] [87473] [ERROR] Error handling request /favicon.ico
To solve this, create a method that returns a simple HTTP response of "Not found.", with a 404 status code:
# api.py
from webob import Request, Response
class API:
...
def default_response(self, response):
response.status_code = 404
response.text = "Not found."
...
Now, use it in the handle_request method:
# api.py
from webob import Request, Response
class API:
...
def handle_request(self, request):
response = Response()
for path, handler in self.routes.items():
if path == request.path:
handler(request, response)
return response
self.default_response(response)
return response
Restart Gunicorn and try some nonexistent routes. You should see a lovely "Not found." page instead of the previous, unhandled error.
Next, refactor out finding a handler to its own method for the sake of readability:
# api.py
from webob import Request, Response
class API:
...
def find_handler(self, request_path):
for path, handler in self.routes.items():
if path == request_path:
return handler
...
Just like before, it iterates over self.route, comparing paths with the request path, and returns the handler if the paths are the same or None if no handler is found.
Now, you can use it in your handle_request method:
# api.py
from webob import Request, Response
class API:
...
def handle_request(self, request):
response = Response()
handler = self.find_handler(request_path=request.path)
if handler is not None:
handler(request, response)
else:
self.default_response(response)
return response
Much better, right? Fairly self-explanatory too.
Restart Gunicorn to ensure that everything works just like before.
Parameterized
At this point, you have routes and handlers. It works, but the routes are simple. They don't support keyword parameters in the URL path. What if we wanted to have a route of @app.route("/hello/{person_name}") and be able to use person_name inside the handlers?
For example:
def say_hello(request, response, person_name):
resp.text = f"Hello, {person_name}"
For that, if someone goes to the /hello/Matthew/, the framework needs to be able to match this path with the registered /hello/{person_name}/ and find the appropriate handler. Thankfully, there is a package called Parse that does exactly that for you. Go ahead and install it:
(venv)$ pip install parse
Test this package out in the Python interpreter. First, open the Python interpreter by running python within your virtual environment, and then test out Parse like so:
>>> from parse import parse
>>> result = parse("Hello, {name}", "Hello, Matthew")
>>> print(result.named)
{'name': 'Matthew'}
As you can see, it parsed the string Hello, Matthew and was able to identify that Matthew corresponds to the provided {name}.
Now, use it in your find_handler method to find not only the method that corresponds to the path but also the keyword params that were provided. Make sure that you import parse first:
# api.py
from parse import parse
from webob import Request, Response
class API:
...
def find_handler(self, request_path):
for path, handler in self.routes.items():
parse_result = parse(path, request_path)
if parse_result is not None:
return handler, parse_result.named
return None, None
...
The find_handler method still iterates over self.routes; but, now instead of comparing the path to the request path, it tries to parse it and if there is a result, it returns both the handler and keyword params as a dictionary.
Now, you can use this inside handle_request to send params to the handlers:
# api.py
from parse import parse
from webob import Request, Response
class API:
...
def handle_request(self, request):
response = Response()
handler, kwargs = self.find_handler(request_path=request.path)
if handler is not None:
handler(request, response, **kwargs)
else:
self.default_response(response)
return response
So, you are now getting both handler and kwargs from self.find_handler and passing kwargs to the handler via **kwargs.
You should now have:
# api.py
from parse import parse
from webob import Request, Response
class API:
def __init__(self):
self.routes = {}
def __call__(self, environ, start_response):
request = Request(environ)
response = self.handle_request(request)
return response(environ, start_response)
def route(self, path):
def wrapper(handler):
self.routes[path] = handler
return handler
return wrapper
def default_response(self, response):
response.status_code = 404
response.text = "Not found."
def find_handler(self, request_path):
for path, handler in self.routes.items():
parse_result = parse(path, request_path)
if parse_result is not None:
return handler, parse_result.named
return None, None
def handle_request(self, request):
response = Response()
handler, kwargs = self.find_handler(request_path=request.path)
if handler is not None:
handler(request, response, **kwargs)
else:
self.default_response(response)
return response
Write a handler with this type of route and try it out in app.py:
# app.py
from api import API
app = API()
...
@app.route("/hello/{name}")
def greeting(request, response, name):
response.text = f"Hello, {name}"
Full file:
# app.py
from api import API
app = API()
@app.route("/home")
def home(request, response):
response.text = "Hello from the HOME page"
@app.route("/about")
def about(request, response):
response.text = "Hello from the ABOUT page"
@app.route("/hello/{name}")
def greeting(request, response, name):
response.text = f"Hello, {name}"
Restart Gunicorn and navigate to http://localhost:8000/hello/Matthew. You should see the wonderful message of Hello, Matthew. Try http://localhost:8000/hello/Mike and you should see Hello, Mike. Awesome, right?
Add a couple more such handlers on your own.
You can also indicate the type of the given params.
For example, you can define a digit type to an age param:
@app.route("/tell/{age:d}")
Another example:
@app.route("/sum/{num_1:d}/{num_2:d}")
def sum(request, response, num_1, num_2):
total = int(num_1) + int(num_2)
response.text = f"{num_1} + {num_2} = {total}"
What happens if you pass in a non-digit? It will not be able to parse it and our
default_responsewill do its job. Try this out.
Conclusion
In this chapter, you started writing your own framework and built the most important features:
- Request handlers
- Simple and parameterized routing
You also learned how to use the WebOb and Parse third-party packages.
We'll look at handling duplicate routes in the next chapter. See you there!
Have you figured out why we refactored out the
responsecreation into its own method calledhandle_requestyet?
✓ Mark as Completed