API
Part 1, Chapter 6
In this chapter, we'll code the first iteration of our search API, while following Test-Driven Development (TDD). Start by deleting the tests.py file in the catalog Django app. From the project's root directory, run the following command:
$ rm server/catalog/tests.py
Replace it with a "tests" directory. (Don't forget to add the __init__.py file.)
$ mkdir -p server/catalog/tests
$ cd server/catalog/tests && touch __init__.py
We're making these changes in order to create a single location for multiple test files.
First Test
Next, create a test_views.py file inside the new "tests" folder with the following code:
# server/catalog/tests/test_views.py
from rest_framework.test import APIClient, APITestCase
from catalog.models import Wine
class ViewTests(APITestCase):
def test_empty_query_returns_everything(self):
wine = Wine.objects.create(
country='US',
description='A delicious bottle of wine.',
points=90,
price=100.00,
variety='Cabernet Sauvignon',
winery='Stag\'s Leap'
)
client = APIClient()
response = client.get('/api/v1/catalog/wines/')
self.assertJSONEqual(response.content, [{
'country': 'US',
'description': 'A delicious bottle of wine.',
'id': str(wine.id),
'points': 90,
'price': '100.00',
'variety': 'Cabernet Sauvignon',
'winery': 'Stag\'s Leap',
}])
Here's what our test is doing:
- Creating a record in the
catalog_winedatabase table - Initializing a testing client
- Making a
GETrequest to thewinesAPI (without providing a query parameter) - Confirming that the response contains every record in the database
When we call our wines API without passing any query string parameters, we get back a list of all records.
From the "server" directory, run the tests.
$ docker-compose exec server python manage.py test
We haven't configured the API yet, so they should fail with the following error:
AssertionError: First argument is not valid JSON: b'\n<!doctype html>\n<html lang="en">\n<head>\n <title>Not Found</title>\n</head>\n<body>\n <h1>Not Found</h1><p>The requested resource was not found on this server.</p>\n</body>\n</html>\n'
Django REST Framework
With the data model in place, it's time to create an API that the client can use to query the database. Our goal is to let the client search the wine catalog and filter the results. In REST, we do this by identifying the wine resource with a URI. When clients send an HTTP GET request to the /api/v1/catalog/wines/ URI, they can expect to receive a response with the representations of the wines in the database. Clients can use query string parameters to filter the results.
Our application uses Django REST Framework. Django REST Framework (DRF) let's you create APIs with very little code and it has built-in support for OpenAPI documentation. Using DRF allows you to not worry about writing the same little details over and over again.
Let's start by editing the settings.py file to install the DRF libraries in our Django project:
# server/perusable/settings.py
THIRD_PARTY_APPS = [
'rest_framework',
'django_filters',
]
Add the following code to the bottom of the file:
# server/perusable/settings.py
REST_FRAMEWORK = {
'DEFAULT_FILTER_BACKENDS': ('django_filters.rest_framework.DjangoFilterBackend',)
}
django-filter is a powerful tool used for filtering Django QuerySets.
This snippet tells Django to always use django_filters as a default filter backend and prevents us from having to explicitly declare it on every API.
Next, add a serializers.py file to the catalog app with the following code:
# server/catalog/serializers.py
from rest_framework import serializers
from .models import Wine
class WineSerializer(serializers.ModelSerializer):
class Meta:
model = Wine
fields = ('id', 'country', 'description', 'points', 'price', 'variety', 'winery',)
This code tells the server how to serialize a Wine resource from a database record to a JSON string.
Next, add a filters.py file to catalog with the following code:
# server/catalog/filters.py
from django.db.models import Q
from django_filters.rest_framework import CharFilter, FilterSet
from .models import Wine
class WineFilterSet(FilterSet):
query = CharFilter(method='filter_query')
def filter_query(self, queryset, name, value):
search_query = Q(
Q(variety__contains=value) |
Q(winery__contains=value) |
Q(description__contains=value)
)
return queryset.filter(search_query)
class Meta:
model = Wine
fields = ('query', 'country', 'points',)
This code transcribes query string parameters into SQL queries via the Django ORM. We're telling the server to look for a query string parameter called query on the incoming HTTP request. The server takes that parameter value and constructs a database query to find all documents that contain the value in the variety, winery, or description fields. Our code also filters the database query by country and points, if those parameters are present in the HTTP request too.
Next, add the following code to the catalog/views.py file:
# server/catalog/views.py
from rest_framework.generics import ListAPIView
from .models import Wine
from .serializers import WineSerializer
from .filters import WineFilterSet
class WinesView(ListAPIView):
queryset = Wine.objects.all()
serializer_class = WineSerializer
filterset_class = WineFilterSet
Note how we handle the query string parameter extraction and SQL query building with the filterset_class. We handle the response payload building with the serializer_class.
Next, create a new urls.py file in the catalog app with the following code:
# server/catalog/urls.py
from django.urls import path
from .views import WinesView
urlpatterns = [
path('wines/', WinesView.as_view()),
]
Edit the server/perusable/urls.py file to include the catalog URLs:
# server/perusable/urls.py
from django.contrib import admin
from django.urls import include, path
urlpatterns = [
path('admin/', admin.site.urls),
path('api/v1/catalog/', include('catalog.urls')),
]
Run the tests again:
$ docker-compose exec server python manage.py test
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
.
----------------------------------------------------------------------
Ran 1 test in 0.011s
OK
Tests pass!
Fixtures
At this point, we have a working API, but we don't have any data.
Create a "fixtures" directory in the "server/catalog" folder.
$ mkdir -p server/catalog/fixtures
Download the wines.zip file to the fixtures directory, unzip it, copy the JSON file to the directory, and then delete the ZIP file.
$ curl -o wines.zip https://raw.githubusercontent.com/testdrivenio/django-postgres-elasticsearch/master/wines.zip
$ unzip wines.zip
$ rm -rf wines.zip
Load the fixture data with the following command. Remember, any files you add to your local machine are automatically copied to your Docker container due to the volumes.
$ docker-compose exec server \
python manage.py loaddata \
catalog/fixtures/wines.json \
--app catalog \
--format json
After a few minutes you should see the following:
Installed 150930 object(s) from 1 fixture(s)
Return to the admin page in your browser and click on the Wines link. You should see a lot of wines.
Now that our wine data is loaded into our database, let's test out our API. From your terminal, run the following command.
$ curl "http://localhost:8003/api/v1/catalog/wines/?query=merlot&country=US&points=90"
You should get back a list with two wines.
Deep Dive
Let's take a look back at our WineFilterSet and re-examine how Django translates a URL into a database query. A FilterSet maps the query string parameters from a URL to fields on a model like so:
class Meta:
model = Wine
fields = ('query', 'country', 'points',)
Since country and points are explicit fields on the Wine model, the FilterSet constructs a QuerySet behind the scenes similar to the following:
queryset = Wine.objects.filter(
country=<country_value_from_url>,
points=<points_value_from_url>
)
The query field, on the other hand, is not a field on the Wine model, and so we need to define custom handling for it. The FilterSet captures the query query string value and passes it to the filter_query() function, which appends another filter() statement onto the QuerySet.
def filter_query(self, queryset, name, value):
search_query = Q(
Q(variety__contains=value) |
Q(winery__contains=value) |
Q(description__contains=value)
)
return queryset.filter(search_query)
The final ORM call looks like this:
queryset = Wine.objects.filter(
country=<country_value_from_url>,
points=<points_value_from_url>
).filter(Q(
Q(variety__contains=value) |
Q(winery__contains=value) |
Q(description__contains=value)
))
You may be wondering about the alternative. If we don't use django-filters, then we have to write a lot of explicit request parsing code ourselves. Without our WineFilterSet, our API changes to the following:
class WinesView(ListAPIView):
serializer_class = WineSerializer
def get_queryset(self):
queryset = Wine.objects.all()
country = self.request.query_params.get('country')
if country is not None:
# Additional type checking...
queryset = queryset.filter(country=country)
points = self.request.query_params.get('points')
if points:
# Additional type checking...
queryset = queryset.filter(points=points)
query = self.request.query_params.get('query')
if query:
queryset = queryset.filter(Q(
Q(variety__contains=query) |
Q(winery__contains=query) |
Q(description__contains=query)
))
return queryset
As you can see, FilterSets handle request parsing, type checking, model field mapping, and more. Also, we can reuse FilterSets between multiple APIs, which allows us to avoid writing the same code over and over again.
At this point, your directory should look like the following:
├── docker-compose.yml
└── server
├── .dockerignore
├── Dockerfile
├── catalog
│ ├── __init__.py
│ ├── admin.py
│ ├── apps.py
│ ├── filters.py
│ ├── fixtures
│ │ └── wines.json
│ ├── migrations
│ │ ├── 0001_initial.py
│ │ └── __init__.py
│ ├── models.py
│ ├── serializers.py
│ ├── tests
│ │ ├── __init__.py
│ │ └── test_views.py
│ ├── urls.py
│ └── views.py
├── manage.py
├── perusable
│ ├── __init__.py
│ ├── asgi.py
│ ├── settings.py
│ ├── urls.py
│ └── wsgi.py
├── requirements.txt
└── start.sh
✓ Mark as Completed