Full-text Search in Django with Postgres and Elasticsearch



Learn how to add full-text search to Django with both Postgres and Elasticsearch!

In this course, you'll learn how to craft a search application that helps users peruse a large catalog of wines. We'll start by building a simple Django project using Docker. Then, we'll do a deep dive into the native search capabilities offered by Postgres, and we'll incorporate them into our app using both the Django library and our own hand rolled constructs. Next, we'll create a React app to drive the user experience. Finally, we'll take our search app to the next level and swap out Postgres for Elasticsearch. And of course we'll be using Test-driven Development every step of the way.

Learning Objectives

By the end of this course, you will be able to:

Part 1

  1. Describe a high-level system design to a friend or colleague
  2. Create a basic Django app that uses a RESTful API
  3. Containerize an app using Docker
  4. Understand basic Django REST Framework concepts (serializers, filters, views, etc.)
  5. Practice Test-driven Development (TDD)

Part 2

  1. List some ways that full-text search matches a user's query better than simple alternatives
  2. Implement full-text search in a Django app using a Postgres database
  3. Sort full-text search results by relevancy
  4. Update a data model to improve full-text search performance
  5. Highlight full-text search matches
  6. Make suggestions to a user who searches for a misspelled word

Part 3

  1. Create a React app using Create React App
  2. Build functional forms with Formik
  3. Communicate with the server through HTTP requests
  4. Test an application end-to-end using Cypress
  5. Build a production version of a React app

Part 4

  1. Configure Elasticsearch for local development
  2. Create an Elasticsearch index and populate it with documents
  3. Perform searches using Query DSL
  4. Highlight search matches
  5. Make suggestions to a user who searches for a misspelled word

What will you build?

Django backend: We'll construct a RESTful API with Django REST Framework that communicates with both Postgres and Elasticsearch data sources

React frontend: We'll build a React app that lets users search a catalog of 150,000+ wines and displays their results

Postgres database: We'll make a data model, explore SQL queries, and learn techniques to help us boost performance

Elasticsearch: We'll create an index, tweak a mapping, and learn how to use the Query DSL

Info

  • Current version: 1.0.0
  • Last updated: March 29th, 2021
  • AuthorJason Parent




What will you learn?

Part 1

In Part 1, you'll learn how to program the server code of the search application. We'll start by developing a basic Django app using a Postgres database in Docker. We'll create the basic data model, code the API, and write tests.

Tools and Technologies: Python, Django, Django REST Framework, Postgres, Docker

Part 2

In Part 2, you'll explore Postgres queries while incorporating them into the search application. We'll go step-by-step in transforming the foundational app we created in Part 1 into a robust search service. We'll start by learning the fundamentals of full-text search, and then we'll slowly build the concepts into our API and demonstrate how they improve the search results. You'll learn how to use database indexes to boost performance, rank results by relevancy, highlight keywords, and offer suggestions for misspelled words. By the end of Part 2, you'll have a good handle on how to write raw SQL queries and the companion Django code.

Tools and Technologies: Python, Django, Django REST Framework, Postgres, Docker

Part 3

In Part 3, you'll code a React app that provides a user interface for the APIs you built in Parts 1 and 2. The UI is broken into two components: a search form and a results list. Along the way, we'll write end-to-end tests using Cypress.

Tools and Technologies: JavaScript, React, Cypress, Docker

Part 4

In Part 4, you'll learn how to use Elasticsearch in place of Postgres to provide comparable functionality. We'll explore how to replicate the same search techniques, such as relevancy boosting and suggestions, using Elasticsearch's concepts and Query DSL. We'll start by configuring an Elasticsearch cluster with a single node in Docker. Next, we'll build an index and walk through how to add features and what affects they have on the search results, just like we did in Part 2 for Postgres. We'll cap things off by using Nginx to serve traffic between the frontend and the backend.

Tools and Technologies: Python, Django, Django REST Framework, Elasticsearch, Nginx, Docker




Table of Contents

Table of contents from Full-text Search in Django with Postgres and Elasticsearch:


Part 1
Part 2
  • Introduction
  • Django Debug Toolbar
  • Postgres Full-text Search
  • Ranking
  • Performance
  • Highlighting
  • Spelling Mistakes
  • Next Steps
Part 3
  • React Setup
  • Cypress Testing
  • Search
  • HTTP Requests
  • More Form Fields
  • Typeahead
  • Docker React
  • Next Steps

Part 4
  • System Design
  • Elasticsearch
  • Elasticsearch Index
  • Elasticsearch API
  • Filtering
  • Ranking
  • Highlighting
  • Suggesters
  • Production
  • Next Steps



Join our mailing list to be notified about course updates and new tutorials.