Debugging a containerized Django application in Jupyter Notebook

Back in the days when I was working as a data analyst, I used to spend hours inside Jupyter notebooks exploring, wrangling, and plotting data to gain insights. However, as I shifted my career gear towards backend software development, my usage of interactive exploratory tools dwindled.

Nowadays, I spend the majority of my time working on a fairly large Django monolith accompanied by a fleet of microservices. Although I love my text editor and terminal emulators, I miss the ability to just start a Jupyter Notebook server and run code snippets interactively. While Django allows you to open up a shell environment and run code snippets interactively, it still isn't as flexible as a notebook.

So, I wanted to see if I could connect a Jupyter notebook server to a containerized Django application running on my local machine and interactively start making queries from there. Turns out, you can do that by integrating three tools into your Dockerized environment: ipykernel, jupyter, and django-extensions. Before I start explaining how everything is tied together, here's a fully working example of a containerized Django application where you can log into the Jupyter server and start debugging the app.

The app is just a Dockerized version of the famous polls-app from the Django tutorial. The directory structure looks as follows:

We define and pin the dependencies required for the Jupyter integration in the requirements-dev.txt file:

The application dependencies are defined in the requirements.txt file:

In the mysite/mysite/_debug_settings.py file, we import the configs from the primary settings file and add the Jupyter configuration attributes there. Here's the full content of the extended _debug_settings.py file:

Notice how we're appending the django_extensions app to the INSTALLED_APPS list defined in the main settings file. Then we're setting the shell to ipython with the SHELL_PLUS attribute. The NOTEBOOK_ARGUMENTS defines the port of the Jupyter server and some auth-specific settings.

Next, in the Dockerfile, we're defining the application like this:

Finally, we're orchestrating the application and the Jupyter server in the docker-compose.yml file. Here's how it looks:

We're orchestrating three services here: webserver, jupyter, and debug. All of them extend the base web service that builds the Dockerfile. The webserver service is where the Django app is run and exposed via the 8000 port. The jupyter service runs the Jupyter server and makes it accessible through your browser via the 8895 port. Additionally, note how we are using our extended version of the main settings by overriding the DJANGO_SETTINGS_MODULE environment variable and setting it to mysite._debug_settings. The debug container is spun up to run the migration commands and perform other maintenance tasks within the container network. All the maintenance commands are defined in the Makefile for your convenience. You can run any of these by running make from the root directory.

And that's it!

If you have Docker and docker-compose installed on your local system, you can give it a try. Clone the fully working example repo, navigate to the root directory and run:

Then run the migration command:

Now head over to your browser and go to http://localhost:8000. You should see an empty page with a simple header like this:

If you go to http://localhost:8895, you'll be able to open a new notebook that automatically connects to your database and allows you to write interactive code immediately.

You can run the following snippet and it'll create two questions and two choices in the database.

If you run this and refresh your application server, you'll that the objects have been created and they appear in the view:

Further reading

• How to access Jupyter notebook in a Docker container