Atom - Jupyter / Hydrogen
I will show you how to use the Atom editor with Jupyter Notebook to debug Python/Django views using Hydrogen plugin.
On the Internet you can find some examples using this setup to debug Python but I found nothing related to Django, so I decided to publish this notes.
This is not a high performance setup, because it involves many technologies at several layers. This is more a proof of concept of nowadays technologies which are available to the programmer's toolbox.
This notes has been written during the development and improvement of the automated tests of the Virtual Library of FAPESP (http://www.bv.fapesp.br).
The environment
The diagram below shows an actual infrastructure to use this setup in a development environment.
Technologies
It is not expected that you know all of the technologies in detail, but there are some trick points. This paper is oriented for Python programmer who already use Atom and Jupyter and enjoy explore the edges of these tools.To make the set of Atom and Jupyter work as expected, it has been used different technologies in terms of products and protocolos. I am not listing all, but the main components.
- Python/Django/virtualenv
- Atom Editor/ Atom plugins
- Hydrogen
- Jupyter Notebook
- Websocket
Jupyter Notebook
Install Jupyter as you prefer 1. I don’t use Anaconda and Conda, but you can install them if you need.Just for the records, I will paste this link that explain very concisely how to install Jupyter on a different virtualenv than your Django project 2. This is very important in my case, because I am in a company that has a large infrastructure and I cannot choose all the components I want. This box runs CentOS and the packages version around the Python environment are not always updated as you need. This link explains that you can use a virtualenv with Python 3 to run Jupyter, even if your project runs in a Python 2.7, like my setup.
Jupyter kernel
Build the Jupyter kernel for the project you want. Here you can choose the environment (virtualenv) that has all the packages your project uses.Jupyter authentication
Before you run Jupyter, you can define the type of authentication you want to use to connect to Jupyter service. The best option, to the user point of view, may be to define a password, but for our test, if you define a password, you need to execute some extra steps. So we will use the default from Jupyter authentication which is to use a token generated when you fire Jupyter.Running Jupyter, nothing new under the sun. As you can check on the image below a Jupyter web interfacer running.
- Jupyter authentication running with secret token
- Kernel I have created
- The root folder is the one where you started Jupyter.
Atom editor/ Hydrogen plugin
Install and configure the Hydrogen plugin. There is only one mandatory setting, which is the Kernel Gateway option. This is the URL for your running Jupyter instance.As you can see on the image below:
- Configure your Kernel Gateway
- After that you can connect to your Jupyter running instance. You will be asked for the secret token, since we are using the default authentication method.
- You can set shortcuts for Hydrogen commands. This is very useful to execute your code inside Atom on Jupyter kernels.
Websocket
Jupyter notebook uses a Tornado websocket server, which is the component responsible for transmit back code to Atom editor.If you open Atom console, which actually is built upon Chrome web browser, you can see this connection attempt on the console.
Unfortunately, you can not trace network connections on Atom network tab inspection. I think that is restricted by design by the Atom or maybe the Hydrogen team.
Django setup
In order to debug a Django view on Hydrogen/Jupyter/Atom, there are some variables we need to set on the views that we are going to execute.HTTP object
We want to execute the views directly from inside Atom, using Hydrogen. And Django is a web framework, which views requires an HTTP object.So we need to instantiate an HTTP object and set manually some parameters that our view requires.
All of the request parameters are optional, but the object itself.
from django.http import HttpRequest request = HttpRequest() request.method = 'GET' request.META['SERVER_NAME'] = '' request.META['SERVER_PORT'] = '' |
Django variables
Again, on this stackoverflow question/comment 3, the author explains how to connect a Django application to Jupyter Notebook, and configure the variables you need.But we are not just running Jupyter Notebook, but it’s plugin for Atom editor, Hydrogen.
So in order to execute the view, we have to call django.setup()4 for a standalone Django usage.
import django
django.setup() |
Debugging
If your configuration is all set, you can debug your Django views inside Atom using Hydrogen/Jupyter.On the image bellow you can see the Atom editor, connected to Jupyter instance. If you are not connected, open Atom menu on Packages-> Hydrogen -> Connect to remote kernels
The ssh connection was made on PUTTY, due to Atom on Windows restrictions. On Linux you can open a SSH connection from within Atom using plugins. This SSH client is connected to your Jupyter instance, where you are going to view your debug messages. We will be there in a minute.
Let me explain what is happening on the image below (follow the red numbers):
- Executing a Django view. Remember that we have created an HTTP object instance to use as an argument to call the view.
To execute any file on Jupyter via Hydrogen, just follow the Atom menu:
Packages-> Hydrogen -> Run - This piece of code is inside the get_context_data method of the view. For this example, I am just recovering all objects from one model and after that I inserted a breakpoint.
- Jupyter throws it’s output to Atom editor. Here Atom opens a dialog box, where you can insert your commands for ipdb. I am just inspecting the model objects Queryset and asking for it’s id.
- Here is the result, at the Jupyter console.
References:
