A few days ago I switched my main development machine from a 2017 15" MacBook Pro with a 4-core 2.9GHz 7820HQ i7, 16GB of RAM and 512GB of SSD to a Lenovo Thinkpad X1 Extreme with a 6-core 8750H i7, 32GB of RAM and 1.5TB of Really Fast SSD.

My reasons for the switch were:

  • The fact that my upgrade path with the MacBook would have been much more expensive.
  • The disappointing MacBook keyboard; the rest of the machine caused many sparks of joy over the past 1.5 years, but the keyboard managed to make me sad Every Single Time.
  • The fact that I like to give my development environment a really good shake-up every few years.

Although the Thinkpad has solid Linux support, I have decided to venture out of my comfort zone to see if I could turn current generation Windows on a top-of-the-line workstation laptop into a productive development environment.

This post deals specifically with the development workflow for one of our main products, a web-app built with Django and React (TypeScript) that is deployed exclusively on Linux servers.

Requirements for the workflow

  • Use of first-class IDE, such as PyCharm.
  • First-class debugging using the above-mentioned IDE.
  • Canonical version of source code remains on the Windows host, can be synced or shared with the Docker container.
  • Auto-refresh has to work. When I make a change in the editor, those changes should be reflected automatically upon first reload of the browser.

Overview of the solution

  • Docker for Windows and Docker Compose are used to configure and orchestrate the Linux container(s) for running the app and its components.
  • The souce code directory on the host is exposed to the running container via bind mount.
  • The front-end system is continuously built on the Windows side and exposed to the Docker container via the same bind mount.
  • Because Docker for Windows currently does not support inotify on a bind mount, I use an ingenious little tool called docker-windows-volume-watcher which efficiently works around this problem.
  • PyCharm has first-class support for both Docker and Docker Compose. I use this for developing and debugging.

Docker setup

I use Docker for Windows to setup and run the Linux containers required for the development of this project.

Ideally, I would have preferred to use the new Linux Containers on Windows (LCOW) functionality, but that’s currently unusable due to a bug resulting in the shutdown of containers taking up to 5 minutes.

Until LCOW is fixed, we make use of the default Moby VM. One of the implications of this is that our app runs as root inside of the container.

Dockerfile configuration

Below is the simplified and redacted Dockerfile for the main app image.

The following aspects are notable:

  • The locale-dance shown here is the best way to ensure that the Ubuntu image is configured as UTF-8.
  • Setting PYTHONBUFFERED is required if you would like to see the standard output of the Django app in the docker-compose console.
  • The pipenv virtual environment is created and all dependencies are installed as part of the image building process.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

FROM ubuntu:18.04

# locales required for update-locale
RUN apt-get update && apt-get install -y software-properties-common locales

# as early as possible: generate and permanently set the correct locale
RUN locale-gen en_US.UTF-8
RUN update-locale LANG=en_US.UTF-8
ENV LANG=en_US.UTF-8
ENV LC_ALL=en_US.UTF-8

# this is required so that Django output will appear on console
# from whence docker-compose is invoked
ENV PYTHONUNBUFFERED 1

# code redacted:
# apt-based installation of python 3.7, yarn, node, etc. using RUN

# install pipenv system-wide
RUN python3.7 -m pip install pipenv

# until docker-compose / docker for windows fix the LCOW long shutdown bug:
# https://github.com/moby/moby/issues/37919
# I have to use the standard mobylinux, which means bind mounts are root
# so we run the whole app as root.

# default dir from here on out; will be created if not present
WORKDIR /home/our_product/app/

# copy python / pipenv dependency specification
COPY Pipfile .
COPY Pipfile.lock .

# ... and then get the virtualenv ready as part of the image
RUN python3.7 -m pipenv install --dev

# this is just the default command that gets executed if nothing is specified
# on the docker command-line
# https://docs.docker.com/engine/reference/builder/#cmd
CMD ["/bin/bash"]

Docker Compose configuration

Docker Compose enables us to coordinate more than just the image specified up above. Here we show a simplified version with just the app image.

Notable here is that we bind mount the source code directory (which also contains the docker configuration files) into the running container.

The pipenv command will run Django in the default WORKDIR specified in the Dockerfile.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

version: '2'
services:
  our_product:
    build:
      context: .
      dockerfile: Dockerfile.dev
    image: our_product:latest
    # only when we use LCOW / WCOW
    # platform: linux

    # this overrides CMD in the Dockerfile
    # https://docs.docker.com/compose/compose-file/#command
    # we've set WORKDIR in the Dockerfile to the app dir
    command: pipenv run python ./manage.py runserver 0.0.0.0:8000
    environment:
      - DJANGO_SETTINGS_MODULE=our_product.settings.dev
    volumes:
      # this is a bind mount
      # inside the container, this gets owner ROOT
      - .:/home/our_product/app/
    ports:
     - "8000:8000"

Starting up the system without PyCharm

Start up the docker container, building any images that might be required:

1

docker-compose up

If you’ve made changes to the Dockerfile, invoke with docker-compose up --build.

Any containers specified in the docker compose config are now up and running.

I tend to treat this whole invocation as a single command: I don’t detach with -d, because I prefer to watch the log output just there, and to press Ctrl-C when I want to stop the whole business.

Starting up continuous front-end building

I do all of the frontend transpilation on the Windows side using the usual suspects: Webpack for coordinating everything, Yarn for package management, and TypeScript (and a bit of Babel) for all of the transpilation.

In short, via a package.json script I keep webpack --watch running. As per usual, when any of the watched source files changes, webpack rebuilds the configured bundles.

This is running on the Windows side, so the usual efficient filesystem monitoring tools are used for this.

All of the built assets are exposed to the running docker container using the existing source code bind mount.

Ensuring that Django edits automatically restart the dev server

With the current Docker for Windows (pre-LCOW) setup, bind mounted host files are network shared to the Linux VM on which the docker containers are actually running.

As mentioned previously, one of the drawbacks of this system is that events on the Windows side are not propagated through to the container as filesystem events that can be detected with the efficient inotify system calls on the Linux side.

Enter docker-windows-volume-watcher!

This Python tool (there are Go versions also) uses efficient filesystem events on the Windows host to detect changed files, and then performs no-op attribute changes on those files within the Linux container to have any inotify clients, such as the Django development server, detect these.

After pip installing the tool, I use the following invocation on the Windows side:

1

docker-volume-watcher.exe --debounce 0.1 -v container_name # see docker ps

PyCharm configuration

  • In the Docker for Windows General settings, ensure that “Expose daemon on tcp://localhost:2375 without TLS” is checked. PyCharm requires this at the moment.
  • In PyCharm, open the source directory on your Windows machine.
  • Under Settings | Project | Project Interpreter add an interpreter of type Docker Compose. After configuring and selecting the docker server, you will need to select the correct docker-compose.yml file and the our_product service. After this, PyCharm will do its usual detection and listing of all installed packages.
  • Finally create a Run / Debug configuration at the top right of the UI. Here it’s important that you use the “Django Server” template and not the “Docker-compose” like I first did. The latter will work for running, but not debugging.

Conclusions

Once you’ve taken care of all of the above, you should have a fairly usable development workflow for a Django / React web-app.

Working auto-refresh is one of the most important components of iterative development. Here, the combination of building the frontend on the Windows side, and the docker-windows-volume-watcher for relaying backend editing events, was a life saver.

I am looking forward to Docker for Windows LCOW one day working even more smoothly than this setup, and obviating the need for additional moving, although quite clever, parts such as the volume watcher.