Inline GraphViz DOT evaluation for graphs using Emacs, org-mode and org-babel

With Emacs, org mode and org-babel, it’s possible to evaluate source code samples embedded in your org files and have the output of said evaluation appear inline. This makes for a beautiful literate programming environment. It also enables one to include graphs in one’s documents (org mode, PDF, HTML presentations or blog posts) by using for example GraphViz.

This blog post (obviously authored using Emacs and Org mode) contains short instructions for doing so.

First follow the org-babel documentation and enable source code evaluation for dot by adding the following to your Emacs init.el:

 '((dot . t)))

Then create a new document, and add something like the following dot source code sample to it:

#+BEGIN_SRC dot :file dot_success.png :cmdline -Kdot -Tpng
  digraph {
  // graph from left to right
  node [shape=box];

  id [label="Install Graphviz"]
  conf [label="Configure org-babel"]
  dot [label="DOT in org-mode"]

  id -> conf
  conf -> dot
  dot -> "Profit"
  dot -> "Success" [style=dotted]

Now press C-c C-c to evaluate this code. Emacs will generate the configured output file dot_success.png and then link to it in an automatically created #+RESULTS section right below it.

You can press C-c C-x C-v to toggle display of inline images to see it directly in Emacs. Alternatively, use M-x org-display-inline-images to switch this on. Whenever you change the DOT source code (press C-c ‘ to edit the dot source code in a separate buffer), just press C-c C-c to re-execute the updated source.

In any exported documents (for examlpe this blog post), only the output graph itself will appear, like this:

Exported as a PDF with C-e l o it looks like this.

In your Emacs, it should look like this:

This approach should work for any of the many languages supported by org-babel. Let me know in the comments what you come up with!

Getting Django Rest Framework to parse docstrings as reStructuredText

The Django REST Framework is awesome, for a whole bunch of reasons, one of them being the browsable HTML version of your API that it automatically generates for you.

As a part of this, it extracts any docstrings that you might have defined for the relevant class (ViewSet or CBV) and adds an HTML version of this documentation to the browsable API, like this:

However, as described in the relevant documentation, it expects Markdown syntax by default. I like Markdown, but the rest of my Python docstrings are all in reStructuredText, which is the default documentation format for many Python projects.

In short, I would prefer to type my view docstrings in reStructuredText and not in the DRF-default markdown, because reStructuredText is what all of my other non-DRF code is using.

Fortunately, DRF has made this relatively easily configurable, if you’re not afraid to do some searching and digging (I couldn’t find a single complete worked-out example). Especially the code for transforming RST text into parsed HTML is not as straight-forward as one would have hoped. The DRF part of the deal is clearly described in its documentation.

Here’s how you can do this:

Define a new DRF view description function

Add the following function to your code somewhere. I added it to the views module in a Django app with reusable library code:

from django.utils.safestring import mark_safe
from docutils import core
from rest_framework.compat import smart_text
from rest_framework.utils import formatting

def get_view_description(view_cls, html=False):
    """Alternative view description function to be used as the DRF
    ``VIEW_DESCRIPTION_FUNCTION`` so that RestructuredText can be used
    instead of the DRF-default MarkDown.

    Except for the RST parts, derived by from the
    DRF default get_view_description function.


    description = view_cls.__doc__ or ''
    description = formatting.dedent(smart_text(description))
    if html:
        # from -- we use the
        # third recipe to get just the HTML parts corresponding to the ReST
        # docstring:
        parts = core.publish_parts(source=description, writer_name='html')
        html = parts['body_pre_docinfo']+parts['fragment']
        # have to use mark_safe so our HTML will get explicitly marked as
        # safe and will be correctly rendered
        return mark_safe(html)

    return description

Configure the view description function

In your settings, activate the new view description function:

    # other REST_FRAMEWORK config here...
    'VIEW_DESCRIPTION_FUNCTION': 'your_app.views.get_view_description'

Enjoy your reStructuredText

If you now reload any of your browsable API pages, you should see your reStructuredText nicely parsed.

Use ADB to bypass dog-slow MTP transfer of files from Android to Linux

Last night I had to backup 2760 photos and videos, about 6.3G worth, from my Nexus 4 Android phone to my Linux laptop.

The Nexus 4, like many other Android phones, only supports the Media Transfer Protocol, or MTP, for transferring files via USB connection. With Ubuntu 14.04, this is a fortunately a plug and play situation: Connect the phone via USB cable, and start dragging and dropping files to and fro using the built-in file manager on the Linux side.

Unfortunately, this turned out to be dog slow. Stabilising at about 1Mbyte/s, this was going to take more time than I had at my disposal. This looks like it could be a Linux-only problem, but I’d like to see that confirmed. Whatever the case may be, I had to find alternatives.

Accessing the AirDroid web-interface on my Android telephone.

My next stop was the ssh server app on the android side. I confirmed that I could ssh in to my unrooted phone (pretty cool that!), and then I confirmed, using the built in ssh:// filesystem support in the Linux file manager, that file transfer throughput was still dog slow (also about 1Mbit/s). After some minutes, it looked like the whole transfer stalled completely.

btsync was of no help in this case, because I have 1.4 on my phone, and use 1.3 on all my other devices (1.4 was completely useless in its beta phase, so I decided to stick with 1.3 until convinced otherwise).

AirDroid is an extremely well done app that enables one to remote control one’s android phone via a super attractive web interface, over its wifi interface. One of its many functions is file transfer, up and down. Selecting to download a whole directory of files results in a huge ZIP file being streamed. This came down at between 2 and 3 Mbyte/s, staying close to 3 for most of the time.

I was still curious whether we could do better via the USB cable, instead of over wifi, so I fired up the Android Debug Bridge, or ADB. This only works if developer mode has been activated on the android phone, and USB debugging mode has been activated. In other words, this is probably not for novice users.

By using adb pull on the whole directory of files, it started downloading all 6.3G worth of photos and videos. At the end of this, the average throughput was 4.2Mbyte/s, the best of all methods I had tested.


If you need to transfer many files from your MTP-only Android telephone to a Linux host, either use AirDroid if you want this to be as easy (and as pretty!) as possible, or use ADB if you want the maximum throughput and don’t mind getting your hands dirty.

Convert dates to different formats in LibreOffice Calc

I’m helping someone process a collection of research data that has been entered by a third party using Excel. We’re using LibreOffice Calc, because research should be reproducible by anyone, not just those in possession of prioprietary software licenses (this also means that we use R, JGR and DeduceR instead of SPSS and Statistica; perhaps more on that later).

After having to fix hundreds of badly entered dates with basic functions (we highly recommend that ISO 8601 format dates, i.e. YYYY-MM-DD, are used from the very start, instead of ambiguous local formats), we ended up with a stubborn subset of DD-MM-YYYY formatted dates in cells that explicitly had ISO 8601 format configured.

What could we do to convert these DD-MM-YYYY dates to the ISO 8601 standard YYYY-MM-DD dates?

The trick here is to use LibreOffice RIGHT, MID and LEFT functions to pull the faulty date apart, and then to put it back together in a new column using the DATE function.

In this screenshot you can see an example:

RIGHT(A2,4) extracts 4 characters from the right of 13-12-1981, yielding 1981, MID(A2,4,2) yields the 2 characters starting at position 4, and LEFT(A2,2) gives us the two characters at the left, in other words the day.

Putting all of this together with the function =DATE(RIGHT(A2,4), MID(A2,4,2), LEFT(A2,2)) will return the date formatted according to the column format setting (right click column heading, select Format Cells and then the ISO 8601 formatting). Obviously you have to replace A2 with the cell broken date format that you want to fix.

Globally, you would create a new column, use the formula above in its first row to fix the first date in the existing broken date column, then replicate the formula all the way down by clicking on it, and then dragging the rectangle at its lower right all the way down.

(Searching for “LibreOffice convert date formats” was of no help, whereas doing the same for Excel yielded at least two good answers (one and two), on which this post is based. I’m putting this out there so that searching for LibreOffice will also turn up something useful.)

CloudFlare full optimizations break MathJax

I’ve just activated CloudFlare (the free tier) for, hoping to do even faster page loads. Most of my WordPress installations already use WP Super Cache to serve mostly static pages when you come here, but CloudFlare should speed this up even further via their content and network traffic optimization, and their servers dotted all over the globe.

Configuration was quite painless (you have to configure your domain’s DNS to point to CloudFlare’s servers, then a few settings using their web interface). It somehow missed my mail server’s DNS record when copying everything over, but that was quick to fix. (It also seems that the SSL-protected roundcube I have on another server in this domain now keeps on logging me out after 2 seconds, but I’m not 100% sure that that’s due to CloudFlare.)

In any case, I soon noticed that the MathJax equations in this post about level sets did not show up. Turns out you need to change the CloudFlare Performance Profile from “CDN + Full Optimizations” to “CDN + Basic Optimizations”, and then your equations should appear again.

In other words, do this:

to see this: