On Wednesday April 3, 2019, I finished migrating this site from Wordpress to Hugo. An important part of this project was finding a new system for handling blog post comments. On my personal blog, I am able to use the free and ad-free tier of Disqus, but because vxlabs.com is strictly-speaking a company site (even although 100% of the posts are non-commercial), that won’t work here. Furthermore, the $9 / month price tag of the lowest Disqus tier is not really justifiable with the relatively small number of comments being handled.
TL;DR: For best results with mixed precision training, use NVIDIA’s Automatic Mixed Precision together with fastai, and remember to set any epsilons, for example in the optimizer, correctly. Background Newer NVIDIA GPUs such as the consumer RTX range, the Tesla V100 and others have hardware support for half-precision / fp16 tensors. This is interesting, because many deep neural networks still function perfectly if you store most of their parameters using the far more compact 16-bit floating point precision.
I have prepared a simple Ansible script which will enable you to convert a clean Ubuntu 18.04 image (as supplied by Google Compute Engine or PaperSpace) into a CUDA 10, PyTorch 1.0 preview, fastai 1.0.x, miniconda3 powerhouse, ready to live the (mixed-precision!) deep learning dream. I built this script specifically in order to be able to do mixed-precision neural network training on NVIDIA’s TensorCores. It currently makes use of the vxlabs.
In a previous post I showed how to get Palantir’s Python Language Server working together with Emacs and lsp-mode. In this post, we look at the brand new elephant in the room, Microsoft’s arguably far more powerful own Python Language Server, and how to integrate it with Emacs. Motivation Since that previous post on Palantir’s language server, I’ve been using Emacs far more intensively for Python coding in tmux on remote machines with GPUs for deep learning.
(The wheel has now been updated to the latest PyTorch 1.0 preview as of December 6, 2018.) You’ve just received a shiny new NVIDIA Turing (RTX 2070, 2080 or 2080 Ti), or maybe even a beautiful Tesla V100, and now you would like to try out mixed precision (well mostly fp16) training on those lovely tensor cores, using PyTorch on an Ubuntu 18.04 LTS x86_64 system. The idea is that these tensor cores chew through fp16 much faster than they do through fp32.
Over the years, I’ve built up quite a collection of notes as Org mode text files. So far, it has proven to be the most expressive and the most robust note-taking modality out of a long list of candidates that I’ve tried. Note-taking using Org mode has one big drawback however: Mobile accessibility. In other words, consulting one’s org mode notes database from a mobile device is painful. This should not be the case; notes should be always and instantly available, even on mobile.
The language server protocol was proposed by Microsoft as a way for different editors and development environments to share language analysis backends This post describes how to configure Emacs, lsp-mode and the palantir python-language-server for improved code intelligence when working on Python projects. (I’m planning a companion post for Emacs, C++ and the cquery language server.) Goal Before starting, it is motivating to see what we are working towards.