Getting EmacsKeys working with QtCreator 3.1.2

Today, Qt 5.3.1 was released along with Qt Creator 3.1.2. Unfortunately, nsf’s EmacsKeys plugin, merged into the Qt trunk a few months ago, was not a part of this release (it should be included in Qt Creator 3.2).

Because the Emacs keybindings are hardwired into my fingers, and I’m using QtCreator for a project at the moment, I spent some time figuring out how to get the plugin built for Qt Creator 3.1.2. This post explains how you too can build it, but, if you’re on Ubuntu 14.04 with Qt 5.3.1 x64, you can just download my binaries and the keymap file (see under requirements).

I’ve tried to setup most of the common Emacs keybindings documented in nsf’s plugin, plus a few more, and I’ve gotten rid of the conflicts. (Thanks nsf for putting me on the right path with that!)

A subset of the Emacs keybindings in the keymap file.

Requirements

You need to have Qt 5.3.1 installed. I used the open source x64 .run files made available by the qt-project.

For both of the following approaches, download and unpack the qtcreator-emacskeys archive I’ve prepared especially for you.

The Easy Way: Install Binaries

  • Copy the archive nsf directory to Qt/Tools/QtCreator/lib/qtcreator/plugins.
  • Copy the archive emacskeys.kms to Qt/Tools/QtCreator/share/qtcreator/schemes.
  • Start QtCreator.
  • In Help | About | Plugins activate EmacsKeys under Utilities.
  • Restart QtCreator.
  • Under Tools | Options | Environment | Keyboard click on Import and then select emacskeys.nsf.

You can scroll down to the Emacs Keys section and check that my choices work for you.

The Hard Way: Build ‘em Yourself

Get the Qt Creator source code by typing:

git clone --recursive https://git.gitorious.org/qt-creator/qt-creator.git

Copy the src/plugins/emacskeys directory somewhere else, out of the whole qt-creator tree, because you’re going to revert to the 3.1.2 release version:

cd qt-creator
git checkout tags/v3.1.2

Copy the emacskeys.pro from my archive into the emacskeys directory that you copied out. Edit QTCREATOR_SOURCES to point to te v3.1.2 qt-creator checkout that you prepared above, and IDE_BUILD_TREE to point to your installed QtCreator directory.

In QtCreator, open the .pro file that you’ve just edited, and build the project. If all goes according to plan, this will put the resultant .so file into the correct plugins directory in a subdirectory called nsf.

Now follow the rest of the steps from the easy way above.

Limitations

I experienced the problem that Alt-W would deselect any existing mark at the press of Alt, so nothing was copied. To get around this, I’ve mapped copy to Esc-W.

Level sets: The practical 10 minute introduction.

Post summary: The level set method is a powerful alternative way to represent N-dimensional surfaces evolving through space.

(This is a significantly extended blog-post version of three slides from my Medical Visualization lecture on image analysis.)

Imagine that you would like to represent a contour in 2D or a surface in 3D, for example to delineate objects in a 2D image or in a 3D volumetric dataset.

Now imagine that for some reason you would also like have this contour or surface move through space, for example to inflate it, or to shrink it, at the same time dynamically morphing the surface to better fit around the object of interest.

Mathematically, we would have for N dimensions the following representation:

\[ C(p,t) = \lbrace x_0(p,t), x_1(p,t), \cdots, x_{N-1}(p,t) \rbrace\tag{1} \]

Where \(C(p,t)\) is the collection of contours that you get when you morph or evolve the initial contour \(C(p,0)\) over time \(t\). At each point in time, each little bit of the contour moves orthogonally to itself (i.e. along its local normal) with a speed \(F(C(p,t))\) that is calculated based on that little bit of contour at that point in time.

This morphing of the contour can thus be described as follows:

\[ \frac{\partial C(p,t)}{\partial t} = F(C(p,t))\vec{n}(C(p,t))\tag{2} \]

In other words, the change in the contour relative to the change in time is defined by that contour- and time-dependent speed \(F\) and the normal to the contour at that point in space and time.

There are at least two ways of representing such contours, both in motion and at rest.

Contours as (moving) points in space: Explicit and Lagrangian.

Most programmers would (sooner or later) come up with the idea of using a bunch of points, and the line segments that connect them, to represent 2D contours, and, analogously, dense meshes consisting of triangles and other polygons to represent 3D surfaces, a la OpenGL.

You would be in good company if you then implemented an algorithm whereby for each iteration in time, you iterated over all points and moved each point a small distance \(F\times \Delta t\), orthogonally to the contour containing it. Such an algorithm could be called active contours, or snakes, and this way of representing a contour or surface as a collection of points moving through space is often called the Lagrangian formulation.

Now imagine that your contour or surface became quite large. You would need to add new points, or a whole bunch of new triangles. This could cause minor headaches. However, your headaches would grow in severity if your contour or surface were to shrink, and you would at some point need to remove, very carefully, extraneous points, edges or triangles. However, this headache pales in comparison to the one you would get if your surface, due to the object it was trying to delineate, would have to split into multiple surfaces, or later would have to merge back into a single surface.

Contours as (changing) measurements of the space around them: Implicit and Eulerian.

However, if you were as clever as, say James Sethian and Stanley Osher, you would decide to sidestep all of that headache and represent your contour implicitly. This can be done by creating a higher-dimensional embedding function or level set function of which the zero level set, or the zero-valued isocontour, is the contour that you’re trying to represent. This is often called the Eulerian formulation, because we’re focusing on specific locations in space as the contour moves through them.

Huh?

(that was what I said when I first read this.)

What’s an isocontour? That’s a line, or a surface (or a hyper-surface in more than 3D) that passes through all locations containing the isovalue. For example, on a contour map you have contour lines going through all positions with the same altitude.

If I were to create a 2D grid, with at each each grid position the floating point closest distance to a 2D contour (by convention, the distance inside the contour is negative and outside positive), then the contour line at value zero (or the zero level set) of that 2D grid would be exactly the 2D contour!

The 2D grid I’ve described above is known as an embedding function \( \phi(x,t)\) of the 2D contour. There are more ways to derive such embedding functions, but the signed distance field (SDF) is a common and practical method.

Let’s summarise what we have up to now: Instead of representing a contour as points, or a surface as triangles, we can represent them as respectively a 2D or 3D grid of signed distance values

For the general case up above, the contour at fixed time \(t_i\) would be:

\[ C(t_i) = \lbrace x \vert \phi(x,t_i) = 0 \rbrace \]

Moving contours are simply point-wise changes in their embedding functions

Instead of directly evolving the contour, it can be implicitly evolved by incrementally modifying its embedding function. Given the initial embedding function \(\phi(x,t=0)\), the contour can be implicitly evolved as follows:

\[ \frac{\partial \phi(x,t)}{\partial t} = - F(x,t) \lvert \nabla \phi(x,t) \rvert \]

where \(F(x,t)\) is a scalar speed function, \(\nabla\) is the gradient operator and \(\lvert \nabla \phi(x,t) \rvert\) is the magnitude of the gradient of the level set function. Note the similarity with the general contour evolution equation 2 above.

In practice, this means that for each iteration, you simply raster scan through the embedding function (2D, 3D or ND grid), and for each grid position you calculate speed \(F(x,y)\), multiply with the negative gradient magnitude of the embedding function, and then add the result to whatever value you found there.

If you were to extract the zero level set at each iteration, you would see the resultant contour (or surface) deform over time according to the speed function \(F\) that you defined.

MAGIC!

A visual confirmation

The figure below is a signed distance field representing two concentric circles in 2D, or a 2D donut. Note that the values are negative inside the donut, and positive elsewhere (in the center of the image, and towards the limits of the grid).

Let’s have a donut!

For the sake of this exposition, let’s define \(F(x,t)=1\), i.e. the donut should get fatter over time. Eventually it will get so fat that the hole in the middle will be closed up.

You can now check by inspection that at each point in the embedding function, or the image, you would have to add a small negative number (\(-1\) multiplied by the gradient magnitude). First positive regions close to the contour will become negative, increasing its size, and regions further away will approach zero from above, and eventually also become negative, making the donut even fatter. Eventually the region in the middle will become all negative, in other words, close up.

Conclusion

Representing surfaces implicitly, and specifically as signed distance fields, has numerous advantages.

  • Contours always have sub-grid-point resolution.
  • Topological changes, such as splitting and merging of N-dimensional objects, is automatically handled.
  • Implementation of N-dimensional contour propagation becomes relatively straight-forward.
  • With this implicit representation, morphing between two N-dimensional contours is a simple point-wise weighted interpolation!

Although iterating through the embedding function is much simpler than managing a whole bunch of triangles and points moving through space, it can be computationally quite demanding. A number of optimization techniques exist, mostly making use of the fact that one only has to maintain the distance field in a narrow region around the evolving contour. These are called narrow-band techniques.

The Insight Segmentation and Registration ToolKit, or ITK, has a very good implementation of a number of different level set method variations. You can also use the open source DeVIDE system to experiment with level set segmentation and 3D volumetric datasets (it makes use of ITK for this).

(I’m planning to make available my MedVis post-graduate lecture exercises. Let me know in the comments how much I should hurry up.)

Huawei E3331 3G USB dongle works on Ubuntu 14.04 Linux

In the store today, I wanted to check that the Huawei E3331 3G USB dongle I was about to buy would work with my Ubuntu Linux laptops. Because I couldn’t find any posts confirming this, I’m writing this one.

Summary: I can confirm that the Huawei E3331 3G USB dongle works, completely out of the box and without any problems, on Ubuntu 14.04.

After inserting the card into a USB slot, I was greeted by this notification:

As per the instructions, I could immediately open the HiLink Web UI at http://192.168.1.1 with my browser, where, after configuring my APN like this:

The home screen showed that I was successfully connected to the 3G network:

No drivers were required. Linux (in my case Ubuntu 14.04 on x86_64) is able to connect to the device using its built-in LAN-over-USB support. This is what the relevant part of the system log looks like when the device is inserted:

[378719.431633] usb 3-3: new high-speed USB device number 73 using xhci_hcd
[378719.450078] usb 3-3: New USB device found, idVendor=12d1, idProduct=14db
[378719.450085] usb 3-3: New USB device strings: Mfr=2, Product=1, SerialNumber=0
[378719.450089] usb 3-3: Product: HUAWEI Mobile
[378719.450092] usb 3-3: Manufacturer: HUAWEI
[378719.461252] cdc_ether 3-3:1.0 eth1: register 'cdc_ether' at usb-0000:00:14.0-3, CDC Ethernet Device, 58:2c:80:13:92:63

Here you have the prerequisite speedtest:

3563342453.png

(This dongle is a connectivity backup. It’s an added bonus that the upstream is 3x that of my ADSL at home.)

Fixing RoundCube Error in DDL upgrade

Just in case you run into the same problem I did when upgrading Roundcube from version 0.9.5 to 1.0.1, I’m posting the exact steps I took to fix the Error in DDL upgrade problem.

So, you’ve followed the Roundcube upgrade instructions, either using the installto.sh or the update.sh script, and then this happens:

Updating database schema (2013042700)... [OK]
Updating database schema (2013052500)... [FAILED]
ERROR: Error in DDL upgrade 2013052500: [1142] CREATE command denied to user 'roundcube'@'localhost' for table 'cache_shared'All done.

You search around and find this forum post by the author of iRedMail. However, if you’re like me, you need a little more detail to be able to apply his advice. Here’s how:

First connect to mysql using the root account (the password is in the iRedMail.tips file that was created when you installed iRedMail):

mysql -u root -p

Grant both the CREATE and ALTER privileges to the roundcube database user:

GRANT CREATE,ALTER on roundcubemail.* to roundcube@localhost;
FLUSH PRIVILEGES;
\q

Then finish the upgrade by starting the update script again. In my case, I started with installto.sh, then ran into the error, then applied the fix and started the update.sh script in the rountdcube target directory:

$ roundcube-dest/bin/update.sh
What version are you upgrading from? Type '?' if you don't know.
0.9.5
Executing database schema update.
Updating database schema (2013061000)... [OK]
Updating database schema (2014042900)... [OK]
This instance of Roundcube is up-to-date.
Have fun!

Configuring Emacs mu4e with nullmailer, offlineimap and multiple identities

mu4e is a mail user agent for your Emacs. After leaving GMail a few months ago, this is the first MUA that I am loving even more. The major reasons for this are the faster than GMail real-time search (e.g. press s, then from:buddy flag:attach design review and watch it search my 68 thousand email archive in a fraction of a second), its Emacs foundation (the more I use it, the better I get at customizing it) and the observation that having my email in an uncluttered, by default text-only interface, somehow helps me to maintain the feeling of control.

A screenshot of my mu4e, showing the results of the search query “flag:list debian”.

This post explains briefly how to setup nullmailer (for sending mail really quickly via your SMTP server), offlineimap (for keeping your local IMAP folders in sync with your IMAP server) and finally mu4e.

nullmailer

mu4e can send mail by connecting directly to your SMTP server, and there’s a commented-out section in the configuration below that shows you how to do this. However, this can sometimes be slow, so I opted to install nullmailer on my laptops. nullmailer is a sendmail lookalike with its own queue, so that mu4e can hand-off your outgoing mail and get back to you immediately whilst nullmailer takes care of the sending in the background.

To support encrypted SMTP (highly recommended), you need at least version 1.10 of nullmailer. On Ubuntu 12.04 I made use of this backport. On Ubuntu 14.04, the nullmailer is up to date and you can simply apt-get install nullmailer.

In /etc/nullmailer/remotes, readable only to its owner, I have the following:

mymailserver.com smtp --port=587 --starttls --user=myusername --pass=mypassword

offlineimap

I’m using offlineimap 6.5.5, built from source.

My ~/.offlineimaprc looks like this:

[general]
accounts = myaccountname
maxsyncaccounts = 3

[Account myaccountname]
localrepository = Local
remoterepository = Remote
# speeds up syncing!
status_backend = sqlite
# -1: always do quick updates
#  0: never do quick updates
quick = -1

[Repository Local]
type = Maildir
localfolders = ~/Maildir

[Repository Remote]
type = IMAP
remotehost = mymailserver.com
remoteuser = myusername
remotepass = mypassword
ssl = yes
# new comodo fingerprint (SHA1)
cert_fingerprint = 8f1e5dcaf69941a8be9fd04c7f4633895f4d2ead
# use at least two threads to sync mail (speeds up)
maxconnections = 2
realdelete = yes

This is for a straight-forward dovecot IMAP server (I manage my own). Please use Google for GMail setups.

mu4e

Install mu and mu4e from source. On Ubuntu, it goes like this:

sudo apt-get install libgmime-2.6-dev libxapian-dev
sudo apt-get install guile-2.0-dev html2text xdg-utils
git clone https://github.com/djcb/mu.git
cd mu
autoreconf -i && ./configure && make
sudo make install

Study the following mu4e configuration, then modify and add to your init.el (or similar).

This configuration (mine) supports multiple identities that can be activated by either pressing a shortcut key in the main screen or headers, or when you’re replying to an email that was sent to an email that’s associated with one of your identities. Each identity is configured by a function that you define. Here that includes the from address and the signature.

;; http://xahlee.blogspot.com/2010/09/elisp-read-file-content-in-one-shot.html
;; we'll use this to read your different signatures from files
(defun get-string-from-file (filePath)
  "Return FILEPATH's file content."
  (with-temp-buffer
    (insert-file-contents filePath)
    (buffer-string)))

;; this is where the install procedure above puts your mu4e
(add-to-list 'load-path "/usr/local/share/emacs/site-lisp/mu4e")

(require 'mu4e)

;; path to our Maildir directory
(setq mu4e-maildir "/home/cpbotha/Maildir")

;; should mu4e use fancy utf characters? NO. they're ugly.
;;(setq mu4e-use-fancy-chars 't)

;; give me ISO(ish) format date-time stamps in the header list
(setq  mu4e-headers-date-format "%Y-%m-%d %H:%M")

;; customize the reply-quote-string
;; M-x find-function RET message-citation-line-format for docs
(setq message-citation-line-format "%N @ %Y-%m-%d %H:%M %Z:\n")
(setq message-citation-line-function 'message-insert-formatted-citation-line)

;; show full addresses in view message (instead of just names)
;; toggle per name with M-RET
(setq mu4e-view-show-addresses 't)

;; without this, "symbol's value as variable is void: mml2014-use" when signing
;; then found http://www.gnu.org/software/emacs/manual/html_node/gnus/Security.html
;; so set to epg and all was good!
;; to sign a mail: M-x mml-secure-sign-pgpmime
(setq mml2015-use 'epg)

;; the next are relative to `mu4e-maildir'
;; instead of strings, they can be functions too, see
;; their docstring or the chapter 'Dynamic folders'
(setq mu4e-sent-folder   "/Archives.2014"
      mu4e-drafts-folder "/Drafts"
      mu4e-trash-folder  "/Trash")

;; the maildirs you use frequently; access them with 'j' ('jump')
(setq   mu4e-maildir-shortcuts
    '(("/Archives.2014"     . ?a)
      ("/INBOX"       . ?i)
      ("/Inbox-WORK"        . ?w)
      ("/Archives.2014"        . ?s)))

;; list of my email addresses.
(setq mu4e-user-mail-address-list '("info@charlbotha.com"
                                    "cpbotha@cpbotha.net"
                                    "cpbotha@vxlabs.com"
                                    "cpbotha@medvis.org"
                                    "charl@treparel.com"
                                    "charl.botha@treparel.com"))


;; when you want to use some external command for html->text
;; conversion, e.g. the 'html2text' program
;; (cpbotha: html2text sees to work better than the built-in one)
(setq mu4e-html2text-command "html2text")

;; mu4e-action-view-in-browser is built into mu4e
;; by adding it to these lists of custom actions
;; it can be invoked by first pressing a, then selecting
(add-to-list 'mu4e-headers-actions
             '("in browser" . mu4e-action-view-in-browser) t)
(add-to-list 'mu4e-view-actions
             '("in browser" . mu4e-action-view-in-browser) t)


;; the headers to show in the headers list -- a pair of a field
;; and its width, with `nil' meaning 'unlimited'
;; (better only use that for the last field.
;; These are the defaults:
(setq mu4e-headers-fields
    '( (:date          .  25)
       (:flags         .   6)
       (:from          .  22)
       (:subject       .  nil)))

;; program to get mail; alternatives are 'fetchmail', 'getmail'
;; isync or your own shellscript. called when 'U' is pressed in
;; main view.

;; If you get your mail without an explicit command,
;; use "true" for the command (this is the default)
(setq mu4e-get-mail-command "offlineimap")

;; setup default identity here:
;; general emacs mail settings; used when composing e-mail
;; the non-mu4e-* stuff is inherited from emacs/message-mode
(setq user-mail-address "info@charlbotha.com"
      user-full-name  "Charl P. Botha")

;; set this to nil so signature is not included by default
;; you can include in message with C-c C-w
(setq mu4e-compose-signature-auto-include 't)
(setq mu4e-compose-signature (with-temp-buffer
                               (insert-file-contents "~/.signature.personal")
                               (buffer-string)))
;; message-signature-file NOT used by mu4e
(setq message-signature-file "~/.signature.personal")

;; many recipes online use an alist with the different email identities
;; I like to use these functions, because then I have more flexibility
(defun cpb-mu4e-personal()
  (interactive)
  (message "personal mail account")
  (setq  user-mail-address "info@charlbotha.com"
         mu4e-compose-signature (get-string-from-file "~/.signature.personal"))
  )

(defun cpb-mu4e-vxlabs()
  (interactive)
  (message "vxlabs mail account")
  (setq  user-mail-address "cpbotha@vxlabs.com"
         mu4e-compose-signature (get-string-from-file "~/.signature.vxlabs"))
)

(defun cpb-mu4e-trep()
  (interactive)
  (message "treparel mail account")
  (setq  user-mail-address "charl.botha@treparel.com"
         mu4e-compose-signature (get-string-from-file "~/.signature.treparel"))
  )

(defun cpb-mu4e-medvis()
  (interactive)
  (message "medvis mail account")
  (setq  user-mail-address "cpbotha@medvis.org"
         mu4e-compose-signature (get-string-from-file "~/.signature.medvis"))
  )

;; quickly change account. got his idea from:
;; https://github.com/skybert/my-little-friends/blob/master/emacs/.emacs.d/tkj-mu4e.el
(define-key mu4e-main-mode-map (kbd "<f1>") 'cpb-mu4e-personal)
(define-key mu4e-main-mode-map (kbd "<f2>") 'cpb-mu4e-vxlabs)
(define-key mu4e-main-mode-map (kbd "<f4>") 'cpb-mu4e-trep)
(define-key mu4e-main-mode-map (kbd "<f6>") 'cpb-mu4e-medvis)
(define-key mu4e-headers-mode-map (kbd "<f1>") 'cpb-mu4e-personal)
(define-key mu4e-headers-mode-map (kbd "<f2>") 'cpb-mu4e-vxlabs)
(define-key mu4e-headers-mode-map (kbd "<f4>") 'cpb-mu4e-trep)
(define-key mu4e-headers-mode-map (kbd "<f6>") 'cpb-mu4e-medvis)


;; for sendmail read this http://www.gnus.org/manual/message_36.html
;; am using nullmailer, so my mail sending just became STUPID fast
(setq message-send-mail-function 'message-send-mail-with-sendmail)

;; smtp mail setting - if you DON'T want to use nullmailer, instead
;; connecting to your smtp server and waiting...
;; (setq
;;    message-send-mail-function 'smtpmail-send-it
;;    smtpmail-stream-type 'starttls
;;    smtpmail-default-smtp-server "mymailserver.com"
;;    smtpmail-smtp-server "mymailserver.com"
;;    smtpmail-smtp-service 587

;;    ;; if you need offline mode, set these -- and create the queue dir
;;    ;; with 'mu mkdir', i.e.. mu mkdir /home/user/Maildir/queue
;;    smtpmail-queue-mail  nil
;;    smtpmail-queue-dir  "/home/user/Maildir/queue/cur")

;; don't keep message buffers around
(setq message-kill-buffer-on-exit t)
;; attachments go here
(setq mu4e-attachment-dir  "~/Downloads")

;; when you reply to a message, use the identity that the mail was sent to
;; the cpbotha variation -- function that checks to, cc and bcc fields
(defun cpb-mu4e-is-message-to (msg rx)
  "Check if to, cc or bcc field in MSG has any address in RX."
  (or (mu4e-message-contact-field-matches msg :to rx)
      (mu4e-message-contact-field-matches msg :cc rx)
      (mu4e-message-contact-field-matches msg :bcc rx)))

;; we only do something if we recognize something (i.e. no stupid default)
(add-hook 'mu4e-compose-pre-hook
          (defun my-set-from-address ()
            "Set current identity based on to, cc, bcc of original."
            (let ((msg mu4e-compose-parent-message)) ;; msg is shorter...
              (if msg
                  (cond
                   ((cpb-mu4e-is-message-to msg (list "cpbotha@cpbotha.net"
                                                      "info@charlbotha.com"))
                    (cpb-mu4e-personal))
                   ((cpb-mu4e-is-message-to msg (list "cpbotha@vxlabs.com"
                                                      "charl@stonethree.com"))
                    (cpb-mu4e-vxlabs))
                   ((cpb-mu4e-is-message-to msg (list "charl.botha@treparel.com"
                                                      "charl@treparel.com"))
                    (cpb-mu4e-trep))
                   ((cpb-mu4e-is-message-to msg "cpbotha@medvis.org")
                    (cpb-mu4e-medvis)))))))

;; convenience function for starting the whole mu4e in its own frame
;; posted by the author of mu4e on the mailing list
(defun mu4e-in-new-frame ()
  "Start mu4e in new frame."
  (interactive)
  (select-frame (make-frame))
  (mu4e))

Getting started

After everything has been configured, download and index your email by doing:

# download all imap email, this can take while
offlineimap
# index all of the mail -- initial index takes a while, then fast!
mu index --maildir=~/Maildir

In the future, press U on the mu4e main screen, or C-S-u on the headers or view screen to retrieve and index new mail from within Emacs.

Conclusion

I hope that you find this howto and configuration useful. Please let me know in the comments if you got it working, or if you have any questions.

Enjoy your mail again with mu4e!