mu4e 0.9.18: E-Mailing with Emacs now even better.

In this post I talk about three great new features in the latest release of mu4e, an email programme that runs in Emacs. I also show my mu4e configuration as an example to others who would like a similar setup.

mu4e-0.9.18-screenshot.png
mu4e 0.9.18 screenshot showing selected context (bottom right of main) and visual-line-mode (long lines) which is activated by the format=flowed support.

After recently discovering that plaintext format=flowed in Thunderbird works only partially, it was time to check back in on mu4e after my previous happy stint using it.

I was pleasantly surprised to see that mu4e development had been quite active, and that this Emacs mail user agent had acquired a number of highly useful new features.

Three features I find particularly interesting are:

Full and deterministic format=flowed support

As I explained in another blog post, format=flowed is a really clever extension to old-school plain text emails that enables any receiving mail app that supports the standard to reflow emails for better display on smaller or larger displays.

mu4e now has first-class and deterministic support of the format=flowed feature.

I do understand that HTML has won the email format war.

However, HTML formatting gives users entirely too much freedom when writing their emails, resulting in misguided typesetting and other unnecessary visual elements, analogous to and just as irritating as chart junk, that regularly damage sensitive eyes.

This is why I prefer writing in plain text. With reliable format=flowed support, mu4e enables me to focus on the content whilst fixing plain-text email’s only real flaw: Reflowing on mobile devices.

Compose emails in a separate window

This, just like anything else, one could previously implement by oneself in emacs-lisp. However, it’s now a welcome core feature activated via the mu4e-compose-in-new-frame variable.

When you press C to compose a new email, a separate frame (this is what Emacs calls a window) is opened. After sending the email with C-c C-c the frame is automatically closed.

Contexts

In my previous setup, I had made a bunch of emacs-lisp functions to switch between different email identities (from addresses).

This worked, but it was not nearly as convenient and well thought-out as mu4e’s new contexts.

I now have a separate context defined for each email identity. I can switch between them using ; followed by the self-configured shortcut character.

They also activate automatically based on a configurable match function, for example examining the destination address of an email I am replying to. My configuration example below shows this in a little more detail.

My configuration

Because of these new features and a number of other changes, I put together this new configuration from scratch based on mu4e’s excellent documentation and examples.

I am still using `offlineimap` to synchronise my email with the IMAP server (fastmail in my case), but I’m currently sending email using the Emacs smtpmail package.

Here is a slightly shortened and simplified version of my configuration:

;; I installed mu4e from the 0.9.18 tarball
;; this is where make install put the emacs bits
(add-to-list 'load-path "/usr/local/share/emacs/site-lisp/mu4e")

(require 'mu4e)

;; I want my format=flowed thank you very much
;; mu4e sets up visual-line-mode and also fill (M-q) to do the right thing
;; each paragraph is a single long line; at sending, emacs will add the
;; special line continuation characters.
(setq mu4e-compose-format-flowed t)

;; every new email composition gets its own frame! (window)
(setq mu4e-compose-in-new-frame t)

;; give me ISO(ish) format date-time stamps in the header list
(setq mu4e-headers-date-format "%Y-%m-%d %H:%M")

;; show full addresses in view message (instead of just names)
;; toggle per name with M-RET
(setq mu4e-view-show-addresses 't)

;; path to our Maildir directory
(setq mu4e-maildir "~/Maildir")

;; the next are relative to `mu4e-maildir'
;; instead of strings, they can be functions too, see
;; their docstring or the chapter 'Dynamic folders'
(setq mu4e-sent-folder   "/Sent"
      mu4e-drafts-folder "/Drafts"
      mu4e-trash-folder  "/Trash")

;; the maildirs you use frequently; access them with 'j' ('jump')
(setq   mu4e-maildir-shortcuts
    '(("/Archive"     . ?a)
      ("/INBOX"       . ?i)
      ("/Sent"        . ?s)))

;; the list of all of my e-mail addresses
(setq mu4e-user-mail-address-list '("me@home.com"
                                    "me@work.com"
                                    "me@org.org"))

;; the headers to show in the headers list -- a pair of a field
;; and its width, with `nil' meaning 'unlimited'
;; (better only use that for the last field.
;; These are the defaults:
(setq mu4e-headers-fields
    '( (:date          .  25)    ;; alternatively, use :human-date
       (:flags         .   6)
       (:from          .  22)
       (:subject       .  nil))) ;; alternatively, use :thread-subject

;; program to get mail; alternatives are 'fetchmail', 'getmail'
;; isync or your own shellscript. called when 'U' is pressed in
;; main view.

;; If you get your mail without an explicit command,
;; use "true" for the command (this is the default)
;; when I press U in the main view, or C-c C-u elsewhere,
;; this command is called, followed by the mu indexer
(setq mu4e-get-mail-command "offlineimap")

;; not using smtp-async yet
;; some of these variables will get overridden by the contexts
(setq
 send-mail-function 'smtpmail-send-it
 message-send-mail-function 'smtpmail-send-it
 smtpmail-smtp-server "smtp.fastmail.com"
 smtpmail-smtp-service 465
 smtpmail-stream-type 'ssl
 )

;; don't keep message buffers around
(setq message-kill-buffer-on-exit t)

;; here come the contexts
;; I have about 5 of these, chopped down to 2 for demonstration purposes
;; each context can set any number of variables (see :vars)
;; for example below here I'm using two different SMTP servers depending on identity
(setq mu4e-contexts
      `( ,(make-mu4e-context
           :name "c me@home.com"
           :enter-func (lambda () (mu4e-message "Enter me@home.com context"))
           :leave-func (lambda () (mu4e-message "Leave me@home.com context"))
           ;; we match based on the contact-fields of the message (that we are replying to)
           ;; https://www.djcbsoftware.nl/code/mu/mu4e/What-are-contexts.html#What-are-contexts
           :match-func (lambda (msg)
                         (when msg 
                           (mu4e-message-contact-field-matches msg 
                                                               :to "me@home.com")))
           :vars '( ( user-mail-address      . "me@home.com"  )
                    ( user-full-name         . "Charl P. Botha" )
                    ( smtpmail-smtp-server   . "smtp.fastmail.com" )
                    ( mu4e-compose-signature .
                                             (concat
                                              "dr. charl p. botha\n"
                                              "http://charlbotha.com/\n"))))

         ,(make-mu4e-context
           :name "s me@org.org"
           :enter-func (lambda () (mu4e-message "Enter me@org.org context"))
           ;; no leave-func
           ;; we match based on the contact-fields of the message
           :match-func (lambda (msg)
                         (when msg 
                           (mu4e-message-contact-field-matches msg 
                                                               :to "me@org.org")))
           :vars '( ( user-mail-address       . "me@org.org" )
                    ( user-full-name          . "Charl P. Botha" )
                    ( smtpmail-smtp-server    . "smtp.gmail.com" )
                    ( mu4e-compose-signature  .
                                              (concat
                                               "Dr. Charl P. Botha\n"
                                               "Science Officer :: Stone Three Venture Technology\n"
                                               "https://www.stonethree.com\n"))))         

         ))

;; start with the first (default) context; 
(setq mu4e-context-policy 'pick-first)

;; compose with the current context if no context matches;
(setq mu4e-compose-context-policy nil)

;; these are the standard mu4e search bookmarks
;; I've only added the fourth one to pull up flagged emails in my inbox
;; I sometimes use this to shortlist emails I need to get around to ASAP
(setq mu4e-bookmarks
  `( ,(make-mu4e-bookmark
       :name  "Unread messages"
       :query "flag:unread AND NOT flag:trashed"
       :key ?u)
     ,(make-mu4e-bookmark
       :name "Today's messages"
       :query "date:today..now"
       :key ?t)
     ,(make-mu4e-bookmark
       :name "Last 7 days"
       :query "date:7d..now"
       :key ?w)
     ,(make-mu4e-bookmark
       :name "Flagged in INBOX"
       :query "maildir:\"/INBOX\" and flag:flagged"
       :key ?f)))

Conclusion

This is currently running smoothly on two of my Linux workstations as well as my early 2015 retina MacBook Pro.

The general setup procedure is to install offlineimap using pip2 install --user offlineimap (the offlineimap configuration file hasn’t changed much since the previous post), run offlineimap for the first time, then build and install mu4e from its tarball, and finally to run mu index to initialise the index.

After this, your mail, and plain-text nirvana, is a simple M-x mu4e away.

Sending emails with math and source code

Org mode is great for authoring rich documents with syntax highlighted source code, LaTeX math and images. It even supports evaluating live snippets of code embedded in the text. It does all of this whilst remaining a plain text format.

Imagine how useful it would be to author programming-related or technical emails using this functionality?

Imagine no more! org-mime, part of the org mode contrib, does this for a number of emacs-based mail clients. However, our preference is for mu4e, which is not part of that list.

Fortunately, it seems mu4e has orgmode support integrated. To get this working, configure the following in your ~/.emacs.d/init.el:

;; configure orgmode support in mu4e
(require 'org-mu4e)
;; when mail is sent, automatically convert org body to HTML
(setq org-mu4e-convert-to-html t)

When composing a new email, switch on the special mu4e / orgmode mode with M-x org~mu4e-mime-switch-headers-or-body (it will automatically switch between compose mode and orgmode depending on whether your cursor is on the headers or in the body).

You should add the following options to your org mode body. The first configures LaTeX (math) to be converted using imagemagick (instead of MathJax or dvipng), and the second removes the table of contents.

#+OPTIONS: tex:imagemagick
#+OPTIONS: toc:0

When you want to send the email, move to the headers, and then do C-c C-c. Because org-mu4e-convert-to-html is set to true, the orgmode body will automatically be converted to HTML before being sent.

Until my pull request is merged in, you will have to make the following change to function org~mu4e-mime-convert-to-html() in org-mu4e.el:

(insert (org~mu4e-mime-multipart
         body html (mapconcat 'identity html-images "\n")))

should be:

(insert (org~mu4e-mime-multipart
         raw-body html (mapconcat 'identity html-images "\n")))

In other words, body becomes raw-body, else your outgoing mails will have HTML in their text parts, instead of plain text.

In that same function, you might also want to change:

(org-export-preserve-breaks t)

to

(org-export-preserve-breaks nil)

So that the HTML email does not get hard linebreaks.

Following are examples of some of the elements you might include in such a rich text email. They should appear in your rich text capable email client as follows:

org-mu4e-example.png

Here are the originals:

A section with some math

We can easily include math using LaTeX:

\[ C(p,t) = \lbrace x_0(p,t), x_1(p,t), \cdots, x_{N-1}(p,t) \rbrace\tag{1} \]

Some syntax highlighted C++

Syntax highlighted source code, for all of the languages supported by emacs (a very long list):

int main() {
    // Recently discovered trick, local functions in C++!
    struct Kludge {
        static int doSomething(int x) {
            return x * 2;
        }
    };

    for (int i=0; i < 10; i++) {
        cout << Kludge::doSomething(i) << endl;
    }

    return 0;
}

Configuring Emacs mu4e with nullmailer, offlineimap and multiple identities

mu4e is a mail user agent for your Emacs. After leaving GMail a few months ago, this is the first MUA that I am loving even more. The major reasons for this are the faster than GMail real-time search (e.g. press s, then from:buddy flag:attach design review and watch it search my 68 thousand email archive in a fraction of a second), its Emacs foundation (the more I use it, the better I get at customizing it) and the observation that having my email in an uncluttered, by default text-only interface, somehow helps me to maintain the feeling of control.

mu4e_screenie.png
A screenshot of my mu4e, showing the results of the search query “flag:list debian”.

This post explains briefly how to setup nullmailer (for sending mail really quickly via your SMTP server), offlineimap (for keeping your local IMAP folders in sync with your IMAP server) and finally mu4e.

nullmailer

mu4e can send mail by connecting directly to your SMTP server, and there’s a commented-out section in the configuration below that shows you how to do this. However, this can sometimes be slow, so I opted to install nullmailer on my laptops. nullmailer is a sendmail lookalike with its own queue, so that mu4e can hand-off your outgoing mail and get back to you immediately whilst nullmailer takes care of the sending in the background.

To support encrypted SMTP (highly recommended), you need at least version 1.10 of nullmailer. On Ubuntu 12.04 I made use of this backport. On Ubuntu 14.04, the nullmailer is up to date and you can simply apt-get install nullmailer.

In /etc/nullmailer/remotes, readable only to its owner, I have the following:

mymailserver.com smtp --port=587 --starttls --user=myusername --pass=mypassword

offlineimap

I’m using offlineimap 6.5.5, built from source.

My ~/.offlineimaprc looks like this:

[general]
accounts = myaccountname
maxsyncaccounts = 3

[Account myaccountname]
localrepository = Local
remoterepository = Remote
# speeds up syncing!
status_backend = sqlite
# -1: always do quick updates
#  0: never do quick updates
quick = -1

[Repository Local]
type = Maildir
localfolders = ~/Maildir

[Repository Remote]
type = IMAP
remotehost = mymailserver.com
remoteuser = myusername
remotepass = mypassword
ssl = yes
# new comodo fingerprint (SHA1)
cert_fingerprint = 8f1e5dcaf69941a8be9fd04c7f4633895f4d2ead
# use at least two threads to sync mail (speeds up)
maxconnections = 2
realdelete = yes

This is for a straight-forward dovecot IMAP server (I manage my own). Please use Google for GMail setups.

mu4e

Install mu and mu4e from source. On Ubuntu, it goes like this:

sudo apt-get install libgmime-2.6-dev libxapian-dev
sudo apt-get install guile-2.0-dev html2text xdg-utils
git clone https://github.com/djcb/mu.git
cd mu
autoreconf -i && ./configure && make
sudo make install

Study the following mu4e configuration, then modify and add to your init.el (or similar).

This configuration (mine) supports multiple identities that can be activated by either pressing a shortcut key in the main screen or headers, or when you’re replying to an email that was sent to an email that’s associated with one of your identities. Each identity is configured by a function that you define. Here that includes the from address and the signature.

;; http://xahlee.blogspot.com/2010/09/elisp-read-file-content-in-one-shot.html
;; we'll use this to read your different signatures from files
(defun get-string-from-file (filePath)
  "Return FILEPATH's file content."
  (with-temp-buffer
    (insert-file-contents filePath)
    (buffer-string)))

;; this is where the install procedure above puts your mu4e
(add-to-list 'load-path "/usr/local/share/emacs/site-lisp/mu4e")

(require 'mu4e)

;; path to our Maildir directory
(setq mu4e-maildir "/home/cpbotha/Maildir")

;; should mu4e use fancy utf characters? NO. they're ugly.
;;(setq mu4e-use-fancy-chars 't)

;; give me ISO(ish) format date-time stamps in the header list
(setq  mu4e-headers-date-format "%Y-%m-%d %H:%M")

;; customize the reply-quote-string
;; M-x find-function RET message-citation-line-format for docs
(setq message-citation-line-format "%N @ %Y-%m-%d %H:%M %Z:\n")
(setq message-citation-line-function 'message-insert-formatted-citation-line)

;; show full addresses in view message (instead of just names)
;; toggle per name with M-RET
(setq mu4e-view-show-addresses 't)

;; without this, "symbol's value as variable is void: mml2014-use" when signing
;; then found http://www.gnu.org/software/emacs/manual/html_node/gnus/Security.html
;; so set to epg and all was good!
;; to sign a mail: M-x mml-secure-sign-pgpmime
(setq mml2015-use 'epg)

;; the next are relative to `mu4e-maildir'
;; instead of strings, they can be functions too, see
;; their docstring or the chapter 'Dynamic folders'
(setq mu4e-sent-folder   "/Archives.2014"
      mu4e-drafts-folder "/Drafts"
      mu4e-trash-folder  "/Trash")

;; the maildirs you use frequently; access them with 'j' ('jump')
(setq   mu4e-maildir-shortcuts
    '(("/Archives.2014"     . ?a)
      ("/INBOX"       . ?i)
      ("/Inbox-WORK"        . ?w)
      ("/Archives.2014"        . ?s)))

;; list of my email addresses.
(setq mu4e-user-mail-address-list '("info@charlbotha.com"
                                    "cpbotha@cpbotha.net"
                                    "cpbotha@vxlabs.com"
                                    "cpbotha@medvis.org"
                                    "charl@treparel.com"
                                    "charl.botha@treparel.com"))


;; when you want to use some external command for html->text
;; conversion, e.g. the 'html2text' program
;; (cpbotha: html2text sees to work better than the built-in one)
(setq mu4e-html2text-command "html2text")

;; mu4e-action-view-in-browser is built into mu4e
;; by adding it to these lists of custom actions
;; it can be invoked by first pressing a, then selecting
(add-to-list 'mu4e-headers-actions
             '("in browser" . mu4e-action-view-in-browser) t)
(add-to-list 'mu4e-view-actions
             '("in browser" . mu4e-action-view-in-browser) t)


;; the headers to show in the headers list -- a pair of a field
;; and its width, with `nil' meaning 'unlimited'
;; (better only use that for the last field.
;; These are the defaults:
(setq mu4e-headers-fields
    '( (:date          .  25)
       (:flags         .   6)
       (:from          .  22)
       (:subject       .  nil)))

;; program to get mail; alternatives are 'fetchmail', 'getmail'
;; isync or your own shellscript. called when 'U' is pressed in
;; main view.

;; If you get your mail without an explicit command,
;; use "true" for the command (this is the default)
(setq mu4e-get-mail-command "offlineimap")

;; setup default identity here:
;; general emacs mail settings; used when composing e-mail
;; the non-mu4e-* stuff is inherited from emacs/message-mode
(setq user-mail-address "info@charlbotha.com"
      user-full-name  "Charl P. Botha")

;; set this to nil so signature is not included by default
;; you can include in message with C-c C-w
(setq mu4e-compose-signature-auto-include 't)
(setq mu4e-compose-signature (with-temp-buffer
                               (insert-file-contents "~/.signature.personal")
                               (buffer-string)))
;; message-signature-file NOT used by mu4e
(setq message-signature-file "~/.signature.personal")

;; many recipes online use an alist with the different email identities
;; I like to use these functions, because then I have more flexibility
(defun cpb-mu4e-personal()
  (interactive)
  (message "personal mail account")
  (setq  user-mail-address "info@charlbotha.com"
         mu4e-compose-signature (get-string-from-file "~/.signature.personal"))
  )

(defun cpb-mu4e-vxlabs()
  (interactive)
  (message "vxlabs mail account")
  (setq  user-mail-address "cpbotha@vxlabs.com"
         mu4e-compose-signature (get-string-from-file "~/.signature.vxlabs"))
)

(defun cpb-mu4e-trep()
  (interactive)
  (message "treparel mail account")
  (setq  user-mail-address "charl.botha@treparel.com"
         mu4e-compose-signature (get-string-from-file "~/.signature.treparel"))
  )

(defun cpb-mu4e-medvis()
  (interactive)
  (message "medvis mail account")
  (setq  user-mail-address "cpbotha@medvis.org"
         mu4e-compose-signature (get-string-from-file "~/.signature.medvis"))
  )

;; quickly change account. got his idea from:
;; https://github.com/skybert/my-little-friends/blob/master/emacs/.emacs.d/tkj-mu4e.el
(define-key mu4e-main-mode-map (kbd "<f1>") 'cpb-mu4e-personal)
(define-key mu4e-main-mode-map (kbd "<f2>") 'cpb-mu4e-vxlabs)
(define-key mu4e-main-mode-map (kbd "<f4>") 'cpb-mu4e-trep)
(define-key mu4e-main-mode-map (kbd "<f6>") 'cpb-mu4e-medvis)
(define-key mu4e-headers-mode-map (kbd "<f1>") 'cpb-mu4e-personal)
(define-key mu4e-headers-mode-map (kbd "<f2>") 'cpb-mu4e-vxlabs)
(define-key mu4e-headers-mode-map (kbd "<f4>") 'cpb-mu4e-trep)
(define-key mu4e-headers-mode-map (kbd "<f6>") 'cpb-mu4e-medvis)


;; for sendmail read this http://www.gnus.org/manual/message_36.html
;; am using nullmailer, so my mail sending just became STUPID fast
(setq message-send-mail-function 'message-send-mail-with-sendmail)

;; smtp mail setting - if you DON'T want to use nullmailer, instead
;; connecting to your smtp server and waiting...
;; (setq
;;    message-send-mail-function 'smtpmail-send-it
;;    smtpmail-stream-type 'starttls
;;    smtpmail-default-smtp-server "mymailserver.com"
;;    smtpmail-smtp-server "mymailserver.com"
;;    smtpmail-smtp-service 587

;;    ;; if you need offline mode, set these -- and create the queue dir
;;    ;; with 'mu mkdir', i.e.. mu mkdir /home/user/Maildir/queue
;;    smtpmail-queue-mail  nil
;;    smtpmail-queue-dir  "/home/user/Maildir/queue/cur")

;; don't keep message buffers around
(setq message-kill-buffer-on-exit t)
;; attachments go here
(setq mu4e-attachment-dir  "~/Downloads")

;; when you reply to a message, use the identity that the mail was sent to
;; the cpbotha variation -- function that checks to, cc and bcc fields
(defun cpb-mu4e-is-message-to (msg rx)
  "Check if to, cc or bcc field in MSG has any address in RX."
  (or (mu4e-message-contact-field-matches msg :to rx)
      (mu4e-message-contact-field-matches msg :cc rx)
      (mu4e-message-contact-field-matches msg :bcc rx)))

;; we only do something if we recognize something (i.e. no stupid default)
(add-hook 'mu4e-compose-pre-hook
          (defun my-set-from-address ()
            "Set current identity based on to, cc, bcc of original."
            (let ((msg mu4e-compose-parent-message)) ;; msg is shorter...
              (if msg
                  (cond
                   ((cpb-mu4e-is-message-to msg (list "cpbotha@cpbotha.net"
                                                      "info@charlbotha.com"))
                    (cpb-mu4e-personal))
                   ((cpb-mu4e-is-message-to msg (list "cpbotha@vxlabs.com"
                                                      "charl@stonethree.com"))
                    (cpb-mu4e-vxlabs))
                   ((cpb-mu4e-is-message-to msg (list "charl.botha@treparel.com"
                                                      "charl@treparel.com"))
                    (cpb-mu4e-trep))
                   ((cpb-mu4e-is-message-to msg "cpbotha@medvis.org")
                    (cpb-mu4e-medvis)))))))

;; convenience function for starting the whole mu4e in its own frame
;; posted by the author of mu4e on the mailing list
(defun mu4e-in-new-frame ()
  "Start mu4e in new frame."
  (interactive)
  (select-frame (make-frame))
  (mu4e))

Getting started

After everything has been configured, download and index your email by doing:

# download all imap email, this can take while
offlineimap
# index all of the mail -- initial index takes a while, then fast!
mu index --maildir=~/Maildir

In the future, press U on the mu4e main screen, or C-S-u on the headers or view screen to retrieve and index new mail from within Emacs.

Conclusion

I hope that you find this howto and configuration useful. Please let me know in the comments if you got it working, or if you have any questions.

Enjoy your mail again with mu4e!