1 OFFLINEIMAP(1) OfflineIMAP manual OFFLINEIMAP(1)
6 OfflineIMAP - Powerful IMAP/Maildir synchronization and reader support
9 offlineimap [ -1 ] [ -P profiledir ] [ -a accountlist ] [ -c configfile
10 ] [ -d debugtype[,debugtype...] ] [ -o ] [ -u interface ]
12 offlineimap -h | --help
15 OfflineIMAP is a tool to simplify your e-mail reading. With
16 OfflineIMAP, you can read the same mailbox from multiple computers.
17 You get a current copy of your messages on each computer, and changes
18 you make one place will be visible on all other systems. For instance,
19 you can delete a message on your home computer, and it will appear
20 deleted on your work computer as well. OfflineIMAP is also useful if
21 you want to use a mail reader that does not have IMAP support, has poor
22 IMAP support, or does not provide disconnected operation.
24 OfflineIMAP is FAST; it synchronizes my two accounts with over 50 fold-
25 ers in 3 seconds. Other similar tools might take over a minute, and
26 achieve a less-reliable result. Some mail readers can take over 10
27 minutes to do the same thing, and some don't even support it at all.
28 Unlike other mail tools, OfflineIMAP features a multi-threaded synchro-
29 nization algorithm that can dramatically speed up performance in many
30 situations by synchronizing several different things simultaneously.
32 OfflineIMAP is FLEXIBLE; you can customize which folders are synced via
33 regular expressions, lists, or Python expressions; a versatile and com-
34 prehensive configuration file is used to control behavior; two user
35 interfaces are built-in; fine-tuning of synchronization performance is
36 possible; internal or external automation is supported; SSL and PREAUTH
37 tunnels are both supported; offline (or "unplugged") reading is sup-
38 ported; and esoteric IMAP features are supported to ensure compatibil-
39 ity with the widest variety of IMAP servers.
41 OfflineIMAP is SAFE; it uses an algorithm designed to prevent mail loss
42 at all costs. Because of the design of this algorithm, even program-
43 ming errors should not result in loss of mail. I am so confident in
44 the algorithm that I use my own personal and work accounts for testing
45 of OfflineIMAP pre-release, development, and beta releases.
48 OfflineIMAP operates by maintaining a hierarchy of mail folders in
49 Maildir format locally. Your own mail reader will read mail from this
50 tree, and need never know that the mail comes from IMAP. OfflineIMAP
51 will detect changes to the mail folders on your IMAP server and your
52 own computer and bi-directionally synchronize them, copying, marking,
53 and deleting messages as necessary.
56 If you are reading this document via the "man" command, it is likely
57 that you have no installation tasks to perform; your system administra-
58 tor has already installed it. If you need to install it yourself, you
59 have three options: a system-wide installation with Debian, system-wide
60 installation with other systems, and a single-user installation. You
61 can download the latest version of OfflineIMAP from
62 http://quux.org/devel/offlineimap/.
65 In order to use OfflineIMAP, you need to have these conditions satis-
68 o Your mail server must support IMAP. Most Internet Service
69 Providers and corporate networks do, and most operating systems
70 have an IMAP implementation readily available.
72 o You must have Python version 2.2.1 or above installed. If you
73 are running on Debian GNU/Linux, this requirement will automati-
74 cally be taken care of for you. If you do not have Python
75 already, check with your system administrator or operating sys-
76 tem vendor; or, download it from http://www.python.org/. If you
77 intend to use the Tk interface, you must have Tkinter (python-
78 tk) installed. If you intend to use the SSL interface, your
79 Python must have been built with SSL support.
81 o Have a mail reader that supports the Maildir mailbox format.
82 Most modern mail readers have this support built-in, so you can
83 choose from a wide variety of mail servers. This format is also
84 known as the "qmail" format, so any mail reader compatible with
85 it will work with OfflineIMAP.
87 DEBIAN SYSTEM-WIDE INSTALLATION
88 If you are tracking Debian unstable, you may install OfflineIMAP by
89 simply running the following command as root:
91 apt-get install offlineimap
93 If you are not tracking Debian unstable, download the Debian .deb pack-
94 age from the OfflineIMAP website and then run dpkg -i to install the
95 downloaded package. Then, go to CONFIGURATION below. You will type
96 offlineimap to invoke the program.
98 OTHER SYSTEM-WIDE INSTALLATION
99 Download the tar.gz version of the package from the website. Then run
100 these commands, making sure that you are the "root" user first:
102 tar -zxvf offlineimap_x.y.z.tar.gz
104 python2.2 setup.py install
106 Some systems will need to use python instead of python2.2. Next, pro-
107 ceed to configuration. You will type offlineimap to invoke the pro-
110 SINGLE-ACCOUNT INSTALLATION
111 Download the tar.gz version of the package from the website. Then run
114 tar -zxvf offlineimap-x.y.z.tar.gz
117 When you want to run OfflineIMAP, you will issue the cd command as
118 above and then type ./offlineimap.py; there is no installation step
122 OfflineIMAP is regulated by a configuration file that is normally
123 stored in ~/.offlineimaprc. OfflineIMAP ships with a file named
124 offlineimap.conf that you should copy to that location and then edit.
125 This file is vital to proper operation of the system; it sets every-
126 thing you need to run OfflineIMAP. Full documentation for the configu-
127 ration file is included within the sample file.
129 OfflineIMAP also ships a file named offlineimap.conf.minimal that you
130 can also try. It's useful if you want to get started with the most
131 basic feature set, and you can read about other features later with
135 Most configuration is done via the configuration file. Nevertheless,
136 there are a few options that you may set for OfflineIMAP.
138 -1 Disable all multithreading operations and use solely a single-
139 thread sync. This effectively sets the maxsyncaccounts and all
140 maxconnections configuration file variables to 1.
143 Sets OfflineIMAP into profile mode. The program will create
144 profiledir (it must not already exist). As it runs, Python pro-
145 filing information about each thread is logged into profiledir.
146 Please note: This option is present for debugging and optimiza-
147 tion only, and should NOT be used unless you have a specific
148 reason to do so. It will significantly slow program perfor-
149 mance, may reduce reliability, and can generate huge amounts of
150 data. You must use the -1 option when you use -P.
154 Overrides the accounts section in the config file. Lets you
155 specify a particular account or set of accounts to sync without
156 having to edit the config file. You might use this to exclude
157 certain accounts, or to sync some accounts that you normally
161 Specifies a configuration file to use in lieu of the default,
164 -d debugtype[,debugtype...]
165 Enables debugging for OfflineIMAP. This is useful if you are
166 trying to track down a malfunction or figure out what is going
167 on under the hood. I suggest that you use this with -1 in order
168 to make the results more sensible.
170 -d now requires one or more debugtypes, separated by commas.
171 These define what exactly will be debugged, and so far include
172 two options: imap and maildir. The imap option will enable IMAP
173 protocol stream and parsing debugging. Note that the output may
174 contain passwords, so take care to remove that from the debug-
175 ging output before sending it to anyone else. The maildir
176 option will enable debugging for certain Maildir operations.
178 -o Run only once, ignoring any autorefresh setting in the config
182 Show summary of options.
185 Specifies an alternative user interface module to use. This
186 overrides the default specified in the configuration file. The
187 UI specified with -u will be forced to be used, even if its
188 isuable() method states that it cannot be. Use this option with
189 care. The pre-defined options are listed in the USER INTERFACES
193 OfflineIMAP has a pluggable user interface system that lets you choose
194 how the program communicates information to you. There are two graphi-
195 cal interfaces, two terminal interfaces, and two noninteractive inter-
196 faces suitable for scripting or logging purposes. The ui option in the
197 configuration file specifies the user interface preferences. The -u
198 command-line option can override the configuration file. The available
199 values for the configuration file or command-line are describef in this
202 Tk.Blinkenlights or Curses.Blinkenlights
203 This is an interface designed to be sleek, fun to watch, and informa-
204 tive of the overall picture of what OfflineIMAP is doing. I consider
205 it to be the best general-purpose interface in OfflineIMAP.
206 Tk.Blinkenlights contains, by default, a small window with a row of
207 LEDs and a row of command buttons. The total size of the window is
208 very small, so it uses little desktop space, yet it is quite func-
209 tional. There is also an optional, toggable, log that shows more
210 detail about what is happening and is color-coded to match the color of
213 Curses.Blinkenlights is an interface very similar to Tk.Blinkenlights,
214 but is designed to be run in a console window (an xterm, Linux virtual
215 terminal, etc.) Since it doesn't have access to graphics, it isn't
216 quite as pretty, but it still gets the job done.
218 Tk.Blinkenlights is the only user interface that has configurable
219 parameters; see the example offlineimap.conf for more details.
221 Each light in the Tk.Blinkenlights or Curses.Blinkenlights interface
222 represents a thread of execution -- that is, a particular task that
223 OfflineIMAP is performing right now. The color indicates what task the
224 particular thread is performing, and are as follows:
226 Black indicates that this light's thread has terminated; it will light
227 up again later when new threads start up. So, black indicates
231 is the color of the main program's thread, which basically does
232 nothing but monitor the others. It might remind you of HAL 9000
235 Gray indicates that the thread is establishing a new connection to
238 Purple is the color of an account synchronization thread that is moni-
239 toring the progress of the folders in that account (not generat-
242 Cyan indicates that the thread is syncing a folder.
244 Green means that a folder's message list is being loaded.
246 Blue is the color of a message synchronization controller thread.
248 Orange indicates that an actual message is being copied.
251 indicates that a message is being deleted.
253 Yellow (bright orange) indicates that message flags are being added.
255 Pink (bright red) indicates that message flags are being removed.
258 corresponds to the countdown timer that runs between synchro-
261 The name of this interface derives from a bit of computer science his-
262 tory. Eric Raymond's Jargon File defines blinkenlights, in part, as:
264 Front-panel diagnostic lights on a computer, esp. a dinosaur.
265 Now that dinosaurs are rare, this term usually refers to status
266 lights on a modem, network hub, or the like.
268 This term derives from the last word of the famous blackletter-
269 Gothic sign in mangled pseudo-German that once graced about half
270 the computer rooms in the English-speaking world. One version
271 ran in its entirety as follows:
273 ACHTUNG! ALLES LOOKENSPEEPERS!
275 Das computermachine ist nicht fuer gefingerpoken und mitten-
276 grabben. Ist easy schnappen der springenwerk, blowenfusen und
277 poppencorken mit spitzensparken. Ist nicht fuer gewerken bei
278 das dumpkopfen. Das rubbernecken sichtseeren keepen das cotten-
279 pickenen hans in das pockets muss; relaxen und watchen das
283 This interface (formerly known as Tk.TkUI) is a graphical interface
284 that presents a variable-sized window. In the window, each currently-
285 executing thread has a section where its name and current status are
286 displayed. This interface is best suited to people running on slower
287 connections, as you get a lot of detail, but for fast connections, the
288 detail may go by too quickly to be useful. People with fast connec-
289 tions may wish to use Tk.Blinkenlights instead.
292 This interface is the default for people running in terminals. It
293 prints out basic status messages and is generally friendly to use on a
297 This interface is designed for situations where OfflineIMAP will be run
298 non-attended and the status of its execution will be logged. You might
299 use it, for instance, to have the system run automatically and e-mail
300 you the results of the synchronization. This user interface is not
301 capable of reading a password from the keyboard; account passwords must
302 be specified using one of the configuration file options.
305 This interface is designed for non-attended running in situations where
306 normal status messages are not desired. It will output nothing except
307 errors and serious warnings. Like Noninteractive.Basic, this user
308 interface is not capable of reading a password from the keyboard;
309 account passwords must be specified using one of the configuration file
313 Here is an example configuration for a particularly complex situation;
314 more examples will be added later.
316 MULTIPLE ACCOUNTS WITH MUTT
317 This example shows you how to set up OfflineIMAP to synchronize multi-
318 ple accounts with the mutt mail reader.
320 Start by creating a directory to hold your folders:
323 In your ~/.offlineimaprc, specify this:
324 accounts = Personal, Work
326 Make sure that you have both a [Personal] and a [Work] section, with
327 different localfolder pathnames and enable [mbnames].
329 In each account section, do something like this:
330 localfolders = ~/Mail/Personal
332 Add these lines to your ~/.muttrc:
333 source ~/path-to-mbnames-muttrc-mailboxes
334 folder-hook Personal set from="youremail@personal.com"
335 folder-hook Work set from="youremail@work.com"
336 set mbox_type=Maildir
337 set folder=$HOME/Mail
338 set spoolfile=+Personal/INBOX
342 UW-IMAPD AND REFERENCES
343 Some users with a UW-IMAPD server need to use OfflineIMAP's "reference"
344 feature to get at their mailboxes, specifying a reference of "~/Mail"
345 or "#mh/" depending on the configuration. The below configuration from
346 docwhat@gerf.org shows using a reference of Mail, a nametrans that
347 strips the leading Mail/ off incoming folder names, and a folderfilter
348 that limits the folders synced to just three.
351 localfolders = ~/Mail
352 remotehost = gerf.org
356 # Trims off the preceeding Mail on all the folder names.
357 nametrans = lambda foldername: \
358 re.sub('^Mail/', '', foldername)
359 # Yeah, you have to mention the Mail dir, even though it
360 # would seem intuitive that reference would trim it.
361 folderfilter = lambda foldername: foldername in [
363 'Mail/list/zaurus-general',
364 'Mail/list/zaurus-dev',
367 holdconnectionopen = no
369 PYTHONFILE CONFIGURATION FILE OPTION
370 You can have OfflineIMAP load up a Python file before evaluating the
371 configuration file options that are Python expressions. This example
372 is based on one supplied by Tommi Virtanen for this feature.
374 In ~/.offlineimap.rc, he adds these options:
377 pythonfile=~/.offlineimap.py
381 Then, the ~/.offlineimap.py file will contain:
383 prioritized = ['INBOX', 'personal', 'announce', 'list']
386 for prefix in prioritized:
387 if x.startswith(prefix):
389 elif y.startswith(prefix):
396 dir(os.path.expanduser('~/data/mail/tv@hq.yok.utu.fi'))
400 This code snippet illustrates how the foldersort option can be cus-
401 tomized with a Python function from the pythonfile to always synchro-
402 nize certain folders first.
405 If you get one of some frequently-encountered or confusing errors,
406 please check this section.
408 UID validity problem for folder
409 IMAP servers use a unique ID (UID) to refer to a specific message.
410 This number is guaranteed to be unique to a particular message FOREVER.
411 No other message in the same folder will ever get the same UID. UIDs
412 are an integral part of OfflineIMAP's synchronization scheme; they are
413 used to match up messages on your computer to messages on the server.
415 Sometimes, the UIDs on the server might get reset. Usually this will
416 happen if you delete and then recreate a folder. When you create a
417 folder, the server will often start the UID back from 1. But
418 OfflineIMAP might still have the UIDs from the previous folder by the
419 same name stored. OfflineIMAP will detect this condition and skip the
420 folder. This is GOOD, because it prevents data loss.
422 You can fix it by removing your local folder and cache data. For
423 instance, if your folders are under ~/Folders and the folder with the
424 problem is INBOX, you'd type this:
426 rm -r ~/Folders/INBOX
427 rm ~/.offlineimap/AccountName/INBOX
429 (replacing AccountName with the account name as specified in
432 Next time you run OfflineIMAP, it will re-download the folder with the
433 new UIDs. Note that the procedure specified above will lose any local
434 changes made to the folder.
436 Some IMAP servers are broken and do not support UIDs properly. If you
437 continue to get this error for all your folders even after performing
438 the above procedure, it is likely that your IMAP server falls into this
439 category. OfflineIMAP is incompatible with such servers. Using
440 OfflineIMAP with them will not destroy any mail, but at the same time,
441 it will not actually synchronize it either. (OfflineIMAP will detect
442 this condition and abort prior to synchronization)
445 OTHER FREQUENTLY ASKED QUESTIONS
446 There are some other FAQs that might not fit into another section of
447 this document, and they are enumerated here.
449 What platforms does OfflineIMAP run on?
450 It should run on most platforms supported by Python, which are
453 I'm using Mutt. Other IMAP sync programs require me to use set
454 maildir_trash=yes . Do I need to do that with OfflineIMAP?
455 No. OfflineIMAP is smart enough to figure out message deletion
456 without this extra crutch. You'll get the best results if you
457 don't use this setting, in fact.
459 How do I specify the names of my folders?
460 You do not need to. OfflineIMAP is smart enough to automati-
461 cally figure out what folders are present on the IMAP server and
462 synchronize them. You can use the folderfilter and foldertrans
463 configuration file options to request certain folders and rename
464 them as they come in if you like.
466 How can I prevent certain folders from being synced?
467 Use the folderfilter option in the configuration file.
469 How can I add or delete a folder?
470 OfflineIMAP does not currently provide this feature, but if you
471 create a new folder on the IMAP server, it will be created
472 locally automatically.
474 Are there any other warnings that I should be aware of?
475 Yes; see the NOTES section below.
477 What is the mailbox name recorder (mbnames) for?
478 The Mutt mail reader is not capable of automatically determining
479 the names of your mailboxes. OfflineIMAP can help it (or many
480 other) programs out be writing these names out in a format you
481 specify. See the example offlineimap.conf file for details.
483 Can I synchronize multiple accounts with OfflineIMAP?
484 Sure. Just name them all in the accounts line in the general
485 section of the config file, and add a per-account section for
488 Does OfflineIMAP support POP?
489 No. POP is not robust enough to do a completely reliable multi-
490 machine synchronization like OfflineIMAP can do. OfflineIMAP
493 Do you support mailbox formats other than Maildir?
494 Not at present. There is no technical reason not to; just no
495 demand yet. Maildir is a superior format anyway.
497 [technical] Why are your Maildir message filenames so huge?
498 OfflineIMAP has two relevant principles: 1) never modifying your
499 messages in any way and 2) ensuring 100% reliable synchroniza-
500 tions. In order to do a reliable sync, OfflineIMAP must have a
501 way to uniquely identify each e-mail. Three pieces of informa-
502 tion are required to do this: your account name, the folder
503 name, and the message UID. The account name can be calculated
504 from the path in which your messages are. The folder name can
505 usually be as well, BUT some mail clients move messages between
506 folders by simply moving the file, leaving the name intact.
508 So, OfflineIMAP must store both a UID folder ID. The folder ID
509 is necessary so OfflineIMAP can detect a message moved to a dif-
510 ferent folder. OfflineIMAP stores the UID (U= number) and an
511 md5sum of the foldername (FMD5= number) to facilitate this.
513 What is the speed of OfflineIMAP's sync?
514 OfflineIMAP versions 2.0 and above contain a multithreaded sys-
515 tem. A good way to experiment is by setting maxsyncaccounts to
516 3 and maxconnections to 3 in each account clause.
518 This lets OfflineIMAP open up multiple connections simultane-
519 ously. That will let it process multiple folders and messages
520 at once. In most cases, this will increase performance of the
523 Don't set the number too high. If you do that, things might
524 actually slow down as your link gets saturated. Also, too many
525 connections can cause mail servers to have excessive load.
526 Administrators might take unkindly to this, and the server might
527 bog down. There are many variables in the optimal setting;
528 experimentation may help.
530 An informal benchmark yields these results for my setup:
532 10 minutes with MacOS X Mail.app "manual cache"
533 5 minutes with GNUS agent sync
534 20 seconds with OfflineIMAP 1.x
535 9 seconds with OfflineIMAP 2.x
536 3 seconds with OfflineIMAP 3.x "cold start"
537 2 seconds with OfflineIMAP 3.x "held connection"
540 o Internet Message Access Protocol version 4rev1 (IMAP 4rev1) as
543 o CRAM-MD5 as specified in RFC2195
545 o Maildir as specified in http://www.qmail.org/qmail-manual-
546 html/man5/maildir.html and http://cr.yp.to/proto/maildir.html.
548 o Standard Python 2.2.1 as implemented on POSIX-compliant systems.
551 DELETING LOCAL FOLDERS
552 OfflineIMAP does a two-way synchronization. That is, if you make a
553 change to the mail on the server, it will be propogated to your local
554 copy, and vise-versa. Some people might think that it would be wise to
555 just delete all their local mail folders periodically. If you do this
556 with OfflineIMAP, remember to also remove your local status cache
557 (~/.offlineimap by default). Otherwise, OfflineIMAP will take this as
558 an intentional deletion of many messages and will interpret your action
559 as requesting them to be deleted from the server as well. (If you
560 don't understand this, don't worry; you probably won't encounter this
563 COPYING MESSAGES BETWEEN FOLDERS
564 Normally, when you copy a message between folders or add a new message
565 to a folder locally, OfflineIMAP will just do the right thing. How-
566 ever, sometimes this can be tricky -- if your IMAP server does not pro-
567 vide the SEARCH command, or does not return something useful,
568 OfflineIMAP cannot determine the new UID of the message. So, in these
569 rare instances, OfflineIMAP will upload the message to the IMAP server
570 and delete it from your local folder. Then, on your next sync, the
571 message will be re-downloaded with the proper UID. OfflineIMAP makes
572 sure that the message was properly uploaded before deleting it, so
573 there should be no risk of data loss.
576 OfflineIMAP can work with Evolution. To do so, first configure your
577 OfflineIMAP account to have:
579 in its configuration. Then, configure Evolution with the "Maildir-for-
580 mat mail directories" server type. For the path, you will need to
581 specify the name of the top-level folder inside your OfflineIMAP stor-
582 age location. You're now set!
585 At this time, I believe that OfflineIMAP is not compatible with KMail.
586 KMail cannot work in any mode other than to move all messages out of
587 all folders immediately, which (besides being annoying and fundamen-
588 tally broken) is incompatible with OfflineIMAP.
591 There is an OfflineIMAP mailing list available.
593 To subscribe, send the text "Subscribe" in the subject of a mail to
594 offlineimap-request@complete.org. To post, send the message to
595 offlineimap@complete.org.
598 Reports of bugs should be sent via e-mail to the OfflineIMAP bug-track-
599 ing system (BTS) at offlineimap@bugs.complete.org or submitted on-line
600 using the Web interface at http://bugs.complete.org/. The Web site
601 also lists all current bugs, where you can check their status or con-
602 tribute to fixing them.
605 OfflineIMAP, and this manual, are Copyright (C) 2002, 2003 John
608 This program is free software; you can redistribute it and/or modify it
609 under the terms of the GNU General Public License as published by the
610 Free Software Foundation; either version 2 of the License, or (at your
611 option) any later version.
613 This program is distributed in the hope that it will be useful, but
614 WITHOUT ANY WARRANTY; without even the implied warranty of MER-
615 CHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
616 Public License for more details.
618 You should have received a copy of the GNU General Public License along
619 with this program; if not, write to:
621 Free Software Foundation, Inc.
624 Boston, MA 02111-1307
628 OfflineIMAP, its libraries, documentation, and all included files,
629 except where noted, was written by John Goerzen <jgoerzen@complete.org>
630 and copyright is held as stated in the COPYRIGHT section.
632 OfflineIMAP may be downloaded, and information found, from its homepage
633 via either Gopher or HTTP:
635 gopher://quux.org/1/devel/offlineimap
636 http://quux.org/devel/offlineimap
638 OfflineIMAP may also be downloaded using Subversion. Additionally, the
639 distributed tar.gz may be updated with a simple "svn update" command;
640 it is ready to go. For information on getting OfflineIMAP with Subver-
643 http://svn.complete.org/
650 John Goerzen July 12, 2002 OFFLINEIMAP(1)