Courier-Cygwin

Prerequisites

Courier-IMAP requires the installation of the Courier Authentication Library and the following cygwin tools and packages

• patch, tar, make, gcc
• crypt
• libgdbm-devel
• libtool
• inetutils
• cygrunsrv

Run the Cygwin setup program. Locate, select and install the required packages.

Building and Installing Courier Authentication Library

Installation of Courier Authentication Library using the module userdb to manage mail accounts. All other modules are disabled.

Download and untar courier-authlib in your favorite sandbox. Version used: 0.61.0

$ tar -xvjf courier-authlib-0.61.0.tar.bz2
$ cd courier-authlib-0.61.0

Apply the [{{#file:courier-authlib-0.61.0-cygwin.patch}} patch] before configuration to correct following issues:

1. incorrect usage of EXEEXT in makedat/Makefile
2. add libtool flag (-no-undefined) required for DLL creation ([1])
3. add missing dependencies
4. add PATH in start script to access new DLL (installed in a different directory)

--- courier-authlib-0.61.0/makedat/Makefile.in	2008-05-24 16:21:09.000000000 +0200
+++ courier-authlib-0.61.0-cygwin/makedat/Makefile.in	2008-10-21 16:02:38.709166700 +0200
@@ -182,7 +182,7 @@
 libexecdir = @libexecdir@
 localedir = @localedir@
 localstatedir = @localstatedir@
-makedatprog_target = @makedatprog_target@
+makedatprog_target = @makedatprog_target@$(EXEEXT)
 makedatprogpath = @makedatprogpath@
 mandir = @mandir@
 mkdir_p = @mkdir_p@
@@ -198,7 +198,7 @@
 target_alias = @target_alias@
 top_builddir = @top_builddir@
 top_srcdir = @top_srcdir@
-noinst_PROGRAMS = @makedatprog_target@
+noinst_PROGRAMS = @makedatprog_target@$(EXEEXT)
 makedatprog_SOURCES = makedatprog.c
 makedatprog_DEPENDENCIES = @dblibrary@
 makedatprog_LDADD = @dblibrary@
--- courier-authlib-0.61.0/Makefile.in	2008-07-12 21:41:08.000000000 +0200
+++ courier-authlib-0.61.0-cygwin/Makefile.in	2008-10-23 22:59:03.843750000 +0200
@@ -213,9 +213,10 @@
 LTCOMPILE = $(LIBTOOL) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) \
 	--mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) \
 	$(AM_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS)
-CCLD = $(CC)
+CCLD = $(CC) -no-undefined
+CCLDEXE = $(CC)
 LINK = $(LIBTOOL) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) \
-	--mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) $(AM_LDFLAGS) \
+	--mode=link $(CCLDEXE) $(AM_CFLAGS) $(CFLAGS) $(AM_LDFLAGS) \
 	$(LDFLAGS) -o $@
 SOURCES = $(libauthcustom_la_SOURCES) $(libauthldap_la_SOURCES) \
 	$(libauthmysql_la_SOURCES) $(libauthpam_la_SOURCES) \
@@ -452,9 +453,9 @@
 	README.authdebug.html
 
 DISTCLEANFILES = dbobj.config README_authlib.html
-commonlibdep = libcourierauthcommon.la
+commonlibdep = libcourierauthcommon.la libcourierauth.la
 commonldflags = -module -rpath $(pkglibdir) -export-symbols-regex 'courier_auth.*_init' -avoid-version
-commonlibadd = libcourierauthcommon.la
+commonlibadd = libcourierauthcommon.la libcourierauth.la
 libcourierauthcommon_t = @CRYPTLIBS@
 libcourierauthcommon_la_SOURCES = \
 	auth.h courierauth.h \
--- courier-authlib-0.61.0/authdaemond.in	2005-07-05 14:25:08.000000000 +0200
+++ courier-authlib-0.61.0-cygwin/authdaemond.in	2008-10-25 12:03:32.140625000 +0200
@@ -15,4 +15,10 @@
 set -a
 . @authdaemonrc@
 
+# Some shared libraries (DLL) are installed in @libdir@/bin
+# instead of @libdir@/@PACKAGE@
+# Setting LD_LIBRARY_PATH at runtime or LR_RUN_PATH at linktime doesn't
+# work in cygwin, only setting PATH works. 
+export PATH=$PATH:@libdir@/bin
+
 exec ${sbindir}/courierlogger -pid=@authdaemonvar@/pid $LOGGEROPTS -$1 @libexecdir@/courier-authlib/authdaemond

Install the patch

$ patch -Np1 < courier-authlib-0.61.0-cygwin.patch

Configure the package without most authentication modules, keeping only userdb. Replace mailuser by an existing user name (in your /etc/passwd file). I used my own username.

$ ./configure --disable-root-check --with-waitfunc=wait --without-authpam --without-authldap --without-authpwd \
--without-authshadow --without-authcustom --without-authpipe --without-authmysql --without-authpgsql --with-mailuser=mailuser \
--with-mailgroup=mkgroup-l-d

Take a long pause... and when ready build, check the result and install the libraries.

$ make
$ make check
$ make install

NOTES: list of related topics found during investigation for compilation

• authldap: configure: error: -lresolve is needed for res_query... http://www.cygwin.com/ml/cygwin/2005-04/msg00419.html

Building and Installing Courier-IMAP

Download and untar package courier-imap in your favorite sandbox. Version used: 4.4.1.20080920

$ tar -xvjf courier-imap-4.4.1.tar.bz2
$ cd courier-imap-4.4.1

Apply the [{{#file:courier-imap-4.4.1-cygwin.patch}} patch] before configuration to correct following issues:

1. incorrect usage of EXEEXT in makedat/Makefile
2. incorrect usage of EXTEXT in main Makefile
3. remove usage of /usr/lib/env in start/stop script (/usr/lib/env does not seem to work under Cygwin)

--- courier-imap-4.4.1.20080920/makedat/Makefile.in	2008-08-24 19:52:51.000000000 +0200
+++ courier-imap-4.4.1.20080920-cygwin/makedat/Makefile.in	2008-10-21 20:43:27.500000000 +0200
@@ -182,7 +182,7 @@
 libexecdir = @libexecdir@
 localedir = @localedir@
 localstatedir = @localstatedir@
-makedatprog_target = @makedatprog_target@
+makedatprog_target = @makedatprog_target@$(EXEEXT)
 makedatprogpath = @makedatprogpath@
 mandir = @mandir@
 mkdir_p = @mkdir_p@
@@ -198,7 +198,7 @@
 target_alias = @target_alias@
 top_builddir = @top_builddir@
 top_srcdir = @top_srcdir@
-noinst_PROGRAMS = @makedatprog_target@
+noinst_PROGRAMS = @makedatprog_target@$(EXEEXT)
 makedatprog_SOURCES = makedatprog.c
 makedatprog_DEPENDENCIES = @dblibrary@
 makedatprog_LDADD = @dblibrary@
--- courier-imap-4.4.1.20080920/Makefile.in	2008-09-20 14:48:46.000000000 +0200
+++ courier-imap-4.4.1.20080920-cygwin/Makefile.in	2008-10-21 20:59:45.156250000 +0200
@@ -247,9 +247,9 @@
 CLEANFILES = $(databin_SCRIPTS) $(man_MANS) $(sysconf_DATA) $(sbin_SCRIPTS)
 databindir = $(datadir)
 databin_SCRIPTS = mkimapdcert mkpop3dcert
-binPROGRAMS = imapd pop3d maildirmake maildiracl deliverquota maildirkw
-sbinPROGRAMS = imaplogin pop3login
-libexecPROGRAMS = makedatprog couriertcpd
+binPROGRAMS = imapd$(EXEEXT) pop3d$(EXEEXT) maildirmake$(EXEEXT) maildiracl$(EXEEXT) deliverquota$(EXEEXT) maildirkw$(EXEEXT)
+sbinPROGRAMS = imaplogin$(EXEEXT) pop3login$(EXEEXT)
+libexecPROGRAMS = makedatprog$(EXEEXT) couriertcpd$(EXEEXT)
 bin_PROGRAMS = @binPROGRAMS_exec@
 sbin_PROGRAMS = @sbinPROGRAMS_exec@
 libexec_PROGRAMS = @libexecPROGRAMS_exec@
--- courier-imap-4.4.1.20080920/imapd.rc.in	2008-05-04 15:12:47.000000000 +0200
+++ courier-imap-4.4.1.20080920-cygwin/imapd.rc.in	2008-10-26 10:17:51.359375000 +0100
@@ -22,6 +22,15 @@
 	exit 1
 fi
 
+# Location for some shared libraries (cygcourierauth*.dll) from authlib
+AUTHLIBDIR=/usr/local
+if test ! -f $AUTHLIBDIR/lib/bin/cygcourierauth.dll
+then
+	echo "$AUTHLIBDIR/lib/bin/cygcourierauth.dll not found."
+	exit 1
+fi
+export PATH=$PATH:$AUTHLIBDIR/lib/bin
+
 TLS_CACHEFILE=""
 . @sysconfdir@/imapd-ssl
 . @sysconfdir@/imapd
@@ -35,7 +44,7 @@
 
 	umask $IMAP_UMASK
 	@ULIMIT@ $IMAP_ULIMITD
-	@SETENV@ -i @SHELL@ -c " set -a ;
+	@SHELL@ -c " set -a ;
 			prefix=@prefix@ ;
 			exec_prefix=@exec_prefix@ ;
 			bindir=@bindir@ ;
--- courier-imap-4.4.1.20080920/pop3d.rc.in	2008-05-04 15:12:47.000000000 +0200
+++ courier-imap-4.4.1.20080920-cygwin/pop3d.rc.in	2008-10-26 10:18:02.187500000 +0100
@@ -22,12 +22,21 @@
 	exit 1
 fi
 
+# Location for some shared libraries (cygcourierauth*.dll) from authlib
+AUTHLIBDIR=/usr/local
+if test ! -f $AUTHLIBDIR/lib/bin/cygcourierauth.dll
+then
+	echo "$AUTHLIBDIR/lib/bin/cygcourierauth.dll not found."
+	exit 1
+fi
+export PATH=$PATH:$AUTHLIBDIR/lib/bin
+
 . @sysconfdir@/pop3d-ssl
 . @sysconfdir@/pop3d
 
 case $1 in
 start)
-	@SETENV@ -i @SHELL@ -c " set -a ;
+	@SHELL@ -c " set -a ;
 		prefix=@prefix@ ;
 		exec_prefix=@exec_prefix@ ;
 		bindir=@bindir@ ;

Install the patch

$ patch -Np1 < courier-imap-4.4.1-cygwin.patch

Configure the package

$ ./configure --disable-root-check --with-waitfunc=wait

Take a walk or some coffee... and when ready build the package and install it.

$ make
$ make install
$ make install-configure

Configuring Courier

Configuration Files and Scripts

If not present yet, add /usr/local/bin and /usr/local/sbin to your path (ideally in /etc/bash.bashrc):

PATH=/usr/local/bin:/usr/local/sbin:$PATH

Create configuration file for authdaemond (/usr/local/etc/authlib/authdaemonrc)

$ cp /usr/local/etc/authlib/authdaemonrc.dist /usr/local/etc/authlib/authdaemonrc

And edit its content to change number of daemons (I use three):

# The number of daemon processes that are started.
daemons=3

Edit the file /usr/lib/courier-imap/etc/imapd, taking care to adapt IMAP_ULIMITD accordingly:

ADDRESS=127.0.0.1                    #Typ. use 0 or 127.0.0.1
...
IMAP_ULIMITD=2097152                 #Set same value as returned by 'ulimit -v' or imapd.rc will complain

Create link to authlib and imap daemon in /etc/rc.d/init.d

ln -s /usr/local/sbin/authdaemond /etc/rc.d/init.d/courier-authdaemon
ln -s /usr/lib/courier-imap/libexec/imapd.rc /etc/rc.d/init.d/courier-imap

IMAP Accounts

User accounts and settings are managed by userdb. man makeuserdb for details.

First create user mailbox (replace username with user account name) with maildirmake. Repeat for all users:

mkdir -p /home/username                          #if don't exist yet...
cd /home/username
for i in "" .Drafts .Sent .Trash .Templates; do /usr/lib/courier-imap/bin/maildirmake ~username/MailDir/$i; done
# Create also default Maildir for new users
for i in "" .Drafts .Sent .Trash .Templates; do /usr/lib/courier-imap/bin/maildirmake /etc/skel/MailDir/$i; done

Next create the user authentication database /usr/local/etc/authlib/userdb:

touch /usr/local/etc/authlib/userdb
chmod 700 /usr/local/etc/authlib/userdb
#Repeat for each username:
pw2userdb | grep "^username">>/usr/local/etc/authlib/userdb   # Fill userdb from /etc/passwd user entry
userdb username set mail=~username/MailDir                    # Specify user MailDir location
userdbpw | userdb username set systempw                       # Set user account password
...
makeuserdb                                                    # Create binary database

You should now have the following files created:

• /usr/local/etc/authlib/userdb.dat
• /usr/local/etc/authlib/userdbshadow.dat

Configuration Summary

$ /usr/local/bin/courierauthconfig.exe --version
$ /usr/local/bin/courierauthconfig.exe --ldflags
$ /usr/local/bin/courierauthconfig.exe --cppflags
$ /usr/local/bin/courierauthconfig.exe --configfiles

Running

Start the daemons with the following commands:

/etc/rc.d/init.d/courier-authdaemon start
/etc/rc.d/init.d/courier-imap start

Stop the daemons with the following commands:

/etc/rc.d/init.d/courier-imap stop
/etc/rc.d/init.d/courier-authdaemon stop

Testing

If not done yet, install and configure syslogd. Run the Cygwin setup program and install inetutils package (contains syslogd). Configure syslogd with the following command, which will create the file /etc/syslogd.conf file and install a windows service using cygrunsrv:

$ syslogd-config                   #if not configured yet
$ net start syslogd                #if not started yet

Edit authdaemond configuration file /usr/local/etc/authlib/authdaemonrc to enable full debug messages:

# DEBUG_LOGIN=2   - turn on debugging + log passwords too
DEBUG_LOGIN=2

and restart courier-authdaemon to take changes into effect:

$ /etc/rc.d/init.d/courier-authdaemon stop
$ /etc/rc.d/init.d/courier-authdaemon start

Testing Courier Authentication Library with authtest:

$ export PATH=$PATH:/usr/local/lib/bin               #because authtest needs a .DLL library there (found with strace)
$ /usr/local/sbin/authtest username CCCCCC

This should produce something similar to the transcript below (where CCCCCC is your cleartext password, and XXXXXX is your encrypted password as provided in /usr/local/etc/authlib/userdb):

 Authentication succeeded.
 
      Authenticated: username  (uid 1001, gid 10545)
     Home Directory: /home/username
            Maildir: /home/username/Maildir
              Quota: (none)
 Encrypted Password: XXXXXX
 Cleartext Password: CCCCCC
            Options: (none)

Check that everything is fine in /var/log/messages (see [2] for details):

$ cat /var/log/messages

Testing Courier-IMAP through telnet:

$ telnet localhost 143
</source

In the ''telnet'' session, type the following test scripts (replace ''username'' and ''password'' accordingly):

<source lang="text">
001 LOGIN username password
002 EXAMINE INBOX
003 LOGOUT

Server must reply with OK messages along with some more information (see also [3] for details).

Hint: if the connection is immediately closed by foreign host it probably means that imapd does not have access to all required DLLs. Check your PATH.

When testing is done, don't forget to reset debugging in /usr/local/etc/authlib/authdaemonrc:

# DEBUG_LOGIN=2   - turn on debugging + log passwords too
DEBUG_LOGIN=0

Troubleshooting

Should you have a doubt on the execution of a command (ie: no output) you can check if the application is able to load required DLL. This error is not reported. Using strace, windows will complain about DLL not found, if any. To solve this problem you may need to add some path in your PATH. For example, in order to test authlib with authtest:

$ strace /usr/local/sbin/authtest             # This will open a dialog box to complain about DLL.
$ export PATH=$PATH:/usr/local/lib/bin
$ /usr/local/sbin/authtest ausername

Installing as a Windows Service

If everything works fine you can install Courier as a Windows Service. Create the [{{#file:imapd-srv.sh}} service script]

#!/bin/sh
# File:     imapd-service.sh
# Purpose:  Courier imap daemon service script.
#
# NOTE:     This script must be excutable by SYSTEM
username=username	# <<<< Change me !!!
pidsleep=''

# Function to handle QUIT signal.
# Stop imapd daemon and kill background sleep.
function handleQuit
{
	echo "STOPPING: /etc/rc.d/init.d/courier-authdaemon"
	su $username -c "/etc/rc.d/init.d/courier-authdaemon stop"
	echo "STOPPING: /etc/rc.d/init.d/courier-imap"
	su $username -c "/etc/rc.d/init.d/courier-imap stop"
	test -n "$pidsleep" && kill -9 $pidsleep
	exit
}

# Interruptible sleep
# Note: sleep is an external command (not builtin command), hence
#       not interruptible by the QUIT signal.
function intsleep
{
	sleep $1 &
	pidsleep=$!
	wait $pidsleep
}

echo "STARTING: /etc/rc.d/init.d/courier-authdaemon as $username at "`date`
su $username -c "/etc/rc.d/init.d/courier-authdaemon start"
echo "STARTING: /etc/rc.d/init.d/courier-imap as $username at "`date`
su $username -c "/etc/rc.d/init.d/courier-imap start"

trap "handleQuit" SIGQUIT

echo "WAITING: "$$
while true; do intsleep 1d; done

Change username and save it to your favorite location, say /usr/local/sbin. The service script must be executable by SYSTEM, ie: the user that run the windows services.

$ chmod +rx /usr/local/sbin/imapd-service.sh

Install the new service with the following command.

$ cygrunsrv --install imapd --desc "Courier IMAP daemon" --disp "CYGWIN imapd" \
  --path /usr/local/sbin/imapd-service.sh --termsig QUIT --type auto --shutdown

Finally, just start the service

$ cygrunsrv --start imapd
# or
$ net start imapd

You should now have couple of processes related to courier-imap

$ ps -a

NOTES:

• If the service could not be started, it's probably because the service script is not executable by SYSTEM

 $ net start imapd
 The CYGWIN imapd service is starting.
 The CYGWIN imapd service could not be started. 
 
 The service did not report an error.
 
 More help is available by typing NET HELPMSG 3534.

• The combination of background sleep and builtin wait command is used to make the waiting statment interruptible. The original script found on the net, was simply using a 'sleep 1' which turned to consume ~10% of the CPU.

Other Interesting Packages

Maildir

maildirmake is part of the courier-imap package

$ /usr/lib/courier-imap/bin/maildirmake

Mairix - index and search mail folders

Run the Cygwin setup program. Locate, select and install the package mairix.

Create the configuration file in your home directory: ~/.mairixrc

base=/home/username
maildir=Maildir...
omit=Maildir/.Mairix**
mfolder=Maildir/.Mairix
database=/home/username/.mairix_database

The create the mairix index

$ mairix
and
$ mairix --fast-index  # on daily base

See Mail_Tips for additional details.

Offlineimap

Requires python, so run the Cygwin setup program. Locate, select and install the package python.

Download offlineimap here. I use version 6.0.3. Extract files in you favorite sandbox.

$ tar xvfz offlineimap_6.0.3.tar.gz
$ cd offlineimap

Apply the following [{{#file:offlineimap_6.0.3-cygwin.patch}} patch] to correct the following issues:

• Suffix separator character colon (:) is not allowed in Windows. Courier-imap for cygwin is using (!)
• While used by courier-imap for cygwin Maildir mail files should not use CRLF, rather only LF. The issue appears only in mail with attachment, that is the raw message is not interpreted and displayed as it in the mail body (under Thunderbird).

--- offlineimap.conf	2008-08-13 20:56:04.000000000 +0200
+++ offlineimap-cygwin.conf	2008-11-06 10:44:31.066750700 +0100
@@ -215,6 +215,19 @@
 
 restoreatime = no
 
+# Cygwin/Windows users may have issues with the Maildir mail filename
+# containing the suffix separator character (:). In this case, they
+# could change the suffix separator character to whatever required.
+# For example, Courier-IMAP for Cygwin is using (!)
+
+# suffixsep = !
+
+# Cygwin/Windows users may need to create Maildir mail filename in
+# binary mode. In this case, set the 'filemode' to 'binary'. Default is
+# 'text'. Courier-IMAP for Cygwin works better with binary mode.
+
+# filemode = binary
+
 [Repository RemoteExample]
 
 # And this is the remote repository.  We only support IMAP or Gmail here.
--- offlineimap/folder/Maildir.py	2008-08-13 20:55:48.000000000 +0200
+++ offlineimap/folder/Maildir-cygwin.py	2008-11-06 10:44:05.344202700 +0100
@@ -23,7 +23,6 @@
 import os.path, os, re, time, socket, md5
 
 uidmatchre = re.compile(',U=(\d+)')
-flagmatchre = re.compile(':.*2,([A-Z]+)')
 
 timeseq = 0
 lasttime = long(0)
@@ -54,6 +53,12 @@
         self.messagelist = None
         self.repository = repository
         self.accountname = accountname
+        self.filemode = {
+          'binary': 'wb',
+          'text': 'wt',
+        }[repository.getconf("filemode", 'text')]
+        self.suffixsep = repository.getconf("suffixsep",':')
+        self.flagmatchre = re.compile('%s.*2,([A-Z]+)' % self.suffixsep)
         BaseFolder.__init__(self)
 
     def getaccountname(self):
@@ -102,7 +107,7 @@
                     nouidcounter -= 1
                 else:
                     uid = long(uidmatch.group(1))
-            flagmatch = flagmatchre.search(messagename)
+            flagmatch = self.flagmatchre.search(messagename)
             flags = []
             if flagmatch:
                 flags = [x for x in flagmatch.group(1)]
@@ -180,7 +185,7 @@
                 break
         tmpmessagename = messagename.split(',')[0]
         ui.debug('maildir', 'savemessage: using temporary name %s' % tmpmessagename)
-        file = open(os.path.join(tmpdir, tmpmessagename), "wt")
+        file = open(os.path.join(tmpdir, tmpmessagename), self.filemode)
         file.write(content)
 
         # Make sure the data hits the disk
@@ -226,11 +231,11 @@
             newpath = os.path.join(self.getfullname(), 'cur')
         else:
             newpath = os.path.join(self.getfullname(), 'new')
-        infostr = ':'
-        infomatch = re.search('(:.*)$', newname)
+        infostr = self.suffixsep
+        infomatch = re.search('(%s.*)$' % self.suffixsep, newname)
         if infomatch:                   # If the info string is present..
             infostr = infomatch.group(1)
-            newname = newname.split(':')[0] # Strip off the info string.
+            newname = newname.split(self.suffixsep)[0] # Strip off the info string.
         infostr = re.sub('2,[A-Z]*', '', infostr)
         flags.sort()
         infostr += '2,' + ''.join(flags)

$ patch -Np0 < offlineimap_6.0.3-cygwin.patch

Install offlineimap

$ python setup.py install

Configure offlineimap as described in Offlineimap and launch offlineimap. CAUTION: Add the NEW suffixsep and filemode options in the Maildir repository section of the configuration file.

$ offlineimap
or
$ offlineimap -1 -o -u Noninteractive.Basic
or
$ offlineimap -c /home/username/offlineimap.conf

Here is a simple [{{#file:offlineimap.conf}} offlineimap configuration file].

[general]
metadata = /home/username/.offlineimap
accounts = Test
maxsyncaccounts = 1
ui = Curses.Blinkenlights, TTY.TTYUI,
     Noninteractive.Basic, Noninteractive.Quiet
ignore-readonly = no

[mbnames]
enabled = no

[ui.Curses.Blinkenlights]
statuschar = .

[Account Test]
localrepository = LocalExample
remoterepository = RemoteExample

[Repository LocalExample]
type = Maildir
localfolders = /home/username/Maildir
sep = .
restoreatime = no

suffixsep = !                    # <<< NEW
filemode = binary                # <<< NEW

[Repository RemoteExample]
type = IMAP
remotehost = imapserver
ssl = no
remoteport = 143
remoteuser = username
remotepass = XXXXXX
maxconnections = 1
holdconnectionopen = no
nametrans = lambda foldername: re.sub('^INBOX\.*', '.', foldername)
folderfilter = lambda foldername: not re.search('(SPAM$|Mairix)', foldername)