Installing The Cyrus Murder

Note that Cyrus Murder is still relatively young in the grand scheme of things, and if you choose to deploy you are doing so at your own risk. Many of the failure modes can be difficult to track without a detailed understanding of the mupdate protocol and IMAP in general, and thus even considering a deployment is not for the faint at heart.

At the same time, we are using it successfully in production at Carnegie Mellon.

Introduction & Assumptions

This document is intended to be a guide to the configuration of a Cyrus IMAP Aggregator, aka Cyrus Murder. It is recommended that you review this document to be familliar with the concepts at work. This document is a work in progress and is at this point incomplete.

This document assumes that you have successfully been able to setup atleast one Cyrus IMAP server. This server will become your first backend server. It also assumes that you are familliar with the administration and day to day operations of the Cyrus IMAP server and the SASL authentication library. If you feel uncomfortable with this, please refer to the rest of the documentation first.

There is a diagram that shows the interactions of the various components of the Cyrus Murder which may be helpful in understanding the "big picture".

Installation

You will need to build Cyrus IMAPd with the --enable-murder configure option. This builds the proxyds and the associated utilities.

Requirements

Configuring the MUPDATE Master

The mupdate master server needs to be running the mupdate service in master mode. Note that you can have the MUPDATE master be one of your frontend machines, just do not configure a slave mupdate process on this machine.

To configure an mupdate master, you will want a cyrus.conf that includes a line similar to the following in the SERVICES section:

mupdate cmd="/usr/cyrus/bin/mupdate -m" listen=3905 prefork=1 Note the "-m" option to tell mupdate that it should start in master mode.

You will also need to configure atleast a skeleton imapd.conf that defines the configdirectory, a bogus partition-default and the admins that can authenticate to the server. Note that slave mupdate servers as well as the backend servers will need to be able to authenticate as admins on the master. Here is a very simple imapd.conf for a master server:

configdirectory: /imap/conf partition-default: /tmp admins: mupdateslave1 backend1 You will also need to configure SASL to properly allow authentication in your enviornment.

Setting up the backends to push changes to the MUPDATE Master

On the backends, configuration to be a part of a murder is easy. You just need to configure the backend to be a part of the murder. To do this, set the mupdate_server option in imapd.conf. Depending on what authentication mechanisms you are using, you may also want to set some or all of the following: Once these settings are successfully made, any mailbox operation on the backend will be sent to the mupdate master for confirmation and entry into the mupdate database.

You will also want to configure atleast one user/group using the proxyservers imapd.conf option. This user should not be an administrator, since this means that anyone who can get ahold of your proxy servers now has full administrative control on your backend. Example:

admins: cyrus proxyservers: murder Keep in mind that you will need to create the proxy user(s) and be sure that they can authenticate to the backend as well.

Importing the database from the backend

Importing the current mailboxes database is easy, as there is a ctl_mboxlist option to do so. To do the first synchronization, simply change to the cyrus user, and issue a ctl_mboxlist -m.

Note that you may wish to issue a ctl_mboxlist -mw first to be sure you understand all the operations that this command will perform, sicne it does require tha all mailboxes are unique in the murder namespace.

If everything is configured properly, the mailbox database of the current host will dump to the mupdate master. If there are problems, the most likely cause is a misconfiguration of the authentication settings, or that mupdate might not be running on the master. Using mupdatetest may be helpful in this case (it establishes an authenticated connection to the mupdate server, if it can).

It is also useful to have the backends automatically resync the state of their local mailboxes database with the master on start up. You can configure this by adding the following to the START section of cyrus.conf on the backends:

mupdatepush cmd="ctl_mboxlist -m" This will perform synchronization with the mupdate master each time the backend restarts, bringing the mupdate database up to date with the contents of the backend (and performing ACTIVATE and DELETES as needed to do so).

Warning: If somehow a mailbox exists on two (or more) backend servers, each time one of them synchronizes its database that backend server will become authoritative. Though this should not happen during normal operation of the murder (because of the consistancy guarantees of the MUPDATE protocol, and the fact that mailbox operations are denied if the mupdate master is down), it is possible when first creating the mupdate database or when bringing a new backend server into the murder.

Configuring the frontends

Configuring the frontends is a two step process. First, you want to set mupdate_server (and friends) as you did for the backends above. However, because the frontends only talk to the mupdate master via a slave running on the local machine, you will also need to set up a slave on the same machine, in the SERVICES section of cyrus.conf, like so:

# mupdate database service - must prefork atleast 1 mupdate cmd="mupdate" listen=3905 prefork=1 Note that as this is a threaded service, you must prefork atleast 1 of them so that the database can be synchronized at startup. Otherwise, the service will not start running until after you recieve an mupdate client connection to the slave (which is not a recommended configuration at this point).

You will also want to change all of your imapd entries to be proxyd, and all of your pop3d entries to be pop3proxyd. That is, you will probabally have a SERVICES section that looks more like this now:

mupdate cmd="/usr/cyrus/bin/mupdate" listen=3905 prefork=1 imap cmd="proxyd" listen="imap" prefork=5 imaps cmd="proxyd -s" listen="imaps" prefork=1 pop3 cmd="pop3proxyd" listen="pop3" prefork=0 pop3s cmd="pop3proxyd -s" listen="pop3s" prefork=0 kpop cmd="pop3proxyd -k" listen="kpop" prefork=0 sieve cmd="timsieved" listen="sieve" prefork=0 lmtp cmd="lmtpproxyd" listen="/var/imap/socket/lmtp" prefork=0 Note that timsieved does not need a proxy daemon, the managesieve protocol deals with the murder with referrals to the backends internally.

Additionally, you will need entries in imapd.conf to indicate the proxy auth name and passwords (if you are using a SASL mechanism that requires them) to the backends, for example, if your backends are mail1.andrew.cmu.edu and mail2.andrew.cmu.edu with passwords of foo and bar, and an auth name of murder:

mail1_password: foo mail2_password: bar proxy_authname: murder If your SASL mechanism does not require authnames or passwords (e.g. KERBEROS_V4), then this is not required. Note that we used the same authname as the configured in the proxyservers line in the backend's imapd.conf above.

When you start master on the frontend, a local mailboxes database should automatically synchronize itself with the contents of the mupdate master, and you should be ready to go. Your clients should connect to the frontends, and the frontends will proxy or refer as applicable to the blackend servers.

Additional backend configuration

If your authentication system requires usernames, passwords, etc, to authenticate (e.g. it isn't Kerberos), then you will also need to specify proxy_authname (and friends) in the backend imapd.confs as well. This is so that the backends can authenticate to eachother to facilitate maibox moves. (Backend machines will need to be full admins).

Delivering mail

To deliver mail to your Murder, configure your MTA just as you did before, but instead of connecting directly to lmtpd, it should connect to lmtpproxyd. You can connect to the lmtpproxyd running on the frontend machines, or you can install master and lmtpproxyd on your SMTP servers.

Administration

Keeping the database synced

Consistancy in the database is maintained by pushing the current status of the backends to the master, and having the frontends stay up to date with the master's database. Since the frontends resync themselves entirely when they startup, downtime should not at all be a problem. (While they are up they should be continously recieving database updates, as well as when they lose connection to the master, they will try to reconnect and resync their database upon reconnection)

Provided that the namespace of the backend servers is kept discrete (with no mailboxes existing on the same server), it is not a big deal to resync the mupdate master using ctl_mboxlist -m. If two servers do have the same mailbox, this will need to be resolved before database consistancy can be guranteed.

Moving Mailboxes between backends

There is currently no 100% foolproof way to do this, however, if you issue a rename command to a frontend (as you would to move a mailbox between partitions), and replace the partition name with the name of the new backend, it will move the mailbox to the indicated backend. You can also use the format backend.domain.com!partition to move to a specific partition (otherwise the default partition will be used). In cyradm, this looks like:

cyrus.andrew.cmu.edu> rename user.bcyrus user.bcyrus mail2.andrew.cmu.edu!u2 Note that since seen state is stored per-user, it is possible that when moving a shared mailbox users will have strange effects. The general rule is that moving an INBOX will move the entire user (including all sub-mailboxes to the INBOX, and seen state, and subscriptions, and sieve scripts, etc). The seen state is merged with the seen state on the new backend, so that no data is lost (seen state is also the only part left behind on the source backend). In the case of any other mailbox, however, only that individual mailbox is moved. If it is a quota root, the new quota root is instated on the new server, but otherwise quotas can appear to be violated, since each backend only takes care of its own quota.

In general, its better to leave trees of mailboxes on the same server, and not move submailboxes of inboxes between servers.

Adding additional backend servers

This is very easy to do, simply configure an empty backend server and set its mupdate_server parameter to point at the mupdate master. Then, issue mailbox creates to it as you would any other backend server.

Backups

xxx, need to write stuff. You don't need to really backup the data on the mupdate master or slaves, since this data can all be generated directly from the backends quite easily.

Gotchyas

Troubleshooting & when things go wrong

References


last modified: $Date: 2003/10/29 19:28:21 $