[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Exim is designed to work efficiently on systems that are permanently connected to the Internet and are handling a general mix of mail. In such circumstances, most messages can be delivered immediately. Consequently, Exim does not maintain independent queues of messages for specific domains or hosts, though it does try to send several messages in a single SMTP connection after a host has been down, and it also maintains per-host retry information.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Policy controls are now an important feature of MTAs that are connected to the Internet. Perhaps their most important job is to stop MTAs being abused as "open relays" by misguided individuals who send out vast amounts of unsolicited junk, and want to disguise its source. Exim provides flexible facilities for specifying policy controls on incoming mail:
local_scan()
can be run to inspect the message and decide
whether to accept it or not (see chapter Adding a local scan function to Exim). If the message
is accepted, the list of recipients can be modified by the function.
local_scan()
mechanism is another way of calling external scanner
software. The SA-Exim
add-on package works this way. It does not require
Exim to be compiled with the content-scanning extension.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
In a conventional Exim configuration, users are able to run private filters by
setting up appropriate ‘.forward’ files in their home directories. See
chapter The redirect router (about the redirect
router) for the
configuration needed to support this, and the separate document entitled
Exim's interfaces to mail filtering for user details. Two different kinds
of filtering are available:
User filters are run as part of the routing process, described below.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Every message handled by Exim is given a message id which is sixteen characters long. It is divided into three parts, separated by hyphens, for example ‘16VDhn-0001bo-D3’. Each part is a sequence of letters and digits, normally encoding numbers in base 62. However, in the Darwin operating system (Mac OS X) and when Exim is compiled to run under Cygwin, base 36 (avoiding the use of lower case letters) is used instead, because the message id is used to construct file names, and the names of files in those systems are not always case-sensitive.
The detail of the contents of the message id have changed as Exim has evolved. Earlier versions relied on the operating system not re-using a process id (pid) within one second. On modern operating systems, this assumption can no longer be made, so the algorithm had to be changed. To retain backward compatibility, the format of the message id was retained, which is why the following rules are somewhat eccentric:
localhost_number
is not set, this value is the fractional part of the
time of reception, normally in units of 1/2000 of a second, but for systems
that must use base 36 instead of base 62 (because of case-insensitive file
systems), the units are 1/1000 of a second.
localhost_number
is set, it is multiplied by 200 (100) and added to
the fractional part of the time, which in this case is in units of 1/200
(1/100) of a second.
After a message has been received, Exim waits for the clock to tick at the appropriate resolution before proceeding, so that if another message is received by the same process, or by another process with the same (re-used) pid, it is guaranteed that the time will be different. In most cases, the clock will already have ticked while the message was being received.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
The only way Exim can receive mail from another host is using SMTP over TCP/IP, in which case the sender and recipient addresses are transferred using SMTP commands. However, from a locally running process (such as a user's MUA), there are several possibilities:
-bm
option, the message is read
non-interactively (usually via a pipe), with the recipients taken from the
command line, or from the body of the message if -t
is also used.
-bS
option, the message is also read
non-interactively, but in this case the recipients are listed at the start of
the message in a series of SMTP RCPT commands, terminated by a DATA
command. This is so-called "batch SMTP" format,
but it isn't really SMTP. The SMTP commands are just another way of passing
envelope addresses in a non-interactive submission.
-bs
option, the message is read
interactively, using the SMTP protocol. A two-way pipe is normally used for
passing data between the local process and the Exim process.
This is "real" SMTP and is handled in the same way as SMTP over TCP/IP. For
example, the ACLs for SMTP commands are used for this form of submission.
In the three cases that do not involve TCP/IP, the sender address is
constructed from the login name of the user that called Exim and a default
qualification domain (which can be set by the qualify_domain
configuration
option). For local or batch SMTP, a sender address that is passed using the
SMTP MAIL command is ignored. However, the system administrator may allow
certain users ("trusted users") to specify a different sender address
unconditionally, or all users to specify certain forms of different sender
address. The -f
option or the SMTP MAIL command is used to specify these
different addresses. See section Trusted and admin users for details of trusted
users, and the untrusted_set_sender
option for a way of allowing untrusted
users to change sender addresses.
Messages received by either of the non-interactive mechanisms are subject to
checking by the non-SMTP ACL, if one is defined. Messages received using SMTP
(either over TCP/IP, or interacting with a local process) can be checked by a
number of ACLs that operate at different times during the SMTP session. Either
individual recipients, or the entire message, can be rejected if local policy
requirements are not met. The local_scan()
function (see chapter
Adding a local scan function to Exim) is run for all incoming messages.
Exim can be configured not to start a delivery process when a message is received; this can be unconditional, or depend on the number of incoming SMTP connections or the system load. In these situations, new messages wait on the queue until a queue runner process picks them up. However, in standard configurations under normal conditions, delivery is started as soon as a message is received.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
When Exim accepts a message, it writes two files in its spool directory. The first contains the envelope information, the current status of the message, and the header lines, and the second contains the body of the message. The names of the two spool files consist of the message id, followed by ‘-H’ for the file containing the envelope and header, and ‘-D’ for the data file.
By default all these message files are held in a single directory called
‘input’ inside the general Exim spool directory. Some operating systems do
not perform very well if the number of files in a directory gets large; to
improve performance in such cases, the split_spool_directory
option can be
used. This causes Exim to split up the input files into 62 sub-directories
whose names are single letters or digits. When this is done, the queue is
processed one sub-directory at a time instead of all at once, which can improve
overall performance even when there are not enough files in each directory to
affect file system performance.
The envelope information consists of the address of the message's sender and the addresses of the recipients. This information is entirely separate from any addresses contained in the header lines. The status of the message includes a list of recipients who have already received the message. The format of the first spool file is described in chapter Format of spool files.
Address rewriting that is specified in the rewrite section of the configuration (see chapter Address rewriting) is done once and for all on incoming addresses, both in the header lines and the envelope, at the time the message is accepted. If during the course of delivery additional addresses are generated (for example, via aliasing), these new addresses are rewritten as soon as they are generated. At the time a message is actually delivered (transported) further rewriting can take place; because this is a transport option, it can be different for different forms of delivery. It is also possible to specify the addition or removal of certain header lines at the time the message is delivered (see chapters Generic options for routers and Generic options for transports).
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
A message remains in the spool directory until it is completely delivered to its recipients or to an error address, or until it is deleted by an administrator or by the user who originally created it. In cases when delivery cannot proceed - for example, when a message can neither be delivered to its recipients nor returned to its sender, the message is marked "frozen" on the spool, and no more deliveries are attempted.
An administrator can "thaw" such messages when the problem has been corrected, and can also freeze individual messages by hand if necessary. In addition, an administrator can force a delivery error, causing a bounce message to be sent.
There are options called ignore_bounce_errors_after
and
timeout_frozen_after
, which discard frozen messages after a certain time.
The first applies only to frozen bounces, the second to any frozen messages.
While Exim is working on a message, it writes information about each delivery
attempt to its main log file. This includes successful, unsuccessful, and
delayed deliveries for each recipient (see chapter Log files). The log
lines are also written to a separate message log file for each message.
These logs are solely for the benefit of the administrator, and are normally
deleted along with the spool files when processing of a message is complete.
The use of individual message logs can be disabled by setting
no_message_logs
; this might give an improvement in performance on very busy
systems.
All the information Exim itself needs to set up a delivery is kept in the first spool file, along with the header lines. When a successful delivery occurs, the address is immediately written at the end of a journal file, whose name is the message id followed by ‘-J’. At the end of a delivery run, if there are some addresses left to be tried again later, the first spool file (the ‘-H’ file) is updated to indicate which these are, and the journal file is then deleted. Updating the spool file is done by writing a new file and renaming it, to minimize the possibility of data loss.
Should the system or the program crash after a successful delivery but before the spool file has been updated, the journal is left lying around. The next time Exim attempts to deliver the message, it reads the journal file and updates the spool file before proceeding. This minimizes the chances of double deliveries caused by crashes.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
The main delivery processing elements of Exim are called routers and transports, and collectively these are known as drivers. Code for a number of them is provided in the source distribution, and compile-time options specify which ones are included in the binary. Run time options specify which ones are actually used for delivering messages.
Each driver that is specified in the run time configuration is an instance
of that particular driver type. Multiple instances are allowed; for example,
you can set up several different smtp
transports, each with different
option values that might specify different ports or different timeouts. Each
instance has its own identifying name. In what follows we will normally use the
instance name when discussing one particular instance (that is, one specific
configuration of the driver), and the generic driver name when discussing
the driver's features in general.
A router is a driver that operates on an address, either determining how its delivery should happen, by assigning it to a specific transport, or converting the address into one or more new addresses (for example, via an alias file). A router may also explicitly choose to fail an address, causing it to be bounced.
A transport is a driver that transmits a copy of the message from Exim's spool to some destination. There are two kinds of transport: for a local transport, the destination is a file or a pipe on the local host, whereas for a remote transport the destination is some other host. A message is passed to a specific transport as a result of successful routing. If a message has several recipients, it may be passed to a number of different transports.
An address is processed by passing it to each configured router instance in turn, subject to certain preconditions, until a router accepts the address or specifies that it should be bounced. We will describe this process in more detail shortly. First, as a simple example, we consider how each recipient address in a message is processed in a small configuration of three routers.
To make this a more concrete example, it is described in terms of some actual routers, but remember, this is only an example. You can configure Exim's routers in many different ways, and there may be any number of routers in a configuration.
The first router that is specified in a configuration is often one that handles addresses in domains that are not recognized specially by the local host. These are typically addresses for arbitrary domains on the Internet. A precondition is set up which looks for the special domains known to the host (for example, its own domain name), and the router is run for addresses that do not match. Typically, this is a router that looks up domains in the DNS in order to find the hosts to which this address routes. If it succeeds, the address is assigned to a suitable SMTP transport; if it does not succeed, the router is configured to fail the address.
The second router is reached only when the domain is recognized as one that "belongs" to the local host. This router does redirection - also known as aliasing and forwarding. When it generates one or more new addresses from the original, each of them is routed independently from the start. Otherwise, the router may cause an address to fail, or it may simply decline to handle the address, in which case the address is passed to the next router.
The final router in many configurations is one that checks to see if the address belongs to a local mailbox. The precondition may involve a check to see if the local part is the name of a login account, or it may look up the local part in a file or a database. If its preconditions are not met, or if the router declines, we have reached the end of the routers. When this happens, the address is bounced.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
As well as being used to decide how to deliver to an address, Exim's routers
are also used for address verification. Verification can be requested as
one of the checks to be performed in an ACL for incoming messages, on both
sender and recipient addresses, and it can be tested using the -bv
and
-bvs
command line options.
When an address is being verified, the routers are run in "verify mode". This
does not affect the way the routers work, but it is a state that can be
detected. By this means, a router can be skipped or made to behave differently
when verifying. A common example is a configuration in which the first router
sends all messages to a message-scanning program, unless they have been
previously scanned. Thus, the first router accepts all addresses without any
checking, making it useless for verifying. Normally, the no_verify
option
would be set for such a router, causing it to be skipped in verify mode.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
As explained in the example above, a number of preconditions are checked before running a router. If any are not met, the router is skipped, and the address is passed to the next router. When all the preconditions on a router are met, the router is run. What happens next depends on the outcome, which is one of the following:
unseen
option is set on the router. This option
can be used to set up multiple deliveries with different routing (for example,
for keeping archive copies of messages). When unseen
is set, the address is
passed to the next router. Normally, however, an accept return marks the
end of routing.
Any child addresses generated by the router are processed independently,
starting with the first router by default. It is possible to change this by
setting the redirect_router
option to specify which router to start at for
child addresses. Unlike pass_router
(see below) the router specified by
redirect_router
may be anywhere in the router configuration.
pass_router
option. However, (unlike redirect_router
) the named router
must be below the current router (to avoid loops).
no_more
option. When no_more
is
set, all the remaining routers are skipped. In effect, no_more
converts
decline into fail.
unseen
is set on the router.
If an address reaches the end of the routers without having been accepted by
any of them, it is bounced as unrouteable. The default error message in this
situation is "unrouteable address", but you can set your own message by
making use of the cannot_route_message
option. This can be set for any
router; the value from the last router that "saw" the address is used.
Sometimes while routing you want to fail a delivery when some conditions are
met but others are not, instead of passing the address on for further routing.
You can do this by having a second router that explicitly fails the delivery
when the relevant conditions are met. The redirect
router has a "fail"
facility for this purpose.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Once routing is complete, Exim scans the addresses that are assigned to local
and remote transports, and discards any duplicates that it finds. During this
check, local parts are treated as case-sensitive. This happens only when
actually delivering a message; when testing routers with -bt
, all the
routed addresses are shown.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
The preconditions that are tested for each router are listed below, in the order in which they are tested. The individual configuration options are described in more detail in chapter Generic options for routers.
local_part_prefix
and local_part_suffix
options can specify that
the local parts handled by the router may or must have certain prefixes and/or
suffixes. If a mandatory affix (prefix or suffix) is not present, the router is
skipped. These conditions are tested first. When an affix is present, it is
removed from the local part before further processing, including the evaluation
of any other conditions.
verify
option is set false, the router is skipped when Exim is verifying an
address.
Setting the verify
option actually sets two options, verify_sender
and
verify_recipient
, which independently control the use of the router for
sender and recipient verification. You can set these options directly if
you want a router to be used for only one type of verification.
address_test
option is set false, the router is skipped when Exim is
run with the -bt
option to test an address routing. This can be helpful
when the first router sends all new messages to a scanner of some sort; it
makes it possible to use -bt
to test subsequent delivery routing without
having to simulate the effect of the scanner.
verify_only
option controls this.
expn
option).
domains
option is set, the domain of the address must be in the set
of domains that it defines.
local_parts
option is set, the local part of the address must be in
the set of local parts that it defines. If local_part_prefix
or
local_part_suffix
is in use, the prefix or suffix is removed from the local
part before this check. If you want to do precondition tests on local parts
that include affixes, you can do so by using a condition
option (see below)
that uses the variables $local_part
, $local_part_prefix
, and
$local_part_suffix
as necessary.
check_local_user
option is set, the local part must be the name of
an account on the local host. If this check succeeds, the uid and gid of the
local user are placed in $local_user_uid
and $local_user_gid
and the
user's home directory is placed in $home
; these values can be used in the
remaining preconditions.
router_home_directory
option is set, it is expanded at this point,
because it overrides the value of $home
. If this expansion were left till
later, the value of $home
as set by check_local_user
would be used in
subsequent tests. Having two different values of $home
in the same router
could lead to confusion.
senders
option is set, the envelope sender address must be in the
set of addresses that it defines.
require_files
option is set, the existence or non-existence of
specified files is tested.
condition
option is set, it is evaluated and tested. This option
uses an expanded string to allow you to set up your own custom preconditions.
Expanded strings are described in chapter String expansions.
Note that require_files
comes near the end of the list, so you cannot use
it to check for the existence of a file in which to lookup up a domain, local
part, or sender. However, as these options are all expanded, you can use the
exists
expansion condition to make such tests within each condition. The
require_files
option is intended for checking files that the router may be
going to use internally, or which are needed by a specific transport (for
example, ‘.procmailrc’).
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
When a message is to be delivered, the sequence of events is as follows:
Some additional features are available in system filters - see chapter
System-wide message filtering for details. Note that a message is passed to the system
filter only once per delivery attempt, however many recipients it has. However,
if there are several delivery attempts because one or more addresses could not
be immediately delivered, the system filter is run each time. The filter
condition first_delivery
can be used to detect the first run of the system
filter.
remote_max_parallel
option.
The order in which deliveries are done is not defined, except that all local
deliveries happen before any remote deliveries.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Exim's mechanism for retrying messages that fail to get delivered at the first
attempt is the queue runner process. You must either run an Exim daemon that
uses the -q
option with a time interval to start queue runners at regular
intervals, or use some other means (such as cron) to start them. If you do
not arrange for queue runners to be run, messages that fail temporarily at the
first attempt will remain on your queue for ever. A queue runner process works
its way through the queue, one message at a time, trying each delivery that has
passed its retry time.
You can run several queue runners at once.
Exim uses a set of configured rules to determine when next to retry the failing address (see chapter Retry configuration). These rules also specify when Exim should give up trying to deliver to the address, at which point it generates a bounce message. If no retry rules are set for a particular host, address, and error combination, no retries are attempted, and temporary errors are treated as permanent.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
There are many reasons why a message may not be immediately deliverable to a particular address. Failure to connect to a remote machine (because it, or the connection to it, is down) is one of the most common. Temporary failures may be detected during routing as well as during the transport stage of delivery. Local deliveries may be delayed if NFS files are unavailable, or if a mailbox is on a file system where the user is over quota. Exim can be configured to impose its own quotas on local mailboxes; where system quotas are set they will also apply.
If a host is unreachable for a period of time, a number of messages may be waiting for it by the time it recovers, and sending them in a single SMTP connection is clearly beneficial. Whenever a delivery to a remote host is deferred,
Exim makes a note in its hints database, and whenever a successful SMTP delivery has happened, it looks to see if any other messages are waiting for the same host. If any are found, they are sent over the same SMTP connection, subject to a configuration limit as to the maximum number in any one connection.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
When a message cannot be delivered to some or all of its intended recipients, a bounce message is generated. Temporary delivery failures turn into permanent errors when their timeout expires. All the addresses that fail in a given delivery attempt are listed in a single message. If the original message has many recipients, it is possible for some addresses to fail in one delivery attempt and others to fail subsequently, giving rise to more than one bounce message. The wording of bounce messages can be customized by the administrator. See chapter Customizing bounce and warning messages for details.
Bounce messages contain an X-Failed-Recipients: header line that lists the failed addresses, for the benefit of programs that try to analyse such messages automatically.
A bounce message is normally sent to the sender of the original message, as obtained from the message's envelope. For incoming SMTP messages, this is the address given in the MAIL command. However, when an address is expanded via a forward or alias file, an alternative address can be specified for delivery failures of the generated addresses. For a mailing list expansion (see section Using Exim to handle mailing lists) it is common to direct bounce messages to the manager of the list.
[ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
If a bounce message (either locally generated or received from a remote host)
itself suffers a permanent delivery failure, the message is left on the queue,
but it is frozen, awaiting the attention of an administrator. There are options
that can be used to make Exim discard such failed messages, or to keep them
for only a short time (see timeout_frozen_after
and
ignore_bounce_errors_after
).
[ << ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
This document was generated on January, 24 2008 using texi2html 1.78.