Code for verify=header_names_ascii
[exim.git] / doc / doc-txt / experimental-spec.txt
CommitLineData
7bafa7d9
TK
1From time to time, experimental features may be added to Exim.
2While a feature is experimental, there will be a build-time
3option whose name starts "EXPERIMENTAL_" that must be set in
4order to include the feature. This file contains information
d36a0501
PP
5about experimental features, all of which are unstable and
6liable to incompatible change.
ee161e8f
PH
7
8
fd98a5c6
JH
9PRDR support
10--------------------------------------------------------------
11
12Per-Recipient Data Reponse is an SMTP extension proposed by Eric Hall
13in a (now-expired) IETF draft from 2007. It's not hit mainstream
14use, but has apparently been implemented in the META1 MTA.
15
16There is mention at http://mail.aegee.org/intern/sendmail.html
17of a patch to sendmail "to make it PRDR capable".
18
19 ref: http://www.eric-a-hall.com/specs/draft-hall-prdr-00.txt
20
21If Exim is built with EXPERIMENTAL_PRDR there is a new config
22boolean "prdr_enable" which controls whether PRDR is advertised
23as part of an EHLO response, a new "acl_data_smtp_prdr" ACL
24(called for each recipient, after data arrives but before the
25data ACL), and a new smtp transport option "hosts_try_prdr".
26
27PRDR may be used to support per-user content filtering. Without it
28one must defer any recipient after the first that has a different
29content-filter configuration. With PRDR, the RCPT-time check
30for this can be disabled when the MAIL-time $smtp_command included
31"PRDR". Any required difference in behaviour of the main DATA-time
32ACL should however depend on the PRDR-time ACL having run, as Exim
33will avoid doing so in some situations (eg. single-recipient mails).
34
35
36
3f7eeb86
PP
37OCSP Stapling support
38--------------------------------------------------------------
39
d36a0501 40X.509 PKI certificates expire and can be revoked; to handle this, the
3f7eeb86
PP
41clients need some way to determine if a particular certificate, from a
42particular Certificate Authority (CA), is still valid. There are three
43main ways to do so.
44
45The simplest way is to serve up a Certificate Revocation List (CRL) with
46an ordinary web-server, regenerating the CRL before it expires. The
47downside is that clients have to periodically re-download a potentially
48huge file from every certificate authority it knows of.
49
50The way with most moving parts at query time is Online Certificate
51Status Protocol (OCSP), where the client verifies the certificate
52against an OCSP server run by the CA. This lets the CA track all
53usage of the certs. This requires running software with access to the
54private key of the CA, to sign the responses to the OCSP queries. OCSP
55is based on HTTP and can be proxied accordingly.
56
57The only widespread OCSP server implementation (known to this writer)
58comes as part of OpenSSL and aborts on an invalid request, such as
59connecting to the port and then disconnecting. This requires
60re-entering the passphrase each time some random client does this.
61
62The third way is OCSP Stapling; in this, the server using a certificate
63issued by the CA periodically requests an OCSP proof of validity from
64the OCSP server, then serves it up inline as part of the TLS
65negotiation. This approach adds no extra round trips, does not let the
66CA track users, scales well with number of certs issued by the CA and is
67resilient to temporary OCSP server failures, as long as the server
68starts retrying to fetch an OCSP proof some time before its current
69proof expires. The downside is that it requires server support.
70
71If Exim is built with EXPERIMENTAL_OCSP and it was built with OpenSSL,
f5d78688 72then it gains a new global option: "tls_ocsp_file".
3f7eeb86
PP
73
74The file specified therein is expected to be in DER format, and contain
75an OCSP proof. Exim will serve it as part of the TLS handshake. This
76option will be re-expanded for SNI, if the tls_certificate option
77contains $tls_sni, as per other TLS options.
78
79Exim does not at this time implement any support for fetching a new OCSP
80proof. The burden is on the administrator to handle this, outside of
81Exim. The file specified should be replaced atomically, so that the
82contents are always valid. Exim will expand the "tls_ocsp_file" option
83on each connection, so a new file will be handled transparently on the
84next connection.
85
98a3b527 86Exim will check for a valid next update timestamp in the OCSP proof;
3f7eeb86
PP
87if not present, or if the proof has expired, it will be ignored.
88
f5d78688
JH
89Also, given EXPERIMENTAL_OCSP and OpenSSL, the smtp transport gains
90a "hosts_require_ocsp" option; a host-list for which an OCSP Stapling
91is requested and required for the connection to proceed. The host(s)
92should also be in "hosts_require_tls", and "tls_verify_certificates"
93configured for the transport.
94
880496ef
JH
95For the client to be able to verify the stapled OCSP the server must
96also supply, in its stapled information, any intermediate
97certificates for the chain leading to the OCSP proof from the signer
98of the server certificate. There may be zero or one such. These
99intermediate certificates should be added to the server OCSP stapling
100file (named by tls_ocsp_file).
101
3f7eeb86
PP
102At this point in time, we're gathering feedback on use, to determine if
103it's worth adding complexity to the Exim daemon to periodically re-fetch
f5d78688 104OCSP files and somehow handling multiple files.
3f7eeb86 105
880496ef
JH
106 A helper script "ocsp_fetch.pl" for fetching a proof from a CA
107 OCSP server is supplied. The server URL may be included in the
108 server certificate, if the CA is helpful.
109
110 One fail mode seen was the OCSP Signer cert expiring before the end
111 of vailidity of the OCSP proof. The checking done by Exim/OpenSSL
112 noted this as invalid overall, but the re-fetch script did not.
113
3f7eeb86
PP
114
115
116
0b23848a 117Brightmail AntiSpam (BMI) suppport
ee161e8f
PH
118--------------------------------------------------------------
119
120Brightmail AntiSpam is a commercial package. Please see
121http://www.brightmail.com for more information on
122the product. For the sake of clarity, we'll refer to it as
123"BMI" from now on.
124
125
1260) BMI concept and implementation overview
127
128In contrast to how spam-scanning with SpamAssassin is
129implemented in exiscan-acl, BMI is more suited for per
130-recipient scanning of messages. However, each messages is
131scanned only once, but multiple "verdicts" for multiple
132recipients can be returned from the BMI server. The exiscan
133implementation passes the message to the BMI server just
134before accepting it. It then adds the retrieved verdicts to
135the messages header file in the spool. These verdicts can then
136be queried in routers, where operation is per-recipient
137instead of per-message. To use BMI, you need to take the
138following steps:
139
140 1) Compile Exim with BMI support
3ec3e3bb 141 2) Set up main BMI options (top section of Exim config file)
ee161e8f
PH
142 3) Set up ACL control statement (ACL section of the config
143 file)
144 4) Set up your routers to use BMI verdicts (routers section
145 of the config file).
146 5) (Optional) Set up per-recipient opt-in information.
147
8ff3788c 148These four steps are explained in more details below.
ee161e8f
PH
149
1501) Adding support for BMI at compile time
151
152 To compile with BMI support, you need to link Exim against
153 the Brighmail client SDK, consisting of a library
154 (libbmiclient_single.so) and a header file (bmi_api.h).
155 You'll also need to explicitly set a flag in the Makefile to
156 include BMI support in the Exim binary. Both can be achieved
157 with these lines in Local/Makefile:
158
159 EXPERIMENTAL_BRIGHTMAIL=yes
47bbda99 160 CFLAGS=-I/path/to/the/dir/with/the/includefile
ee161e8f 161 EXTRALIBS_EXIM=-L/path/to/the/dir/with/the/library -lbmiclient_single
8ff3788c 162
ee161e8f
PH
163 If you use other CFLAGS or EXTRALIBS_EXIM settings then
164 merge the content of these lines with them.
165
7c0c8547 166 Note for BMI6.x users: You'll also have to add -lxml2_single
ee161e8f
PH
167 to the EXTRALIBS_EXIM line. Users of 5.5x do not need to do
168 this.
8ff3788c 169
ee161e8f
PH
170 You should also include the location of
171 libbmiclient_single.so in your dynamic linker configuration
172 file (usually /etc/ld.so.conf) and run "ldconfig"
173 afterwards, or else the produced Exim binary will not be
174 able to find the library file.
175
176
3ec3e3bb 1772) Setting up BMI support in the Exim main configuration
ee161e8f 178
3ec3e3bb 179 To enable BMI support in the main Exim configuration, you
ee161e8f
PH
180 should set the path to the main BMI configuration file with
181 the "bmi_config_file" option, like this:
8ff3788c 182
ee161e8f 183 bmi_config_file = /opt/brightmail/etc/brightmail.cfg
8ff3788c 184
3ec3e3bb 185 This must go into section 1 of Exim's configuration file (You
ee161e8f
PH
186 can put it right on top). If you omit this option, it
187 defaults to /opt/brightmail/etc/brightmail.cfg.
188
189 Note for BMI6.x users: This file is in XML format in V6.xx
190 and its name is /opt/brightmail/etc/bmiconfig.xml. So BMI
191 6.x users MUST set the bmi_config_file option.
8ff3788c 192
ee161e8f
PH
193
1943) Set up ACL control statement
195
196 To optimize performance, it makes sense only to process
197 messages coming from remote, untrusted sources with the BMI
198 server. To set up a messages for processing by the BMI
199 server, you MUST set the "bmi_run" control statement in any
200 ACL for an incoming message. You will typically do this in
201 an "accept" block in the "acl_check_rcpt" ACL. You should
202 use the "accept" block(s) that accept messages from remote
203 servers for your own domain(s). Here is an example that uses
3ec3e3bb 204 the "accept" blocks from Exim's default configuration file:
8ff3788c 205
ee161e8f
PH
206
207 accept domains = +local_domains
208 endpass
209 verify = recipient
210 control = bmi_run
211
212 accept domains = +relay_to_domains
213 endpass
214 verify = recipient
215 control = bmi_run
8ff3788c 216
ee161e8f
PH
217 If bmi_run is not set in any ACL during reception of the
218 message, it will NOT be passed to the BMI server.
219
220
2214) Setting up routers to use BMI verdicts
222
223 When a message has been run through the BMI server, one or
224 more "verdicts" are present. Different recipients can have
225 different verdicts. Each recipient is treated individually
226 during routing, so you can query the verdicts by recipient
3ec3e3bb 227 at that stage. From Exim's view, a verdict can have the
ee161e8f 228 following outcomes:
8ff3788c 229
ee161e8f
PH
230 o deliver the message normally
231 o deliver the message to an alternate location
232 o do not deliver the message
8ff3788c 233
ee161e8f
PH
234 To query the verdict for a recipient, the implementation
235 offers the following tools:
8ff3788c
TK
236
237
ee161e8f
PH
238 - Boolean router preconditions. These can be used in any
239 router. For a simple implementation of BMI, these may be
240 all that you need. The following preconditions are
241 available:
8ff3788c 242
ee161e8f 243 o bmi_deliver_default
8ff3788c 244
ee161e8f
PH
245 This precondition is TRUE if the verdict for the
246 recipient is to deliver the message normally. If the
247 message has not been processed by the BMI server, this
248 variable defaults to TRUE.
8ff3788c 249
ee161e8f 250 o bmi_deliver_alternate
8ff3788c 251
ee161e8f
PH
252 This precondition is TRUE if the verdict for the
253 recipient is to deliver the message to an alternate
254 location. You can get the location string from the
255 $bmi_alt_location expansion variable if you need it. See
256 further below. If the message has not been processed by
257 the BMI server, this variable defaults to FALSE.
8ff3788c 258
ee161e8f 259 o bmi_dont_deliver
8ff3788c 260
ee161e8f
PH
261 This precondition is TRUE if the verdict for the
262 recipient is NOT to deliver the message to the
263 recipient. You will typically use this precondition in a
264 top-level blackhole router, like this:
8ff3788c 265
ee161e8f
PH
266 # don't deliver messages handled by the BMI server
267 bmi_blackhole:
268 driver = redirect
269 bmi_dont_deliver
270 data = :blackhole:
8ff3788c 271
ee161e8f
PH
272 This router should be on top of all others, so messages
273 that should not be delivered do not reach other routers
274 at all. If the message has not been processed by
275 the BMI server, this variable defaults to FALSE.
8ff3788c
TK
276
277
ee161e8f
PH
278 - A list router precondition to query if rules "fired" on
279 the message for the recipient. Its name is "bmi_rule". You
280 use it by passing it a colon-separated list of rule
281 numbers. You can use this condition to route messages that
282 matched specific rules. Here is an example:
8ff3788c 283
ee161e8f
PH
284 # special router for BMI rule #5, #8 and #11
285 bmi_rule_redirect:
286 driver = redirect
287 bmi_rule = 5:8:11
288 data = postmaster@mydomain.com
8ff3788c
TK
289
290
ee161e8f
PH
291 - Expansion variables. Several expansion variables are set
292 during routing. You can use them in custom router
293 conditions, for example. The following variables are
294 available:
8ff3788c 295
ee161e8f 296 o $bmi_base64_verdict
8ff3788c 297
ee161e8f
PH
298 This variable will contain the BASE64 encoded verdict
299 for the recipient being routed. You can use it to add a
300 header to messages for tracking purposes, for example:
8ff3788c 301
ee161e8f
PH
302 localuser:
303 driver = accept
304 check_local_user
305 headers_add = X-Brightmail-Verdict: $bmi_base64_verdict
306 transport = local_delivery
8ff3788c 307
ee161e8f
PH
308 If there is no verdict available for the recipient being
309 routed, this variable contains the empty string.
8ff3788c 310
ee161e8f 311 o $bmi_base64_tracker_verdict
8ff3788c 312
ee161e8f
PH
313 This variable will contain a BASE64 encoded subset of
314 the verdict information concerning the "rules" that
315 fired on the message. You can add this string to a
316 header, commonly named "X-Brightmail-Tracker". Example:
8ff3788c 317
ee161e8f
PH
318 localuser:
319 driver = accept
320 check_local_user
321 headers_add = X-Brightmail-Tracker: $bmi_base64_tracker_verdict
322 transport = local_delivery
8ff3788c 323
ee161e8f
PH
324 If there is no verdict available for the recipient being
325 routed, this variable contains the empty string.
8ff3788c 326
ee161e8f 327 o $bmi_alt_location
8ff3788c 328
ee161e8f
PH
329 If the verdict is to redirect the message to an
330 alternate location, this variable will contain the
331 alternate location string returned by the BMI server. In
332 its default configuration, this is a header-like string
333 that can be added to the message with "headers_add". If
334 there is no verdict available for the recipient being
335 routed, or if the message is to be delivered normally,
336 this variable contains the empty string.
8ff3788c 337
ee161e8f 338 o $bmi_deliver
8ff3788c 339
ee161e8f
PH
340 This is an additional integer variable that can be used
341 to query if the message should be delivered at all. You
342 should use router preconditions instead if possible.
8ff3788c 343
ee161e8f
PH
344 $bmi_deliver is '0': the message should NOT be delivered.
345 $bmi_deliver is '1': the message should be delivered.
8ff3788c
TK
346
347
ee161e8f
PH
348 IMPORTANT NOTE: Verdict inheritance.
349 The message is passed to the BMI server during message
350 reception, using the target addresses from the RCPT TO:
351 commands in the SMTP transaction. If recipients get expanded
352 or re-written (for example by aliasing), the new address(es)
353 inherit the verdict from the original address. This means
354 that verdicts also apply to all "child" addresses generated
355 from top-level addresses that were sent to the BMI server.
8ff3788c
TK
356
357
ee161e8f
PH
3585) Using per-recipient opt-in information (Optional)
359
360 The BMI server features multiple scanning "profiles" for
361 individual recipients. These are usually stored in a LDAP
362 server and are queried by the BMI server itself. However,
363 you can also pass opt-in data for each recipient from the
364 MTA to the BMI server. This is particularly useful if you
3ec3e3bb 365 already look up recipient data in Exim anyway (which can
ee161e8f
PH
366 also be stored in a SQL database or other source). This
367 implementation enables you to pass opt-in data to the BMI
368 server in the RCPT ACL. This works by setting the
369 'bmi_optin' modifier in a block of that ACL. If should be
370 set to a list of comma-separated strings that identify the
371 features which the BMI server should use for that particular
372 recipient. Ideally, you would use the 'bmi_optin' modifier
373 in the same ACL block where you set the 'bmi_run' control
374 flag. Here is an example that will pull opt-in data for each
375 recipient from a flat file called
376 '/etc/exim/bmi_optin_data'.
8ff3788c 377
ee161e8f 378 The file format:
8ff3788c 379
ee161e8f
PH
380 user1@mydomain.com: <OPTIN STRING1>:<OPTIN STRING2>
381 user2@thatdomain.com: <OPTIN STRING3>
8ff3788c
TK
382
383
ee161e8f 384 The example:
8ff3788c 385
ee161e8f
PH
386 accept domains = +relay_to_domains
387 endpass
388 verify = recipient
389 bmi_optin = ${lookup{$local_part@$domain}lsearch{/etc/exim/bmi_optin_data}}
8ff3788c
TK
390 control = bmi_run
391
ee161e8f 392 Of course, you can also use any other lookup method that
3ec3e3bb 393 Exim supports, including LDAP, Postgres, MySQL, Oracle etc.,
ee161e8f
PH
394 as long as the result is a list of colon-separated opt-in
395 strings.
8ff3788c 396
ee161e8f
PH
397 For a list of available opt-in strings, please contact your
398 Brightmail representative.
ee161e8f 399
8ff3788c
TK
400
401
402
0b23848a 403Sender Policy Framework (SPF) support
ee161e8f
PH
404--------------------------------------------------------------
405
f413481d 406To learn more about SPF, visit http://www.openspf.org. This
ee161e8f
PH
407document does not explain the SPF fundamentals, you should
408read and understand the implications of deploying SPF on your
409system before doing so.
410
8ff3788c 411SPF support is added via the libspf2 library. Visit
ee161e8f
PH
412
413 http://www.libspf2.org/
8ff3788c 414
ee161e8f
PH
415to obtain a copy, then compile and install it. By default,
416this will put headers in /usr/local/include and the static
417library in /usr/local/lib.
418
3ec3e3bb 419To compile Exim with SPF support, set these additional flags in
ee161e8f
PH
420Local/Makefile:
421
422EXPERIMENTAL_SPF=yes
423CFLAGS=-DSPF -I/usr/local/include
424EXTRALIBS_EXIM=-L/usr/local/lib -lspf2
425
426This assumes that the libspf2 files are installed in
427their default locations.
428
429You can now run SPF checks in incoming SMTP by using the "spf"
430ACL condition in either the MAIL, RCPT or DATA ACLs. When
d36a0501 431using it in the RCPT ACL, you can make the checks dependent on
ee161e8f
PH
432the RCPT address (or domain), so you can check SPF records
433only for certain target domains. This gives you the
434possibility to opt-out certain customers that do not want
435their mail to be subject to SPF checking.
436
437The spf condition takes a list of strings on its right-hand
438side. These strings describe the outcome of the SPF check for
439which the spf condition should succeed. Valid strings are:
440
441 o pass The SPF check passed, the sending host
442 is positively verified by SPF.
443 o fail The SPF check failed, the sending host
444 is NOT allowed to send mail for the domain
445 in the envelope-from address.
446 o softfail The SPF check failed, but the queried
447 domain can't absolutely confirm that this
448 is a forgery.
449 o none The queried domain does not publish SPF
450 records.
451 o neutral The SPF check returned a "neutral" state.
452 This means the queried domain has published
453 a SPF record, but wants to allow outside
454 servers to send mail under its domain as well.
455 o err_perm This indicates a syntax error in the SPF
456 record of the queried domain. This should be
457 treated like "none".
458 o err_temp This indicates a temporary error during all
3ec3e3bb 459 processing, including Exim's SPF processing.
ee161e8f 460 You may defer messages when this occurs.
8ff3788c 461
ee161e8f
PH
462You can prefix each string with an exclamation mark to invert
463is meaning, for example "!fail" will match all results but
464"fail". The string list is evaluated left-to-right, in a
465short-circuit fashion. When a string matches the outcome of
466the SPF check, the condition succeeds. If none of the listed
467strings matches the outcome of the SPF check, the condition
468fails.
469
f413481d
TK
470Here is an example to fail forgery attempts from domains that
471publish SPF records:
ee161e8f
PH
472
473/* -----------------
f413481d
TK
474deny message = $sender_host_address is not allowed to send mail from ${if def:sender_address_domain {$sender_address_domain}{$sender_helo_name}}. \
475 Please see http://www.openspf.org/Why?scope=${if def:sender_address_domain {mfrom}{helo}};identity=${if def:sender_address_domain {$sender_address}{$sender_helo_name}};ip=$sender_host_address
ee161e8f
PH
476 spf = fail
477--------------------- */
478
479You can also give special treatment to specific domains:
480
481/* -----------------
482deny message = AOL sender, but not from AOL-approved relay.
483 sender_domains = aol.com
484 spf = fail:neutral
485--------------------- */
486
487Explanation: AOL publishes SPF records, but is liberal and
488still allows non-approved relays to send mail from aol.com.
489This will result in a "neutral" state, while mail from genuine
490AOL servers will result in "pass". The example above takes
491this into account and treats "neutral" like "fail", but only
492for aol.com. Please note that this violates the SPF draft.
493
494When the spf condition has run, it sets up several expansion
495variables.
496
497 $spf_header_comment
498 This contains a human-readable string describing the outcome
499 of the SPF check. You can add it to a custom header or use
500 it for logging purposes.
8ff3788c 501
ee161e8f 502 $spf_received
8fe685ad 503 This contains a complete Received-SPF: header that can be
ee161e8f
PH
504 added to the message. Please note that according to the SPF
505 draft, this header must be added at the top of the header
506 list. Please see section 10 on how you can do this.
8ff3788c 507
65a7d8c3
NM
508 Note: in case of "Best-guess" (see below), the convention is
509 to put this string in a header called X-SPF-Guess: instead.
510
ee161e8f
PH
511 $spf_result
512 This contains the outcome of the SPF check in string form,
513 one of pass, fail, softfail, none, neutral, err_perm or
514 err_temp.
8ff3788c 515
ee161e8f
PH
516 $spf_smtp_comment
517 This contains a string that can be used in a SMTP response
518 to the calling party. Useful for "fail".
8ff3788c 519
65a7d8c3
NM
520In addition to SPF, you can also perform checks for so-called
521"Best-guess". Strictly speaking, "Best-guess" is not standard
522SPF, but it is supported by the same framework that enables SPF
523capability. Refer to http://www.openspf.org/FAQ/Best_guess_record
524for a description of what it means.
525
526To access this feature, simply use the spf_guess condition in place
527of the spf one. For example:
528
529/* -----------------
530deny message = $sender_host_address doesn't look trustworthy to me
531 spf_guess = fail
532--------------------- */
533
534In case you decide to reject messages based on this check, you
535should note that although it uses the same framework, "Best-guess"
536is NOT SPF, and therefore you should not mention SPF at all in your
537reject message.
538
539When the spf_guess condition has run, it sets up the same expansion
540variables as when spf condition is run, described above.
541
d36a0501 542Additionally, since Best-guess is not standardized, you may redefine
65a7d8c3
NM
543what "Best-guess" means to you by redefining spf_guess variable in
544global config. For example, the following:
545
546/* -----------------
547spf_guess = v=spf1 a/16 mx/16 ptr ?all
548--------------------- */
549
550would relax host matching rules to a broader network range.
8ff3788c 551
ee161e8f 552
0b23848a 553SRS (Sender Rewriting Scheme) Support
ee161e8f
PH
554--------------------------------------------------------------
555
556Exiscan currently includes SRS support via Miles Wilton's
8ff3788c 557libsrs_alt library. The current version of the supported
ee161e8f
PH
558library is 0.5.
559
560In order to use SRS, you must get a copy of libsrs_alt from
561
562http://srs.mirtol.com/
563
564Unpack the tarball, then refer to MTAs/README.EXIM
565to proceed. You need to set
566
567EXPERIMENTAL_SRS=yes
568
569in your Local/Makefile.
570
571
0e1ccf44
PP
572DCC Support
573--------------------------------------------------------------
574
575*) Building exim
576
577In order to build exim with DCC support add
578
579EXPERIMENTAL_DCC=yes
580
581to your Makefile. (Re-)build/install exim. exim -d should show
582EXPERIMENTAL_DCC under "Support for".
583
584
585*) Configuration
586
587In the main section of exim.cf add at least
588 dccifd_address = /usr/local/dcc/var/dccifd
589or
590 dccifd_address = <ip> <port>
591
592In the DATA ACL you can use the new condition
593 dcc = *
594
595After that "$dcc_header" contains the X-DCC-Header.
596
d36a0501 597Return values are:
0e1ccf44
PP
598 fail for overall "R", "G" from dccifd
599 defer for overall "T" from dccifd
600 accept for overall "A", "S" from dccifd
601
602dcc = */defer_ok works as for spamd.
603
604The "$dcc_result" variable contains the overall result from DCC
605answer. There will an X-DCC: header added to the mail.
606
607Usually you'll use
608 defer !dcc = *
609to greylist with DCC.
610
611If you set, in the main section,
612 dcc_direct_add_header = true
613then the dcc header will be added "in deep" and if the spool
614file was already written it gets removed. This forces Exim to
615write it again if needed. This helps to get the DCC Header
616through to eg. SpamAssassin.
617
618If you want to pass even more headers in the middle of the
619DATA stage you can set
620 $acl_m_dcc_add_header
05c39afa 621to tell the DCC routines to add more information; eg, you might set
0e1ccf44
PP
622this to some results from ClamAV. Be careful. Header syntax is
623not checked and is added "as is".
624
05c39afa
JH
625In case you've troubles with sites sending the same queue items from several
626hosts and fail to get through greylisting you can use
627$acl_m_dcc_override_client_ip
628
629Setting $acl_m_dcc_override_client_ip to an IP address overrides the default
630of $sender_host_address. eg. use the following ACL in DATA stage:
631
632 warn set acl_m_dcc_override_client_ip = \
633 ${lookup{$sender_helo_name}nwildlsearch{/etc/mail/multipleip_sites}{$value}{}}
634 condition = ${if def:acl_m_dcc_override_client_ip}
635 log_message = dbg: acl_m_dcc_override_client_ip set to \
636 $acl_m_dcc_override_client_ip
637
638Then set something like
639# cat /etc/mail/multipleip_sites
640mout-xforward.gmx.net 82.165.159.12
641mout.gmx.net 212.227.15.16
642
643Use a reasonable IP. eg. one the sending cluster acutally uses.
0e1ccf44 644
1899bab2
TL
645DMARC Support
646--------------------------------------------------------------
647
648DMARC combines feedback from SPF, DKIM, and header From: in order
649to attempt to provide better indicators of the authenticity of an
650email. This document does not explain the fundamentals, you
651should read and understand how it works by visiting the website at
652http://www.dmarc.org/.
653
654DMARC support is added via the libopendmarc library. Visit:
655
656 http://sourceforge.net/projects/opendmarc/
657
658to obtain a copy, or find it in your favorite rpm package
659repository. If building from source, this description assumes
660that headers will be in /usr/local/include, and that the libraries
661are in /usr/local/lib.
662
6631. To compile Exim with DMARC support, you must first enable SPF.
664Please read the above section on enabling the EXPERIMENTAL_SPF
665feature. You must also have DKIM support, so you cannot set the
666DISABLE_DKIM feature. Once both of those conditions have been met
667you can enable DMARC in Local/Makefile:
668
669EXPERIMENTAL_DMARC=yes
670LDFLAGS += -lopendmarc
671# CFLAGS += -I/usr/local/include
672# LDFLAGS += -L/usr/local/lib
673
674The first line sets the feature to include the correct code, and
675the second line says to link the libopendmarc libraries into the
676exim binary. The commented out lines should be uncommented if you
677built opendmarc from source and installed in the default location.
678Adjust the paths if you installed them elsewhere, but you do not
679need to uncomment them if an rpm (or you) installed them in the
680package controlled locations (/usr/include and /usr/lib).
681
682
6832. Use the following global settings to configure DMARC:
684
685Required:
686dmarc_tld_file Defines the location of a text file of valid
687 top level domains the opendmarc library uses
688 during domain parsing. Maintained by Mozilla,
689 the most current version can be downloaded
690 from a link at http://publicsuffix.org/list/.
691
692Optional:
693dmarc_history_file Defines the location of a file to log results
694 of dmarc verification on inbound emails. The
695 contents are importable by the opendmarc tools
696 which will manage the data, send out DMARC
697 reports, and expire the data. Make sure the
698 directory of this file is writable by the user
699 exim runs as.
700
701dmarc_forensic_sender The email address to use when sending a
702 forensic report detailing alignment failures
703 if a sender domain's dmarc record specifies it
704 and you have configured Exim to send them.
705 Default: do-not-reply@$default_hostname
706
707
7083. By default, the DMARC processing will run for any remote,
709non-authenticated user. It makes sense to only verify DMARC
710status of messages coming from remote, untrusted sources. You can
711use standard conditions such as hosts, senders, etc, to decide that
712DMARC verification should *not* be performed for them and disable
713DMARC with a control setting:
714
12d0043d 715 control = dmarc_disable_verify
1899bab2
TL
716
717A DMARC record can also specify a "forensic address", which gives
718exim an email address to submit reports about failed alignment.
719Exim does not do this by default because in certain conditions it
720results in unintended information leakage (what lists a user might
721be subscribed to, etc). You must configure exim to submit forensic
722reports to the owner of the domain. If the DMARC record contains a
723forensic address and you specify the control statement below, then
724exim will send these forensic emails. It's also advised that you
725configure a dmarc_forensic_sender because the default sender address
726construction might be inadequate.
727
728 control = dmarc_forensic_enable
729
730(AGAIN: You can choose not to send these forensic reports by simply
731not putting the dmarc_forensic_enable control line at any point in
732your exim config. If you don't tell it to send them, it will not
733send them.)
734
735There are no options to either control. Both must appear before
736the DATA acl.
737
738
7394. You can now run DMARC checks in incoming SMTP by using the
740"dmarc_status" ACL condition in the DATA ACL. You are required to
741call the spf condition first in the ACLs, then the "dmarc_status"
742condition. Putting this condition in the ACLs is required in order
743for a DMARC check to actually occur. All of the variables are set
744up before the DATA ACL, but there is no actual DMARC check that
745occurs until a "dmarc_status" condition is encountered in the ACLs.
746
747The dmarc_status condition takes a list of strings on its
748right-hand side. These strings describe recommended action based
749on the DMARC check. To understand what the policy recommendations
750mean, refer to the DMARC website above. Valid strings are:
751
752 o accept The DMARC check passed and the library recommends
753 accepting the email.
754 o reject The DMARC check failed and the library recommends
755 rejecting the email.
756 o quarantine The DMARC check failed and the library recommends
757 keeping it for further inspection.
7a8678e6 758 o none The DMARC check passed and the library recommends
759 no specific action, neutral.
1899bab2
TL
760 o norecord No policy section in the DMARC record for this
761 sender domain.
762 o nofrom Unable to determine the domain of the sender.
7a8678e6 763 o temperror Library error or dns error.
05070e30 764 o off The DMARC check was disabled for this email.
1899bab2
TL
765
766You can prefix each string with an exclamation mark to invert its
767meaning, for example "!accept" will match all results but
768"accept". The string list is evaluated left-to-right in a
769short-circuit fashion. When a string matches the outcome of the
770DMARC check, the condition succeeds. If none of the listed
771strings matches the outcome of the DMARC check, the condition
772fails.
773
774Of course, you can also use any other lookup method that Exim
775supports, including LDAP, Postgres, MySQL, etc, as long as the
776result is a list of colon-separated strings;
777
778Several expansion variables are set before the DATA ACL is
779processed, and you can use them in this ACL. The following
780expansion variables are available:
781
782 o $dmarc_status
783 This is a one word status indicating what the DMARC library
784 thinks of the email.
785
786 o $dmarc_status_text
787 This is a slightly longer, human readable status.
788
789 o $dmarc_used_domain
790 This is the domain which DMARC used to look up the DMARC
791 policy record.
792
793 o $dmarc_ar_header
794 This is the entire Authentication-Results header which you can
795 add using an add_header modifier.
796
797
7985. How to enable DMARC advanced operation:
799By default, Exim's DMARC configuration is intended to be
800non-intrusive and conservative. To facilitate this, Exim will not
801create any type of logging files without explicit configuration by
802you, the admin. Nor will Exim send out any emails/reports about
803DMARC issues without explicit configuration by you, the admin (other
804than typical bounce messages that may come about due to ACL
805processing or failure delivery issues).
806
807In order to log statistics suitable to be imported by the opendmarc
808tools, you need to:
809a. Configure the global setting dmarc_history_file.
810b. Configure cron jobs to call the appropriate opendmarc history
811 import scripts and truncating the dmarc_history_file.
812
813In order to send forensic reports, you need to:
814a. Configure the global setting dmarc_forensic_sender.
815b. Configure, somewhere before the DATA ACL, the control option to
816 enable sending DMARC forensic reports.
817
818
8196. Example usage:
820(RCPT ACL)
821 warn domains = +local_domains
822 hosts = +local_hosts
12d0043d 823 control = dmarc_disable_verify
1899bab2
TL
824
825 warn !domains = +screwed_up_dmarc_records
826 control = dmarc_enable_forensic
827
828(DATA ACL)
829 warn dmarc_status = accept : none : off
830 !authenticated = *
831 log_message = DMARC DEBUG: $dmarc_status $dmarc_used_domain
832 add_header = $dmarc_ar_header
833
834 warn dmarc_status = !accept
835 !authenticated = *
836 log_message = DMARC DEBUG: '$dmarc_status' for $dmarc_used_domain
837
838 warn dmarc_status = quarantine
839 !authenticated = *
840 set $acl_m_quarantine = 1
841 # Do something in a transport with this flag variable
842
843 deny dmarc_status = reject
844 !authenticated = *
845 message = Message from $domain_used_domain failed sender's DMARC policy, REJECT
846
847
848
d68218c7
JH
849Transport post-delivery actions
850--------------------------------------------------------------
851
852An arbitrary per-transport string can be expanded on successful delivery,
853and (for SMTP transports) a second string on deferrals caused by a host error.
854This feature may be used, for example, to write exim internal log information
855(not available otherwise) into a database.
856
857In order to use the feature, you must set
858
859EXPERIMENTAL_TPDA=yes
860
861in your Local/Makefile
862
863and define the expandable strings in the runtime config file, to
864be executed at end of delivery.
865
866Additionally, there are 6 more variables, available at end of
867delivery:
868
869tpda_delivery_ip IP of host, which has accepted delivery
870tpda_delivery_port Port of remote host which has accepted delivery
871tpda_delivery_fqdn FQDN of host, which has accepted delivery
872tpda_delivery_local_part local part of address being delivered
873tpda_delivery_domain domain part of address being delivered
874tpda_delivery_confirmation SMTP confirmation message
875
876In case of a deferral caused by a host-error:
877tpda_defer_errno Error number
878tpda_defer_errstr Error string possibly containing more details
879
880The $router_name and $transport_name variables are also usable.
881
882
883To take action after successful deliveries, set the following option
884on any transport of interest.
885
886tpda_delivery_action
887
888An example might look like:
889
890tpda_delivery_action = \
891${lookup pgsql {SELECT * FROM record_Delivery( \
892 '${quote_pgsql:$sender_address_domain}',\
893 '${quote_pgsql:${lc:$sender_address_local_part}}', \
894 '${quote_pgsql:$tpda_delivery_domain}', \
895 '${quote_pgsql:${lc:$tpda_delivery_local_part}}', \
896 '${quote_pgsql:$tpda_delivery_ip}', \
897 '${quote_pgsql:${lc:$tpda_delivery_fqdn}}', \
898 '${quote_pgsql:$message_exim_id}')}}
899
900The string is expanded after the delivery completes and any
901side-effects will happen. The result is then discarded.
902Note that for complex operations an ACL expansion can be used.
903
904
905In order to log host deferrals, add the following option to an SMTP
906transport:
907
908tpda_host_defer_action
909
910This is a private option of the SMTP transport. It is intended to
911log failures of remote hosts. It is executed only when exim has
912attempted to deliver a message to a remote host and failed due to
913an error which doesn't seem to be related to the individual
914message, sender, or recipient address.
7c261cc6 915See section 47.2 of the exim documentation for more details on how
d68218c7
JH
916this is determined.
917
918Example:
919
920tpda_host_defer_action = \
921${lookup mysql {insert into delivlog set \
922 msgid = '${quote_mysql:$message_exim_id}', \
923 senderlp = '${quote_mysql:${lc:$sender_address_local_part}}', \
924 senderdom = '${quote_mysql:$sender_address_domain}', \
925 delivlp = '${quote_mysql:${lc:$tpda_delivery_local_part}}', \
926 delivdom = '${quote_mysql:$tpda_delivery_domain}', \
927 delivip = '${quote_mysql:$tpda_delivery_ip}', \
928 delivport = '${quote_mysql:$tpda_delivery_port}', \
929 delivfqdn = '${quote_mysql:$tpda_delivery_fqdn}', \
930 deliverrno = '${quote_mysql:$tpda_defer_errno}', \
931 deliverrstr = '${quote_mysql:$tpda_defer_errstr}' \
932 }}
1899bab2 933
9bdd29ad
TL
934
935Redis Lookup
936--------------------------------------------------------------
937
938Redis is open source advanced key-value data store. This document
939does not explain the fundamentals, you should read and understand how
940it works by visiting the website at http://www.redis.io/.
941
942Redis lookup support is added via the hiredis library. Visit:
943
944 https://github.com/redis/hiredis
945
946to obtain a copy, or find it in your operating systems package repository.
947If building from source, this description assumes that headers will be in
948/usr/local/include, and that the libraries are in /usr/local/lib.
949
9501. In order to build exim with Redis lookup support add
951
952EXPERIMENTAL_REDIS=yes
953
954to your Local/Makefile. (Re-)build/install exim. exim -d should show
955Experimental_Redis in the line "Support for:".
956
957EXPERIMENTAL_REDIS=yes
958LDFLAGS += -lhiredis
959# CFLAGS += -I/usr/local/include
960# LDFLAGS += -L/usr/local/lib
961
962The first line sets the feature to include the correct code, and
963the second line says to link the hiredis libraries into the
964exim binary. The commented out lines should be uncommented if you
965built hiredis from source and installed in the default location.
966Adjust the paths if you installed them elsewhere, but you do not
967need to uncomment them if an rpm (or you) installed them in the
968package controlled locations (/usr/include and /usr/lib).
969
970
9712. Use the following global settings to configure Redis lookup support:
972
973Required:
974redis_servers This option provides a list of Redis servers
975 and associated connection data, to be used in
976 conjunction with redis lookups. The option is
977 only available if Exim is configured with Redis
978 support.
979
980For example:
981
982redis_servers = 127.0.0.1/10/ - using database 10 with no password
983redis_servers = 127.0.0.1//password - to make use of the default database of 0 with a password
984redis_servers = 127.0.0.1// - for default database of 0 with no password
985
9863. Once you have the Redis servers defined you can then make use of the
987experimental Redis lookup by specifying ${lookup redis{}} in a lookup query.
988
9894. Example usage:
990
991(Host List)
992hostlist relay_from_ips = <\n ${lookup redis{SMEMBERS relay_from_ips}}
993
994Where relay_from_ips is a Redis set which contains entries such as "192.168.0.0/24" "10.0.0.0/8" and so on.
995The result set is returned as
996192.168.0.0/24
99710.0.0.0/8
998..
999.
1000
1001(Domain list)
1002domainlist virtual_domains = ${lookup redis {HGET $domain domain}}
1003
1004Where $domain is a hash which includes the key 'domain' and the value '$domain'.
1005
1006(Adding or updating an existing key)
1007set acl_c_spammer = ${if eq{${lookup redis{SPAMMER_SET}}}{OK}}
1008
1009Where SPAMMER_SET is a macro and it is defined as
1010
1011"SET SPAMMER <some_value>"
1012
1013(Getting a value from Redis)
1014
1015set acl_c_spam_host = ${lookup redis{GET...}}
1016
1017
a3c86431
TL
1018Proxy Protocol Support
1019--------------------------------------------------------------
1020
1021Exim now has Experimental "Proxy Protocol" support. It was built on
1022specifications from:
1023http://haproxy.1wt.eu/download/1.5/doc/proxy-protocol.txt
1024
1025The purpose of this function is so that an application load balancer,
1026such as HAProxy, can sit in front of several Exim servers and Exim
1027will log the IP that is connecting to the proxy server instead of
1028the IP of the proxy server when it connects to Exim. It resets the
1029$sender_address_host and $sender_address_port to the IP:port of the
1030connection to the proxy. It also re-queries the DNS information for
1031this new IP address so that the original sender's hostname and IP
1032get logged in the Exim logfile. There is no logging if a host passes or
1033fails Proxy Protocol negotiation, but it can easily be determined and
1034recorded in an ACL (example is below).
1035
10361. To compile Exim with Proxy Protocol support, put this in
1037Local/Makefile:
1038
1039EXPERIMENTAL_PROXY=yes
1040
10412. Global configuration settings:
1042
1043proxy_required_hosts = HOSTLIST
1044
1045The proxy_required_hosts option will require any IP in that hostlist
1046to use Proxy Protocol. The specification of Proxy Protocol is very
1047strict, and if proxy negotiation fails, Exim will not allow any SMTP
1048command other than QUIT. (See end of this section for an example.)
1049The option is expanded when used, so it can be a hostlist as well as
1050string of IP addresses. Since it is expanded, specifying an alternate
1051separator is supported for ease of use with IPv6 addresses.
1052
1053To log the IP of the proxy in the incoming logline, add:
1054 log_selector = +proxy
1055
1056A default incoming logline (wrapped for appearance) will look like this:
1057
1058 2013-11-04 09:25:06 1VdNti-0001OY-1V <= me@example.net
1059 H=mail.example.net [1.2.3.4] P=esmtp S=433
1060
1061With the log selector enabled, an email that was proxied through a
1062Proxy Protocol server at 192.168.1.2 will look like this:
1063
1064 2013-11-04 09:25:06 1VdNti-0001OY-1V <= me@example.net
1065 H=mail.example.net [1.2.3.4] P=esmtp PRX=192.168.1.2 S=433
1066
10673. In the ACL's the following expansion variables are available.
1068
a3bddaa8
TL
1069proxy_host_address The src IP of the proxy server making the connection
1070proxy_host_port The src port the proxy server is using
1071proxy_session Boolean, yes/no, the connected host is required to use
1072 Proxy Protocol.
a3c86431
TL
1073
1074There is no expansion for a failed proxy session, however you can detect
1075it by checking if $proxy_session is true but $proxy_host is empty. As
1076an example, in my connect ACL, I have:
1077
1078 warn condition = ${if and{ {bool{$proxy_session}} \
a3bddaa8 1079 {eq{$proxy_host_address}{}} } }
a3c86431
TL
1080 log_message = Failed required proxy protocol negotiation \
1081 from $sender_host_name [$sender_host_address]
1082
1083 warn condition = ${if and{ {bool{$proxy_session}} \
a3bddaa8 1084 {!eq{$proxy_host_address}{}} } }
a3c86431 1085 # But don't log health probes from the proxy itself
a3bddaa8 1086 condition = ${if eq{$proxy_host_address}{$sender_host_address} \
a3c86431
TL
1087 {false}{true}}
1088 log_message = Successfully proxied from $sender_host_name \
1089 [$sender_host_address] through proxy protocol \
a3bddaa8 1090 host $proxy_host_address
a3c86431
TL
1091
10924. Runtime issues to be aware of:
1093 - Since the real connections are all coming from your proxy, and the
1094 per host connection tracking is done before Proxy Protocol is
1095 evaluated, smtp_accept_max_per_host must be set high enough to
1096 handle all of the parallel volume you expect per inbound proxy.
1097 - The proxy has 3 seconds (hard-coded in the source code) to send the
1098 required Proxy Protocol header after it connects. If it does not,
1099 the response to any commands will be:
1100 "503 Command refused, required Proxy negotiation failed"
1101 - If the incoming connection is configured in Exim to be a Proxy
1102 Protocol host, but the proxy is not sending the header, the banner
1103 does not get sent until the timeout occurs. If the sending host
1104 sent any input (before the banner), this causes a standard Exim
1105 synchronization error (i.e. trying to pipeline before PIPELINING
1106 was advertised).
1107 - This is not advised, but is mentioned for completeness if you have
1108 a specific internal configuration that you want this: If the Exim
1109 server only has an internal IP address and no other machines in your
1110 organization will connect to it to try to send email, you may
1111 simply set the hostlist to "*", however, this will prevent local
1112 mail programs from working because that would require mail from
1113 localhost to use Proxy Protocol. Again, not advised!
1114
11155. Example of a refused connection because the Proxy Protocol header was
1116not sent from a host configured to use Proxy Protocol. In the example,
1117the 3 second timeout occurred (when a Proxy Protocol banner should have
1118been sent), the banner was displayed to the user, but all commands are
1119rejected except for QUIT:
1120
1121# nc mail.example.net 25
1122220-mail.example.net, ESMTP Exim 4.82+proxy, Mon, 04 Nov 2013 10:45:59
1123220 -0800 RFC's enforced
1124EHLO localhost
1125503 Command refused, required Proxy negotiation failed
1126QUIT
1127221 mail.example.net closing connection
1128
1129
9bdd29ad 1130
ee161e8f
PH
1131--------------------------------------------------------------
1132End of file
1133--------------------------------------------------------------