Eliminate one foolish way to break the build
[exim.git] / doc / doc-txt / experimental-spec.txt
CommitLineData
7bafa7d9
TK
1From time to time, experimental features may be added to Exim.
2While a feature is experimental, there will be a build-time
3option whose name starts "EXPERIMENTAL_" that must be set in
4order to include the feature. This file contains information
d36a0501
PP
5about experimental features, all of which are unstable and
6liable to incompatible change.
ee161e8f
PH
7
8
3f7eeb86
PP
9OCSP Stapling support
10--------------------------------------------------------------
11
d36a0501 12X.509 PKI certificates expire and can be revoked; to handle this, the
3f7eeb86
PP
13clients need some way to determine if a particular certificate, from a
14particular Certificate Authority (CA), is still valid. There are three
15main ways to do so.
16
17The simplest way is to serve up a Certificate Revocation List (CRL) with
18an ordinary web-server, regenerating the CRL before it expires. The
19downside is that clients have to periodically re-download a potentially
20huge file from every certificate authority it knows of.
21
22The way with most moving parts at query time is Online Certificate
23Status Protocol (OCSP), where the client verifies the certificate
24against an OCSP server run by the CA. This lets the CA track all
25usage of the certs. This requires running software with access to the
26private key of the CA, to sign the responses to the OCSP queries. OCSP
27is based on HTTP and can be proxied accordingly.
28
29The only widespread OCSP server implementation (known to this writer)
30comes as part of OpenSSL and aborts on an invalid request, such as
31connecting to the port and then disconnecting. This requires
32re-entering the passphrase each time some random client does this.
33
34The third way is OCSP Stapling; in this, the server using a certificate
35issued by the CA periodically requests an OCSP proof of validity from
36the OCSP server, then serves it up inline as part of the TLS
37negotiation. This approach adds no extra round trips, does not let the
38CA track users, scales well with number of certs issued by the CA and is
39resilient to temporary OCSP server failures, as long as the server
40starts retrying to fetch an OCSP proof some time before its current
41proof expires. The downside is that it requires server support.
42
43If Exim is built with EXPERIMENTAL_OCSP and it was built with OpenSSL,
2b4a568d
JH
44or with GnuTLS 3.1.3 or later, then it gains a new global option:
45"tls_ocsp_file".
3f7eeb86
PP
46
47The file specified therein is expected to be in DER format, and contain
48an OCSP proof. Exim will serve it as part of the TLS handshake. This
49option will be re-expanded for SNI, if the tls_certificate option
50contains $tls_sni, as per other TLS options.
51
52Exim does not at this time implement any support for fetching a new OCSP
53proof. The burden is on the administrator to handle this, outside of
54Exim. The file specified should be replaced atomically, so that the
55contents are always valid. Exim will expand the "tls_ocsp_file" option
56on each connection, so a new file will be handled transparently on the
57next connection.
58
44662487
JH
59Under OpenSSL Exim will check for a valid next update timestamp in the
60OCSP proof; if not present, or if the proof has expired, it will be
61ignored.
62
63Also, given EXPERIMENTAL_OCSP, the smtp transport gains two options:
64- "hosts_require_ocsp"; a host-list for which an OCSP Stapling
65is requested and required for the connection to proceed. The default
66value is empty.
67- "hosts_request_ocsp"; a host-list for which (additionally) an OCSP
68Stapling is requested (but not necessarily verified). The default
69value is "*" meaning that requests are made unless configured
70otherwise.
71
72The host(s) should also be in "hosts_require_tls", and
73"tls_verify_certificates" configured for the transport.
f5d78688 74
880496ef
JH
75For the client to be able to verify the stapled OCSP the server must
76also supply, in its stapled information, any intermediate
77certificates for the chain leading to the OCSP proof from the signer
78of the server certificate. There may be zero or one such. These
79intermediate certificates should be added to the server OCSP stapling
80file (named by tls_ocsp_file).
81
2b4a568d
JH
82Note that the proof only covers the terminal server certificate,
83not any of the chain from CA to it.
84
3f7eeb86
PP
85At this point in time, we're gathering feedback on use, to determine if
86it's worth adding complexity to the Exim daemon to periodically re-fetch
f5d78688 87OCSP files and somehow handling multiple files.
3f7eeb86 88
880496ef
JH
89 A helper script "ocsp_fetch.pl" for fetching a proof from a CA
90 OCSP server is supplied. The server URL may be included in the
91 server certificate, if the CA is helpful.
92
2b4a568d
JH
93 One failure mode seen was the OCSP Signer cert expiring before the end
94 of validity of the OCSP proof. The checking done by Exim/OpenSSL
880496ef
JH
95 noted this as invalid overall, but the re-fetch script did not.
96
3f7eeb86
PP
97
98
99
0b23848a 100Brightmail AntiSpam (BMI) suppport
ee161e8f
PH
101--------------------------------------------------------------
102
103Brightmail AntiSpam is a commercial package. Please see
104http://www.brightmail.com for more information on
105the product. For the sake of clarity, we'll refer to it as
106"BMI" from now on.
107
108
1090) BMI concept and implementation overview
110
111In contrast to how spam-scanning with SpamAssassin is
112implemented in exiscan-acl, BMI is more suited for per
113-recipient scanning of messages. However, each messages is
114scanned only once, but multiple "verdicts" for multiple
115recipients can be returned from the BMI server. The exiscan
116implementation passes the message to the BMI server just
117before accepting it. It then adds the retrieved verdicts to
118the messages header file in the spool. These verdicts can then
119be queried in routers, where operation is per-recipient
120instead of per-message. To use BMI, you need to take the
121following steps:
122
123 1) Compile Exim with BMI support
3ec3e3bb 124 2) Set up main BMI options (top section of Exim config file)
ee161e8f
PH
125 3) Set up ACL control statement (ACL section of the config
126 file)
127 4) Set up your routers to use BMI verdicts (routers section
128 of the config file).
129 5) (Optional) Set up per-recipient opt-in information.
130
8ff3788c 131These four steps are explained in more details below.
ee161e8f
PH
132
1331) Adding support for BMI at compile time
134
135 To compile with BMI support, you need to link Exim against
136 the Brighmail client SDK, consisting of a library
137 (libbmiclient_single.so) and a header file (bmi_api.h).
138 You'll also need to explicitly set a flag in the Makefile to
139 include BMI support in the Exim binary. Both can be achieved
140 with these lines in Local/Makefile:
141
142 EXPERIMENTAL_BRIGHTMAIL=yes
47bbda99 143 CFLAGS=-I/path/to/the/dir/with/the/includefile
ee161e8f 144 EXTRALIBS_EXIM=-L/path/to/the/dir/with/the/library -lbmiclient_single
8ff3788c 145
ee161e8f
PH
146 If you use other CFLAGS or EXTRALIBS_EXIM settings then
147 merge the content of these lines with them.
148
7c0c8547 149 Note for BMI6.x users: You'll also have to add -lxml2_single
ee161e8f
PH
150 to the EXTRALIBS_EXIM line. Users of 5.5x do not need to do
151 this.
8ff3788c 152
ee161e8f
PH
153 You should also include the location of
154 libbmiclient_single.so in your dynamic linker configuration
155 file (usually /etc/ld.so.conf) and run "ldconfig"
156 afterwards, or else the produced Exim binary will not be
157 able to find the library file.
158
159
3ec3e3bb 1602) Setting up BMI support in the Exim main configuration
ee161e8f 161
3ec3e3bb 162 To enable BMI support in the main Exim configuration, you
ee161e8f
PH
163 should set the path to the main BMI configuration file with
164 the "bmi_config_file" option, like this:
8ff3788c 165
ee161e8f 166 bmi_config_file = /opt/brightmail/etc/brightmail.cfg
8ff3788c 167
3ec3e3bb 168 This must go into section 1 of Exim's configuration file (You
ee161e8f
PH
169 can put it right on top). If you omit this option, it
170 defaults to /opt/brightmail/etc/brightmail.cfg.
171
172 Note for BMI6.x users: This file is in XML format in V6.xx
173 and its name is /opt/brightmail/etc/bmiconfig.xml. So BMI
174 6.x users MUST set the bmi_config_file option.
8ff3788c 175
ee161e8f
PH
176
1773) Set up ACL control statement
178
179 To optimize performance, it makes sense only to process
180 messages coming from remote, untrusted sources with the BMI
181 server. To set up a messages for processing by the BMI
182 server, you MUST set the "bmi_run" control statement in any
183 ACL for an incoming message. You will typically do this in
184 an "accept" block in the "acl_check_rcpt" ACL. You should
185 use the "accept" block(s) that accept messages from remote
186 servers for your own domain(s). Here is an example that uses
3ec3e3bb 187 the "accept" blocks from Exim's default configuration file:
8ff3788c 188
ee161e8f
PH
189
190 accept domains = +local_domains
191 endpass
192 verify = recipient
193 control = bmi_run
194
195 accept domains = +relay_to_domains
196 endpass
197 verify = recipient
198 control = bmi_run
8ff3788c 199
ee161e8f
PH
200 If bmi_run is not set in any ACL during reception of the
201 message, it will NOT be passed to the BMI server.
202
203
2044) Setting up routers to use BMI verdicts
205
206 When a message has been run through the BMI server, one or
207 more "verdicts" are present. Different recipients can have
208 different verdicts. Each recipient is treated individually
209 during routing, so you can query the verdicts by recipient
3ec3e3bb 210 at that stage. From Exim's view, a verdict can have the
ee161e8f 211 following outcomes:
8ff3788c 212
ee161e8f
PH
213 o deliver the message normally
214 o deliver the message to an alternate location
215 o do not deliver the message
8ff3788c 216
ee161e8f
PH
217 To query the verdict for a recipient, the implementation
218 offers the following tools:
8ff3788c
TK
219
220
ee161e8f
PH
221 - Boolean router preconditions. These can be used in any
222 router. For a simple implementation of BMI, these may be
223 all that you need. The following preconditions are
224 available:
8ff3788c 225
ee161e8f 226 o bmi_deliver_default
8ff3788c 227
ee161e8f
PH
228 This precondition is TRUE if the verdict for the
229 recipient is to deliver the message normally. If the
230 message has not been processed by the BMI server, this
231 variable defaults to TRUE.
8ff3788c 232
ee161e8f 233 o bmi_deliver_alternate
8ff3788c 234
ee161e8f
PH
235 This precondition is TRUE if the verdict for the
236 recipient is to deliver the message to an alternate
237 location. You can get the location string from the
238 $bmi_alt_location expansion variable if you need it. See
239 further below. If the message has not been processed by
240 the BMI server, this variable defaults to FALSE.
8ff3788c 241
ee161e8f 242 o bmi_dont_deliver
8ff3788c 243
ee161e8f
PH
244 This precondition is TRUE if the verdict for the
245 recipient is NOT to deliver the message to the
246 recipient. You will typically use this precondition in a
247 top-level blackhole router, like this:
8ff3788c 248
ee161e8f
PH
249 # don't deliver messages handled by the BMI server
250 bmi_blackhole:
251 driver = redirect
252 bmi_dont_deliver
253 data = :blackhole:
8ff3788c 254
ee161e8f
PH
255 This router should be on top of all others, so messages
256 that should not be delivered do not reach other routers
257 at all. If the message has not been processed by
258 the BMI server, this variable defaults to FALSE.
8ff3788c
TK
259
260
ee161e8f
PH
261 - A list router precondition to query if rules "fired" on
262 the message for the recipient. Its name is "bmi_rule". You
263 use it by passing it a colon-separated list of rule
264 numbers. You can use this condition to route messages that
265 matched specific rules. Here is an example:
8ff3788c 266
ee161e8f
PH
267 # special router for BMI rule #5, #8 and #11
268 bmi_rule_redirect:
269 driver = redirect
270 bmi_rule = 5:8:11
271 data = postmaster@mydomain.com
8ff3788c
TK
272
273
ee161e8f
PH
274 - Expansion variables. Several expansion variables are set
275 during routing. You can use them in custom router
276 conditions, for example. The following variables are
277 available:
8ff3788c 278
ee161e8f 279 o $bmi_base64_verdict
8ff3788c 280
ee161e8f
PH
281 This variable will contain the BASE64 encoded verdict
282 for the recipient being routed. You can use it to add a
283 header to messages for tracking purposes, for example:
8ff3788c 284
ee161e8f
PH
285 localuser:
286 driver = accept
287 check_local_user
288 headers_add = X-Brightmail-Verdict: $bmi_base64_verdict
289 transport = local_delivery
8ff3788c 290
ee161e8f
PH
291 If there is no verdict available for the recipient being
292 routed, this variable contains the empty string.
8ff3788c 293
ee161e8f 294 o $bmi_base64_tracker_verdict
8ff3788c 295
ee161e8f
PH
296 This variable will contain a BASE64 encoded subset of
297 the verdict information concerning the "rules" that
298 fired on the message. You can add this string to a
299 header, commonly named "X-Brightmail-Tracker". Example:
8ff3788c 300
ee161e8f
PH
301 localuser:
302 driver = accept
303 check_local_user
304 headers_add = X-Brightmail-Tracker: $bmi_base64_tracker_verdict
305 transport = local_delivery
8ff3788c 306
ee161e8f
PH
307 If there is no verdict available for the recipient being
308 routed, this variable contains the empty string.
8ff3788c 309
ee161e8f 310 o $bmi_alt_location
8ff3788c 311
ee161e8f
PH
312 If the verdict is to redirect the message to an
313 alternate location, this variable will contain the
314 alternate location string returned by the BMI server. In
315 its default configuration, this is a header-like string
316 that can be added to the message with "headers_add". If
317 there is no verdict available for the recipient being
318 routed, or if the message is to be delivered normally,
319 this variable contains the empty string.
8ff3788c 320
ee161e8f 321 o $bmi_deliver
8ff3788c 322
ee161e8f
PH
323 This is an additional integer variable that can be used
324 to query if the message should be delivered at all. You
325 should use router preconditions instead if possible.
8ff3788c 326
ee161e8f
PH
327 $bmi_deliver is '0': the message should NOT be delivered.
328 $bmi_deliver is '1': the message should be delivered.
8ff3788c
TK
329
330
ee161e8f
PH
331 IMPORTANT NOTE: Verdict inheritance.
332 The message is passed to the BMI server during message
333 reception, using the target addresses from the RCPT TO:
334 commands in the SMTP transaction. If recipients get expanded
335 or re-written (for example by aliasing), the new address(es)
336 inherit the verdict from the original address. This means
337 that verdicts also apply to all "child" addresses generated
338 from top-level addresses that were sent to the BMI server.
8ff3788c
TK
339
340
ee161e8f
PH
3415) Using per-recipient opt-in information (Optional)
342
343 The BMI server features multiple scanning "profiles" for
344 individual recipients. These are usually stored in a LDAP
345 server and are queried by the BMI server itself. However,
346 you can also pass opt-in data for each recipient from the
347 MTA to the BMI server. This is particularly useful if you
3ec3e3bb 348 already look up recipient data in Exim anyway (which can
ee161e8f
PH
349 also be stored in a SQL database or other source). This
350 implementation enables you to pass opt-in data to the BMI
351 server in the RCPT ACL. This works by setting the
352 'bmi_optin' modifier in a block of that ACL. If should be
353 set to a list of comma-separated strings that identify the
354 features which the BMI server should use for that particular
355 recipient. Ideally, you would use the 'bmi_optin' modifier
356 in the same ACL block where you set the 'bmi_run' control
357 flag. Here is an example that will pull opt-in data for each
358 recipient from a flat file called
359 '/etc/exim/bmi_optin_data'.
8ff3788c 360
ee161e8f 361 The file format:
8ff3788c 362
ee161e8f
PH
363 user1@mydomain.com: <OPTIN STRING1>:<OPTIN STRING2>
364 user2@thatdomain.com: <OPTIN STRING3>
8ff3788c
TK
365
366
ee161e8f 367 The example:
8ff3788c 368
ee161e8f
PH
369 accept domains = +relay_to_domains
370 endpass
371 verify = recipient
372 bmi_optin = ${lookup{$local_part@$domain}lsearch{/etc/exim/bmi_optin_data}}
8ff3788c
TK
373 control = bmi_run
374
ee161e8f 375 Of course, you can also use any other lookup method that
3ec3e3bb 376 Exim supports, including LDAP, Postgres, MySQL, Oracle etc.,
ee161e8f
PH
377 as long as the result is a list of colon-separated opt-in
378 strings.
8ff3788c 379
ee161e8f
PH
380 For a list of available opt-in strings, please contact your
381 Brightmail representative.
ee161e8f 382
8ff3788c
TK
383
384
385
0b23848a 386Sender Policy Framework (SPF) support
ee161e8f
PH
387--------------------------------------------------------------
388
f413481d 389To learn more about SPF, visit http://www.openspf.org. This
ee161e8f
PH
390document does not explain the SPF fundamentals, you should
391read and understand the implications of deploying SPF on your
392system before doing so.
393
8ff3788c 394SPF support is added via the libspf2 library. Visit
ee161e8f
PH
395
396 http://www.libspf2.org/
8ff3788c 397
ee161e8f
PH
398to obtain a copy, then compile and install it. By default,
399this will put headers in /usr/local/include and the static
400library in /usr/local/lib.
401
3ec3e3bb 402To compile Exim with SPF support, set these additional flags in
ee161e8f
PH
403Local/Makefile:
404
405EXPERIMENTAL_SPF=yes
406CFLAGS=-DSPF -I/usr/local/include
407EXTRALIBS_EXIM=-L/usr/local/lib -lspf2
408
409This assumes that the libspf2 files are installed in
410their default locations.
411
412You can now run SPF checks in incoming SMTP by using the "spf"
413ACL condition in either the MAIL, RCPT or DATA ACLs. When
d36a0501 414using it in the RCPT ACL, you can make the checks dependent on
ee161e8f
PH
415the RCPT address (or domain), so you can check SPF records
416only for certain target domains. This gives you the
417possibility to opt-out certain customers that do not want
418their mail to be subject to SPF checking.
419
420The spf condition takes a list of strings on its right-hand
421side. These strings describe the outcome of the SPF check for
422which the spf condition should succeed. Valid strings are:
423
424 o pass The SPF check passed, the sending host
425 is positively verified by SPF.
426 o fail The SPF check failed, the sending host
427 is NOT allowed to send mail for the domain
428 in the envelope-from address.
429 o softfail The SPF check failed, but the queried
430 domain can't absolutely confirm that this
431 is a forgery.
432 o none The queried domain does not publish SPF
433 records.
434 o neutral The SPF check returned a "neutral" state.
435 This means the queried domain has published
436 a SPF record, but wants to allow outside
437 servers to send mail under its domain as well.
8ddef691
TL
438 This should be treated like "none".
439 o permerror This indicates a syntax error in the SPF
440 record of the queried domain. You may deny
441 messages when this occurs. (Changed in 4.83)
442 o temperror This indicates a temporary error during all
3ec3e3bb 443 processing, including Exim's SPF processing.
ee161e8f 444 You may defer messages when this occurs.
8ddef691 445 (Changed in 4.83)
982650ec
TL
446 o err_temp Same as permerror, deprecated in 4.83, will be
447 removed in a future release.
448 o err_perm Same as temperror, deprecated in 4.83, will be
449 removed in a future release.
8ff3788c 450
ee161e8f 451You can prefix each string with an exclamation mark to invert
982650ec 452its meaning, for example "!fail" will match all results but
ee161e8f
PH
453"fail". The string list is evaluated left-to-right, in a
454short-circuit fashion. When a string matches the outcome of
455the SPF check, the condition succeeds. If none of the listed
456strings matches the outcome of the SPF check, the condition
457fails.
458
f413481d
TK
459Here is an example to fail forgery attempts from domains that
460publish SPF records:
ee161e8f
PH
461
462/* -----------------
f413481d
TK
463deny message = $sender_host_address is not allowed to send mail from ${if def:sender_address_domain {$sender_address_domain}{$sender_helo_name}}. \
464 Please see http://www.openspf.org/Why?scope=${if def:sender_address_domain {mfrom}{helo}};identity=${if def:sender_address_domain {$sender_address}{$sender_helo_name}};ip=$sender_host_address
ee161e8f
PH
465 spf = fail
466--------------------- */
467
468You can also give special treatment to specific domains:
469
470/* -----------------
471deny message = AOL sender, but not from AOL-approved relay.
472 sender_domains = aol.com
473 spf = fail:neutral
474--------------------- */
475
476Explanation: AOL publishes SPF records, but is liberal and
477still allows non-approved relays to send mail from aol.com.
478This will result in a "neutral" state, while mail from genuine
479AOL servers will result in "pass". The example above takes
480this into account and treats "neutral" like "fail", but only
481for aol.com. Please note that this violates the SPF draft.
482
483When the spf condition has run, it sets up several expansion
484variables.
485
486 $spf_header_comment
487 This contains a human-readable string describing the outcome
488 of the SPF check. You can add it to a custom header or use
489 it for logging purposes.
8ff3788c 490
ee161e8f 491 $spf_received
8fe685ad 492 This contains a complete Received-SPF: header that can be
ee161e8f
PH
493 added to the message. Please note that according to the SPF
494 draft, this header must be added at the top of the header
495 list. Please see section 10 on how you can do this.
8ff3788c 496
65a7d8c3
NM
497 Note: in case of "Best-guess" (see below), the convention is
498 to put this string in a header called X-SPF-Guess: instead.
499
ee161e8f
PH
500 $spf_result
501 This contains the outcome of the SPF check in string form,
8ddef691
TL
502 one of pass, fail, softfail, none, neutral, permerror or
503 temperror.
8ff3788c 504
ee161e8f
PH
505 $spf_smtp_comment
506 This contains a string that can be used in a SMTP response
507 to the calling party. Useful for "fail".
8ff3788c 508
65a7d8c3
NM
509In addition to SPF, you can also perform checks for so-called
510"Best-guess". Strictly speaking, "Best-guess" is not standard
511SPF, but it is supported by the same framework that enables SPF
512capability. Refer to http://www.openspf.org/FAQ/Best_guess_record
513for a description of what it means.
514
515To access this feature, simply use the spf_guess condition in place
516of the spf one. For example:
517
518/* -----------------
519deny message = $sender_host_address doesn't look trustworthy to me
520 spf_guess = fail
521--------------------- */
522
523In case you decide to reject messages based on this check, you
524should note that although it uses the same framework, "Best-guess"
525is NOT SPF, and therefore you should not mention SPF at all in your
526reject message.
527
528When the spf_guess condition has run, it sets up the same expansion
529variables as when spf condition is run, described above.
530
d36a0501 531Additionally, since Best-guess is not standardized, you may redefine
65a7d8c3
NM
532what "Best-guess" means to you by redefining spf_guess variable in
533global config. For example, the following:
534
535/* -----------------
536spf_guess = v=spf1 a/16 mx/16 ptr ?all
537--------------------- */
538
539would relax host matching rules to a broader network range.
8ff3788c 540
ee161e8f 541
0b23848a 542SRS (Sender Rewriting Scheme) Support
ee161e8f
PH
543--------------------------------------------------------------
544
545Exiscan currently includes SRS support via Miles Wilton's
8ff3788c 546libsrs_alt library. The current version of the supported
ee161e8f
PH
547library is 0.5.
548
549In order to use SRS, you must get a copy of libsrs_alt from
550
551http://srs.mirtol.com/
552
553Unpack the tarball, then refer to MTAs/README.EXIM
554to proceed. You need to set
555
556EXPERIMENTAL_SRS=yes
557
558in your Local/Makefile.
559
560
0e1ccf44
PP
561DCC Support
562--------------------------------------------------------------
563
564*) Building exim
565
566In order to build exim with DCC support add
567
568EXPERIMENTAL_DCC=yes
569
570to your Makefile. (Re-)build/install exim. exim -d should show
571EXPERIMENTAL_DCC under "Support for".
572
573
574*) Configuration
575
576In the main section of exim.cf add at least
577 dccifd_address = /usr/local/dcc/var/dccifd
578or
579 dccifd_address = <ip> <port>
580
581In the DATA ACL you can use the new condition
582 dcc = *
583
584After that "$dcc_header" contains the X-DCC-Header.
585
d36a0501 586Return values are:
0e1ccf44
PP
587 fail for overall "R", "G" from dccifd
588 defer for overall "T" from dccifd
589 accept for overall "A", "S" from dccifd
590
591dcc = */defer_ok works as for spamd.
592
593The "$dcc_result" variable contains the overall result from DCC
594answer. There will an X-DCC: header added to the mail.
595
596Usually you'll use
597 defer !dcc = *
598to greylist with DCC.
599
600If you set, in the main section,
601 dcc_direct_add_header = true
602then the dcc header will be added "in deep" and if the spool
603file was already written it gets removed. This forces Exim to
604write it again if needed. This helps to get the DCC Header
605through to eg. SpamAssassin.
606
607If you want to pass even more headers in the middle of the
608DATA stage you can set
609 $acl_m_dcc_add_header
05c39afa 610to tell the DCC routines to add more information; eg, you might set
0e1ccf44
PP
611this to some results from ClamAV. Be careful. Header syntax is
612not checked and is added "as is".
613
05c39afa
JH
614In case you've troubles with sites sending the same queue items from several
615hosts and fail to get through greylisting you can use
616$acl_m_dcc_override_client_ip
617
618Setting $acl_m_dcc_override_client_ip to an IP address overrides the default
619of $sender_host_address. eg. use the following ACL in DATA stage:
620
621 warn set acl_m_dcc_override_client_ip = \
622 ${lookup{$sender_helo_name}nwildlsearch{/etc/mail/multipleip_sites}{$value}{}}
623 condition = ${if def:acl_m_dcc_override_client_ip}
624 log_message = dbg: acl_m_dcc_override_client_ip set to \
625 $acl_m_dcc_override_client_ip
626
627Then set something like
628# cat /etc/mail/multipleip_sites
629mout-xforward.gmx.net 82.165.159.12
630mout.gmx.net 212.227.15.16
631
632Use a reasonable IP. eg. one the sending cluster acutally uses.
0e1ccf44 633
1899bab2
TL
634DMARC Support
635--------------------------------------------------------------
636
637DMARC combines feedback from SPF, DKIM, and header From: in order
638to attempt to provide better indicators of the authenticity of an
639email. This document does not explain the fundamentals, you
640should read and understand how it works by visiting the website at
641http://www.dmarc.org/.
642
643DMARC support is added via the libopendmarc library. Visit:
644
645 http://sourceforge.net/projects/opendmarc/
646
647to obtain a copy, or find it in your favorite rpm package
648repository. If building from source, this description assumes
649that headers will be in /usr/local/include, and that the libraries
650are in /usr/local/lib.
651
6521. To compile Exim with DMARC support, you must first enable SPF.
653Please read the above section on enabling the EXPERIMENTAL_SPF
654feature. You must also have DKIM support, so you cannot set the
655DISABLE_DKIM feature. Once both of those conditions have been met
656you can enable DMARC in Local/Makefile:
657
658EXPERIMENTAL_DMARC=yes
659LDFLAGS += -lopendmarc
660# CFLAGS += -I/usr/local/include
661# LDFLAGS += -L/usr/local/lib
662
663The first line sets the feature to include the correct code, and
664the second line says to link the libopendmarc libraries into the
665exim binary. The commented out lines should be uncommented if you
666built opendmarc from source and installed in the default location.
667Adjust the paths if you installed them elsewhere, but you do not
668need to uncomment them if an rpm (or you) installed them in the
669package controlled locations (/usr/include and /usr/lib).
670
671
6722. Use the following global settings to configure DMARC:
673
674Required:
675dmarc_tld_file Defines the location of a text file of valid
676 top level domains the opendmarc library uses
677 during domain parsing. Maintained by Mozilla,
678 the most current version can be downloaded
679 from a link at http://publicsuffix.org/list/.
680
681Optional:
682dmarc_history_file Defines the location of a file to log results
683 of dmarc verification on inbound emails. The
684 contents are importable by the opendmarc tools
685 which will manage the data, send out DMARC
686 reports, and expire the data. Make sure the
687 directory of this file is writable by the user
688 exim runs as.
689
690dmarc_forensic_sender The email address to use when sending a
691 forensic report detailing alignment failures
692 if a sender domain's dmarc record specifies it
693 and you have configured Exim to send them.
694 Default: do-not-reply@$default_hostname
695
696
6973. By default, the DMARC processing will run for any remote,
698non-authenticated user. It makes sense to only verify DMARC
699status of messages coming from remote, untrusted sources. You can
700use standard conditions such as hosts, senders, etc, to decide that
701DMARC verification should *not* be performed for them and disable
702DMARC with a control setting:
703
12d0043d 704 control = dmarc_disable_verify
1899bab2
TL
705
706A DMARC record can also specify a "forensic address", which gives
707exim an email address to submit reports about failed alignment.
708Exim does not do this by default because in certain conditions it
709results in unintended information leakage (what lists a user might
710be subscribed to, etc). You must configure exim to submit forensic
711reports to the owner of the domain. If the DMARC record contains a
712forensic address and you specify the control statement below, then
713exim will send these forensic emails. It's also advised that you
714configure a dmarc_forensic_sender because the default sender address
715construction might be inadequate.
716
717 control = dmarc_forensic_enable
718
719(AGAIN: You can choose not to send these forensic reports by simply
720not putting the dmarc_forensic_enable control line at any point in
721your exim config. If you don't tell it to send them, it will not
722send them.)
723
724There are no options to either control. Both must appear before
725the DATA acl.
726
727
7284. You can now run DMARC checks in incoming SMTP by using the
729"dmarc_status" ACL condition in the DATA ACL. You are required to
730call the spf condition first in the ACLs, then the "dmarc_status"
731condition. Putting this condition in the ACLs is required in order
732for a DMARC check to actually occur. All of the variables are set
733up before the DATA ACL, but there is no actual DMARC check that
734occurs until a "dmarc_status" condition is encountered in the ACLs.
735
736The dmarc_status condition takes a list of strings on its
737right-hand side. These strings describe recommended action based
738on the DMARC check. To understand what the policy recommendations
739mean, refer to the DMARC website above. Valid strings are:
740
741 o accept The DMARC check passed and the library recommends
742 accepting the email.
743 o reject The DMARC check failed and the library recommends
744 rejecting the email.
745 o quarantine The DMARC check failed and the library recommends
746 keeping it for further inspection.
7a8678e6 747 o none The DMARC check passed and the library recommends
748 no specific action, neutral.
1899bab2
TL
749 o norecord No policy section in the DMARC record for this
750 sender domain.
751 o nofrom Unable to determine the domain of the sender.
7a8678e6 752 o temperror Library error or dns error.
05070e30 753 o off The DMARC check was disabled for this email.
1899bab2
TL
754
755You can prefix each string with an exclamation mark to invert its
756meaning, for example "!accept" will match all results but
757"accept". The string list is evaluated left-to-right in a
758short-circuit fashion. When a string matches the outcome of the
759DMARC check, the condition succeeds. If none of the listed
760strings matches the outcome of the DMARC check, the condition
761fails.
762
763Of course, you can also use any other lookup method that Exim
764supports, including LDAP, Postgres, MySQL, etc, as long as the
8c8b8274 765result is a list of colon-separated strings.
1899bab2
TL
766
767Several expansion variables are set before the DATA ACL is
768processed, and you can use them in this ACL. The following
769expansion variables are available:
770
771 o $dmarc_status
772 This is a one word status indicating what the DMARC library
8c8b8274
TL
773 thinks of the email. It is a combination of the results of
774 DMARC record lookup and the SPF/DKIM/DMARC processing results
775 (if a DMARC record was found). The actual policy declared
776 in the DMARC record is in a separate expansion variable.
1899bab2
TL
777
778 o $dmarc_status_text
779 This is a slightly longer, human readable status.
780
781 o $dmarc_used_domain
782 This is the domain which DMARC used to look up the DMARC
783 policy record.
784
8c8b8274
TL
785 o $dmarc_domain_policy
786 This is the policy declared in the DMARC record. Valid values
787 are "none", "reject" and "quarantine". It is blank when there
788 is any error, including no DMARC record.
789
1899bab2
TL
790 o $dmarc_ar_header
791 This is the entire Authentication-Results header which you can
792 add using an add_header modifier.
793
794
7955. How to enable DMARC advanced operation:
796By default, Exim's DMARC configuration is intended to be
797non-intrusive and conservative. To facilitate this, Exim will not
798create any type of logging files without explicit configuration by
799you, the admin. Nor will Exim send out any emails/reports about
800DMARC issues without explicit configuration by you, the admin (other
801than typical bounce messages that may come about due to ACL
802processing or failure delivery issues).
803
804In order to log statistics suitable to be imported by the opendmarc
805tools, you need to:
806a. Configure the global setting dmarc_history_file.
807b. Configure cron jobs to call the appropriate opendmarc history
808 import scripts and truncating the dmarc_history_file.
809
810In order to send forensic reports, you need to:
811a. Configure the global setting dmarc_forensic_sender.
812b. Configure, somewhere before the DATA ACL, the control option to
813 enable sending DMARC forensic reports.
814
815
8166. Example usage:
817(RCPT ACL)
818 warn domains = +local_domains
819 hosts = +local_hosts
12d0043d 820 control = dmarc_disable_verify
1899bab2
TL
821
822 warn !domains = +screwed_up_dmarc_records
823 control = dmarc_enable_forensic
824
8c8b8274
TL
825 warn condition = (lookup if destined to mailing list)
826 set acl_m_mailing_list = 1
827
1899bab2
TL
828(DATA ACL)
829 warn dmarc_status = accept : none : off
830 !authenticated = *
831 log_message = DMARC DEBUG: $dmarc_status $dmarc_used_domain
832 add_header = $dmarc_ar_header
833
834 warn dmarc_status = !accept
835 !authenticated = *
836 log_message = DMARC DEBUG: '$dmarc_status' for $dmarc_used_domain
837
838 warn dmarc_status = quarantine
839 !authenticated = *
840 set $acl_m_quarantine = 1
841 # Do something in a transport with this flag variable
842
8c8b8274
TL
843 deny condition = ${if eq{$dmarc_domain_policy}{reject}}
844 condition = ${if eq{$acl_m_mailing_list}{1}}
845 message = Messages from $dmarc_used_domain break mailing lists
846
1899bab2
TL
847 deny dmarc_status = reject
848 !authenticated = *
849 message = Message from $domain_used_domain failed sender's DMARC policy, REJECT
850
851
852
d68218c7
JH
853Transport post-delivery actions
854--------------------------------------------------------------
855
856An arbitrary per-transport string can be expanded on successful delivery,
857and (for SMTP transports) a second string on deferrals caused by a host error.
858This feature may be used, for example, to write exim internal log information
859(not available otherwise) into a database.
860
861In order to use the feature, you must set
862
863EXPERIMENTAL_TPDA=yes
864
865in your Local/Makefile
866
867and define the expandable strings in the runtime config file, to
868be executed at end of delivery.
869
870Additionally, there are 6 more variables, available at end of
871delivery:
872
873tpda_delivery_ip IP of host, which has accepted delivery
874tpda_delivery_port Port of remote host which has accepted delivery
875tpda_delivery_fqdn FQDN of host, which has accepted delivery
876tpda_delivery_local_part local part of address being delivered
877tpda_delivery_domain domain part of address being delivered
878tpda_delivery_confirmation SMTP confirmation message
879
880In case of a deferral caused by a host-error:
881tpda_defer_errno Error number
882tpda_defer_errstr Error string possibly containing more details
883
884The $router_name and $transport_name variables are also usable.
885
886
887To take action after successful deliveries, set the following option
888on any transport of interest.
889
890tpda_delivery_action
891
892An example might look like:
893
894tpda_delivery_action = \
895${lookup pgsql {SELECT * FROM record_Delivery( \
896 '${quote_pgsql:$sender_address_domain}',\
897 '${quote_pgsql:${lc:$sender_address_local_part}}', \
898 '${quote_pgsql:$tpda_delivery_domain}', \
899 '${quote_pgsql:${lc:$tpda_delivery_local_part}}', \
900 '${quote_pgsql:$tpda_delivery_ip}', \
901 '${quote_pgsql:${lc:$tpda_delivery_fqdn}}', \
902 '${quote_pgsql:$message_exim_id}')}}
903
904The string is expanded after the delivery completes and any
905side-effects will happen. The result is then discarded.
906Note that for complex operations an ACL expansion can be used.
907
908
909In order to log host deferrals, add the following option to an SMTP
910transport:
911
912tpda_host_defer_action
913
914This is a private option of the SMTP transport. It is intended to
915log failures of remote hosts. It is executed only when exim has
916attempted to deliver a message to a remote host and failed due to
917an error which doesn't seem to be related to the individual
918message, sender, or recipient address.
7c261cc6 919See section 47.2 of the exim documentation for more details on how
d68218c7
JH
920this is determined.
921
922Example:
923
924tpda_host_defer_action = \
925${lookup mysql {insert into delivlog set \
926 msgid = '${quote_mysql:$message_exim_id}', \
927 senderlp = '${quote_mysql:${lc:$sender_address_local_part}}', \
928 senderdom = '${quote_mysql:$sender_address_domain}', \
929 delivlp = '${quote_mysql:${lc:$tpda_delivery_local_part}}', \
930 delivdom = '${quote_mysql:$tpda_delivery_domain}', \
931 delivip = '${quote_mysql:$tpda_delivery_ip}', \
932 delivport = '${quote_mysql:$tpda_delivery_port}', \
933 delivfqdn = '${quote_mysql:$tpda_delivery_fqdn}', \
934 deliverrno = '${quote_mysql:$tpda_defer_errno}', \
935 deliverrstr = '${quote_mysql:$tpda_defer_errstr}' \
936 }}
1899bab2 937
9bdd29ad
TL
938
939Redis Lookup
940--------------------------------------------------------------
941
942Redis is open source advanced key-value data store. This document
943does not explain the fundamentals, you should read and understand how
944it works by visiting the website at http://www.redis.io/.
945
946Redis lookup support is added via the hiredis library. Visit:
947
948 https://github.com/redis/hiredis
949
950to obtain a copy, or find it in your operating systems package repository.
951If building from source, this description assumes that headers will be in
952/usr/local/include, and that the libraries are in /usr/local/lib.
953
9541. In order to build exim with Redis lookup support add
955
956EXPERIMENTAL_REDIS=yes
957
958to your Local/Makefile. (Re-)build/install exim. exim -d should show
959Experimental_Redis in the line "Support for:".
960
961EXPERIMENTAL_REDIS=yes
962LDFLAGS += -lhiredis
963# CFLAGS += -I/usr/local/include
964# LDFLAGS += -L/usr/local/lib
965
966The first line sets the feature to include the correct code, and
967the second line says to link the hiredis libraries into the
968exim binary. The commented out lines should be uncommented if you
969built hiredis from source and installed in the default location.
970Adjust the paths if you installed them elsewhere, but you do not
971need to uncomment them if an rpm (or you) installed them in the
972package controlled locations (/usr/include and /usr/lib).
973
974
9752. Use the following global settings to configure Redis lookup support:
976
977Required:
978redis_servers This option provides a list of Redis servers
979 and associated connection data, to be used in
980 conjunction with redis lookups. The option is
981 only available if Exim is configured with Redis
982 support.
983
984For example:
985
986redis_servers = 127.0.0.1/10/ - using database 10 with no password
987redis_servers = 127.0.0.1//password - to make use of the default database of 0 with a password
988redis_servers = 127.0.0.1// - for default database of 0 with no password
989
9903. Once you have the Redis servers defined you can then make use of the
991experimental Redis lookup by specifying ${lookup redis{}} in a lookup query.
992
9934. Example usage:
994
995(Host List)
996hostlist relay_from_ips = <\n ${lookup redis{SMEMBERS relay_from_ips}}
997
998Where relay_from_ips is a Redis set which contains entries such as "192.168.0.0/24" "10.0.0.0/8" and so on.
999The result set is returned as
1000192.168.0.0/24
100110.0.0.0/8
1002..
1003.
1004
1005(Domain list)
1006domainlist virtual_domains = ${lookup redis {HGET $domain domain}}
1007
1008Where $domain is a hash which includes the key 'domain' and the value '$domain'.
1009
1010(Adding or updating an existing key)
1011set acl_c_spammer = ${if eq{${lookup redis{SPAMMER_SET}}}{OK}}
1012
1013Where SPAMMER_SET is a macro and it is defined as
1014
1015"SET SPAMMER <some_value>"
1016
1017(Getting a value from Redis)
1018
1019set acl_c_spam_host = ${lookup redis{GET...}}
1020
1021
a3c86431
TL
1022Proxy Protocol Support
1023--------------------------------------------------------------
1024
1025Exim now has Experimental "Proxy Protocol" support. It was built on
1026specifications from:
1027http://haproxy.1wt.eu/download/1.5/doc/proxy-protocol.txt
36719342
TL
1028Above URL revised May 2014 to change version 2 spec:
1029http://git.1wt.eu/web?p=haproxy.git;a=commitdiff;h=afb768340c9d7e50d8e
a3c86431
TL
1030
1031The purpose of this function is so that an application load balancer,
1032such as HAProxy, can sit in front of several Exim servers and Exim
1033will log the IP that is connecting to the proxy server instead of
1034the IP of the proxy server when it connects to Exim. It resets the
1035$sender_address_host and $sender_address_port to the IP:port of the
1036connection to the proxy. It also re-queries the DNS information for
1037this new IP address so that the original sender's hostname and IP
1038get logged in the Exim logfile. There is no logging if a host passes or
1039fails Proxy Protocol negotiation, but it can easily be determined and
1040recorded in an ACL (example is below).
1041
10421. To compile Exim with Proxy Protocol support, put this in
1043Local/Makefile:
1044
1045EXPERIMENTAL_PROXY=yes
1046
10472. Global configuration settings:
1048
1049proxy_required_hosts = HOSTLIST
1050
1051The proxy_required_hosts option will require any IP in that hostlist
1052to use Proxy Protocol. The specification of Proxy Protocol is very
1053strict, and if proxy negotiation fails, Exim will not allow any SMTP
1054command other than QUIT. (See end of this section for an example.)
1055The option is expanded when used, so it can be a hostlist as well as
1056string of IP addresses. Since it is expanded, specifying an alternate
1057separator is supported for ease of use with IPv6 addresses.
1058
1059To log the IP of the proxy in the incoming logline, add:
1060 log_selector = +proxy
1061
1062A default incoming logline (wrapped for appearance) will look like this:
1063
1064 2013-11-04 09:25:06 1VdNti-0001OY-1V <= me@example.net
1065 H=mail.example.net [1.2.3.4] P=esmtp S=433
1066
1067With the log selector enabled, an email that was proxied through a
1068Proxy Protocol server at 192.168.1.2 will look like this:
1069
1070 2013-11-04 09:25:06 1VdNti-0001OY-1V <= me@example.net
1071 H=mail.example.net [1.2.3.4] P=esmtp PRX=192.168.1.2 S=433
1072
10733. In the ACL's the following expansion variables are available.
1074
eb57651e
TL
1075proxy_host_address The (internal) src IP of the proxy server
1076 making the connection to the Exim server.
1077proxy_host_port The (internal) src port the proxy server is
1078 using to connect to the Exim server.
1079proxy_target_address The dest (public) IP of the remote host to
1080 the proxy server.
1081proxy_target_port The dest port the remote host is using to
1082 connect to the proxy server.
1083proxy_session Boolean, yes/no, the connected host is required
1084 to use Proxy Protocol.
a3c86431
TL
1085
1086There is no expansion for a failed proxy session, however you can detect
1087it by checking if $proxy_session is true but $proxy_host is empty. As
1088an example, in my connect ACL, I have:
1089
1090 warn condition = ${if and{ {bool{$proxy_session}} \
a3bddaa8 1091 {eq{$proxy_host_address}{}} } }
a3c86431
TL
1092 log_message = Failed required proxy protocol negotiation \
1093 from $sender_host_name [$sender_host_address]
1094
1095 warn condition = ${if and{ {bool{$proxy_session}} \
a3bddaa8 1096 {!eq{$proxy_host_address}{}} } }
a3c86431 1097 # But don't log health probes from the proxy itself
a3bddaa8 1098 condition = ${if eq{$proxy_host_address}{$sender_host_address} \
a3c86431
TL
1099 {false}{true}}
1100 log_message = Successfully proxied from $sender_host_name \
1101 [$sender_host_address] through proxy protocol \
a3bddaa8 1102 host $proxy_host_address
a3c86431 1103
eb57651e
TL
1104 # Possibly more clear
1105 warn logwrite = Remote Source Address: $sender_host_address:$sender_host_port
1106 logwrite = Proxy Target Address: $proxy_target_address:$proxy_target_port
1107 logwrite = Proxy Internal Address: $proxy_host_address:$proxy_host_port
1108 logwrite = Internal Server Address: $received_ip_address:$received_port
1109
1110
a3c86431
TL
11114. Runtime issues to be aware of:
1112 - Since the real connections are all coming from your proxy, and the
1113 per host connection tracking is done before Proxy Protocol is
1114 evaluated, smtp_accept_max_per_host must be set high enough to
1115 handle all of the parallel volume you expect per inbound proxy.
1116 - The proxy has 3 seconds (hard-coded in the source code) to send the
1117 required Proxy Protocol header after it connects. If it does not,
1118 the response to any commands will be:
1119 "503 Command refused, required Proxy negotiation failed"
1120 - If the incoming connection is configured in Exim to be a Proxy
1121 Protocol host, but the proxy is not sending the header, the banner
1122 does not get sent until the timeout occurs. If the sending host
1123 sent any input (before the banner), this causes a standard Exim
1124 synchronization error (i.e. trying to pipeline before PIPELINING
1125 was advertised).
1126 - This is not advised, but is mentioned for completeness if you have
1127 a specific internal configuration that you want this: If the Exim
1128 server only has an internal IP address and no other machines in your
1129 organization will connect to it to try to send email, you may
1130 simply set the hostlist to "*", however, this will prevent local
1131 mail programs from working because that would require mail from
1132 localhost to use Proxy Protocol. Again, not advised!
1133
11345. Example of a refused connection because the Proxy Protocol header was
1135not sent from a host configured to use Proxy Protocol. In the example,
1136the 3 second timeout occurred (when a Proxy Protocol banner should have
1137been sent), the banner was displayed to the user, but all commands are
1138rejected except for QUIT:
1139
1140# nc mail.example.net 25
1141220-mail.example.net, ESMTP Exim 4.82+proxy, Mon, 04 Nov 2013 10:45:59
1142220 -0800 RFC's enforced
1143EHLO localhost
1144503 Command refused, required Proxy negotiation failed
1145QUIT
1146221 mail.example.net closing connection
1147
1148
9bdd29ad 1149
e51c7be2
JH
1150Certificate name checking
1151--------------------------------------------------------------
1152The X509 certificates used for TLS are supposed be verified
1153that they are owned by the expected host. The coding of TLS
1154support to date has not made these checks.
1155
1156If built with EXPERIMENTAL_CERTNAMES defined, code is
1157included to do so, and a new smtp transport option
1158"tls_verify_cert_hostname" supported which takes a list of
1159names for which the checks must be made. The host must
1160also be in "tls_verify_hosts".
1161
1162Both Subject and Subject-Alternate-Name certificate fields
1163are supported, as are wildcard certificates (limited to
1164a single wildcard being the initial component of a 3-or-more
1165component FQDN).
1166
1167
1168
ee161e8f
PH
1169--------------------------------------------------------------
1170End of file
1171--------------------------------------------------------------