LDAP: Check for errors of TLS initialisation
[exim.git] / doc / doc-txt / experimental-spec.txt
CommitLineData
7bafa7d9
TK
1From time to time, experimental features may be added to Exim.
2While a feature is experimental, there will be a build-time
3option whose name starts "EXPERIMENTAL_" that must be set in
4order to include the feature. This file contains information
5about experimenatal features, all of which are unstable and
ee161e8f
PH
6liable to incompatibile change.
7
8
0b23848a 9Brightmail AntiSpam (BMI) suppport
ee161e8f
PH
10--------------------------------------------------------------
11
12Brightmail AntiSpam is a commercial package. Please see
13http://www.brightmail.com for more information on
14the product. For the sake of clarity, we'll refer to it as
15"BMI" from now on.
16
17
180) BMI concept and implementation overview
19
20In contrast to how spam-scanning with SpamAssassin is
21implemented in exiscan-acl, BMI is more suited for per
22-recipient scanning of messages. However, each messages is
23scanned only once, but multiple "verdicts" for multiple
24recipients can be returned from the BMI server. The exiscan
25implementation passes the message to the BMI server just
26before accepting it. It then adds the retrieved verdicts to
27the messages header file in the spool. These verdicts can then
28be queried in routers, where operation is per-recipient
29instead of per-message. To use BMI, you need to take the
30following steps:
31
32 1) Compile Exim with BMI support
3ec3e3bb 33 2) Set up main BMI options (top section of Exim config file)
ee161e8f
PH
34 3) Set up ACL control statement (ACL section of the config
35 file)
36 4) Set up your routers to use BMI verdicts (routers section
37 of the config file).
38 5) (Optional) Set up per-recipient opt-in information.
39
8ff3788c 40These four steps are explained in more details below.
ee161e8f
PH
41
421) Adding support for BMI at compile time
43
44 To compile with BMI support, you need to link Exim against
45 the Brighmail client SDK, consisting of a library
46 (libbmiclient_single.so) and a header file (bmi_api.h).
47 You'll also need to explicitly set a flag in the Makefile to
48 include BMI support in the Exim binary. Both can be achieved
49 with these lines in Local/Makefile:
50
51 EXPERIMENTAL_BRIGHTMAIL=yes
47bbda99 52 CFLAGS=-I/path/to/the/dir/with/the/includefile
ee161e8f 53 EXTRALIBS_EXIM=-L/path/to/the/dir/with/the/library -lbmiclient_single
8ff3788c 54
ee161e8f
PH
55 If you use other CFLAGS or EXTRALIBS_EXIM settings then
56 merge the content of these lines with them.
57
7c0c8547 58 Note for BMI6.x users: You'll also have to add -lxml2_single
ee161e8f
PH
59 to the EXTRALIBS_EXIM line. Users of 5.5x do not need to do
60 this.
8ff3788c 61
ee161e8f
PH
62 You should also include the location of
63 libbmiclient_single.so in your dynamic linker configuration
64 file (usually /etc/ld.so.conf) and run "ldconfig"
65 afterwards, or else the produced Exim binary will not be
66 able to find the library file.
67
68
3ec3e3bb 692) Setting up BMI support in the Exim main configuration
ee161e8f 70
3ec3e3bb 71 To enable BMI support in the main Exim configuration, you
ee161e8f
PH
72 should set the path to the main BMI configuration file with
73 the "bmi_config_file" option, like this:
8ff3788c 74
ee161e8f 75 bmi_config_file = /opt/brightmail/etc/brightmail.cfg
8ff3788c 76
3ec3e3bb 77 This must go into section 1 of Exim's configuration file (You
ee161e8f
PH
78 can put it right on top). If you omit this option, it
79 defaults to /opt/brightmail/etc/brightmail.cfg.
80
81 Note for BMI6.x users: This file is in XML format in V6.xx
82 and its name is /opt/brightmail/etc/bmiconfig.xml. So BMI
83 6.x users MUST set the bmi_config_file option.
8ff3788c 84
ee161e8f
PH
85
863) Set up ACL control statement
87
88 To optimize performance, it makes sense only to process
89 messages coming from remote, untrusted sources with the BMI
90 server. To set up a messages for processing by the BMI
91 server, you MUST set the "bmi_run" control statement in any
92 ACL for an incoming message. You will typically do this in
93 an "accept" block in the "acl_check_rcpt" ACL. You should
94 use the "accept" block(s) that accept messages from remote
95 servers for your own domain(s). Here is an example that uses
3ec3e3bb 96 the "accept" blocks from Exim's default configuration file:
8ff3788c 97
ee161e8f
PH
98
99 accept domains = +local_domains
100 endpass
101 verify = recipient
102 control = bmi_run
103
104 accept domains = +relay_to_domains
105 endpass
106 verify = recipient
107 control = bmi_run
8ff3788c 108
ee161e8f
PH
109 If bmi_run is not set in any ACL during reception of the
110 message, it will NOT be passed to the BMI server.
111
112
1134) Setting up routers to use BMI verdicts
114
115 When a message has been run through the BMI server, one or
116 more "verdicts" are present. Different recipients can have
117 different verdicts. Each recipient is treated individually
118 during routing, so you can query the verdicts by recipient
3ec3e3bb 119 at that stage. From Exim's view, a verdict can have the
ee161e8f 120 following outcomes:
8ff3788c 121
ee161e8f
PH
122 o deliver the message normally
123 o deliver the message to an alternate location
124 o do not deliver the message
8ff3788c 125
ee161e8f
PH
126 To query the verdict for a recipient, the implementation
127 offers the following tools:
8ff3788c
TK
128
129
ee161e8f
PH
130 - Boolean router preconditions. These can be used in any
131 router. For a simple implementation of BMI, these may be
132 all that you need. The following preconditions are
133 available:
8ff3788c 134
ee161e8f 135 o bmi_deliver_default
8ff3788c 136
ee161e8f
PH
137 This precondition is TRUE if the verdict for the
138 recipient is to deliver the message normally. If the
139 message has not been processed by the BMI server, this
140 variable defaults to TRUE.
8ff3788c 141
ee161e8f 142 o bmi_deliver_alternate
8ff3788c 143
ee161e8f
PH
144 This precondition is TRUE if the verdict for the
145 recipient is to deliver the message to an alternate
146 location. You can get the location string from the
147 $bmi_alt_location expansion variable if you need it. See
148 further below. If the message has not been processed by
149 the BMI server, this variable defaults to FALSE.
8ff3788c 150
ee161e8f 151 o bmi_dont_deliver
8ff3788c 152
ee161e8f
PH
153 This precondition is TRUE if the verdict for the
154 recipient is NOT to deliver the message to the
155 recipient. You will typically use this precondition in a
156 top-level blackhole router, like this:
8ff3788c 157
ee161e8f
PH
158 # don't deliver messages handled by the BMI server
159 bmi_blackhole:
160 driver = redirect
161 bmi_dont_deliver
162 data = :blackhole:
8ff3788c 163
ee161e8f
PH
164 This router should be on top of all others, so messages
165 that should not be delivered do not reach other routers
166 at all. If the message has not been processed by
167 the BMI server, this variable defaults to FALSE.
8ff3788c
TK
168
169
ee161e8f
PH
170 - A list router precondition to query if rules "fired" on
171 the message for the recipient. Its name is "bmi_rule". You
172 use it by passing it a colon-separated list of rule
173 numbers. You can use this condition to route messages that
174 matched specific rules. Here is an example:
8ff3788c 175
ee161e8f
PH
176 # special router for BMI rule #5, #8 and #11
177 bmi_rule_redirect:
178 driver = redirect
179 bmi_rule = 5:8:11
180 data = postmaster@mydomain.com
8ff3788c
TK
181
182
ee161e8f
PH
183 - Expansion variables. Several expansion variables are set
184 during routing. You can use them in custom router
185 conditions, for example. The following variables are
186 available:
8ff3788c 187
ee161e8f 188 o $bmi_base64_verdict
8ff3788c 189
ee161e8f
PH
190 This variable will contain the BASE64 encoded verdict
191 for the recipient being routed. You can use it to add a
192 header to messages for tracking purposes, for example:
8ff3788c 193
ee161e8f
PH
194 localuser:
195 driver = accept
196 check_local_user
197 headers_add = X-Brightmail-Verdict: $bmi_base64_verdict
198 transport = local_delivery
8ff3788c 199
ee161e8f
PH
200 If there is no verdict available for the recipient being
201 routed, this variable contains the empty string.
8ff3788c 202
ee161e8f 203 o $bmi_base64_tracker_verdict
8ff3788c 204
ee161e8f
PH
205 This variable will contain a BASE64 encoded subset of
206 the verdict information concerning the "rules" that
207 fired on the message. You can add this string to a
208 header, commonly named "X-Brightmail-Tracker". Example:
8ff3788c 209
ee161e8f
PH
210 localuser:
211 driver = accept
212 check_local_user
213 headers_add = X-Brightmail-Tracker: $bmi_base64_tracker_verdict
214 transport = local_delivery
8ff3788c 215
ee161e8f
PH
216 If there is no verdict available for the recipient being
217 routed, this variable contains the empty string.
8ff3788c 218
ee161e8f 219 o $bmi_alt_location
8ff3788c 220
ee161e8f
PH
221 If the verdict is to redirect the message to an
222 alternate location, this variable will contain the
223 alternate location string returned by the BMI server. In
224 its default configuration, this is a header-like string
225 that can be added to the message with "headers_add". If
226 there is no verdict available for the recipient being
227 routed, or if the message is to be delivered normally,
228 this variable contains the empty string.
8ff3788c 229
ee161e8f 230 o $bmi_deliver
8ff3788c 231
ee161e8f
PH
232 This is an additional integer variable that can be used
233 to query if the message should be delivered at all. You
234 should use router preconditions instead if possible.
8ff3788c 235
ee161e8f
PH
236 $bmi_deliver is '0': the message should NOT be delivered.
237 $bmi_deliver is '1': the message should be delivered.
8ff3788c
TK
238
239
ee161e8f
PH
240 IMPORTANT NOTE: Verdict inheritance.
241 The message is passed to the BMI server during message
242 reception, using the target addresses from the RCPT TO:
243 commands in the SMTP transaction. If recipients get expanded
244 or re-written (for example by aliasing), the new address(es)
245 inherit the verdict from the original address. This means
246 that verdicts also apply to all "child" addresses generated
247 from top-level addresses that were sent to the BMI server.
8ff3788c
TK
248
249
ee161e8f
PH
2505) Using per-recipient opt-in information (Optional)
251
252 The BMI server features multiple scanning "profiles" for
253 individual recipients. These are usually stored in a LDAP
254 server and are queried by the BMI server itself. However,
255 you can also pass opt-in data for each recipient from the
256 MTA to the BMI server. This is particularly useful if you
3ec3e3bb 257 already look up recipient data in Exim anyway (which can
ee161e8f
PH
258 also be stored in a SQL database or other source). This
259 implementation enables you to pass opt-in data to the BMI
260 server in the RCPT ACL. This works by setting the
261 'bmi_optin' modifier in a block of that ACL. If should be
262 set to a list of comma-separated strings that identify the
263 features which the BMI server should use for that particular
264 recipient. Ideally, you would use the 'bmi_optin' modifier
265 in the same ACL block where you set the 'bmi_run' control
266 flag. Here is an example that will pull opt-in data for each
267 recipient from a flat file called
268 '/etc/exim/bmi_optin_data'.
8ff3788c 269
ee161e8f 270 The file format:
8ff3788c 271
ee161e8f
PH
272 user1@mydomain.com: <OPTIN STRING1>:<OPTIN STRING2>
273 user2@thatdomain.com: <OPTIN STRING3>
8ff3788c
TK
274
275
ee161e8f 276 The example:
8ff3788c 277
ee161e8f
PH
278 accept domains = +relay_to_domains
279 endpass
280 verify = recipient
281 bmi_optin = ${lookup{$local_part@$domain}lsearch{/etc/exim/bmi_optin_data}}
8ff3788c
TK
282 control = bmi_run
283
ee161e8f 284 Of course, you can also use any other lookup method that
3ec3e3bb 285 Exim supports, including LDAP, Postgres, MySQL, Oracle etc.,
ee161e8f
PH
286 as long as the result is a list of colon-separated opt-in
287 strings.
8ff3788c 288
ee161e8f
PH
289 For a list of available opt-in strings, please contact your
290 Brightmail representative.
ee161e8f 291
8ff3788c
TK
292
293
294
0b23848a 295Sender Policy Framework (SPF) support
ee161e8f
PH
296--------------------------------------------------------------
297
f413481d 298To learn more about SPF, visit http://www.openspf.org. This
ee161e8f
PH
299document does not explain the SPF fundamentals, you should
300read and understand the implications of deploying SPF on your
301system before doing so.
302
8ff3788c 303SPF support is added via the libspf2 library. Visit
ee161e8f
PH
304
305 http://www.libspf2.org/
8ff3788c 306
ee161e8f
PH
307to obtain a copy, then compile and install it. By default,
308this will put headers in /usr/local/include and the static
309library in /usr/local/lib.
310
3ec3e3bb 311To compile Exim with SPF support, set these additional flags in
ee161e8f
PH
312Local/Makefile:
313
314EXPERIMENTAL_SPF=yes
315CFLAGS=-DSPF -I/usr/local/include
316EXTRALIBS_EXIM=-L/usr/local/lib -lspf2
317
318This assumes that the libspf2 files are installed in
319their default locations.
320
321You can now run SPF checks in incoming SMTP by using the "spf"
322ACL condition in either the MAIL, RCPT or DATA ACLs. When
323using it in the RCPT ACL, you can make the checks dependend on
324the RCPT address (or domain), so you can check SPF records
325only for certain target domains. This gives you the
326possibility to opt-out certain customers that do not want
327their mail to be subject to SPF checking.
328
329The spf condition takes a list of strings on its right-hand
330side. These strings describe the outcome of the SPF check for
331which the spf condition should succeed. Valid strings are:
332
333 o pass The SPF check passed, the sending host
334 is positively verified by SPF.
335 o fail The SPF check failed, the sending host
336 is NOT allowed to send mail for the domain
337 in the envelope-from address.
338 o softfail The SPF check failed, but the queried
339 domain can't absolutely confirm that this
340 is a forgery.
341 o none The queried domain does not publish SPF
342 records.
343 o neutral The SPF check returned a "neutral" state.
344 This means the queried domain has published
345 a SPF record, but wants to allow outside
346 servers to send mail under its domain as well.
347 o err_perm This indicates a syntax error in the SPF
348 record of the queried domain. This should be
349 treated like "none".
350 o err_temp This indicates a temporary error during all
3ec3e3bb 351 processing, including Exim's SPF processing.
ee161e8f 352 You may defer messages when this occurs.
8ff3788c 353
ee161e8f
PH
354You can prefix each string with an exclamation mark to invert
355is meaning, for example "!fail" will match all results but
356"fail". The string list is evaluated left-to-right, in a
357short-circuit fashion. When a string matches the outcome of
358the SPF check, the condition succeeds. If none of the listed
359strings matches the outcome of the SPF check, the condition
360fails.
361
f413481d
TK
362Here is an example to fail forgery attempts from domains that
363publish SPF records:
ee161e8f
PH
364
365/* -----------------
f413481d
TK
366deny message = $sender_host_address is not allowed to send mail from ${if def:sender_address_domain {$sender_address_domain}{$sender_helo_name}}. \
367 Please see http://www.openspf.org/Why?scope=${if def:sender_address_domain {mfrom}{helo}};identity=${if def:sender_address_domain {$sender_address}{$sender_helo_name}};ip=$sender_host_address
ee161e8f
PH
368 spf = fail
369--------------------- */
370
371You can also give special treatment to specific domains:
372
373/* -----------------
374deny message = AOL sender, but not from AOL-approved relay.
375 sender_domains = aol.com
376 spf = fail:neutral
377--------------------- */
378
379Explanation: AOL publishes SPF records, but is liberal and
380still allows non-approved relays to send mail from aol.com.
381This will result in a "neutral" state, while mail from genuine
382AOL servers will result in "pass". The example above takes
383this into account and treats "neutral" like "fail", but only
384for aol.com. Please note that this violates the SPF draft.
385
386When the spf condition has run, it sets up several expansion
387variables.
388
389 $spf_header_comment
390 This contains a human-readable string describing the outcome
391 of the SPF check. You can add it to a custom header or use
392 it for logging purposes.
8ff3788c 393
ee161e8f 394 $spf_received
8fe685ad 395 This contains a complete Received-SPF: header that can be
ee161e8f
PH
396 added to the message. Please note that according to the SPF
397 draft, this header must be added at the top of the header
398 list. Please see section 10 on how you can do this.
8ff3788c 399
65a7d8c3
NM
400 Note: in case of "Best-guess" (see below), the convention is
401 to put this string in a header called X-SPF-Guess: instead.
402
ee161e8f
PH
403 $spf_result
404 This contains the outcome of the SPF check in string form,
405 one of pass, fail, softfail, none, neutral, err_perm or
406 err_temp.
8ff3788c 407
ee161e8f
PH
408 $spf_smtp_comment
409 This contains a string that can be used in a SMTP response
410 to the calling party. Useful for "fail".
8ff3788c 411
65a7d8c3
NM
412In addition to SPF, you can also perform checks for so-called
413"Best-guess". Strictly speaking, "Best-guess" is not standard
414SPF, but it is supported by the same framework that enables SPF
415capability. Refer to http://www.openspf.org/FAQ/Best_guess_record
416for a description of what it means.
417
418To access this feature, simply use the spf_guess condition in place
419of the spf one. For example:
420
421/* -----------------
422deny message = $sender_host_address doesn't look trustworthy to me
423 spf_guess = fail
424--------------------- */
425
426In case you decide to reject messages based on this check, you
427should note that although it uses the same framework, "Best-guess"
428is NOT SPF, and therefore you should not mention SPF at all in your
429reject message.
430
431When the spf_guess condition has run, it sets up the same expansion
432variables as when spf condition is run, described above.
433
434Additionally, since Best-guess is not standarized, you may redefine
435what "Best-guess" means to you by redefining spf_guess variable in
436global config. For example, the following:
437
438/* -----------------
439spf_guess = v=spf1 a/16 mx/16 ptr ?all
440--------------------- */
441
442would relax host matching rules to a broader network range.
8ff3788c 443
ee161e8f 444
0b23848a 445SRS (Sender Rewriting Scheme) Support
ee161e8f
PH
446--------------------------------------------------------------
447
448Exiscan currently includes SRS support via Miles Wilton's
8ff3788c 449libsrs_alt library. The current version of the supported
ee161e8f
PH
450library is 0.5.
451
452In order to use SRS, you must get a copy of libsrs_alt from
453
454http://srs.mirtol.com/
455
456Unpack the tarball, then refer to MTAs/README.EXIM
457to proceed. You need to set
458
459EXPERIMENTAL_SRS=yes
460
461in your Local/Makefile.
462
463
464--------------------------------------------------------------
465End of file
466--------------------------------------------------------------