PAM: fix crash in the pam expansion condition. Bug 2489
[exim.git] / doc / doc-txt / experimental-spec.txt
CommitLineData
7bafa7d9
TK
1From time to time, experimental features may be added to Exim.
2While a feature is experimental, there will be a build-time
3option whose name starts "EXPERIMENTAL_" that must be set in
4order to include the feature. This file contains information
d36a0501
PP
5about experimental features, all of which are unstable and
6liable to incompatible change.
ee161e8f
PH
7
8
4c04137d 9Brightmail AntiSpam (BMI) support
ee161e8f
PH
10--------------------------------------------------------------
11
12Brightmail AntiSpam is a commercial package. Please see
13http://www.brightmail.com for more information on
14the product. For the sake of clarity, we'll refer to it as
15"BMI" from now on.
16
17
180) BMI concept and implementation overview
19
20In contrast to how spam-scanning with SpamAssassin is
21implemented in exiscan-acl, BMI is more suited for per
22-recipient scanning of messages. However, each messages is
23scanned only once, but multiple "verdicts" for multiple
24recipients can be returned from the BMI server. The exiscan
25implementation passes the message to the BMI server just
26before accepting it. It then adds the retrieved verdicts to
27the messages header file in the spool. These verdicts can then
28be queried in routers, where operation is per-recipient
29instead of per-message. To use BMI, you need to take the
30following steps:
31
32 1) Compile Exim with BMI support
3ec3e3bb 33 2) Set up main BMI options (top section of Exim config file)
ee161e8f
PH
34 3) Set up ACL control statement (ACL section of the config
35 file)
36 4) Set up your routers to use BMI verdicts (routers section
37 of the config file).
38 5) (Optional) Set up per-recipient opt-in information.
39
8ff3788c 40These four steps are explained in more details below.
ee161e8f
PH
41
421) Adding support for BMI at compile time
43
44 To compile with BMI support, you need to link Exim against
4c04137d 45 the Brightmail client SDK, consisting of a library
ee161e8f
PH
46 (libbmiclient_single.so) and a header file (bmi_api.h).
47 You'll also need to explicitly set a flag in the Makefile to
48 include BMI support in the Exim binary. Both can be achieved
49 with these lines in Local/Makefile:
50
51 EXPERIMENTAL_BRIGHTMAIL=yes
47bbda99 52 CFLAGS=-I/path/to/the/dir/with/the/includefile
ee161e8f 53 EXTRALIBS_EXIM=-L/path/to/the/dir/with/the/library -lbmiclient_single
8ff3788c 54
ee161e8f
PH
55 If you use other CFLAGS or EXTRALIBS_EXIM settings then
56 merge the content of these lines with them.
57
7c0c8547 58 Note for BMI6.x users: You'll also have to add -lxml2_single
ee161e8f
PH
59 to the EXTRALIBS_EXIM line. Users of 5.5x do not need to do
60 this.
8ff3788c 61
ee161e8f
PH
62 You should also include the location of
63 libbmiclient_single.so in your dynamic linker configuration
64 file (usually /etc/ld.so.conf) and run "ldconfig"
65 afterwards, or else the produced Exim binary will not be
66 able to find the library file.
67
68
3ec3e3bb 692) Setting up BMI support in the Exim main configuration
ee161e8f 70
3ec3e3bb 71 To enable BMI support in the main Exim configuration, you
ee161e8f
PH
72 should set the path to the main BMI configuration file with
73 the "bmi_config_file" option, like this:
8ff3788c 74
ee161e8f 75 bmi_config_file = /opt/brightmail/etc/brightmail.cfg
8ff3788c 76
3ec3e3bb 77 This must go into section 1 of Exim's configuration file (You
ee161e8f
PH
78 can put it right on top). If you omit this option, it
79 defaults to /opt/brightmail/etc/brightmail.cfg.
80
81 Note for BMI6.x users: This file is in XML format in V6.xx
82 and its name is /opt/brightmail/etc/bmiconfig.xml. So BMI
83 6.x users MUST set the bmi_config_file option.
8ff3788c 84
ee161e8f
PH
85
863) Set up ACL control statement
87
88 To optimize performance, it makes sense only to process
89 messages coming from remote, untrusted sources with the BMI
90 server. To set up a messages for processing by the BMI
91 server, you MUST set the "bmi_run" control statement in any
92 ACL for an incoming message. You will typically do this in
93 an "accept" block in the "acl_check_rcpt" ACL. You should
94 use the "accept" block(s) that accept messages from remote
95 servers for your own domain(s). Here is an example that uses
3ec3e3bb 96 the "accept" blocks from Exim's default configuration file:
8ff3788c 97
ee161e8f
PH
98
99 accept domains = +local_domains
100 endpass
101 verify = recipient
102 control = bmi_run
103
104 accept domains = +relay_to_domains
105 endpass
106 verify = recipient
107 control = bmi_run
8ff3788c 108
ee161e8f
PH
109 If bmi_run is not set in any ACL during reception of the
110 message, it will NOT be passed to the BMI server.
111
112
1134) Setting up routers to use BMI verdicts
114
115 When a message has been run through the BMI server, one or
116 more "verdicts" are present. Different recipients can have
117 different verdicts. Each recipient is treated individually
118 during routing, so you can query the verdicts by recipient
3ec3e3bb 119 at that stage. From Exim's view, a verdict can have the
ee161e8f 120 following outcomes:
8ff3788c 121
ee161e8f
PH
122 o deliver the message normally
123 o deliver the message to an alternate location
124 o do not deliver the message
8ff3788c 125
ee161e8f
PH
126 To query the verdict for a recipient, the implementation
127 offers the following tools:
8ff3788c
TK
128
129
ee161e8f
PH
130 - Boolean router preconditions. These can be used in any
131 router. For a simple implementation of BMI, these may be
132 all that you need. The following preconditions are
133 available:
8ff3788c 134
ee161e8f 135 o bmi_deliver_default
8ff3788c 136
ee161e8f
PH
137 This precondition is TRUE if the verdict for the
138 recipient is to deliver the message normally. If the
139 message has not been processed by the BMI server, this
140 variable defaults to TRUE.
8ff3788c 141
ee161e8f 142 o bmi_deliver_alternate
8ff3788c 143
ee161e8f
PH
144 This precondition is TRUE if the verdict for the
145 recipient is to deliver the message to an alternate
146 location. You can get the location string from the
147 $bmi_alt_location expansion variable if you need it. See
148 further below. If the message has not been processed by
149 the BMI server, this variable defaults to FALSE.
8ff3788c 150
ee161e8f 151 o bmi_dont_deliver
8ff3788c 152
ee161e8f
PH
153 This precondition is TRUE if the verdict for the
154 recipient is NOT to deliver the message to the
155 recipient. You will typically use this precondition in a
156 top-level blackhole router, like this:
8ff3788c 157
ee161e8f
PH
158 # don't deliver messages handled by the BMI server
159 bmi_blackhole:
160 driver = redirect
161 bmi_dont_deliver
162 data = :blackhole:
8ff3788c 163
ee161e8f
PH
164 This router should be on top of all others, so messages
165 that should not be delivered do not reach other routers
166 at all. If the message has not been processed by
167 the BMI server, this variable defaults to FALSE.
8ff3788c
TK
168
169
ee161e8f
PH
170 - A list router precondition to query if rules "fired" on
171 the message for the recipient. Its name is "bmi_rule". You
172 use it by passing it a colon-separated list of rule
173 numbers. You can use this condition to route messages that
174 matched specific rules. Here is an example:
8ff3788c 175
ee161e8f
PH
176 # special router for BMI rule #5, #8 and #11
177 bmi_rule_redirect:
178 driver = redirect
179 bmi_rule = 5:8:11
180 data = postmaster@mydomain.com
8ff3788c
TK
181
182
ee161e8f
PH
183 - Expansion variables. Several expansion variables are set
184 during routing. You can use them in custom router
185 conditions, for example. The following variables are
186 available:
8ff3788c 187
ee161e8f 188 o $bmi_base64_verdict
8ff3788c 189
ee161e8f
PH
190 This variable will contain the BASE64 encoded verdict
191 for the recipient being routed. You can use it to add a
192 header to messages for tracking purposes, for example:
8ff3788c 193
ee161e8f
PH
194 localuser:
195 driver = accept
196 check_local_user
197 headers_add = X-Brightmail-Verdict: $bmi_base64_verdict
198 transport = local_delivery
8ff3788c 199
ee161e8f
PH
200 If there is no verdict available for the recipient being
201 routed, this variable contains the empty string.
8ff3788c 202
ee161e8f 203 o $bmi_base64_tracker_verdict
8ff3788c 204
ee161e8f
PH
205 This variable will contain a BASE64 encoded subset of
206 the verdict information concerning the "rules" that
207 fired on the message. You can add this string to a
208 header, commonly named "X-Brightmail-Tracker". Example:
8ff3788c 209
ee161e8f
PH
210 localuser:
211 driver = accept
212 check_local_user
213 headers_add = X-Brightmail-Tracker: $bmi_base64_tracker_verdict
214 transport = local_delivery
8ff3788c 215
ee161e8f
PH
216 If there is no verdict available for the recipient being
217 routed, this variable contains the empty string.
8ff3788c 218
ee161e8f 219 o $bmi_alt_location
8ff3788c 220
ee161e8f
PH
221 If the verdict is to redirect the message to an
222 alternate location, this variable will contain the
223 alternate location string returned by the BMI server. In
224 its default configuration, this is a header-like string
225 that can be added to the message with "headers_add". If
226 there is no verdict available for the recipient being
227 routed, or if the message is to be delivered normally,
228 this variable contains the empty string.
8ff3788c 229
ee161e8f 230 o $bmi_deliver
8ff3788c 231
ee161e8f
PH
232 This is an additional integer variable that can be used
233 to query if the message should be delivered at all. You
234 should use router preconditions instead if possible.
8ff3788c 235
ee161e8f
PH
236 $bmi_deliver is '0': the message should NOT be delivered.
237 $bmi_deliver is '1': the message should be delivered.
8ff3788c
TK
238
239
ee161e8f
PH
240 IMPORTANT NOTE: Verdict inheritance.
241 The message is passed to the BMI server during message
242 reception, using the target addresses from the RCPT TO:
243 commands in the SMTP transaction. If recipients get expanded
244 or re-written (for example by aliasing), the new address(es)
245 inherit the verdict from the original address. This means
246 that verdicts also apply to all "child" addresses generated
247 from top-level addresses that were sent to the BMI server.
8ff3788c
TK
248
249
ee161e8f
PH
2505) Using per-recipient opt-in information (Optional)
251
252 The BMI server features multiple scanning "profiles" for
253 individual recipients. These are usually stored in a LDAP
254 server and are queried by the BMI server itself. However,
255 you can also pass opt-in data for each recipient from the
256 MTA to the BMI server. This is particularly useful if you
3ec3e3bb 257 already look up recipient data in Exim anyway (which can
ee161e8f
PH
258 also be stored in a SQL database or other source). This
259 implementation enables you to pass opt-in data to the BMI
260 server in the RCPT ACL. This works by setting the
261 'bmi_optin' modifier in a block of that ACL. If should be
262 set to a list of comma-separated strings that identify the
263 features which the BMI server should use for that particular
264 recipient. Ideally, you would use the 'bmi_optin' modifier
265 in the same ACL block where you set the 'bmi_run' control
266 flag. Here is an example that will pull opt-in data for each
267 recipient from a flat file called
268 '/etc/exim/bmi_optin_data'.
8ff3788c 269
ee161e8f 270 The file format:
8ff3788c 271
ee161e8f
PH
272 user1@mydomain.com: <OPTIN STRING1>:<OPTIN STRING2>
273 user2@thatdomain.com: <OPTIN STRING3>
8ff3788c
TK
274
275
ee161e8f 276 The example:
8ff3788c 277
ee161e8f
PH
278 accept domains = +relay_to_domains
279 endpass
280 verify = recipient
281 bmi_optin = ${lookup{$local_part@$domain}lsearch{/etc/exim/bmi_optin_data}}
8ff3788c
TK
282 control = bmi_run
283
ee161e8f 284 Of course, you can also use any other lookup method that
3ec3e3bb 285 Exim supports, including LDAP, Postgres, MySQL, Oracle etc.,
ee161e8f
PH
286 as long as the result is a list of colon-separated opt-in
287 strings.
8ff3788c 288
ee161e8f
PH
289 For a list of available opt-in strings, please contact your
290 Brightmail representative.
ee161e8f 291
8ff3788c
TK
292
293
294
7ef88aa0 295SRS (Sender Rewriting Scheme) Support (using libsrs_alt)
ee161e8f 296--------------------------------------------------------------
7ef88aa0 297See also below, for an alternative native support implementation.
ee161e8f 298
7ef88aa0 299Exim currently includes SRS support via Miles Wilton's
8ff3788c 300libsrs_alt library. The current version of the supported
f413521a 301library is 0.5, there are reports of 1.0 working.
ee161e8f
PH
302
303In order to use SRS, you must get a copy of libsrs_alt from
304
f413521a
PP
305https://opsec.eu/src/srs/
306
307(not the original source, which has disappeared.)
ee161e8f
PH
308
309Unpack the tarball, then refer to MTAs/README.EXIM
310to proceed. You need to set
311
312EXPERIMENTAL_SRS=yes
313
314in your Local/Makefile.
315
002fcd22
JH
316The following main-section options become available:
317 srs_config string
318 srs_hashlength int
319 srs_hashmin int
320 srs_maxage int
321 srs_secrets string
322 srs_usehash bool
323 srs_usetimestamp bool
324
325The redirect router gains these options (all of type string, unset by default):
326 srs
327 srs_alias
328 srs_condition
329 srs_dbinsert
330 srs_dbselect
331
332The following variables become available:
333 $srs_db_address
334 $srs_db_key
335 $srs_orig_recipient
336 $srs_orig_sender
337 $srs_recipient
338 $srs_status
339
340The predefined feature-macro _HAVE_SRS will be present.
341Additional delivery log line elements, tagged with "SRS=" will show the srs sender.
342For configuration information see https://github.com/Exim/exim/wiki/SRS .
343
344
ee161e8f 345
f413521a 346
7ef88aa0
JH
347SRS (Sender Rewriting Scheme) Support (native)
348--------------------------------------------------------------
349This is less full-featured than the libsrs_alt version above.
350
351The Exim build needs to be done with this in Local/Makefile:
352EXPERIMENTAL_SRS_NATIVE=yes
353
354The following are provided:
355- an expansion item "srs_encode"
356 This takes three arguments:
357 - a site SRS secret
358 - the return_path
359 - the pre-forwarding domain
360
361- an expansion condition "inbound_srs"
362 This takes two arguments: the local_part to check, and a site SRS secret.
363 If the secret is zero-length, only the pattern of the local_part is checked.
364 The $srs_recipient variable is set as a side-effect.
365
366- an expansion variable $srs_recipient
367 This gets the original return_path encoded in the SRS'd local_part
368
369- predefined macros _HAVE_SRS and _HAVE_NATIVE_SRS
370
371Sample usage:
372
373 #macro
374 SRS_SECRET = <pick something unique for your site for this>
375
376 #routers
377
378 outbound:
379 driver = dnslookup
380 # if outbound, and forwarding has been done, use an alternate transport
381 domains = ! +my_domains
382 transport = ${if eq {$local_part@$domain} \
383 {$original_local_part@$original_domain} \
384 {remote_smtp} {remote_forwarded_smtp}}
385
386 inbound_srs:
387 driver = redirect
388 senders = :
389 domains = +my_domains
390 # detect inbound bounces which are SRS'd, and decode them
391 condition = ${if inbound_srs {$local_part} {SRS_SECRET}}
392 data = $srs_recipient
393
394 inbound_srs_failure:
395 driver = redirect
396 senders = :
397 domains = +my_domains
398 # detect inbound bounces which look SRS'd but are invalid
399 condition = ${if inbound_srs {$local_part} {}}
400 allow_fail
401 data = :fail: Invalid SRS recipient address
402
403 #... further routers here
404
405
406 # transport; should look like the non-forward outbound
407 # one, plus the max_rcpt and return_path options
408 remote_forwarded_smtp:
409 driver = smtp
410 # modify the envelope from, for mails that we forward
411 max_rcpt = 1
412 return_path = ${srs_encode {SRS_SECRET} {$return_path} {$original_domain}}
413
414
415
416
0e1ccf44
PP
417DCC Support
418--------------------------------------------------------------
d36254f2 419Distributed Checksum Clearinghouse; http://www.rhyolite.com/dcc/
0e1ccf44
PP
420
421*) Building exim
422
423In order to build exim with DCC support add
424
425EXPERIMENTAL_DCC=yes
426
427to your Makefile. (Re-)build/install exim. exim -d should show
428EXPERIMENTAL_DCC under "Support for".
429
430
431*) Configuration
432
433In the main section of exim.cf add at least
434 dccifd_address = /usr/local/dcc/var/dccifd
435or
436 dccifd_address = <ip> <port>
437
438In the DATA ACL you can use the new condition
439 dcc = *
440
441After that "$dcc_header" contains the X-DCC-Header.
442
d36a0501 443Return values are:
0e1ccf44
PP
444 fail for overall "R", "G" from dccifd
445 defer for overall "T" from dccifd
446 accept for overall "A", "S" from dccifd
447
448dcc = */defer_ok works as for spamd.
449
450The "$dcc_result" variable contains the overall result from DCC
451answer. There will an X-DCC: header added to the mail.
452
453Usually you'll use
454 defer !dcc = *
455to greylist with DCC.
456
457If you set, in the main section,
458 dcc_direct_add_header = true
459then the dcc header will be added "in deep" and if the spool
460file was already written it gets removed. This forces Exim to
461write it again if needed. This helps to get the DCC Header
462through to eg. SpamAssassin.
463
464If you want to pass even more headers in the middle of the
465DATA stage you can set
466 $acl_m_dcc_add_header
05c39afa 467to tell the DCC routines to add more information; eg, you might set
0e1ccf44
PP
468this to some results from ClamAV. Be careful. Header syntax is
469not checked and is added "as is".
470
05c39afa
JH
471In case you've troubles with sites sending the same queue items from several
472hosts and fail to get through greylisting you can use
473$acl_m_dcc_override_client_ip
474
475Setting $acl_m_dcc_override_client_ip to an IP address overrides the default
476of $sender_host_address. eg. use the following ACL in DATA stage:
477
478 warn set acl_m_dcc_override_client_ip = \
479 ${lookup{$sender_helo_name}nwildlsearch{/etc/mail/multipleip_sites}{$value}{}}
480 condition = ${if def:acl_m_dcc_override_client_ip}
481 log_message = dbg: acl_m_dcc_override_client_ip set to \
482 $acl_m_dcc_override_client_ip
483
484Then set something like
485# cat /etc/mail/multipleip_sites
486mout-xforward.gmx.net 82.165.159.12
487mout.gmx.net 212.227.15.16
488
4c04137d 489Use a reasonable IP. eg. one the sending cluster actually uses.
0e1ccf44 490
48155ab3
JH
491
492
895fbaf2
JH
493DSN extra information
494---------------------
495If compiled with EXPERIMENTAL_DSN_INFO extra information will be added
496to DSN fail messages ("bounces"), when available. The intent is to aid
497tracing of specific failing messages, when presented with a "bounce"
498complaint and needing to search logs.
499
500
501The remote MTA IP address, with port number if nonstandard.
502Example:
503 Remote-MTA: X-ip; [127.0.0.1]:587
504Rationale:
505 Several addresses may correspond to the (already available)
506 dns name for the remote MTA.
507
508The remote MTA connect-time greeting.
509Example:
510 X-Remote-MTA-smtp-greeting: X-str; 220 the.local.host.name ESMTP Exim x.yz Tue, 2 Mar 1999 09:44:33 +0000
511Rationale:
512 This string sometimes presents the remote MTA's idea of its
513 own name, and sometimes identifies the MTA software.
514
515The remote MTA response to HELO or EHLO.
516Example:
517 X-Remote-MTA-helo-response: X-str; 250-the.local.host.name Hello localhost [127.0.0.1]
518Limitations:
519 Only the first line of a multiline response is recorded.
520Rationale:
521 This string sometimes presents the remote MTA's view of
522 the peer IP connecting to it.
523
524The reporting MTA detailed diagnostic.
525Example:
526 X-Exim-Diagnostic: X-str; SMTP error from remote mail server after RCPT TO:<d3@myhost.test.ex>: 550 hard error
527Rationale:
4c04137d 528 This string sometimes give extra information over the
895fbaf2
JH
529 existing (already available) Diagnostic-Code field.
530
531
532Note that non-RFC-documented field names and data types are used.
533
534
5bde3efa
ACK
535LMDB Lookup support
536-------------------
537LMDB is an ultra-fast, ultra-compact, crash-proof key-value embedded data store.
4c04137d 538It is modeled loosely on the BerkeleyDB API. You should read about the feature
5bde3efa
ACK
539set as well as operation modes at https://symas.com/products/lightning-memory-mapped-database/
540
541LMDB single key lookup support is provided by linking to the LMDB C library.
542The current implementation does not support writing to the LMDB database.
543
544Visit https://github.com/LMDB/lmdb to download the library or find it in your
545operating systems package repository.
546
547If building from source, this description assumes that headers will be in
548/usr/local/include, and that the libraries are in /usr/local/lib.
549
5501. In order to build exim with LMDB lookup support add or uncomment
551
552EXPERIMENTAL_LMDB=yes
553
554to your Local/Makefile. (Re-)build/install exim. exim -d should show
555Experimental_LMDB in the line "Support for:".
556
557EXPERIMENTAL_LMDB=yes
558LDFLAGS += -llmdb
559# CFLAGS += -I/usr/local/include
560# LDFLAGS += -L/usr/local/lib
561
562The first line sets the feature to include the correct code, and
563the second line says to link the LMDB libraries into the
564exim binary. The commented out lines should be uncommented if you
565built LMDB from source and installed in the default location.
566Adjust the paths if you installed them elsewhere, but you do not
567need to uncomment them if an rpm (or you) installed them in the
568package controlled locations (/usr/include and /usr/lib).
569
5702. Create your LMDB files, you can use the mdb_load utility which is
571part of the LMDB distribution our your favourite language bindings.
572
5733. Add the single key lookups to your exim.conf file, example lookups
574are below.
575
576${lookup{$sender_address_domain}lmdb{/var/lib/baruwa/data/db/relaydomains.mdb}{$value}}
577${lookup{$sender_address_domain}lmdb{/var/lib/baruwa/data/db/relaydomains.mdb}{$value}fail}
578${lookup{$sender_address_domain}lmdb{/var/lib/baruwa/data/db/relaydomains.mdb}}
895fbaf2 579
ed0512a1 580
3369a853
ACK
581Queuefile transport
582-------------------
583Queuefile is a pseudo transport which does not perform final delivery.
584It simply copies the exim spool files out of the spool directory into
585an external directory retaining the exim spool format.
586
587The spool files can then be processed by external processes and then
588requeued into exim spool directories for final delivery.
20751c76
JH
589However, note carefully the warnings in the main documentation on
590qpool file formats.
3369a853
ACK
591
592The motivation/inspiration for the transport is to allow external
593processes to access email queued by exim and have access to all the
594information which would not be available if the messages were delivered
595to the process in the standard email formats.
596
597The mailscanner package is one of the processes that can take advantage
598of this transport to filter email.
599
600The transport can be used in the same way as the other existing transports,
601i.e by configuring a router to route mail to a transport configured with
602the queuefile driver.
603
604The transport only takes one option:
605
606* directory - This is used to specify the directory messages should be
9604413b 607copied to. Expanded.
3369a853
ACK
608
609The generic transport options (body_only, current_directory, disable_logging,
610debug_print, delivery_date_add, envelope_to_add, event_action, group,
611headers_add, headers_only, headers_remove, headers_rewrite, home_directory,
612initgroups, max_parallel, message_size_limit, rcpt_include_affixes,
613retry_use_local_part, return_path, return_path_add, shadow_condition,
614shadow_transport, transport_filter, transport_filter_timeout, user) are
615ignored.
616
617Sample configuration:
618
619(Router)
620
621scan:
622 driver = accept
623 transport = scan
624
625(Transport)
626
627scan:
628 driver = queuefile
629 directory = /var/spool/baruwa-scanner/input
630
631
632In order to build exim with Queuefile transport support add or uncomment
633
634EXPERIMENTAL_QUEUEFILE=yes
635
636to your Local/Makefile. (Re-)build/install exim. exim -d should show
637Experimental_QUEUEFILE in the line "Support for:".
638
639
617d3932
JH
640ARC support
641-----------
642Specification: https://tools.ietf.org/html/draft-ietf-dmarc-arc-protocol-11
643Note that this is not an RFC yet, so may change.
644
dc2d885a
JH
645[RFC 8617 was published 2019/06. Draft 11 was 2018/01. A review of the
646changes has not yet been done]
647
617d3932
JH
648ARC is intended to support the utility of SPF and DKIM in the presence of
649intermediaries in the transmission path - forwarders and mailinglists -
650by establishing a cryptographically-signed chain in headers.
651
652Normally one would only bother doing ARC-signing when functioning as
653an intermediary. One might do verify for local destinations.
654
655ARC uses the notion of a "ADministrative Management Domain" (ADMD).
dc2d885a
JH
656Described in RFC 5598 (section 2.3), this is essentially a set of
657mail-handling systems that mail transits that are all under the control
658of one organisation. A label should be chosen to identify the ADMD.
659Messages should be ARC-verified on entry to the ADMD, and ARC-signed on exit
660from it.
617d3932
JH
661
662
2b2cfa83
JH
663Building with ARC Support
664--
665Enable using EXPERIMENTAL_ARC=yes in your Local/Makefile.
666You must also have DKIM present (not disabled), and you very likely
667want to have SPF enabled.
668
669
617d3932
JH
670Verification
671--
672An ACL condition is provided to perform the "verifier actions" detailed
673in section 6 of the above specification. It may be called from the DATA ACL
674and succeeds if the result matches any of a given list.
675It also records the highest ARC instance number (the chain size)
676and verification result for later use in creating an Authentication-Results:
677standard header.
678
679 verify = arc/<acceptable_list> none:fail:pass
680
681 add_header = :at_start:${authresults {<admd-identifier>}}
682
683 Note that it would be wise to strip incoming messages of A-R headers
684 that claim to be from our own <admd-identifier>.
685
a1d4300b 686There are four new variables:
9cffa436
JH
687
688 $arc_state One of pass, fail, none
689 $arc_state_reason (if fail, why)
08bd2689
JH
690 $arc_domains colon-sep list of ARC chain domains, in chain order.
691 problematic elements may have empty list elements
ea7b1f16 692 $arc_oldest_pass lowest passing instance number of chain
93c931f8 693
bce15b62
JH
694Example:
695 logwrite = oldest-p-ams: <${reduce {$lh_ARC-Authentication-Results:} \
696 {} \
697 {${if = {$arc_oldest_pass} \
698 {${extract {i}{${extract {1}{;}{$item}}}}} \
699 {$item} {$value}}} \
700 }>
701
617d3932
JH
702Receive log lines for an ARC pass will be tagged "ARC".
703
704
705Signing
706--
a93dbf44 707arc_sign = <admd-identifier> : <selector> : <privkey> [ : <options> ]
617d3932
JH
708An option on the smtp transport, which constructs and prepends to the message
709an ARC set of headers. The textually-first Authentication-Results: header
710is used as a basis (you must have added one on entry to the ADMD).
9f7d9fa1 711Expanded as a whole; if unset, empty or forced-failure then no signing is done.
5054c4fd 712If it is set, all of the first three elements must be non-empty.
617d3932 713
a93dbf44 714The fourth element is optional, and if present consists of a comma-separated list
0e83c148
JH
715of options. The options implemented are
716
717 timestamps Add a t= tag to the generated AMS and AS headers, with the
718 current time.
719 expire[=<val>] Add an x= tag to the generated AMS header, with an expiry time.
720 If the value <val> is an plain number it is used unchanged.
721 If it starts with a '+' then the following number is added
722 to the current time, as an offset in seconds.
723 If a value is not given it defaults to a one month offset.
a93dbf44
JH
724
725[As of writing, gmail insist that a t= tag on the AS is mandatory]
726
afcdd656
PP
727Caveats:
728 * There must be an Authentication-Results header, presumably added by an ACL
729 while receiving the message, for the same ADMD, for arc_sign to succeed.
730 This requires careful coordination between inbound and outbound logic.
5054c4fd
JH
731
732 Only one A-R header is taken account of. This is a limitation versus
733 the ARC spec (which says that all A-R headers from within the ADMD must
734 be used).
735
afcdd656
PP
736 * If passing a message to another system, such as a mailing-list manager
737 (MLM), between receipt and sending, be wary of manipulations to headers made
738 by the MLM.
739 + For instance, Mailman with REMOVE_DKIM_HEADERS==3 might improve
740 deliverability in a pre-ARC world, but that option also renames the
741 Authentication-Results header, which breaks signing.
5054c4fd 742
afcdd656
PP
743 * Even if you use multiple DKIM keys for different domains, the ARC concept
744 should try to stick to one ADMD, so pick a primary domain and use that for
745 AR headers and outbound signing.
746
5add7dc4
JH
747Signing is not compatible with cutthrough delivery; any (before expansion)
748value set for the option will result in cutthrough delivery not being
749used via the transport in question.
750
617d3932
JH
751
752
8ac90765 753
b10c87b3
JH
754TLS Session Resumption
755----------------------
43e2db44 756TLS Session Resumption for TLS 1.2 and TLS 1.3 connections can be used (defined
b10c87b3 757in RFC 5077 for 1.2). The support for this can be included by building with
43e2db44
JH
758EXPERIMENTAL_TLS_RESUME defined. This requires GnuTLS 3.6.3 or OpenSSL 1.1.1
759(or later).
b10c87b3
JH
760
761Session resumption (this is the "stateless" variant) involves the server sending
762a "session ticket" to the client on one connection, which can be stored by the
763client and used for a later session. The ticket contains sufficient state for
764the server to reconstruct the TLS session, avoiding some expensive crypto
765calculation and one full packet roundtrip time.
766
767Operational cost/benefit:
768 The extra data being transmitted costs a minor amount, and the client has
68c62739 769 extra costs in storing and retrieving the data.
b10c87b3 770
68c62739
JH
771 In the Exim/Gnutls implementation the extra cost on an initial connection
772 which is TLS1.2 over a loopback path is about 6ms on 2017-laptop class hardware.
773 The saved cost on a subsequent connection is about 4ms; three or more
774 connections become a net win. On longer network paths, two or more
775 connections will have an average lower startup time thanks to the one
776 saved packet roundtrip. TLS1.3 will save the crypto cpu costs but not any
777 packet roundtrips.
778
779 Since a new hints DB is used, the hints DB maintenance should be updated
780 to additionally handle "tls".
b10c87b3
JH
781
782Security aspects:
783 The session ticket is encrypted, but is obviously an additional security
68c62739
JH
784 vulnarability surface. An attacker able to decrypt it would have access
785 all connections using the resumed session.
786 The session ticket encryption key is not committed to storage by the server
dea4b568
JH
787 and is rotated regularly (OpenSSL: 1hr, and one previous key is used for
788 overlap; GnuTLS 6hr but does not specify any overlap).
789 Tickets have limited lifetime (2hr, and new ones issued after 1hr under
790 OpenSSL. GnuTLS 2hr, appears to not do overlap).
b10c87b3 791
68c62739
JH
792 There is a question-mark over the security of the Diffie-Helman parameters
793 used for session negotiation. TBD. q-value; cf bug 1895
b10c87b3
JH
794
795Observability:
796 New log_selector "tls_resumption", appends an asterisk to the tls_cipher "X="
68c62739
JH
797 element.
798
4a1bd6b9 799 Variables $tls_{in,out}_resumption have bits 0-4 indicating respectively
68c62739
JH
800 support built, client requested ticket, client offered session,
801 server issued ticket, resume used. A suitable decode list is provided
802 in the builtin macro _RESUME_DECODE for ${listextract {}{}}.
803
804Issues:
805 In a resumed session:
11c4a22b 806 $tls_{in,out}_cipher will have values different to the original (under GnuTLS)
4f1d23a1 807 $tls_{in,out}_ocsp will be "not requested" or "no response", and
f4e62a87 808 hosts_require_ocsp will fail
b10c87b3
JH
809
810
ee161e8f
PH
811--------------------------------------------------------------
812End of file
813--------------------------------------------------------------