4.52 TK/01
[exim.git] / doc / doc-txt / NewStuff
7d50add3 1$Cambridge: exim/doc/doc-txt/NewStuff,v 1.47 2005/05/25 20:33:28 tom Exp $
3New Features in Exim
6This file contains descriptions of new features that have been added to Exim,
7but have not yet made it into the main manual (which is most conveniently
8updated when there is a relatively large batch of changes). The doc/ChangeLog
9file contains a listing of all changes, including bug fixes.
12Exim version 4.52
15TF/01 Support for checking Client SMTP Authorization has been added. CSA is a
16 system which allows a site to advertise which machines are and are not
17 permitted to send email. This is done by placing special SRV records in
18 the DNS, which are looked up using the client's HELO domain. At this
19 time CSA is still an Internet-Draft.
21 Client SMTP Authorization checks are performed by the ACL condition
22 verify=csa. This will fail if the client is not authorized. If there is
23 a DNS problem, or if no valid CSA SRV record is found, or if the client
24 is authorized, the condition succeeds. These three cases can be
25 distinguished using the expansion variable $csa_status, which can take
26 one of the values "fail", "defer", "unknown", or "ok". The condition
27 does not itself defer because that would be likely to cause problems
28 for legitimate email.
30 The error messages produced by the CSA code include slightly more
31 detail. If $csa_status is "defer" this may be because of problems
32 looking up the CSA SRV record, or problems looking up the CSA target
33 address record. There are four reasons for $csa_status being "fail":
34 the client's host name is explicitly not authorized; the client's IP
35 address does not match any of the CSA target IP addresses; the client's
36 host name is authorized but it has no valid target IP addresses (e.g.
37 the target's addresses are IPv6 and the client is using IPv4); or the
38 client's host name has no CSA SRV record but a parent domain has
39 asserted that all subdomains must be explicitly authorized.
41 The verify=csa condition can take an argument which is the domain to
42 use for the DNS query. The default is verify=csa/$sender_helo_name.
44 This implementation includes an extension to CSA. If the query domain
45 is an address literal such as [], or if it is a bare IP
46 address, Exim will search for CSA SRV records in the reverse DNS as if
47 the HELO domain was e.g. Therefore it is
48 meaningful to say, for example, verify=csa/$sender_host_address - in
49 fact, this is the check that Exim performs if the client does not say
50 HELO. This extension can be turned off by setting the main
51 configuration option dns_csa_use_reverse = false.
53 If a CSA SRV record is not found for the domain itself, then a search
54 is performed through its parent domains for a record which might be
55 making assertions about subdomains. The maximum depth of this search is
56 limited using the main configuration option dns_csa_search_limit, which
57 takes the value 5 by default. Exim does not look for CSA SRV records in
58 a top level domain, so the default settings handle HELO domains as long
59 as seven (hostname.five.four.three.two.one.com) which encompasses the
60 vast majority of legitimate HELO domains.
62 The dnsdb lookup also has support for CSA. Although dnsdb already
63 supports SRV lookups, this is not sufficient because of the extra
64 parent domain search behaviour of CSA, and (as with PTR lookups)
65 dnsdb also turns IP addresses into lookups in the reverse DNS space.
66 The result of ${lookup dnsdb {csa=$sender_helo_name} } has two
67 space-separated fields: an authorization code and a target host name.
68 The authorization code can be "Y" for yes, "N" for no, "X" for explicit
69 authorization required but absent, or "?" for unknown.
71PH/01 The amount of output produced by the "make" process has been reduced,
72 because the compile lines are often rather long, making it all pretty
73 unreadable. The new style is along the lines of the 2.6 Linux kernel:
74 just a short line for each module that is being compiled or linked.
75 However, it is still possible to get the full output, by calling "make"
76 like this:
78 FULLECHO='' make -e
80 The value of FULLECHO defaults to "@", the flag character that suppresses
81 command reflection in "make". When you ask for the full output, it is
82 given in addition to the the short output.
4df1e33e 84TF/02 There have been two changes concerned with submission mode:
87ba3f5f 85
86 Until now submission mode always left the return path alone, whereas
87 locally-submitted messages from untrusted users have the return path
88 fixed to the user's email address. Submission mode now fixes the return
89 path to the same address as is used to create the Sender: header. If
90 /sender_retain is specified then both the Sender: header and the return
91 path are left alone.
87ba3f5f 92
93 Note that the changes caused by submission mode take effect after the
94 predata ACL. This means that any sender checks performed before the
95 fix-ups will use the untrusted sender address specified by the user, not
96 the trusted sender address specified by submission mode. Although this
97 might be slightly unexpected, it does mean that you can configure ACL
98 checks to spot that a user is trying to spoof another's address, for
99 example.
87ba3f5f 100
101 There is also a new /name= option for submission mode which allows you
102 to specify the user's full name to be included in the Sender: header.
103 For example:
105 accept authenticated = *
106 control = submission/name=${lookup {$authenticated_id} \
107 lsearch {/etc/exim/namelist} }
109 The namelist file contains entries like
111 fanf: Tony Finch
113 And the resulting Sender: header looks like
115 Sender: Tony Finch <fanf@exim.org>
117TF/03 The control = fakereject ACL modifier now has a fakedefer counterpart,
118 which works in exactly the same way except it causes a fake SMTP 450
119 response after the message data instead of a fake SMTP 550 response.
120 You must take care when using fakedefer because it will cause messages
121 to be duplicated when the sender retries. Therefore you should not use
122 fakedefer if the message will be delivered normally.
124TF/04 There is a new ratelimit ACL condition which can be used to measure
125 and control the rate at which clients can send email. This is more
126 powerful than the existing smtp_ratelimit_* options, because those
127 options only control the rate of commands in a single SMTP session,
128 whereas the new ratelimit condition works across all connections
129 (concurrent and sequential) to the same host.
131 The syntax of the ratelimit condition is:
133 ratelimit = <m> / <p> / <options> / <key>
135 If the average client sending rate is less than m messages per time
136 period p then the condition is false, otherwise it is true.
138 The parameter p is the smoothing time constant, in the form of an Exim
139 time interval e.g. 8h for eight hours. A larger time constant means it
140 takes Exim longer to forget a client's past behaviour. The parameter m is
141 the maximum number of messages that a client can send in a fast burst. By
142 increasing both m and p but keeping m/p constant, you can allow a client
143 to send more messages in a burst without changing its overall sending
144 rate limit. Conversely, if m and p are both small then messages must be
145 sent at an even rate.
147 The key is used to look up the data used to calcluate the client's
148 average sending rate. This data is stored in a database maintained by
149 Exim in its spool directory alongside the retry database etc. For
150 example, you can limit the sending rate of each authenticated user,
151 independent of the computer they are sending from, by setting the key
152 to $authenticated_id. The default key is $sender_host_address.
154 Each ratelimit condition can have up to two options. The first option
155 specifies what Exim measures the rate of, and the second specifies how
156 Exim handles excessively fast clients.
158 The per_mail option means that it measures the client's rate of sending
159 messages. This is the default if none of the per_* options is specified.
161 The per_conn option means that it measures the client's connection rate.
163 The per_byte option limits the sender's email bandwidth. Note that it
164 is best to use this option in the DATA ACL; if it is used in an earlier
165 ACL it relies on the SIZE parameter on the MAIL command, which may be
166 inaccurate or completely missing. You can follow the limit m in the
167 configuration with K, M, or G to specify limits in kilobytes,
168 megabytes, or gigabytes respectively.
170 The per_cmd option means that Exim recomputes the rate every time the
171 condition is processed, which can be used to limit the SMTP command rate.
172 The alias per_rcpt is provided for use in the RCPT ACL instead of per_cmd
173 to make it clear that the effect is to limit the rate at which recipients
174 are accepted. Note that in this case the rate limiting engine will see a
175 message with many recipients as a large high-speed burst.
177 If a client's average rate is greater than the maximum, the rate
178 limiting engine can react in two possible ways, depending on the
179 presence of the strict or leaky options. This is independent of the
180 other counter-measures (e.g. rejecting the message) that may be
181 specified by the rest of the ACL. The default mode is leaky, which
182 avoids a sender's over-aggressive retry rate preventing it from getting
183 any email through.
185 The strict option means that the client's recorded rate is always
186 updated. The effect of this is that Exim measures the client's average
187 rate of attempts to send email, which can be much higher than the
188 maximum. If the client is over the limit it will be subjected to
189 counter-measures until it slows down below the maximum rate.
191 The leaky option means that the client's recorded rate is not updated
192 if it is above the limit. The effect of this is that Exim measures the
193 client's average rate of successfully sent email, which cannot be
194 greater than the maximum. If the client is over the limit it will
195 suffer some counter-measures, but it will still be able to send email
196 at the configured maximum rate, whatever the rate of its attempts.
198 As a side-effect, the ratelimit condition will set the expansion
199 variables $sender_rate containing the client's computed rate,
200 $sender_rate_limit containing the configured value of m, and
201 $sender_rate_period containing the configured value of p.
203 Exim's other ACL facilities are used to define what counter-measures
204 are taken when the rate limit is exceeded. This might be anything from
205 logging a warning (e.g. while measuring existing sending rates in order
206 to define our policy), through time delays to slow down fast senders,
207 up to rejecting the message. For example,
209 # Log all senders' rates
210 warn
211 ratelimit = 0 / 1h / strict
212 log_message = \
213 Sender rate $sender_rate > $sender_rate_limit / $sender_rate_period
215 # Slow down fast senders
216 warn
217 ratelimit = 100 / 1h / per_rcpt / strict
218 delay = ${eval: 10 * ($sender_rate - $sender_rate_limit) }
220 # Keep authenticated users under control
221 deny
222 ratelimit = 100 / 1d / strict / $authenticated_id
224 # System-wide rate limit
225 defer
226 message = Sorry, too busy. Try again later.
227 ratelimit = 10 / 1s / $primary_hostname
229 # Restrict incoming rate from each host, with a default rate limit
230 # set using a macro and special cases looked up in a table.
231 defer
232 message = Sender rate $sender_rate exceeds \
233 $sender_rate_limit messages per $sender_rate_period
234 ratelimit = ${lookup {$sender_host_address} \
235 cdb {DB/ratelimits.cdb} \
236 {$value} {RATELIMIT} }
238TK/01 Added an 'spf' lookup type that will return an SPF result for a given
239 email address (the key) and an IP address (the database):
241 ${lookup {tom@duncanthrax.net} spf{}}
243 The lookup will return the same result strings as they can appear in
244 $spf_result (pass,fail,softfail,neutral,none,err_perm,err_temp). The
245 lookup is armored in EXPERIMENTAL_SPF. Currently, only IPv4 addresses
246 are supported.
248 Patch submitted by Chris Webb <chris@arachsys.com>.
e5a9dba6 250
251Version 4.51
254PH/01 The format in which GnuTLS parameters are written to the gnutls-param
255 file in the spool directory has been changed. This change has been made
256 to alleviate problems that some people had with the generation of the
257 parameters by Exim when /dev/random was exhausted. In this situation,
258 Exim would hang until /dev/random acquired some more entropy.
260 The new code exports and imports the DH and RSA parameters in PEM
261 format. This means that the parameters can be generated externally using
262 the certtool command that is part of GnuTLS.
264 To replace the parameters with new ones, instead of deleting the file
265 and letting Exim re-create it, you can generate new parameters using
266 certtool and, when this has been done, replace Exim's cache file by
267 renaming. The relevant commands are something like this:
269 # rm -f new.params
270 # touch new.params
271 # chown exim:exim new.params
272 # chmod 0400 new.params
273 # certtool --generate-privkey --bits 512 >new.params
274 # echo "" >>new.params
275 # certtool --generate-dh-params --bits 1024 >> new.params
276 # mv new.params params
278 If Exim never has to generate the parameters itself, the possibility of
279 stalling is removed.
281PH/02 A new expansion item for dynamically loading and calling a locally-
282 written C function is now provided, if Exim is compiled with
286 set in Local/Makefile. The facility is not included by default (a
287 suitable error is given if you try to use it when it is not there.)
289 If you enable EXPAND_DLFUNC, you should also be aware of the new redirect
290 router option forbid_filter_dlfunc. If you have unprivileged users on
291 your system who are permitted to create filter files, you might want to
292 set forbid_filter_dlfunc=true in the appropriate router, to stop them
293 using ${dlfunc to run code within Exim.
295 You load and call an external function like this:
297 ${dlfunc{/some/file}{function}{arg1}{arg2}...}
299 Once loaded, Exim remembers the dynamically loaded object so that it
300 doesn't reload the same object file in the same Exim process (but of
301 course Exim does start new processes frequently).
303 There may be from zero to eight arguments to the function. When compiling
304 a local function that is to be called in this way, local_scan.h should be
305 included. The Exim variables and functions that are defined by that API
306 are also available for dynamically loaded functions. The function itself
307 must have the following type:
309 int dlfunction(uschar **yield, int argc, uschar *argv[])
311 Where "uschar" is a typedef for "unsigned char" in local_scan.h. The
312 function should return one of the following values:
314 OK Success. The string that is placed in "yield" is put into
315 the expanded string that is being built.
317 FAIL A non-forced expansion failure occurs, with the error
318 message taken from "yield", if it is set.
320 FAIL_FORCED A forced expansion failure occurs, with the error message
321 taken from "yield" if it is set.
323 ERROR Same as FAIL, except that a panic log entry is written.
325 When compiling a function that is to be used in this way with gcc,
326 you need to add -shared to the gcc command. Also, in the Exim build-time
327 configuration, you must add -export-dynamic to EXTRALIBS.
b5aea5e1 328
329TF/01 $received_time is a new expansion variable containing the time and date
330 as a number of seconds since the start of the Unix epoch when the
331 current message was received.
b5aea5e1 332
333PH/03 There is a new value for RADIUS_LIB_TYPE that can be set in
334 Local/Makefile. It is RADIUSCLIENTNEW, and it requests that the new API,
335 in use from radiusclient 0.4.0 onwards, be used. It does not appear to be
336 possible to detect the different versions automatically.
338PH/04 There is a new option called acl_not_smtp_mime that allows you to scan
339 MIME parts in non-SMTP messages. It operates in exactly the same way as
340 acl_smtp_mime
342PH/05 It is now possible to redefine a macro within the configuration file.
343 The macro must have been previously defined within the configuration (or
344 an included file). A definition on the command line using the -D option
345 causes all definitions and redefinitions within the file to be ignored.
346 In other words, -D overrides any values that are set in the file.
347 Redefinition is specified by using '==' instead of '='. For example:
349 MAC1 = initial value
350 ...
351 MAC1 == updated value
353 Redefinition does not alter the order in which the macros are applied to
354 the subsequent lines of the configuration file. It is still the same
355 order in which the macros were originally defined. All that changes is
356 the macro's value. Redefinition makes it possible to accumulate values.
357 For example:
359 MAC1 = initial value
360 ...
361 MAC1 == MAC1 and something added
363 This can be helpful in situations where the configuration file is built
364 from a number of other files.
366PH/06 Macros may now be defined or redefined between router, transport,
367 authenticator, or ACL definitions, as well as in the main part of the
368 configuration. They may not, however, be changed within an individual
369 driver or ACL, or in the local_scan, retry, or rewrite sections of the
370 configuration.
372PH/07 $acl_verify_message is now set immediately after the failure of a
373 verification in an ACL, and so is available in subsequent modifiers. In
374 particular, the message can be preserved by coding like this:
376 warn !verify = sender
377 set acl_m0 = $acl_verify_message
379 Previously, $acl_verify_message was set only while expanding "message"
380 and "log_message" when a very denied access.
382PH/08 The redirect router has two new options, sieve_useraddress and
383 sieve_subaddress. These are passed to a Sieve filter to specify the :user
384 and :subaddress parts of an address. Both options are unset by default.
385 However, when a Sieve filter is run, if sieve_useraddress is unset, the
386 entire original local part (including any prefix or suffix) is used for
387 :user. An unset subaddress is treated as an empty subaddress.
475fe28a 388
389PH/09 Quota values can be followed by G as well as K and M.
391PH/10 $message_linecount is a new variable that contains the total number of
392 lines in the header and body of the message. Compare $body_linecount,
393 which is the count for the body only. During the DATA and
394 content-scanning ACLs, $message_linecount contains the number of lines
395 received. Before delivery happens (that is, before filters, routers, and
396 transports run) the count is increased to include the Received: header
397 line that Exim standardly adds, and also any other header lines that are
398 added by ACLs. The blank line that separates the message header from the
399 body is not counted. Here is an example of the use of this variable in a
402 deny message = Too many lines in message header
403 condition = \
404 ${if <{250}{${eval: $message_linecount - $body_linecount}}}
406 In the MAIL and RCPT ACLs, the value is zero because at that stage the
407 message has not yet been received.
409PH/11 In a ${run expansion, the variable $value (which contains the standard
410 output) is now also usable in the "else" string.
412PH/12 In a pipe transport, although a timeout while waiting for the pipe
413 process to complete was treated as a delivery failure, a timeout while
414 writing the message to the pipe was logged, but erroneously treated as a
415 successful delivery. Such timeouts include transport filter timeouts. For
416 consistency with the overall process timeout, these timeouts are now
417 treated as errors, giving rise to delivery failures by default. However,
418 there is now a new Boolean option for the pipe transport called
419 timeout_defer, which, if set TRUE, converts the failures into defers for
420 both kinds of timeout. A transport filter timeout is now identified in
421 the log output.
7766a4f0 423
f7b63901 424Version 4.50
b9e40c51 427The documentation is up-to-date for the 4.50 release.