| 1 | $Cambridge: exim/doc/doc-txt/NewStuff,v 1.23 2004/12/22 12:05:45 ph10 Exp $ |
| 2 | |
| 3 | New Features in Exim |
| 4 | -------------------- |
| 5 | |
| 6 | This file contains descriptions of new features that have been added to Exim, |
| 7 | but have not yet made it into the main manual (which is most conveniently |
| 8 | updated when there is a relatively large batch of changes). The doc/ChangeLog |
| 9 | file contains a listing of all changes, including bug fixes. |
| 10 | |
| 11 | |
| 12 | Version 4.50 |
| 13 | ------------ |
| 14 | |
| 15 | 1. There is a new build-time option called CONFIGURE_GROUP which works like |
| 16 | CONFIGURE_OWNER. It specifies one additional group that is permitted for |
| 17 | the runtime configuration file when the group write permission is set. |
| 18 | |
| 19 | 2. The "control=submission" facility has a new option /sender_retain. This |
| 20 | has the effect of setting local_sender_retain true and local_from_check |
| 21 | false for the incoming message in which it is encountered. |
| 22 | |
| 23 | 3. $recipients is now available in the predata ACL (oversight). |
| 24 | |
| 25 | 4. The value of address_data from a sender verification is now available in |
| 26 | $sender_address_data in subsequent conditions in the ACL statement. Note: |
| 27 | this is just like $address_data. The value does not persist after the end |
| 28 | of the current ACL statement. If you want to preserve it, you can use one |
| 29 | of the ACL variables. |
| 30 | |
| 31 | 5. The redirect router has two new options: forbid_sieve_filter and |
| 32 | forbid_exim_filter. When filtering is enabled by allow_filter, these |
| 33 | options control which type(s) of filtering are permitted. By default, both |
| 34 | Exim and Sieve filters are allowed. |
| 35 | |
| 36 | 6. A new option for callouts makes it possible to set a different (usually |
| 37 | smaller) timeout for making the SMTP connection. The keyword is "connect". |
| 38 | For example: |
| 39 | |
| 40 | verify = sender/callout=5s,connect=1s |
| 41 | |
| 42 | If not specified, it defaults to the general timeout value. |
| 43 | |
| 44 | 7. The new variables $sender_verify_failure and $recipient_verify_failure |
| 45 | contain information about exactly what failed. In an ACL, after one of |
| 46 | these failures, the relevant variable contains one of the following words: |
| 47 | |
| 48 | qualify the address was unqualified (no domain), and the message |
| 49 | was neither local nor came from an exempted host; |
| 50 | |
| 51 | route routing failed; |
| 52 | |
| 53 | mail routing succeeded, and a callout was attempted; rejection |
| 54 | occurred at or before the MAIL command (that is, on initial |
| 55 | connection, HELO, or MAIL); |
| 56 | |
| 57 | recipient the RCPT command in a callout was rejected; |
| 58 | |
| 59 | postmaster the postmaster check in a callout was rejected. |
| 60 | |
| 61 | The main use of these variables is expected to be to distinguish between |
| 62 | rejections of MAIL and rejections of RCPT. |
| 63 | |
| 64 | 8. The command line option -dd behaves exactly like -d except when used on a |
| 65 | command that starts a daemon process. In that case, debugging is turned off |
| 66 | for the subprocesses that the daemon creates. Thus, it is useful for |
| 67 | monitoring the behaviour of the daemon without creating as much output as |
| 68 | full debugging. |
| 69 | |
| 70 | 9. $host_address is now set to the target address during the checking of |
| 71 | ignore_target_hosts. |
| 72 | |
| 73 | 10. There are four new variables called $spool_space, $log_space, |
| 74 | $spool_inodes, and $log_inodes. The first two contain the amount of free |
| 75 | space in the disk partitions where Exim has its spool directory and log |
| 76 | directory, respectively. (When these are in the same partition, the values |
| 77 | will, of course, be the same.) The second two variables contain the numbers |
| 78 | of free inodes in the respective partitions. |
| 79 | |
| 80 | NOTE: Because disks can nowadays be very large, the values in the space |
| 81 | variables are in kilobytes rather than in bytes. Thus, for example, to |
| 82 | check in an ACL that there is at least 50M free on the spool, you would |
| 83 | write: |
| 84 | |
| 85 | condition = ${if > {$spool_space}{50000}{yes}{no}} |
| 86 | |
| 87 | The values are recalculated whenever any of these variables is referenced. |
| 88 | If the relevant file system does not have the concept of inodes, the value |
| 89 | of those variables is -1. If the operating system does not have the ability |
| 90 | to find the amount of free space (only true for experimental systems), the |
| 91 | space value is -1. |
| 92 | |
| 93 | 11. It is now permitted to omit both strings after an "if" condition; if the |
| 94 | condition is true, the result is the string "true". As before, when the |
| 95 | second string is omitted, a false condition yields an empty string. This |
| 96 | makes it less cumbersome to write custom ACL and router conditions. For |
| 97 | example, instead of |
| 98 | |
| 99 | condition = ${if eq {$acl_m4}{1}{yes}{no}} |
| 100 | |
| 101 | or the shorter form |
| 102 | |
| 103 | condition = ${if eq {$acl_m4}{1}{yes}} |
| 104 | |
| 105 | (because the second string has always defaulted to ""), you can now write |
| 106 | |
| 107 | condition = ${if eq {$acl_m4}{1}} |
| 108 | |
| 109 | Previously this was a syntax error. |
| 110 | |
| 111 | 12. There is a new "record type" that can be specified in dnsdb lookups. It |
| 112 | is "zns" (for "zone NS"). It performs a lookup for NS records on the given |
| 113 | domain, but if none are found, it removes the first component of the domain |
| 114 | name, and tries again. This process continues until NS records are found |
| 115 | or there are no more components left (or there's a DNS error). In other |
| 116 | words, it may return the name servers for a top-level domain, but it never |
| 117 | returns the root name servers. If there are no NS records for the top-level |
| 118 | domain, the lookup fails. |
| 119 | |
| 120 | For example, ${lookup dnsdb{zns=xxx.quercite.com}} returns the name |
| 121 | servers for quercite.com, whereas ${lookup dnsdb{zns=xxx.edu}} returns |
| 122 | the name servers for edu, assuming in each case that there are no NS |
| 123 | records for the full domain name. |
| 124 | |
| 125 | You should be careful about how you use this lookup because, unless the |
| 126 | top-level domain does not exist, the lookup will always return some host |
| 127 | names. The sort of use to which this might be put is for seeing if the name |
| 128 | servers for a given domain are on a blacklist. You can probably assume that |
| 129 | the name servers for the high-level domains such as .com or .co.uk are not |
| 130 | going to be on such a list. |
| 131 | |
| 132 | 13. Another new "record type" is "mxh"; this looks up MX records just as "mx" |
| 133 | does, but it returns only the names of the hosts, omitting the priority |
| 134 | values. |
| 135 | |
| 136 | 14. It is now possible to specify a list of domains or IP addresses to be |
| 137 | looked up in a dnsdb lookup. The list is specified in the normal Exim way, |
| 138 | with colon as the default separator, but with the ability to change this. |
| 139 | For example: |
| 140 | |
| 141 | ${lookup dnsdb{one.domain.com:two.domain.com}} |
| 142 | ${lookup dnsdb{a=one.host.com:two.host.com}} |
| 143 | ${lookup dnsdb{ptr = <; 1.2.3.4 ; 4.5.6.8}} |
| 144 | |
| 145 | In order to retain backwards compatibility, there is one special case: if |
| 146 | the lookup type is PTR and no change of separator is specified, Exim looks |
| 147 | to see if the rest of the string is precisely one IPv6 address. In this |
| 148 | case, it does not treat it as a list. |
| 149 | |
| 150 | The data from each lookup is concatenated, with newline separators (by |
| 151 | default - see 14 below), in the same way that multiple DNS records for a |
| 152 | single item are handled. |
| 153 | |
| 154 | The dnsdb lookup fails only if all the DNS lookups fail. If there is a |
| 155 | temporary DNS error for any of them, the behaviour is controlled by |
| 156 | an optional keyword followed by a comma that may appear before the record |
| 157 | type. The possible keywords are "defer_strict", "defer_never", and |
| 158 | "defer_lax". With "strict" behaviour, any temporary DNS error causes the |
| 159 | whole lookup to defer. With "never" behaviour, a temporary DNS error is |
| 160 | ignored, and the behaviour is as if the DNS lookup failed to find anything. |
| 161 | With "lax" behaviour, all the queries are attempted, but a temporary DNS |
| 162 | error causes the whole lookup to defer only if none of the other lookups |
| 163 | succeed. The default is "lax", so the following lookups are equivalent: |
| 164 | |
| 165 | ${lookup dnsdb{defer_lax,a=one.host.com:two.host.com}} |
| 166 | ${lookup dnsdb{a=one.host.com:two.host.com}} |
| 167 | |
| 168 | Thus, in the default case, as long as at least one of the DNS lookups |
| 169 | yields some data, the dnsdb lookup succeeds. |
| 170 | |
| 171 | 15. It is now possible to specify the character to be used as a separator when |
| 172 | a dnsdb lookup returns data from more than one DNS record. The default is a |
| 173 | newline. To specify a different character, put '>' followed by the new |
| 174 | character at the start of the query. For example: |
| 175 | |
| 176 | ${lookup dnsdb{>: a=h1.test.ex:h2.test.ex}} |
| 177 | ${lookup dnsdb{>| mxh=<;m1.test.ex;m2.test.ex}} |
| 178 | |
| 179 | It is permitted to specify a space as the separator character. Note that |
| 180 | more than one DNS record can be found for a single lookup item; this |
| 181 | feature is relevant even when you do not specify a list. |
| 182 | |
| 183 | The same effect could be achieved by wrapping the lookup in ${tr...}; this |
| 184 | feature is just a syntactic simplification. |
| 185 | |
| 186 | 16. It is now possible to supply a list of domains and/or IP addresses to be |
| 187 | lookup up in a DNS blacklist. Previously, only a single domain name could |
| 188 | be given, for example: |
| 189 | |
| 190 | dnslists = black.list.tld/$sender_host_name |
| 191 | |
| 192 | What follows the slash can now be a list. As with all lists, the default |
| 193 | separator is a colon. However, because this is a sublist within the list of |
| 194 | DNS blacklist domains, it is necessary either to double the separators like |
| 195 | this: |
| 196 | |
| 197 | dnslists = black.list.tld/name.1::name.2 |
| 198 | |
| 199 | or to change the separator character, like this: |
| 200 | |
| 201 | dnslists = black.list.tld/<;name.1;name.2 |
| 202 | |
| 203 | If an item in the list is an IP address, it is inverted before the DNS |
| 204 | blacklist domain is appended. If it is not an IP address, no inversion |
| 205 | occurs. Consider this condition: |
| 206 | |
| 207 | dnslists = black.list.tls/<;192.168.1.2;a.domain |
| 208 | |
| 209 | The DNS lookups that occur are for |
| 210 | |
| 211 | 2.1.168.192.black.list.tld and a.domain.black.list.tld |
| 212 | |
| 213 | Once a DNS record has been found (that matches a specific IP return |
| 214 | address, if specified), no further lookups are done. If there is a |
| 215 | temporary DNS error, the rest of the sublist of domains or IP addresses is |
| 216 | tried. The dnslists item itself defers only if none of the other DNS |
| 217 | lookups in this sublist succeeds. In other words, a successful lookup for |
| 218 | any of the items in the sublist overrides a defer for a previous item. |
| 219 | |
| 220 | 17. The log selector queue_time_overall causes Exim to output the time spent on |
| 221 | the queue as an addition to the "Completed" message. Like queue_time (which |
| 222 | puts the queue time on individual delivery lines), the time is tagged with |
| 223 | "QT=", and it is measured from the time that the message starts to be |
| 224 | received, so it includes the reception time. |
| 225 | |
| 226 | 18. It is now possible to use both -bF and -bf on the same command, in order to |
| 227 | test a system filter and a user filter in the same run. For example: |
| 228 | |
| 229 | exim -bF /system/filter -bf /user/filter </test/message |
| 230 | |
| 231 | This is helpful when the system filter adds header lines or sets filter |
| 232 | variables that are used by the user filter. |
| 233 | |
| 234 | 19. The Exiscan patch is now merged into the main source. See src/EDITME for |
| 235 | parameters for the build. |
| 236 | |
| 237 | 20. If the key for a dnsdb lookup is not an IP address, it is used verbatim, |
| 238 | without component reversal and without the addition of in-addr.arpa or |
| 239 | ip6.arpa. |
| 240 | |
| 241 | 21. Two changes related to the smtp_active_hostname option: |
| 242 | |
| 243 | (1) $smtp_active_hostname is now available as a variable. Its value |
| 244 | sticks with the message and is therefore available in routers and |
| 245 | transports at delivery time. |
| 246 | |
| 247 | (2) The default for smtp_banner uses $smtp_active_hostname instead |
| 248 | of $primary_hostname. |
| 249 | |
| 250 | |
| 251 | Version 4.43 |
| 252 | ------------ |
| 253 | |
| 254 | 1. There is a new Boolean global option called mua_wrapper, defaulting false. |
| 255 | This causes Exim to run an a restricted mode, in order to provide a very |
| 256 | specific service. |
| 257 | |
| 258 | Background: On a personal computer, it is a common requirement for all |
| 259 | email to be sent to a smarthost. There are plenty of MUAs that can be |
| 260 | configured to operate that way, for all the popular operating systems. |
| 261 | However, there are MUAs for Unix-like systems that cannot be so configured: |
| 262 | they submit messages using the command line interface of |
| 263 | /usr/sbin/sendmail. In addition, utility programs such as cron submit |
| 264 | messages this way. |
| 265 | |
| 266 | Requirement: The requirement is for something that can provide the |
| 267 | /usr/sbin/sendmail interface and deliver messages to a smarthost, but not |
| 268 | provide any queueing or retrying facilities. Furthermore, the delivery to |
| 269 | the smarthost should be synchronous, so that if it fails, the sending MUA |
| 270 | is immediately informed. In other words, we want something that in effect |
| 271 | converts a command-line MUA into a TCP/SMTP MUA. |
| 272 | |
| 273 | Solutions: There are a number of applications (for example, ssmtp) that do |
| 274 | this job. However, people have found them to be lacking in various ways. |
| 275 | For instance, some sites want to allow aliasing and forwarding before |
| 276 | sending to the smarthost. |
| 277 | |
| 278 | Using Exim: Exim already had the necessary infrastructure for doing this |
| 279 | job. Just a few tweaks were needed to make it behave as required, though it |
| 280 | is somewhat of an overkill to use a fully-featured MTA for this purpose. |
| 281 | |
| 282 | Setting mua_wrapper=true causes Exim to run in a special mode where it |
| 283 | assumes that it is being used to "wrap" a command-line MUA in the manner |
| 284 | just described. |
| 285 | |
| 286 | If you set mua_wrapper=true, you also need to provide a compatible router |
| 287 | and transport configuration. Typically there will be just one router and |
| 288 | one transport, sending everything to a smarthost. |
| 289 | |
| 290 | When run in MUA wrapping mode, the behaviour of Exim changes in the |
| 291 | following ways: |
| 292 | |
| 293 | (a) A daemon cannot be run, nor will Exim accept incoming messages from |
| 294 | inetd. In other words, the only way to submit messages is via the |
| 295 | command line. |
| 296 | |
| 297 | (b) Each message is synchonously delivered as soon as it is received (-odi |
| 298 | is assumed). All queueing options (queue_only, queue_smtp_domains, |
| 299 | control=queue, control=freeze in an ACL etc.) are quietly ignored. The |
| 300 | Exim reception process does not finish until the delivery attempt is |
| 301 | complete. If the delivery was successful, a zero return code is given. |
| 302 | |
| 303 | (c) Address redirection is permitted, but the final routing for all |
| 304 | addresses must be to the same remote transport, and to the same list of |
| 305 | hosts. Furthermore, the return_address must be the same for all |
| 306 | recipients, as must any added or deleted header lines. In other words, |
| 307 | it must be possible to deliver the message in a single SMTP |
| 308 | transaction, however many recipients there are. |
| 309 | |
| 310 | (d) If the conditions in (c) are not met, or if routing any address results |
| 311 | in a failure or defer status, or if Exim is unable to deliver all the |
| 312 | recipients successfully to one of the hosts immediately, delivery of |
| 313 | the entire message fails. |
| 314 | |
| 315 | (e) Because no queueing is allowed, all failures are treated as permanent; |
| 316 | there is no distinction between 4xx and 5xx SMTP response codes from |
| 317 | the smarthost. Furthermore, because only a single yes/no response can |
| 318 | be given to the caller, it is not possible to deliver to some |
| 319 | recipients and not others. If there is an error (temporary or |
| 320 | permanent) for any recipient, all are failed. |
| 321 | |
| 322 | (f) If more than one host is listed, Exim will try another host after a |
| 323 | connection failure or a timeout, in the normal way. However, if this |
| 324 | kind of failure happens for all the hosts, the delivery fails. |
| 325 | |
| 326 | (g) When delivery fails, an error message is written to the standard error |
| 327 | stream (as well as to Exim's log), and Exim exits to the caller with a |
| 328 | return code value 1. The message is expunged from Exim's spool files. |
| 329 | No bounce messages are ever generated. |
| 330 | |
| 331 | (h) No retry data is maintained, and any retry rules are ignored. |
| 332 | |
| 333 | (i) A number of Exim options are overridden: deliver_drop_privilege is |
| 334 | forced true, max_rcpt in the smtp transport is forced to "unlimited", |
| 335 | remote_max_parallel is forced to one, and fallback hosts are ignored. |
| 336 | |
| 337 | The overall effect is that Exim makes a single synchronous attempt to |
| 338 | deliver the message, failing if there is any kind of problem. Because no |
| 339 | local deliveries are done and no daemon can be run, Exim does not need root |
| 340 | privilege. It should be possible to run it setuid=exim instead of |
| 341 | setuid=root. See section 48.3 in the 4.40 manual for a general discussion |
| 342 | about the advantages and disadvantages of running without root privilege. |
| 343 | |
| 344 | 2. There have been problems with DNS servers when SRV records are looked up. |
| 345 | Some mis-behaving servers return a DNS error or timeout when a non-existent |
| 346 | SRV record is sought. Similar problems have in the past been reported for |
| 347 | MX records. The global dns_again_means_nonexist option can help with this |
| 348 | problem, but it is heavy-handed because it is a global option. There are |
| 349 | now two new options for the dnslookup router. They are called |
| 350 | srv_fail_domains and mx_fail_domains. In each case, the value is a domain |
| 351 | list. If an attempt to look up an SRV or MX record results in a DNS failure |
| 352 | or "try again" response, and the domain matches the relevant list, Exim |
| 353 | behaves as if the DNS had responded "no such record". In the case of an SRV |
| 354 | lookup, this means that the router proceeds to look for MX records; in the |
| 355 | case of an MX lookup, it proceeds to look for A or AAAA records, unless the |
| 356 | domain matches mx_domains. |
| 357 | |
| 358 | 3. The following functions are now available in the local_scan() API: |
| 359 | |
| 360 | (a) void header_remove(int occurrence, uschar *name) |
| 361 | |
| 362 | This function removes header lines. If "occurrence" is zero or negative, |
| 363 | all occurrences of the header are removed. If occurrence is greater |
| 364 | than zero, that particular instance of the header is removed. If no |
| 365 | header(s) can be found that match the specification, the function does |
| 366 | nothing. |
| 367 | |
| 368 | (b) BOOL header_testname(header_line *hdr, uschar *name, int length, |
| 369 | BOOL notdel) |
| 370 | |
| 371 | This function tests whether the given header has the given name. It |
| 372 | is not just a string comparison, because whitespace is permitted |
| 373 | between the name and the colon. If the "notdel" argument is TRUE, a |
| 374 | FALSE return is forced for all "deleted" headers; otherwise they are |
| 375 | not treated specially. For example: |
| 376 | |
| 377 | if (header_testname(h, US"X-Spam", 6, TRUE)) ... |
| 378 | |
| 379 | (c) void header_add_at_position(BOOL after, uschar *name, BOOL topnot, |
| 380 | int type, char *format, ...) |
| 381 | |
| 382 | This function adds a new header line at a specified point in the header |
| 383 | chain. If "name" is NULL, the new header is added at the end of the |
| 384 | chain if "after" is TRUE, or at the start if "after" is FALSE. If |
| 385 | "name" is not NULL, the headers are searched for the first non-deleted |
| 386 | header that matches the name. If one is found, the new header is added |
| 387 | before it if "after" is FALSE. If "after" is true, the new header is |
| 388 | added after the found header and any adjacent subsequent ones with the |
| 389 | same name (even if marked "deleted"). If no matching non-deleted header |
| 390 | is found, the "topnot" option controls where the header is added. If it |
| 391 | is TRUE, addition is at the top; otherwise at the bottom. Thus, to add |
| 392 | a header after all the Received: headers, or at the top if there are no |
| 393 | Received: headers, you could use |
| 394 | |
| 395 | header_add_at_position(TRUE, US"Received", TRUE, ' ', "X-xxx: ..."); |
| 396 | |
| 397 | Normally, there is always at least one non-deleted Received: header, |
| 398 | but there may not be if received_header_text expands to an empty |
| 399 | string. |
| 400 | |
| 401 | (d) BOOL receive_remove_recipient(uschar *recipient) |
| 402 | |
| 403 | This is a convenience function to remove a named recipient from the |
| 404 | list of recipients. It returns TRUE if a recipient was removed, and |
| 405 | FALSE if no matching recipient could be found. The argument must be a |
| 406 | complete email address. |
| 407 | |
| 408 | 4. When an ACL "warn" statement adds one or more header lines to a message, |
| 409 | they are added at the end of the existing header lines by default. It is |
| 410 | now possible to specify that any particular header line should be added |
| 411 | right at the start (before all the Received: lines) or immediately after |
| 412 | the first block of Received: lines in the message. This is done by |
| 413 | specifying :at_start: or :after_received: (or, for completeness, :at_end:) |
| 414 | before the text of the header line. (Header text cannot start with a colon, |
| 415 | as there has to be a header name first.) For example: |
| 416 | |
| 417 | warn message = :after_received:X-My-Header: something or other... |
| 418 | |
| 419 | If more than one header is supplied in a single warn statement, each one is |
| 420 | treated independently and can therefore be placed differently. If you add |
| 421 | more than one line at the start, or after the Received: block, they will |
| 422 | end up in reverse order. |
| 423 | |
| 424 | Warning: This facility currently applies only to header lines that are |
| 425 | added in an ACL. It does NOT work for header lines that are added in a |
| 426 | system filter or in a router or transport. |
| 427 | |
| 428 | 5. There is now a new error code that can be used in retry rules. Its name is |
| 429 | "rcpt_4xx", and there are three forms. A literal "rcpt_4xx" matches any 4xx |
| 430 | error received for an outgoing SMTP RCPT command; alternatively, either the |
| 431 | first or both of the x's can be given as digits, for example: "rcpt_45x" or |
| 432 | "rcpt_436". If you want (say) to recognize 452 errors given to RCPT |
| 433 | commands by a particular host, and have only a one-hour retry for them, you |
| 434 | can set up a retry rule of this form: |
| 435 | |
| 436 | the.host.name rcpt_452 F,1h,10m |
| 437 | |
| 438 | Naturally, this rule must come before any others that would match. |
| 439 | |
| 440 | These new errors apply to both outgoing SMTP (the smtp transport) and |
| 441 | outgoing LMTP (either the lmtp transport, or the smtp transport in LMTP |
| 442 | mode). Note, however, that they apply only to responses to RCPT commands. |
| 443 | |
| 444 | 6. The "postmaster" option of the callout feature of address verification has |
| 445 | been extended to make it possible to use a non-empty MAIL FROM address when |
| 446 | checking a postmaster address. The new suboption is called "postmaster_ |
| 447 | mailfrom", and you use it like this: |
| 448 | |
| 449 | require verify = sender/callout=postmaster_mailfrom=abc@x.y.z |
| 450 | |
| 451 | Providing this suboption causes the postmaster check to be done using the |
| 452 | given address. The original "postmaster" option is equivalent to |
| 453 | |
| 454 | require verify = sender/callout=postmaster_mailfrom= |
| 455 | |
| 456 | If both suboptions are present, the rightmost one overrides. |
| 457 | |
| 458 | Important notes: |
| 459 | |
| 460 | (1) If you use a non-empty sender address for postmaster checking, there is |
| 461 | the likelihood that the remote host will itself initiate a callout |
| 462 | check back to your host to check that address. As this is a "normal" |
| 463 | callout check, the sender will most probably be empty, thus avoiding |
| 464 | possible callout loops. However, to be on the safe side it would be |
| 465 | best to set up your own ACLs so that they do not do sender verification |
| 466 | checks when the recipient is the address you use for postmaster callout |
| 467 | checking. |
| 468 | |
| 469 | (2) The caching arrangements for postmaster checking do NOT take account of |
| 470 | the sender address. It is assumed that either the empty address, or a |
| 471 | fixed non-empty address will be used. All that Exim remembers is that |
| 472 | the postmaster check for the domain succeeded or failed. |
| 473 | |
| 474 | 7. When verifying addresses in header lines using the verify=header_sender |
| 475 | option, Exim behaves by default as if the addresses are envelope sender |
| 476 | addresses from a message. Callout verification therefore tests to see |
| 477 | whether a bounce message could be delivered, by using an empty address in |
| 478 | the MAIL FROM command. However, it is arguable that these addresses might |
| 479 | never be used as envelope senders, and could therefore justifiably reject |
| 480 | bounce messages (empty senders). There is now an additional callout option |
| 481 | for verify=header_sender that allows you to specify what address to use in |
| 482 | the MAIL FROM command. You use it as in this example: |
| 483 | |
| 484 | require verify = header_sender/callout=mailfrom=abcd@x.y.z |
| 485 | |
| 486 | Important notes: |
| 487 | |
| 488 | (1) As in the case of postmaster_mailfrom (see above), you should think |
| 489 | about possible loops. |
| 490 | |
| 491 | (2) In this case, as in the case of recipient callouts with non-empty |
| 492 | senders (the use_sender option), caching is done on the basis of a |
| 493 | recipient/sender pair. |
| 494 | |
| 495 | 8. If you build Exim with USE_READLINE=yes in Local/Makefile, it will try to |
| 496 | load libreadline dynamically whenever the -be (test expansion) option is |
| 497 | used without command line arguments. If successful, it will then use |
| 498 | readline() for reading the test data. A line history is supported. By the |
| 499 | time Exim does this, it is running as the calling user, so this should not |
| 500 | cause any security problems. Security is the reason why this is NOT |
| 501 | supported for -bt or -bv, when Exim is running as root or exim, |
| 502 | respectively. Note that this option adds to the size of the Exim binary, |
| 503 | because the dynamic loading library is not otherwise included. On my |
| 504 | desktop it adds about 2.5K. You may need to add -ldl to EXTRA_LIBS when you |
| 505 | set USE_READLINE=yes. |
| 506 | |
| 507 | 9. Added ${str2b64:<string>} to the expansion operators. This operator |
| 508 | converts an arbitrary string into one that is base64 encoded. |
| 509 | |
| 510 | 10. A new authenticator, called cyrus_sasl, has been added. This requires |
| 511 | the presence of the Cyrus SASL library; it authenticates by calling this |
| 512 | library, which supports a number of authentication mechanisms, including |
| 513 | PLAIN and LOGIN, but also several others that Exim does not support |
| 514 | directly. The code for this authenticator was provided by Matthew |
| 515 | Byng-Maddick of A L Digital Ltd (http://www.aldigital.co.uk). Here follows |
| 516 | draft documentation: |
| 517 | |
| 518 | xx. THE CYRUS_SASL AUTHENTICATOR |
| 519 | |
| 520 | The cyrus_sasl authenticator provides server support for the Cyrus library |
| 521 | Implementation of the RFC 2222 "Simple Authentication and Security Layer". |
| 522 | It provides a gatewaying mechanism directly to the Cyrus interface, so if |
| 523 | your Cyrus library can do, for example, CRAM-MD5, then so can the |
| 524 | cyrus_sasl authenticator. By default it uses the public name of the driver |
| 525 | to determine which mechanism to support. |
| 526 | |
| 527 | Where access to some kind of secret file is required, for example in GSSAPI |
| 528 | or CRAM-MD5, it is worth noting that the authenticator runs as the exim |
| 529 | user, and that the Cyrus SASL library has no way of escalating privileges |
| 530 | by default. You may also find you need to set environment variables, |
| 531 | depending on the driver you are using. |
| 532 | |
| 533 | xx.1 Using cyrus_sasl as a server |
| 534 | |
| 535 | The cyrus_sasl authenticator has four private options. It puts the username |
| 536 | (on a successful authentication) into $1. |
| 537 | |
| 538 | server_hostname Type: string* Default: $primary_hostname |
| 539 | |
| 540 | This option selects the hostname that is used when communicating with |
| 541 | the library. It is up to the underlying SASL plug-in what it does with |
| 542 | this data. |
| 543 | |
| 544 | server_mech Type: string Default: public_name |
| 545 | |
| 546 | This option selects the authentication mechanism this driver should |
| 547 | use. It allows you to use a different underlying mechanism from the |
| 548 | advertised name. For example: |
| 549 | |
| 550 | sasl: |
| 551 | driver = cyrus_sasl |
| 552 | public_name = X-ANYTHING |
| 553 | server_mech = CRAM-MD5 |
| 554 | server_set_id = $1 |
| 555 | |
| 556 | server_realm Type: string Default: unset |
| 557 | |
| 558 | This is the SASL realm that the server is claiming to be in. |
| 559 | |
| 560 | server_service Type: string Default: "smtp" |
| 561 | |
| 562 | This is the SASL service that the server claims to implement. |
| 563 | |
| 564 | For straigthforward cases, you do not need to set any of the |
| 565 | authenticator's private options. All you need to do is to specify an |
| 566 | appropriate mechanism as the public name. Thus, if you have a SASL library |
| 567 | that supports CRAM-MD5 and PLAIN, you might have two authenticators as |
| 568 | follows: |
| 569 | |
| 570 | sasl_cram_md5: |
| 571 | driver = cyrus_sasl |
| 572 | public_name = CRAM-MD5 |
| 573 | server_set_id = $1 |
| 574 | |
| 575 | sasl_plain: |
| 576 | driver = cyrus_sasl |
| 577 | public_name = PLAIN |
| 578 | server_set_id = $1 |
| 579 | |
| 580 | 11. There is a new global option called tls_on_connect_ports. Its value must be |
| 581 | a list of port numbers; the most common use is expected to be |
| 582 | |
| 583 | tls_on_connect_ports = 465 |
| 584 | |
| 585 | Setting this option has the same effect as -tls-on-connect on the command |
| 586 | line, but only for the specified ports. It applies to all connections, both |
| 587 | via the daemon and via inetd. You still need to specify all the ports for |
| 588 | the daemon (using daemon_smtp_ports or local_interfaces or the -X command |
| 589 | line option) because this option does not add an extra port -- rather, it |
| 590 | specifies different behaviour on a port that is defined elsewhere. The |
| 591 | -tls-on-connect command line option overrides tls_on_connect_ports, and |
| 592 | forces tls-on-connect for all ports. |
| 593 | |
| 594 | 12. There is a new ACL that is run when a DATA command is received, before the |
| 595 | data itself is received. The ACL is defined by acl_smtp_predata. (Compare |
| 596 | acl_smtp_data, which is run after the data has been received.) |
| 597 | This new ACL allows a negative response to be given to the DATA command |
| 598 | itself. Header lines added by MAIL or RCPT ACLs are not visible at this |
| 599 | time, but any that are defined here are visible when the acl_smtp_data ACL |
| 600 | is run. |
| 601 | |
| 602 | 13. The "control=submission" ACL modifier has an option "/domain=xxx" which |
| 603 | specifies the domain to be used when creating From: or Sender: lines using |
| 604 | the authenticated id as a local part. If the option is supplied with an |
| 605 | empty domain, that is, just "/domain=", Exim assumes that the authenticated |
| 606 | id is a complete email address, and it uses it as is when creating From: |
| 607 | or Sender: lines. |
| 608 | |
| 609 | 14. It is now possible to make retry rules that apply only when the failing |
| 610 | message has a specific sender. In particular, this can be used to define |
| 611 | retry rules that apply only to bounce messages. The syntax is to add a new |
| 612 | third item to a retry rule, of the form "senders=<address list>". The retry |
| 613 | timings themselves then become the fourth item. For example: |
| 614 | |
| 615 | * * senders=: F,1h,30m |
| 616 | |
| 617 | would match all bounce messages. If the address list contains white space, |
| 618 | it must be enclosed in quotes. For example: |
| 619 | |
| 620 | a.domain timeout senders="x@b.dom : y@c.dom" G,8h,10m,1.5 |
| 621 | |
| 622 | When testing retry rules using -brt, you can supply a sender using the -f |
| 623 | command line option, like this: |
| 624 | |
| 625 | exim -f "" -brt user@dom.ain |
| 626 | |
| 627 | If you do not set -f with -brt, a retry rule that contains a senders list |
| 628 | will never be matched. |
| 629 | |
| 630 | 15. Two new control modifiers have been added to ACLs: "control = enforce_sync" |
| 631 | and "control = no_enforce_sync". This makes it possible to be selective |
| 632 | about when SMTP synchronization is enforced. The global option |
| 633 | smtp_enforce_sync now specifies the default state of the switch. These |
| 634 | controls can appear in any ACL, but the most obvious place to put them is |
| 635 | in the ACL defined by acl_smtp_connect, which is run at the start of an |
| 636 | incoming SMTP connection, before the first synchronization check. |
| 637 | |
| 638 | 16. Another two new control modifiers are "control = caseful_local_part" and |
| 639 | "control = caselower_local_part". These are permitted only in the ACL |
| 640 | specified by acl_smtp_rcpt (i.e. during RCPT processing). By default, the |
| 641 | contents of $local_part are lower cased before ACL processing. |
| 642 | After "control = caseful_local_part", any uppercase letters in the original |
| 643 | local part are restored in $local_part for the rest of the ACL, or until |
| 644 | "control = caselower_local_part" is encountered. However, this applies only |
| 645 | to local part handling that takes place directly in the ACL (for example, |
| 646 | as a key in lookups). If a "verify = recipient" test is obeyed, the |
| 647 | case-related handling of the local part during the verification is |
| 648 | controlled by the router configuration (see the caseful_local_part generic |
| 649 | router option). |
| 650 | |
| 651 | This facility could be used, for example, to add a spam score to local |
| 652 | parts containing upper case letters. For example, using $acl_m4 to |
| 653 | accumulate the spam score: |
| 654 | |
| 655 | warn control = caseful_local_part |
| 656 | set acl_m4 = ${eval:\ |
| 657 | $acl_m4 + \ |
| 658 | ${if match{$local_part}{[A-Z]}{1}{0}}\ |
| 659 | } |
| 660 | control = caselower_local_part |
| 661 | |
| 662 | Notice that we put back the lower cased version afterwards, assuming that |
| 663 | is what is wanted for subsequent tests. |
| 664 | |
| 665 | 17. The option hosts_connection_nolog is provided so that certain hosts can be |
| 666 | excepted from logging when the +smtp_connection log selector is set. For |
| 667 | example, you might want not to log SMTP connections from local processes, |
| 668 | or from 127.0.0.1, or from your local LAN. The option is a host list with |
| 669 | an unset default. Because it is consulted in the main loop of the daemon, |
| 670 | you should strive to restrict its value to a short inline list of IP |
| 671 | addresses and networks. To disable logging SMTP connections from local |
| 672 | processes, you must create a host list with an empty item. For example: |
| 673 | |
| 674 | hosts_connection_nolog = : |
| 675 | |
| 676 | If the +smtp_connection log selector is not set, this option has no effect. |
| 677 | |
| 678 | 18. There is now an acl called acl_smtp_quit, which is run for the QUIT |
| 679 | command. The outcome of the ACL does not affect the response code to QUIT, |
| 680 | which is always 221. Thus, the ACL does not in fact control any access. |
| 681 | For this reason, the only verbs that are permitted are "accept" and "warn". |
| 682 | |
| 683 | The ACL can be used for tasks such as custom logging at the end of an SMTP |
| 684 | session. For example, you can use ACL variables in other ACLs to count |
| 685 | messages, recipients, etc., and log the totals at QUIT time using one or |
| 686 | more "logwrite" modifiers on a "warn" command. |
| 687 | |
| 688 | You do not need to have a final "accept", but if you do, you can use a |
| 689 | "message" modifier to specify custom text that is sent as part of the 221 |
| 690 | response. |
| 691 | |
| 692 | This ACL is run only for a "normal" QUIT. For certain kinds of disastrous |
| 693 | failure (for example, failure to open a log file, or when Exim is bombing |
| 694 | out because it has detected an unrecoverable error), all SMTP commands |
| 695 | from the client are given temporary error responses until QUIT is received |
| 696 | or the connection is closed. In these special cases, the ACL is not run. |
| 697 | |
| 698 | 19. The appendfile transport has two new options, mailbox_size and mailbox_ |
| 699 | filecount. If either these options are set, it is expanded, and the result |
| 700 | is taken as the current size of the mailbox or the number of files in the |
| 701 | mailbox, respectively. This makes it possible to use some external means of |
| 702 | maintaining the data about the size of a mailbox for enforcing quota |
| 703 | limits. The result of expanding these option values must be a decimal |
| 704 | number, optionally followed by "K" or "M". |
| 705 | |
| 706 | 20. It seems that there are broken clients in use that cannot handle multiline |
| 707 | SMTP responses. Can't people who implement these braindead programs read? |
| 708 | RFC 821 mentions multiline responses, and it is over 20 years old. They |
| 709 | must handle multiline responses for EHLO, or do they still use HELO? |
| 710 | Anyway, here is YAWFAB (yet another workaround for asinine brokenness). |
| 711 | There's a new ACL switch that can be set by |
| 712 | |
| 713 | control = no_multiline_responses |
| 714 | |
| 715 | If this is set, it suppresses multiline SMTP responses from ACL rejections. |
| 716 | One way of doing this would have been just to put out these responses as |
| 717 | one long line. However, RFC 2821 specifies a maximum of 512 bytes per |
| 718 | response ("use multiline responses for more" it says), and some of the |
| 719 | responses might get close to that. So I have implemented this by doing two |
| 720 | very easy things: |
| 721 | |
| 722 | (1) Extra information that is normally output as part of a rejection |
| 723 | caused by sender verification failure is omitted. Only the final line |
| 724 | (typically "sender verification failed") is now sent. |
| 725 | |
| 726 | (2) If a "message" modifier supplies a multiline response, only the first |
| 727 | line is output. |
| 728 | |
| 729 | The setting of the switch can, of course, be made conditional on the |
| 730 | calling host. |
| 731 | |
| 732 | 21. There is now support for the libradius library that comes with FreeBSD. |
| 733 | This is an alternative to the radiusclient library that Exim already |
| 734 | supports. To use the FreeBSD library, you need to set |
| 735 | |
| 736 | RADIUS_LIB_TYPE=RADLIB |
| 737 | |
| 738 | in Local/Makefile, in addition to RADIUS_CONFIGURE_FILE, and you probably |
| 739 | also need -libradius in EXTRALIBS. |
| 740 | |
| 741 | |
| 742 | Version 4.42 |
| 743 | ------------ |
| 744 | |
| 745 | 1. The "personal" filter test is brought up-to-date with recommendations from |
| 746 | the Sieve specification: (a) The list of non-personal From: addresses now |
| 747 | includes "listserv", "majordomo", and "*-request"; (b) If the message |
| 748 | contains any header line starting with "List=-" it is treated as |
| 749 | non-personal. |
| 750 | |
| 751 | 2. The Sieve functionality has been extended to support the "copy" and |
| 752 | "vacation" extensions, and comparison tests. |
| 753 | |
| 754 | 3. There is now an overall timeout for performing a callout verification. It |
| 755 | defaults to 4 times the callout timeout, which applies to individual SMTP |
| 756 | commands during the callout. The overall timeout applies when there is more |
| 757 | than one host that can be tried. The timeout is checked before trying the |
| 758 | next host. This prevents very long delays if there are a large number of |
| 759 | hosts and all are timing out (e.g. when the network connections are timing |
| 760 | out). The value of the overall timeout can be changed by specifying an |
| 761 | additional sub-option for "callout", called "maxwait". For example: |
| 762 | |
| 763 | verify = sender/callout=5s,maxwait=20s |
| 764 | |
| 765 | 4. Changes to the "personal" filter test: |
| 766 | |
| 767 | (1) The list of non-personal local parts in From: addresses has been |
| 768 | extended to include "listserv", "majordomo", "*-request", and "owner-*", |
| 769 | taken from the Sieve specification recommendations. |
| 770 | |
| 771 | (2) If the message contains any header line starting with "List-" it is |
| 772 | treated as non-personal. |
| 773 | |
| 774 | (3) The test for "circular" in the Subject: header line has been removed |
| 775 | because it now seems ill-conceived. |
| 776 | |
| 777 | 5. The autoreply transport has a new option called never_mail. This is an |
| 778 | address list. If any run of the transport creates a message with a |
| 779 | recipient that matches any item in the list, that recipient is quietly |
| 780 | discarded. If all recipients are discarded, no message is created. |
| 781 | |
| 782 | |
| 783 | Version 4.40 |
| 784 | ------------ |
| 785 | |
| 786 | The documentation is up-to-date for the 4.40 release. What follows here is a |
| 787 | brief list of the new features that have been added since 4.30. |
| 788 | |
| 789 | 1. log_incoming_interface affects more log lines. |
| 790 | |
| 791 | 2. New ACL modifier "control = submission". |
| 792 | |
| 793 | 3. CONFIGURE_OWNER can be set at build time to define an alternative owner for |
| 794 | the configuration file, in addition to root and exim. |
| 795 | |
| 796 | 4. Added expansion variables $body_zerocount, $recipient_data, and |
| 797 | $sender_data. |
| 798 | |
| 799 | 5. The time of last modification of the "new" subdirectory is now used as the |
| 800 | "mailbox time last read" when there is a quota error for a maildir |
| 801 | delivery. |
| 802 | |
| 803 | 6. The special item "+ignore_unknown" may now appear in host lists. |
| 804 | |
| 805 | 7. The special domain-matching patterns @mx_any, @mx_primary, and |
| 806 | @mx_secondary can now be followed by "/ignore=<ip list>". |
| 807 | |
| 808 | 8. New expansion conditions: match_domain, match_address, match_local_part, |
| 809 | lt, lti, le, lei, gt, gti, ge, and new expansion operators time_interval, |
| 810 | eval10, and base62d. |
| 811 | |
| 812 | 9. New lookup type called "iplsearch". |
| 813 | |
| 814 | 10. New log selectors ident_timeout, tls_certificate_verified, queue_time, |
| 815 | deliver_time, outgoing_port, return_path_on_delivery. |
| 816 | |
| 817 | 11. New global options smtp_active_hostname and tls_require_ciphers. |
| 818 | |
| 819 | 12. Exinext has -C and -D options. |
| 820 | |
| 821 | 13. "domainlist_cache" forces caching of an apparently variable list. |
| 822 | |
| 823 | 14. For compatibility with Sendmail, the command line option -prval:sval |
| 824 | is equivalent to -oMr rval -oMs sval. |
| 825 | |
| 826 | 15. New callout options use_sender and use_postmaster for use when verifying |
| 827 | recipients. |
| 828 | |
| 829 | 16. John Jetmore's "exipick" utility has been added to the distribution. |
| 830 | |
| 831 | 17. The TLS code now supports CRLs. |
| 832 | |
| 833 | 18. The dnslookup router and the dnsdb lookup type now support the use of SRV |
| 834 | records. |
| 835 | |
| 836 | 19. The redirect router has a new option called qualify_domain. |
| 837 | |
| 838 | 20. exigrep's output now also includes lines that are not related to any |
| 839 | particular message, but which do match the pattern. |
| 840 | |
| 841 | 21. New global option write_rejectlog. If it is set false, Exim no longer |
| 842 | writes anything to the reject log. |
| 843 | |
| 844 | **** |