| 1 | $Cambridge: exim/src/README.UPDATING,v 1.10 2005/12/12 15:58:53 ph10 Exp $ |
| 2 | |
| 3 | This document contains detailed information about incompatibilities that might |
| 4 | be encountered when upgrading from one release of Exim to another. The |
| 5 | information is in reverse order of release numbers. Mostly these are relatively |
| 6 | small points, and the configuration file is normally upwards compatible, but |
| 7 | there have been two big upheavals... |
| 8 | |
| 9 | |
| 10 | ************************************************************************** |
| 11 | * There was a big reworking of the way mail routing works for release * |
| 12 | * 4.00. Previously used "directors" were abolished, and all routing is * |
| 13 | * now done by routers. Policy controls for incoming mail are now done by * |
| 14 | * Access Control Lists instead of separate options. All this means that * |
| 15 | * pre-4.00 configuration files have to be massively converted. If you * |
| 16 | * are coming from a 3.xx release, please read the document in the file * |
| 17 | * doc/Exim4.upgrade, and allow some time to complete the upgrade. * |
| 18 | * * |
| 19 | * There was a big reworking of the way domain/host/net/address lists are * |
| 20 | * handled at release 3.00. If you are coming from a pre-3.00 release, it * |
| 21 | * might be easier to start again from a default configuration. Otherwise * |
| 22 | * you need to read doc/Exim3.upgrade and do a double conversion of your * |
| 23 | * configuration file. * |
| 24 | ************************************************************************** |
| 25 | |
| 26 | |
| 27 | The rest of this document contains information about changes in 4.xx releases |
| 28 | that might affect a running system. |
| 29 | |
| 30 | |
| 31 | Exim version 4.61 |
| 32 | ----------------- |
| 33 | |
| 34 | The default number of ACL variables of each type has been increased to 20, and |
| 35 | it's possible to compile Exim with more. You can safely upgrade to this release |
| 36 | if you already have messages on the queue with saved ACL variable values. |
| 37 | However, if you downgrade from this release with messages on the queue, any |
| 38 | saved ACL values they may have will be lost. |
| 39 | |
| 40 | |
| 41 | Exim version 4.54 |
| 42 | ----------------- |
| 43 | |
| 44 | There was a problem with 4.52/TF/02 in that a "name=" option on control= |
| 45 | submission terminated at the next slash, thereby not allowing for slashes in |
| 46 | the name. This has been changed so that "name=" takes the rest of the string as |
| 47 | its data. It must therefore be the last option. |
| 48 | |
| 49 | |
| 50 | Version 4.53 |
| 51 | ------------ |
| 52 | |
| 53 | If you are using the experimental Domain Keys support, you must upgrade to |
| 54 | at least libdomainkeys 0.67 in order to run this release of Exim. |
| 55 | |
| 56 | |
| 57 | Version 4.51 |
| 58 | ------------ |
| 59 | |
| 60 | 1. The format in which GnuTLS parameters are cached (in the file gnutls-params |
| 61 | in the spool directory) has been changed. The new format can also be generated |
| 62 | externally, so it is now possible to update the values from outside Exim. This |
| 63 | has been implemented in an upwards, BUT NOT downwards, compatible manner. |
| 64 | Upgrading should be seamless: when Exim finds that it cannot understand an |
| 65 | existing cache file, it generates new parameters and writes them to the cache |
| 66 | in the new format. If, however, you downgrade from 4.51 to a previous release, |
| 67 | you MUST delete the gnutls-params file in the spool directory, because the |
| 68 | older Exim will not recognize the new format. |
| 69 | |
| 70 | 2. When doing a callout as part of verifying an address, Exim was not paying |
| 71 | attention to any local part prefix or suffix that was matched by the router |
| 72 | that accepted the address. It now behaves in the same way as it does for |
| 73 | delivery: the affixes are removed from the local part unless |
| 74 | rcpt_include_affixes is set on the transport. If you have a configuration that |
| 75 | uses prefixes or suffixes on addresses that could be used for callouts, and you |
| 76 | want the affixes to be retained, you must make sure that rcpt_include_affixes |
| 77 | is set on the transport. |
| 78 | |
| 79 | 3. Bounce and delay warning messages no longer contain details of delivery |
| 80 | errors, except for explicit messages (e.g. generated by :fail:) and SMTP |
| 81 | responses from remote hosts. |
| 82 | |
| 83 | |
| 84 | Version 4.50 |
| 85 | ------------ |
| 86 | |
| 87 | The exicyclog script has been updated to use three-digit numbers in rotated log |
| 88 | files if the maximum number to keep is greater than 99. If you are already |
| 89 | keeping more than 99, there will be an incompatible change when you upgrade. |
| 90 | You will probably want to rename your old log files to the new form before |
| 91 | running the new exicyclog. |
| 92 | |
| 93 | |
| 94 | Version 4.42 |
| 95 | ------------ |
| 96 | |
| 97 | RFC 3848 specifies standard names for the "with" phrase in Received: header |
| 98 | lines when AUTH and/or TLS are in use. This is the "received protocol" |
| 99 | field. Exim used to use "asmtp" for authenticated SMTP, without any |
| 100 | indication (in the protocol name) for TLS use. Now it follows the RFC and |
| 101 | uses "esmtpa" if the connection is authenticated, "esmtps" if it is |
| 102 | encrypted, and "esmtpsa" if it is both encrypted and authenticated. These names |
| 103 | appear in log lines as well as in Received: header lines. |
| 104 | |
| 105 | |
| 106 | Version 4.34 |
| 107 | ------------ |
| 108 | |
| 109 | Change 4.31/2 gave problems to data ACLs and local_scan() functions that |
| 110 | expected to see a Received: header. I have changed to yet another scheme. The |
| 111 | Received: header is now generated after the body is received, but before the |
| 112 | ACL or local_scan() is called. After they have run, the timestamp in the |
| 113 | Received: header is updated. |
| 114 | |
| 115 | Thus, change (a) of 4.31/2 has been reversed, but change (b) is still true, |
| 116 | which is lucky, since I decided it was a bug fix. |
| 117 | |
| 118 | |
| 119 | Version 4.33 |
| 120 | ------------ |
| 121 | |
| 122 | If an expansion in a condition on a "warn" statement fails because a lookup |
| 123 | defers, the "warn" statement is abandoned, and the next ACL statement is |
| 124 | processed. Previously this caused the whole ACL to be aborted. |
| 125 | |
| 126 | |
| 127 | Version 4.32 |
| 128 | ------------ |
| 129 | |
| 130 | Change 4.31/2 has been reversed, as it proved contentious. Recipient callout |
| 131 | verification now uses <> in the MAIL command by default, as it did before. A |
| 132 | new callout option, "use_sender", has been added to request the other |
| 133 | behaviour. |
| 134 | |
| 135 | |
| 136 | Version 4.31 |
| 137 | ------------ |
| 138 | |
| 139 | 1. If you compile Exim to use GnuTLS, it now requires the use of release 1.0.0 |
| 140 | or greater. The interface to the obsolete 0.8.x releases is no longer |
| 141 | supported. There is one externally visible change: the format for the |
| 142 | display of Distinguished Names now uses commas as a separator rather than a |
| 143 | slash. This is to comply with RFC 2253. |
| 144 | |
| 145 | 2. When a message is received, the Received: header line is now generated when |
| 146 | reception is complete, instead of at the start of reception. For messages |
| 147 | that take a long time to come in, this changes the meaning of the timestamp. |
| 148 | There are several side-effects of this change: |
| 149 | |
| 150 | (a) If a message is rejected by a DATA or non-SMTP ACL, or by local_scan(), |
| 151 | the logged header lines no longer include the local Received: line, |
| 152 | because it has not yet been created. If the message is a non-SMTP one, |
| 153 | and the error is processed by sending a message to the sender, the copy |
| 154 | of the original message that is returned does not have an added |
| 155 | Received: line. |
| 156 | |
| 157 | (b) When a filter file is tested using -bf, no additional Received: header |
| 158 | is added to the test message. After some thought, I decided that this |
| 159 | is a bug fix. |
| 160 | |
| 161 | The contents of $received_for are not affected by this change. This |
| 162 | variable still contains the single recipient of a message, copied after |
| 163 | addresses have been rewritten, but before local_scan() is run. |
| 164 | |
| 165 | 2. Recipient callout verification, like sender verification, was using <> in |
| 166 | the MAIL FROM command. This isn't really the right thing, since the actual |
| 167 | sender may affect whether the remote host accepts the recipient or not. I |
| 168 | have changed it to use the actual sender in the callout; this means that |
| 169 | the cache record is now keyed on a recipient/sender pair, not just the |
| 170 | recipient address. There doesn't seem to be a real danger of callout loops, |
| 171 | since a callout by the remote host to check the sender would use <>. |
| 172 | |
| 173 | |
| 174 | Version 4.30 |
| 175 | ------------ |
| 176 | |
| 177 | 1. I have abolished timeout_DNS as an error that can be detected in retry |
| 178 | rules, because it has never worked. Despite the fact that it has been |
| 179 | documented since at least release 1.62, there was no code to support it. |
| 180 | If you have used it in your retry rules, you will now get a warning message |
| 181 | to the log and panic log. It is now treated as plain "timeout". |
| 182 | |
| 183 | 2. After discussion on the mailing list, Exim no longer adds From:, Date:, or |
| 184 | Message-Id: header lines to messages that do not originate locally, that is, |
| 185 | messages that have an associated sending host address. |
| 186 | |
| 187 | 3. When looking up a host name from an IP address, Exim now tries the DNS |
| 188 | first, and only if that fails does it use gethostbyaddr() (or equivalent). |
| 189 | This change was made because on some OS, not all the names are given for |
| 190 | addresses with multiple PTR records via the gethostbyaddr() interface. The |
| 191 | order of lookup can be changed by setting host_lookup_order. |
| 192 | |
| 193 | |
| 194 | Version 4.23 |
| 195 | ------------ |
| 196 | |
| 197 | 1. The new FIXED_NEVER_USERS build-time option creates a list of "never users" |
| 198 | that cannot be overridden. The default in the distributed EDITME is "root". |
| 199 | If for some reason you were (against advice) running deliveries as root, you |
| 200 | will have to ensure that FIXED_NEVER_USERS is not set in your |
| 201 | Local/Makefile. |
| 202 | |
| 203 | 2. The ${quote: operator now quotes an empty string, which it did not before. |
| 204 | |
| 205 | 3. Version 4.23 saves the contents of the ACL variables with the message, so |
| 206 | that they can be used later. If one of these variables contains a newline, |
| 207 | there will be a newline character in the spool that will not be interpreted |
| 208 | correctely by a previous version of Exim. (Exim ignores keyed spool file |
| 209 | items that it doesn't understand - precisely for this kind of problem - but |
| 210 | it expects them all to be on one line.) |
| 211 | |
| 212 | So the bottom line is: if you have newlines in your ACL variables, you |
| 213 | cannot retreat from 4.23. |
| 214 | |
| 215 | |
| 216 | Version 4.21 |
| 217 | ------------ |
| 218 | |
| 219 | 1. The idea of the "warn" ACL verb is that it adds a header or writes to the |
| 220 | log only when "message" or "log_message" are set. However, if one of the |
| 221 | conditions was an address verification, or a call to a nested ACL, the |
| 222 | messages generated by the underlying test were being passed through. This |
| 223 | no longer happens. The underlying message is available in $acl_verify_ |
| 224 | message for both "message" and "log_message" expansions, so it can be |
| 225 | passed through if needed. |
| 226 | |
| 227 | 2. The way that the $h_ (and $header_) expansions work has been changed by the |
| 228 | addition of RFC 2047 decoding. See the main documentation (the NewStuff file |
| 229 | until release 4.30, then the manual) for full details. Briefly, there are |
| 230 | now three forms: |
| 231 | |
| 232 | $rh_xxx: and $rheader_xxx: give the original content of the header |
| 233 | line(s), with no processing at all. |
| 234 | |
| 235 | $bh_xxx: and $bheader_xxx: remove leading and trailing white space, and |
| 236 | then decode base64 or quoted-printable "words" within the header text, |
| 237 | but do not do charset translation. |
| 238 | |
| 239 | $h_xxx: and $header_xxx: attempt to translate the $bh_ string to a |
| 240 | standard character set. |
| 241 | |
| 242 | If you have previously been using $h_ expansions to access the raw |
| 243 | characters, you should change to $rh_ instead. |
| 244 | |
| 245 | 3. When Exim creates an RFC 2047 encoded word in a header line, it labels it |
| 246 | with the default character set from the headers_charset option instead of |
| 247 | always using iso-8859-1. |
| 248 | |
| 249 | 4. If TMPDIR is defined in Local/Makefile (default in src/EDITME is |
| 250 | TMPDIR="/tmp"), Exim checks for the presence of an environment variable |
| 251 | called TMPDIR, and if it finds it is different, it changes its value. |
| 252 | |
| 253 | 5. Following a discussion on the list, the rules by which Exim recognises line |
| 254 | endings on incoming messages have been changed. The -dropcr and drop_cr |
| 255 | options are now no-ops, retained only for backwards compatibility. The |
| 256 | following line terminators are recognized: LF CRLF CR. However, special |
| 257 | processing applies to CR: |
| 258 | |
| 259 | (i) The sequence CR . CR does *not* terminate an incoming SMTP message, |
| 260 | nor a local message in the state where . is a terminator. |
| 261 | |
| 262 | (ii) If a bare CR is encountered in a header line, an extra space is added |
| 263 | after the line terminator so as not to end the header. The reasoning |
| 264 | behind this is that bare CRs in header lines are most likely either |
| 265 | to be mistakes, or people trying to play silly games. |
| 266 | |
| 267 | 6. The code for using daemon_smtp_port, local_interfaces, and the -oX options |
| 268 | has been reorganized. It is supposed to be backwards compatible, but it is |
| 269 | mentioned here just in case I've screwed up. |
| 270 | |
| 271 | |
| 272 | |
| 273 | Version 4.20 |
| 274 | ------------ |
| 275 | |
| 276 | 1. I have tidied and re-organized the code that uses alarm() for imposing time |
| 277 | limits on various things. It shouldn't affect anything, but if you notice |
| 278 | processes getting stuck, it may be that I've broken something. |
| 279 | |
| 280 | 2. The "arguments" log selector now also logs the current working directory |
| 281 | when Exim is called. |
| 282 | |
| 283 | 3. An incompatible change has been made to the appendfile transport. This |
| 284 | affects the case when it is used for file deliveries that are set up by |
| 285 | .forward and filter files. Previously, any settings of the "file" or |
| 286 | "directory" options were ignored. It is hoped that, like the address_file |
| 287 | transport in the default configuration, these options were never in fact set |
| 288 | on such transports, because they were of no use. |
| 289 | |
| 290 | Now, if either of these options is set, it is used. The path that is passed |
| 291 | by the router is in $address_file (this is not new), so it can be used as |
| 292 | part of a longer path, or modified in any other way that expansion permits. |
| 293 | |
| 294 | If neither "file" nor "directory" is set, the behaviour is unchanged. |
| 295 | |
| 296 | 4. Related to the above: in a filter, if a "save" command specifies a non- |
| 297 | absolute path, the value of $home/ is pre-pended. This no longer happens if |
| 298 | $home is unset or is set to an empty string. |
| 299 | |
| 300 | 5. Multiple file deliveries from a filter or .forward file can never be |
| 301 | batched; the value of batch_max on the transport is ignored for file |
| 302 | deliveries. I'm assuming that nobody ever actually set batch_max on the |
| 303 | address_file transport - it would have had odd effects previously. |
| 304 | |
| 305 | 6. DESTDIR is the more common variable that ROOT for use when installing |
| 306 | software under a different root filing system. The Exim install script now |
| 307 | recognizes DESTDIR first; if it is not set, ROOT is used. |
| 308 | |
| 309 | 7. If DESTDIR is set when installing Exim, it no longer prepends its value to |
| 310 | the path of the system aliases file that appears in the default |
| 311 | configuration (when a default configuration is installed). If an aliases |
| 312 | file is actually created, its name *does* use the prefix. |
| 313 | |
| 314 | |
| 315 | Version 4.14 |
| 316 | ------------ |
| 317 | |
| 318 | 1. The default for the maximum number of unknown SMTP commands that Exim will |
| 319 | accept before dropping a connection has been reduced from 5 to 3. However, you |
| 320 | can now change the value by setting smtp_max_unknown_commands. |
| 321 | |
| 322 | 2. The ${quote: operator has been changed so that it turns newline and carriage |
| 323 | return characters into \n and \r, respectively. |
| 324 | |
| 325 | 3. The file names used for maildir messages now include the microsecond time |
| 326 | fraction as well as the time in seconds, to cope with systems where the process |
| 327 | id can be re-used within the same second. The format is now |
| 328 | |
| 329 | <time>.H<microsec>P<pid>.<host> |
| 330 | |
| 331 | This should be a compatible change, but is noted here just in case. |
| 332 | |
| 333 | 4. The rules for creating message ids have changed, to cope with systems where |
| 334 | the process id can be re-used within the same second. The format, however, is |
| 335 | unchanged, so this should not cause any problems, except as noted in the next |
| 336 | item. |
| 337 | |
| 338 | 5. The maximum value for localhost_number has been reduced from 255 to 16, in |
| 339 | order to implement the new message id rules. For operating systems that have |
| 340 | case-insensitive file systems (Cygwin and Darwin), the limit is 10. |
| 341 | |
| 342 | 6. verify = header_syntax was allowing unqualified addresses in all cases. Now |
| 343 | it allows them only for locally generated messages and from hosts that match |
| 344 | sender_unqualified_hosts or recipient_unqualified_hosts, respectively. |
| 345 | |
| 346 | 7. For reasons lost in the mists of time, when a pipe transport was run, the |
| 347 | environment variable MESSAGE_ID was set to the message ID preceded by 'E' (the |
| 348 | form used in Message-ID: header lines). The 'E' has been removed. |
| 349 | |
| 350 | |
| 351 | Version 4.11 |
| 352 | ------------ |
| 353 | |
| 354 | 1. The handling of lines in the configuration file has changed. Previously, |
| 355 | macro expansion was applied to logical lines, after continuations had been |
| 356 | joined on. This meant that it could not be used in .include lines, which are |
| 357 | handled as physical rather than logical lines. Macro expansion is now done on |
| 358 | physical lines rather than logical lines. This means there are two |
| 359 | incompatibilities: |
| 360 | |
| 361 | (a) A macro that expands to # to turn a line into a comment now applies only |
| 362 | to the physical line where it appears. Previously, it would have caused |
| 363 | any following continuations also to be ignored. |
| 364 | |
| 365 | (b) A macro name can no longer be split over the boundary between a line and |
| 366 | its continuation. Actually, this is more of a bug fix. :-) |
| 367 | |
| 368 | 2. The -D command line option must now all be within one command line item. |
| 369 | This makes it possible to use -D to set a macro to the empty string by commands |
| 370 | such as |
| 371 | |
| 372 | exim -DABC ... |
| 373 | exim -DABC= ... |
| 374 | |
| 375 | Previously, these items would have moved on to the next item on the command |
| 376 | line. To include spaces in a macro definition item, quotes must be used, in |
| 377 | which case you can also have spaces after -D and surrounding the equals. For |
| 378 | example: |
| 379 | |
| 380 | exim '-D ABC = something' ... |
| 381 | |
| 382 | 3. The way that addresses that redirect to themselves are handled has been |
| 383 | changed, in order to fix an obscure bug. This should not cause any problems |
| 384 | except in the case of wanting to go back from a 4.11 (or later) release to an |
| 385 | earlier release. If there are undelivered messages on the spool that contain |
| 386 | addresses which redirect to themselves, and the redirected addresses have |
| 387 | already been delivered, you might get a duplicate delivery if you revert to an |
| 388 | earlier Exim. |
| 389 | |
| 390 | 4. The default way of looking up IP addresses for hosts in the manualroute and |
| 391 | queryprogram routers has been changed. If "byname" or "bydns" is explicitly |
| 392 | specified, there is no change, but if no method is specified, Exim now behaves |
| 393 | as follows: |
| 394 | |
| 395 | First, a DNS lookup is done. If this yields anything other than |
| 396 | HOST_NOT_FOUND, that result is used. Otherwise, Exim goes on to try a call to |
| 397 | getipnodebyname() (or gethostbyname() on older systems) and the result of the |
| 398 | lookup is the result of that call. |
| 399 | |
| 400 | This change has been made because it has been discovered that on some systems, |
| 401 | if a DNS lookup called via getipnodebyname() times out, HOST_NOT_FOUND is |
| 402 | returned instead of TRY_AGAIN. Thus, it is safest to try a DNS lookup directly |
| 403 | first, and only if that gives a definite "no such host" to try the local |
| 404 | function. |
| 405 | |
| 406 | 5. In fixing the minor security problem with pid_file_path, I have removed some |
| 407 | backwards-compatible (undocumented) code which was present to ease conversion |
| 408 | from Exim 3. In Exim 4, pid_file_path is a literal; in Exim 3 it was allowed to |
| 409 | contain "%s", which was replaced by the port number for daemons listening on |
| 410 | non-standard ports. In Exim 4, such daemons do not write a pid file. The |
| 411 | backwards compatibility feature was to replace "%s" by nothing if it occurred |
| 412 | in an Exim 4 setting of pid_file_path. The bug was in this code. I have solved |
| 413 | the problem by removing the backwards compatibility feature. Thus, if you still |
| 414 | have "%s" somewhere in a setting of pid_file_path, you should remove it. |
| 415 | |
| 416 | 6. There has been an extension to lsearch files. The keys in these files may |
| 417 | now be quoted in order to allow for whitespace and colons in them. This means |
| 418 | that if you were previously using keys that began with a doublequote, you will |
| 419 | now have to wrap them with extra quotes and escape the internal quotes. The |
| 420 | possibility that anybody is actually doing this seems extremely remote, but it |
| 421 | is documented just in case. |
| 422 | |
| 423 | |
| 424 | Version 4.10 |
| 425 | ------------ |
| 426 | |
| 427 | The build-time parameter EXIWHAT_KILL_ARG has been renamed EXIWHAT_KILL_SIGNAL |
| 428 | to better reflect its function. The OS-specific files have been updated. Only |
| 429 | if you have explicitly set this in your Makefile (highly unlikely) do you need |
| 430 | to change anything. |
| 431 | |
| 432 | **** |