| 1 | $Cambridge: exim/src/README.UPDATING,v 1.5 2005/03/08 11:38:21 ph10 Exp $ |
| 2 | |
| 3 | This document contains detailed information about incompatibilities that might |
| 4 | be encountered when upgrading from one release of Exim to another. The |
| 5 | information is in reverse order of release numbers. Mostly these are relatively |
| 6 | small points, and the configuration file is normally upwards compatible, but |
| 7 | there have been two big upheavals... |
| 8 | |
| 9 | |
| 10 | ************************************************************************** |
| 11 | * There was a big reworking of the way mail routing works for release * |
| 12 | * 4.00. Previously used "directors" were abolished, and all routing is * |
| 13 | * now done by routers. Policy controls for incoming mail are now done by * |
| 14 | * Access Control Lists instead of separate options. All this means that * |
| 15 | * pre-4.00 configuration files have to be massively converted. If you * |
| 16 | * are coming from a 3.xx release, please read the document in the file * |
| 17 | * doc/Exim4.upgrade, and allow some time to complete the upgrade. * |
| 18 | * * |
| 19 | * There was a big reworking of the way domain/host/net/address lists are * |
| 20 | * handled at release 3.00. If you are coming from a pre-3.00 release, it * |
| 21 | * might be easier to start again from a default configuration. Otherwise * |
| 22 | * you need to read doc/Exim3.upgrade and do a double conversion of your * |
| 23 | * configuration file. * |
| 24 | ************************************************************************** |
| 25 | |
| 26 | |
| 27 | The rest of this document contains information about changes in 4.xx releases |
| 28 | that might affect a running system. |
| 29 | |
| 30 | |
| 31 | Version 4.51 |
| 32 | ------------ |
| 33 | |
| 34 | The format in which GnuTLS parameters are cached (in the file gnutls-params in |
| 35 | the spool directory) has been changed. The new format can also be generated |
| 36 | externally, so it is now possible to update the values from outside Exim. This |
| 37 | has been implemented in an upwards, BUT NOT downwards, compatible manner. |
| 38 | Upgrading should be seamless: when Exim finds that it cannot understand an |
| 39 | existing cache file, it generates new parameters and writes them to the cache |
| 40 | in the new format. If, however, you downgrade from 4.51 to a previous release, |
| 41 | you MUST delete the gnutls-params file in the spool directory, because the |
| 42 | older Exim will not recognize the new format. |
| 43 | |
| 44 | |
| 45 | Version 4.50 |
| 46 | ------------ |
| 47 | |
| 48 | The exicyclog script has been updated to use three-digit numbers in rotated log |
| 49 | files if the maximum number to keep is greater than 99. If you are already |
| 50 | keeping more than 99, there will be an incompatible change when you upgrade. |
| 51 | You will probably want to rename your old log files to the new form before |
| 52 | running the new exicyclog. |
| 53 | |
| 54 | |
| 55 | Version 4.42 |
| 56 | ------------ |
| 57 | |
| 58 | RFC 3848 specifies standard names for the "with" phrase in Received: header |
| 59 | lines when AUTH and/or TLS are in use. This is the "received protocol" |
| 60 | field. Exim used to use "asmtp" for authenticated SMTP, without any |
| 61 | indication (in the protocol name) for TLS use. Now it follows the RFC and |
| 62 | uses "esmtpa" if the connection is authenticated, "esmtps" if it is |
| 63 | encrypted, and "esmtpsa" if it is both encrypted and authenticated. These names |
| 64 | appear in log lines as well as in Received: header lines. |
| 65 | |
| 66 | |
| 67 | Version 4.34 |
| 68 | ------------ |
| 69 | |
| 70 | Change 4.31/2 gave problems to data ACLs and local_scan() functions that |
| 71 | expected to see a Received: header. I have changed to yet another scheme. The |
| 72 | Received: header is now generated after the body is received, but before the |
| 73 | ACL or local_scan() is called. After they have run, the timestamp in the |
| 74 | Received: header is updated. |
| 75 | |
| 76 | Thus, change (a) of 4.31/2 has been reversed, but change (b) is still true, |
| 77 | which is lucky, since I decided it was a bug fix. |
| 78 | |
| 79 | |
| 80 | Version 4.33 |
| 81 | ------------ |
| 82 | |
| 83 | If an expansion in a condition on a "warn" statement fails because a lookup |
| 84 | defers, the "warn" statement is abandoned, and the next ACL statement is |
| 85 | processed. Previously this caused the whole ACL to be aborted. |
| 86 | |
| 87 | |
| 88 | Version 4.32 |
| 89 | ------------ |
| 90 | |
| 91 | Change 4.31/2 has been reversed, as it proved contentious. Recipient callout |
| 92 | verification now uses <> in the MAIL command by default, as it did before. A |
| 93 | new callout option, "use_sender", has been added to request the other |
| 94 | behaviour. |
| 95 | |
| 96 | |
| 97 | Version 4.31 |
| 98 | ------------ |
| 99 | |
| 100 | 1. If you compile Exim to use GnuTLS, it now requires the use of release 1.0.0 |
| 101 | or greater. The interface to the obsolete 0.8.x releases is no longer |
| 102 | supported. There is one externally visible change: the format for the |
| 103 | display of Distinguished Names now uses commas as a separator rather than a |
| 104 | slash. This is to comply with RFC 2253. |
| 105 | |
| 106 | 2. When a message is received, the Received: header line is now generated when |
| 107 | reception is complete, instead of at the start of reception. For messages |
| 108 | that take a long time to come in, this changes the meaning of the timestamp. |
| 109 | There are several side-effects of this change: |
| 110 | |
| 111 | (a) If a message is rejected by a DATA or non-SMTP ACL, or by local_scan(), |
| 112 | the logged header lines no longer include the local Received: line, |
| 113 | because it has not yet been created. If the message is a non-SMTP one, |
| 114 | and the error is processed by sending a message to the sender, the copy |
| 115 | of the original message that is returned does not have an added |
| 116 | Received: line. |
| 117 | |
| 118 | (b) When a filter file is tested using -bf, no additional Received: header |
| 119 | is added to the test message. After some thought, I decided that this |
| 120 | is a bug fix. |
| 121 | |
| 122 | The contents of $received_for are not affected by this change. This |
| 123 | variable still contains the single recipient of a message, copied after |
| 124 | addresses have been rewritten, but before local_scan() is run. |
| 125 | |
| 126 | 2. Recipient callout verification, like sender verification, was using <> in |
| 127 | the MAIL FROM command. This isn't really the right thing, since the actual |
| 128 | sender may affect whether the remote host accepts the recipient or not. I |
| 129 | have changed it to use the actual sender in the callout; this means that |
| 130 | the cache record is now keyed on a recipient/sender pair, not just the |
| 131 | recipient address. There doesn't seem to be a real danger of callout loops, |
| 132 | since a callout by the remote host to check the sender would use <>. |
| 133 | |
| 134 | |
| 135 | Version 4.30 |
| 136 | ------------ |
| 137 | |
| 138 | 1. I have abolished timeout_DNS as an error that can be detected in retry |
| 139 | rules, because it has never worked. Despite the fact that it has been |
| 140 | documented since at least release 1.62, there was no code to support it. |
| 141 | If you have used it in your retry rules, you will now get a warning message |
| 142 | to the log and panic log. It is now treated as plain "timeout". |
| 143 | |
| 144 | 2. After discussion on the mailing list, Exim no longer adds From:, Date:, or |
| 145 | Message-Id: header lines to messages that do not originate locally, that is, |
| 146 | messages that have an associated sending host address. |
| 147 | |
| 148 | 3. When looking up a host name from an IP address, Exim now tries the DNS |
| 149 | first, and only if that fails does it use gethostbyaddr() (or equivalent). |
| 150 | This change was made because on some OS, not all the names are given for |
| 151 | addresses with multiple PTR records via the gethostbyaddr() interface. The |
| 152 | order of lookup can be changed by setting host_lookup_order. |
| 153 | |
| 154 | |
| 155 | Version 4.23 |
| 156 | ------------ |
| 157 | |
| 158 | 1. The new FIXED_NEVER_USERS build-time option creates a list of "never users" |
| 159 | that cannot be overridden. The default in the distributed EDITME is "root". |
| 160 | If for some reason you were (against advice) running deliveries as root, you |
| 161 | will have to ensure that FIXED_NEVER_USERS is not set in your |
| 162 | Local/Makefile. |
| 163 | |
| 164 | 2. The ${quote: operator now quotes an empty string, which it did not before. |
| 165 | |
| 166 | 3. Version 4.23 saves the contents of the ACL variables with the message, so |
| 167 | that they can be used later. If one of these variables contains a newline, |
| 168 | there will be a newline character in the spool that will not be interpreted |
| 169 | correctely by a previous version of Exim. (Exim ignores keyed spool file |
| 170 | items that it doesn't understand - precisely for this kind of problem - but |
| 171 | it expects them all to be on one line.) |
| 172 | |
| 173 | So the bottom line is: if you have newlines in your ACL variables, you |
| 174 | cannot retreat from 4.23. |
| 175 | |
| 176 | |
| 177 | Version 4.21 |
| 178 | ------------ |
| 179 | |
| 180 | 1. The idea of the "warn" ACL verb is that it adds a header or writes to the |
| 181 | log only when "message" or "log_message" are set. However, if one of the |
| 182 | conditions was an address verification, or a call to a nested ACL, the |
| 183 | messages generated by the underlying test were being passed through. This |
| 184 | no longer happens. The underlying message is available in $acl_verify_ |
| 185 | message for both "message" and "log_message" expansions, so it can be |
| 186 | passed through if needed. |
| 187 | |
| 188 | 2. The way that the $h_ (and $header_) expansions work has been changed by the |
| 189 | addition of RFC 2047 decoding. See the main documentation (the NewStuff file |
| 190 | until release 4.30, then the manual) for full details. Briefly, there are |
| 191 | now three forms: |
| 192 | |
| 193 | $rh_xxx: and $rheader_xxx: give the original content of the header |
| 194 | line(s), with no processing at all. |
| 195 | |
| 196 | $bh_xxx: and $bheader_xxx: remove leading and trailing white space, and |
| 197 | then decode base64 or quoted-printable "words" within the header text, |
| 198 | but do not do charset translation. |
| 199 | |
| 200 | $h_xxx: and $header_xxx: attempt to translate the $bh_ string to a |
| 201 | standard character set. |
| 202 | |
| 203 | If you have previously been using $h_ expansions to access the raw |
| 204 | characters, you should change to $rh_ instead. |
| 205 | |
| 206 | 3. When Exim creates an RFC 2047 encoded word in a header line, it labels it |
| 207 | with the default character set from the headers_charset option instead of |
| 208 | always using iso-8859-1. |
| 209 | |
| 210 | 4. If TMPDIR is defined in Local/Makefile (default in src/EDITME is |
| 211 | TMPDIR="/tmp"), Exim checks for the presence of an environment variable |
| 212 | called TMPDIR, and if it finds it is different, it changes its value. |
| 213 | |
| 214 | 5. Following a discussion on the list, the rules by which Exim recognises line |
| 215 | endings on incoming messages have been changed. The -dropcr and drop_cr |
| 216 | options are now no-ops, retained only for backwards compatibility. The |
| 217 | following line terminators are recognized: LF CRLF CR. However, special |
| 218 | processing applies to CR: |
| 219 | |
| 220 | (i) The sequence CR . CR does *not* terminate an incoming SMTP message, |
| 221 | nor a local message in the state where . is a terminator. |
| 222 | |
| 223 | (ii) If a bare CR is encountered in a header line, an extra space is added |
| 224 | after the line terminator so as not to end the header. The reasoning |
| 225 | behind this is that bare CRs in header lines are most likely either |
| 226 | to be mistakes, or people trying to play silly games. |
| 227 | |
| 228 | 6. The code for using daemon_smtp_port, local_interfaces, and the -oX options |
| 229 | has been reorganized. It is supposed to be backwards compatible, but it is |
| 230 | mentioned here just in case I've screwed up. |
| 231 | |
| 232 | |
| 233 | |
| 234 | Version 4.20 |
| 235 | ------------ |
| 236 | |
| 237 | 1. I have tidied and re-organized the code that uses alarm() for imposing time |
| 238 | limits on various things. It shouldn't affect anything, but if you notice |
| 239 | processes getting stuck, it may be that I've broken something. |
| 240 | |
| 241 | 2. The "arguments" log selector now also logs the current working directory |
| 242 | when Exim is called. |
| 243 | |
| 244 | 3. An incompatible change has been made to the appendfile transport. This |
| 245 | affects the case when it is used for file deliveries that are set up by |
| 246 | .forward and filter files. Previously, any settings of the "file" or |
| 247 | "directory" options were ignored. It is hoped that, like the address_file |
| 248 | transport in the default configuration, these options were never in fact set |
| 249 | on such transports, because they were of no use. |
| 250 | |
| 251 | Now, if either of these options is set, it is used. The path that is passed |
| 252 | by the router is in $address_file (this is not new), so it can be used as |
| 253 | part of a longer path, or modified in any other way that expansion permits. |
| 254 | |
| 255 | If neither "file" nor "directory" is set, the behaviour is unchanged. |
| 256 | |
| 257 | 4. Related to the above: in a filter, if a "save" command specifies a non- |
| 258 | absolute path, the value of $home/ is pre-pended. This no longer happens if |
| 259 | $home is unset or is set to an empty string. |
| 260 | |
| 261 | 5. Multiple file deliveries from a filter or .forward file can never be |
| 262 | batched; the value of batch_max on the transport is ignored for file |
| 263 | deliveries. I'm assuming that nobody ever actually set batch_max on the |
| 264 | address_file transport - it would have had odd effects previously. |
| 265 | |
| 266 | 6. DESTDIR is the more common variable that ROOT for use when installing |
| 267 | software under a different root filing system. The Exim install script now |
| 268 | recognizes DESTDIR first; if it is not set, ROOT is used. |
| 269 | |
| 270 | 7. If DESTDIR is set when installing Exim, it no longer prepends its value to |
| 271 | the path of the system aliases file that appears in the default |
| 272 | configuration (when a default configuration is installed). If an aliases |
| 273 | file is actually created, its name *does* use the prefix. |
| 274 | |
| 275 | |
| 276 | Version 4.14 |
| 277 | ------------ |
| 278 | |
| 279 | 1. The default for the maximum number of unknown SMTP commands that Exim will |
| 280 | accept before dropping a connection has been reduced from 5 to 3. However, you |
| 281 | can now change the value by setting smtp_max_unknown_commands. |
| 282 | |
| 283 | 2. The ${quote: operator has been changed so that it turns newline and carriage |
| 284 | return characters into \n and \r, respectively. |
| 285 | |
| 286 | 3. The file names used for maildir messages now include the microsecond time |
| 287 | fraction as well as the time in seconds, to cope with systems where the process |
| 288 | id can be re-used within the same second. The format is now |
| 289 | |
| 290 | <time>.H<microsec>P<pid>.<host> |
| 291 | |
| 292 | This should be a compatible change, but is noted here just in case. |
| 293 | |
| 294 | 4. The rules for creating message ids have changed, to cope with systems where |
| 295 | the process id can be re-used within the same second. The format, however, is |
| 296 | unchanged, so this should not cause any problems, except as noted in the next |
| 297 | item. |
| 298 | |
| 299 | 5. The maximum value for localhost_number has been reduced from 255 to 16, in |
| 300 | order to implement the new message id rules. For operating systems that have |
| 301 | case-insensitive file systems (Cygwin and Darwin), the limit is 10. |
| 302 | |
| 303 | 6. verify = header_syntax was allowing unqualified addresses in all cases. Now |
| 304 | it allows them only for locally generated messages and from hosts that match |
| 305 | sender_unqualified_hosts or recipient_unqualified_hosts, respectively. |
| 306 | |
| 307 | 7. For reasons lost in the mists of time, when a pipe transport was run, the |
| 308 | environment variable MESSAGE_ID was set to the message ID preceded by 'E' (the |
| 309 | form used in Message-ID: header lines). The 'E' has been removed. |
| 310 | |
| 311 | |
| 312 | Version 4.11 |
| 313 | ------------ |
| 314 | |
| 315 | 1. The handling of lines in the configuration file has changed. Previously, |
| 316 | macro expansion was applied to logical lines, after continuations had been |
| 317 | joined on. This meant that it could not be used in .include lines, which are |
| 318 | handled as physical rather than logical lines. Macro expansion is now done on |
| 319 | physical lines rather than logical lines. This means there are two |
| 320 | incompatibilities: |
| 321 | |
| 322 | (a) A macro that expands to # to turn a line into a comment now applies only |
| 323 | to the physical line where it appears. Previously, it would have caused |
| 324 | any following continuations also to be ignored. |
| 325 | |
| 326 | (b) A macro name can no longer be split over the boundary between a line and |
| 327 | its continuation. Actually, this is more of a bug fix. :-) |
| 328 | |
| 329 | 2. The -D command line option must now all be within one command line item. |
| 330 | This makes it possible to use -D to set a macro to the empty string by commands |
| 331 | such as |
| 332 | |
| 333 | exim -DABC ... |
| 334 | exim -DABC= ... |
| 335 | |
| 336 | Previously, these items would have moved on to the next item on the command |
| 337 | line. To include spaces in a macro definition item, quotes must be used, in |
| 338 | which case you can also have spaces after -D and surrounding the equals. For |
| 339 | example: |
| 340 | |
| 341 | exim '-D ABC = something' ... |
| 342 | |
| 343 | 3. The way that addresses that redirect to themselves are handled has been |
| 344 | changed, in order to fix an obscure bug. This should not cause any problems |
| 345 | except in the case of wanting to go back from a 4.11 (or later) release to an |
| 346 | earlier release. If there are undelivered messages on the spool that contain |
| 347 | addresses which redirect to themselves, and the redirected addresses have |
| 348 | already been delivered, you might get a duplicate delivery if you revert to an |
| 349 | earlier Exim. |
| 350 | |
| 351 | 4. The default way of looking up IP addresses for hosts in the manualroute and |
| 352 | queryprogram routers has been changed. If "byname" or "bydns" is explicitly |
| 353 | specified, there is no change, but if no method is specified, Exim now behaves |
| 354 | as follows: |
| 355 | |
| 356 | First, a DNS lookup is done. If this yields anything other than |
| 357 | HOST_NOT_FOUND, that result is used. Otherwise, Exim goes on to try a call to |
| 358 | getipnodebyname() (or gethostbyname() on older systems) and the result of the |
| 359 | lookup is the result of that call. |
| 360 | |
| 361 | This change has been made because it has been discovered that on some systems, |
| 362 | if a DNS lookup called via getipnodebyname() times out, HOST_NOT_FOUND is |
| 363 | returned instead of TRY_AGAIN. Thus, it is safest to try a DNS lookup directly |
| 364 | first, and only if that gives a definite "no such host" to try the local |
| 365 | function. |
| 366 | |
| 367 | 5. In fixing the minor security problem with pid_file_path, I have removed some |
| 368 | backwards-compatible (undocumented) code which was present to ease conversion |
| 369 | from Exim 3. In Exim 4, pid_file_path is a literal; in Exim 3 it was allowed to |
| 370 | contain "%s", which was replaced by the port number for daemons listening on |
| 371 | non-standard ports. In Exim 4, such daemons do not write a pid file. The |
| 372 | backwards compatibility feature was to replace "%s" by nothing if it occurred |
| 373 | in an Exim 4 setting of pid_file_path. The bug was in this code. I have solved |
| 374 | the problem by removing the backwards compatibility feature. Thus, if you still |
| 375 | have "%s" somewhere in a setting of pid_file_path, you should remove it. |
| 376 | |
| 377 | 6. There has been an extension to lsearch files. The keys in these files may |
| 378 | now be quoted in order to allow for whitespace and colons in them. This means |
| 379 | that if you were previously using keys that began with a doublequote, you will |
| 380 | now have to wrap them with extra quotes and escape the internal quotes. The |
| 381 | possibility that anybody is actually doing this seems extremely remote, but it |
| 382 | is documented just in case. |
| 383 | |
| 384 | |
| 385 | Version 4.10 |
| 386 | ------------ |
| 387 | |
| 388 | The build-time parameter EXIWHAT_KILL_ARG has been renamed EXIWHAT_KILL_SIGNAL |
| 389 | to better reflect its function. The OS-specific files have been updated. Only |
| 390 | if you have explicitly set this in your Makefile (highly unlikely) do you need |
| 391 | to change anything. |
| 392 | |
| 393 | **** |