doc/doc-txt/README.SIEVE

   1               Notes on the Sieve implementation for Exim
   2
   3 Exim Filter Versus Sieve Filter
   4
   5 Exim supports two incompatible filters: The traditional Exim filter and
   6 the Sieve filter. Since Sieve is a extensible language, it is important
   7 to understand "Sieve" in this context as "the specific implementation
   8 of Sieve for Exim".
   9
  10 The Exim filter contains more features, such as variable expansion, and
  11 better integration with the host environment, like external processes
  12 and pipes.
  13
  14 Sieve is a standard for interoperable filters, defined in RFC 5228,
  15 with multiple implementations around. If interoperability is important,
  16 then there is no way around it.
  17
  18
  19 Exim Implementation
  20
  21 The Exim Sieve implementation offers the core as defined by RFC 5228,
  22 the "encoded-character" extension (RFC 5228), the "envelope" test (RFC
  23 5228), the "fileinto" action (5228), the "copy" parameter (RFC 3894), the
  24 "vacation" action (5230), the "notify" action (draft-ietf-sieve-notify-12)
  25 with mailto URIs (draft-ietf-sieve-notify-mailto-05), the
  26 "i;ascii-numeric" comparator (RFC 2244) and the subaddress parameter
  27 (RFC 5233).
  28
  29 The Sieve filter is integrated in Exim and works very similar to the
  30 Exim filter: Sieve scripts are recognized by the first line containing
  31 "# sieve filter".  When using "keep" or "fileinto" to save a mail into a
  32 folder, the resulting string is available as the variable $address_file
  33 in the transport that stores it.  The following routers and transport
  34 show a typical use of Sieve:
  35
  36 begin routers
  37
  38 localuser_verify:
  39   driver = accept
  40   domains = +localdomains
  41   local_part_suffix = "-*"
  42   local_part_suffix_optional
  43   check_local_user
  44   require_files = $home/.forward
  45   verify_only = true
  46
  47 localuser_deliver:
  48   driver = redirect
  49   domains = +localdomains
  50   local_part_suffix = "-*"
  51   local_part_suffix_optional
  52   sieve_subaddress = "${sg{$local_part_suffix}{^-}{}}"
  53   sieve_useraddress = "$local_part"
  54   check_local_user
  55   require_files = $home/.forward
  56   file = $home/.forward
  57   check_ancestor
  58   allow_filter
  59   file_transport = localuser
  60   reply_transport = vacation
  61   sieve_vacation_directory = $home/mail/vacation
  62   verify = false
  63
  64 begin transports
  65
  66 localuser:
  67   driver = appendfile
  68   file = ${if eq{$address_file}{inbox} \
  69               {/var/mail/$local_part} \
  70               {${if eq{${substr_0_1:$address_file}}{/} \
  71                     {$address_file} \
  72                     {$home/mail/$address_file} \
  73               }} \
  74          }
  75   delivery_date_add
  76   envelope_to_add
  77   return_path_add
  78   mode = 0600
  79
  80 vacation:
  81   driver = autoreply
  82
  83 Absolute files are stored where specified, relative files are stored
  84 relative to $home/mail and "inbox" goes to the standard mailbox location.
  85 To enable "vacation", sieve_vacation_directory is set to the directory
  86 where vacation databases are held (don't put anything else in that
  87 directory) and point reply_transport to an autoreply transport.
  88 Setting the Sieve useraddress and subaddress allows to use the subaddress
  89 extension.
  90
  91
  92 RFC Compliance
  93
  94 Exim requires the first line to be "# sieve filter".  Of course the RFC
  95 does not enforce that line.  Don't expect examples to work without adding
  96 it, though.
  97
  98 RFC 5228 requires using CRLF to terminate the end of a line.
  99 The rationale was that CRLF is universally used in network protocols
 100 to mark the end of the line.  This implementation does not embed Sieve
 101 in a network protocol, but uses Sieve scripts as part of the Exim MTA.
 102 Since all parts of Exim use \n as newline character, this implementation
 103 does, too.  You can change this by defining the macro RFC_EOL at compile
 104 time to enforce CRLF being used.
 105
 106 The folder specified by "fileinto" must not contain the character
 107 sequence ".." to avoid security problems.  RFC 5228 does not specify the
 108 syntax of folders apart from keep being equivalent to fileinto "INBOX".
 109 This implementation uses "inbox" instead.
 110
 111 Sieve script errors currently cause that messages are silently filed into
 112 "inbox".  RFC 5228 requires that the user is notified of that condition.
 113 This may be implemented in future by adding a header line to mails that
 114 are filed into "inbox" due to an error in the filter.
 115
 116 The automatic replies generated by "vacation" do not contain an updated
 117 "references" header field.
 118
 119
 120 Semantics Of Keep
 121
 122 The keep command is equivalent to fileinto "inbox": It saves the
 123 message and resets the implicit keep flag.  It does not set the
 124 implicit keep flag; there is no command to set it once it has
 125 been reset.
 126
 127
 128 Semantics Of Fileinto
 129
 130 RFC 5228 does not specify if "fileinto" tries to create a mail folder,
 131 in case it does not exist.  This implementation allows to configure
 132 that aspect using the appendfile transport options "create_directory",
 133 "create_file" and "file_must_exist".  See the appendfile transport in
 134 the Exim specification for details.
 135
 136
 137 Allof And Anyof Test
 138
 139 RFC 5228 does not specify if these tests use shortcut/lazy evaluation.
 140 Exim uses shortcut evaluation.
 141
 142
 143 Action Reordering
 144
 145 RFC 5228 does not specify if actions may be executed out of order.
 146 Exim may execute them out of order, e.g. messages may be filed to
 147 folders or forwarded in a different order than specified, because
 148 those actions only setup delivery, but do not execute it themselves.
 149
 150
 151 Sieve Syntax And Semantics
 152
 153 RFC 5228 uses a generic grammar as syntax for commands and tests and
 154 performs many checks during semantic analysis.  Syntax is specified
 155 by grammar rules, semantics by natural language.  The intention is to
 156 provide a framework for the syntax that describes current commands as
 157 well as future extensions, and describing commands by semantics.
 158
 159 The following replacement for section 8.2 gives a grammar for specific
 160 commands of this implementation, thus removing most of the semantic
 161 analysis.  Since the parser can not parse unsupported extensions, the
 162 result is strict error checking of any executed and not executed code
 163 until "stop" is executed or the end of the script is reached.
 164
 165 8.2. Grammar
 166
 167 The grammar is specified in ABNF with two extensions to describe tagged
 168 arguments that can be reordered and grammar extensions: { } denotes a
 169 sequence of symbols that may appear in any order.  Example:
 170
 171   options = a b c
 172   start   = { options }
 173
 174 is equivalent to:
 175
 176   start   =  ( a b c ) / ( a c b ) / ( b a c ) / ( b c a ) / ( c a b ) / ( c b a )
 177
 178 The symbol =) is used to append to a rule:
 179
 180   start =  a
 181   start =) b
 182
 183 is equivalent to
 184
 185   start =  a b
 186
 187 The basic Sieve commands are specified using the following grammar, which
 188 language is a subset of the generic grammar above.  The start symbol is
 189 "start".
 190
 191   address-part     =  ":localpart" / ":domain" / ":all"
 192   comparator       =  ":comparator" string
 193   match-type       =  ":is" / ":contains" / ":matches"
 194   string           =  quoted-string / multi-line
 195   string-list      =  "[" string *("," string) "]" / string
 196   address-test     =  "address" { [address-part] [comparator] [match-type] }
 197                       string-list string-list
 198   test-list        =  "(" test *("," test) ")"
 199   allof-test       =  "allof" test-list
 200   anyof-test       =  "anyof" test-list
 201   exists-test      =  "exists" string-list
 202   false-test       =  "false"
 203   true=test        =  "true"
 204   header-test      =  "header" { [comparator] [match-type] }
 205                       string-list string-list
 206   not-test         =  "not" test
 207   relop            =  ":over" / ":under"
 208   size-test        =  "size" relop number
 209   block            =  "{" commands "}"
 210   if-command       =  "if" test block *( "elsif" test block ) [ "else" block ]
 211   stop-command     =  "stop" { stop-options } ";"
 212   stop-options     =
 213   keep-command     =  "keep" { keep-options } ";"
 214   keep-options     =
 215   discard-command  =  "discard" { discard-options } ";"
 216   discard-options  =
 217   redirect-command =  "redirect" { redirect-options } string ";"
 218   redirect-options =
 219   require-command  =  "require" { require-options } string-list ";"
 220   require-options  =
 221   test             =  address-test / allof-test / anyof-test / exists-test
 222                       / false-test / true-test / header-test / not-test
 223                       / size-test
 224   command          =  if-command / stop-command / keep-command
 225                       / discard-command / redirect-command
 226   commands         =  *command
 227   start            =  *require-command commands
 228
 229 The extensions "envelope" and "fileinto" are specified using the following
 230 grammar extension.
 231
 232   envelope-test    =  "envelope" { [comparator] [address-part] [match-type] }
 233                       string-list string-list
 234   test             =/ envelope-test
 235
 236   fileinto-command =  "fileinto" { fileinto-options } string ";"
 237   fileinto-options =
 238   command          =/ fileinto-command
 239
 240 The extension "copy" is specified as:
 241
 242   fileinto-options =) ":copy"
 243   redirect-options =) ":copy"
 244
 245
 246 The i;ascii-numeric Comparator
 247
 248 RFC 2244 describes this comparator and specifies that non-numeric strings
 249 are considered equal with an ordinal value higher than any numeric string.
 250 Although not stated explicitly, this includes the empty string.  A range
 251 of at least 2^31 is required.  This implementation does not limit the
 252 range, because it does not convert numbers to binary representation
 253 before comparing them.
 254
 255
 256 The vacation extension
 257
 258 The extension "vacation" is specified using the following grammar
 259 extension.
 260
 261   vacation-command =  "vacation" { vacation-options } <reason: string>
 262   vacation-options =  [":days" number]
 263                       [":subject" string]
 264                       [":from" string]
 265                       [":addresses" string-list]
 266                       [":mime"]
 267                       [":handle" string]
 268   command          =/ vacation-command
 269
 270
 271 Semantics Of ":mime"
 272
 273 The draft does not specify how strings using MIME entities are used
 274 to compose messages.  As a result, different implementations generate
 275 different mails.  The Exim Sieve implementation splits the reason into
 276 header and body.  It adds the header to the mail header and uses the body
 277 as mail body.  Be aware, that other imlementations compose a multipart
 278 structure with the reason as only part.  Both conform to the specification
 279 (or lack thereof).
 280
 281
 282 Semantics Of Not Using ":mime"
 283
 284 Sieve scripts are written in UTF-8, so is the reason string in this
 285 case.  This implementation adds MIME headers to indicate that.  This
 286 is not required by the vacation draft, which does not specify how
 287 the UTF-8 reason is processed to compose the resulting message.
 288
 289
 290 Default Subject
 291
 292 RFC 5230 specifies that the default message subject is "Auto: " plus
 293 the old subject.  Using this subject is dangerous, because many mailing
 294 lists verify addresses by sending a secret key in the subject of a
 295 message, asking to reply to the message for confirmation.  Using the
 296 default vacation subject confirms any subscription request of this kind,
 297 allowing to subscribe a third party to any mailing list, either to annoy
 298 the user or to declare spam as legitimate mail by proving to use opt-in.
 299
 300
 301 Rate Limiting Responses
 302
 303 In absence of a handle, this implementation hashes the reason,
 304 ":subject" option, ":mime" option and ":from" option and uses the hex
 305 string representation as filename within the "sieve_vacation_directory"
 306 to store the recipient addresses for this vacation parameter set.
 307
 308 The draft specifies that sites may define a minimum ":days" value than 1.
 309 This implementation uses 1.  The maximum value MUST greater than 7,
 310 and SHOULD be greater than 30.  This implementation uses a maximum of 31.
 311
 312 Vacation recipient address databases older than 31 days are automatically
 313 removed.  Users do not have to remove them manually when modifying their
 314 scripts.  Don't put anything but vacation databases in that directory
 315 or you risk that it will be removed, too!
 316
 317
 318 Global Reply Address Blacklist
 319
 320 The draft requires that each implementation offers a global black list
 321 of addresses that will never be replied to.  Exim offers this as option
 322 "never_mail" in the autoreply transport.
 323
 324
 325 The enotify extension
 326
 327 The extension "enotify" is specified using the following grammar
 328 extension.
 329
 330   notify-command =  "notify" { notify-options } <method: string>
 331   notify-options =  [":from" string]
 332                     [":importance" <"1" / "2" / "3">]
 333                     [":options" 1*(string-list / number)]
 334                     [":message" string]
 335
 336   command          =/ notify-command
 337
 338   valid_notify_method = "valid_notify_method"
 339                         <notification-uris: string-list>
 340
 341   test             =/ valid_notify_method
 342
 343 Only the mailto URI scheme is implemented.