Testsuite: bounce_message_file and warn_message_file
[exim.git] / test / README
CommitLineData
151b83f8
PH
1EXPORTABLE EXIM TEST SUITE
2--------------------------
3
4This document last updated for:
5
33390b6a
GF
6Test Suite Version: 4.87
7Date: 30 January 2016
151b83f8
PH
8
9
10BACKGROUND
11----------
12
13For a long time, the Exim test suite was confined to Philip Hazel's
14workstation, because it relied on that particular environment. The problem is
15that an MTA such as Exim interacts a great deal with its environment, so if you
16run it somewhere else, the output will be different, which makes automatic
17checking difficult. Even in a single environment, things are not all that easy.
18For instance, if Exim delivers a message, the log line (which one would want to
19compare) contains a timestamp and an Exim message id that will be different
20each time. This issue is dealt with by a Perl script that munges the output by
21recognizing changing sequences and replacing them with fixed values before
22doing a comparison. Another problem with exporting the original test suite is
23that it assumes a version of Exim with more or less every optional feature
24enabled.
25
26This README describes a new test suite that is intended to be exportable and to
1b781f48
PH
27run in a number of different environments. The tests themselves are in no
28particular order; they accumulated over the years as Exim was extended and
29modified. They vary greatly in size and complexity. Some were specifically
30constructed to test new features; others were made to demonstrate that a bug
31had been fixed.
151b83f8
PH
32
33A few of the original tests have had to be omitted from this more general
34suite because differences in operating system behaviour make it impossible to
35generalize them. An example is a test that uses a version of Exim that is
36setuid to the Exim user rather than root, with the deliver_drop_privilege
37option set. In Linux, such a binary is able to deliver a message as the caller
38of Exim, because it can revert to the caller's uid. In FreeBSD this is not the
39case.
40
151b83f8
PH
41
42REQUIREMENTS
43------------
44
45In order to run this test suite, the following requirements must be met:
46
1b781f48 47(1) You should run the tests on a matching version of Exim, because the suite
151b83f8
PH
48 is continuously updated to test the latest features and bug fixes. The
49 version you test does not, however, have to be installed as the live
1b781f48
PH
50 version. You can of course try the tests on any version of Exim, but some
51 may fail. In particular, the test suite will fall apart horrible with
52 versions of Exim prior to 4.54.
151b83f8
PH
53
54(2) You can use any non-root login to run the tests, but there must be access
55 via "sudo" to root from this login. Privilege is required to override
56 configuration change checks and for things like cleaning up spool files,
57 but on the other hand, the tests themselves need to call Exim from a
58 non-root process. The use of "sudo" is the easiest way to achieve all this.
59 The test script uses "sudo" to do a number of things as root, so it is best
60 if you set a sudo timeout so that you do not have to keep typing a
61 password. For example, if you put
62
63 Defaults timestamp_timeout=480
64
65 in /etc/sudoers, a password lasts for 8 hours (a working day). It is
a56f166d
JJ
66 not permitted to run the tests as the Exim user because the test suite
67 tracks the two users independently. Using the same user would result
68 in false positives on some tests.
151b83f8 69
8121f028
PP
70 Further, some tests invoke sudo in an environment where there might not be
71 a TTY, so tickets should be global, not per-TTY. Taking this all together
72 and assuming a user of "exim-build", you might have this in sudoers:
73
74 Defaults:exim-build timestamp_timeout=480,!tty_tickets
75
b43517ed
JH
76(3) The login under which you run the tests must have the exim group as a
77 secondary so that it has access to logs, spool files, etc. However, it
78 should have a different primary group (eg. "users" vs. "eximgroup"). The
79 login should not be one of the names "userx", "usery", "userz", or a few
80 other simple ones such as "abcd" and "xyz" and single letters that are used
81 in the tests. The test suite expects the login to have a gecos name; I think
82 it will now run if the gecos field is empty but there may be anomalies.
8d42c836
HSHR
83 The login must not contain a dash or an equal sign. (Otherwise some tests
84 about local_from_{suffix,prefix} will fail.)
151b83f8
PH
85
86(4) The directory into which you unpack the test suite must be accessible by
1a2a87af
JH
87 the Exim user, so that code running as exim can access the files therein.
88 This includes search-access on all path elements leading to it. A
1b781f48
PH
89 world-readable directory is fine. However, there may be problems if the
90 path name of the directory is excessively long. This is because it
91 sometimes appears in log lines or debug output, and if it is truncated, it
151b83f8
PH
92 is no longer recognized.
93
94(5) Exim must be built with its user and group specified at build time, and
95 with certain minimum facilities, namely:
96
3d41333d
TL
97 Routers: accept, dnslookup, manualroute, redirect
98 Transports: appendfile, autoreply, pipe, smtp
99 Lookups: lsearch
100 Authenticators: plaintext
151b83f8
PH
101
102 Most Exim binaries will have these included.
103
104(6) A C compiler is needed to build some test programs, and the test script is
105 written in Perl, so you need that.
106
107(7) Some of the tests run Exim as a daemon, and others use a testing server
108 (described below). These require TCP ports. In the configurations and
109 scripts, the ports are parameterized, but at present, fixed values are
110 written into the controlling script. These are ports 1224 to 1229. If these
111 ports are not available for use, some of the tests will fail.
112
113(8) There is an underlying assumption that the host on which the tests are
114 being run has an IPv4 address (which the test script seeks out). If there
115 is also an IPv6 address, additional tests are run when the Exim binary
116 contains IPv6 support. There are checks in the scripts for a running IPv4
117 interface; when one is not found, some tests are skipped (with a warning
5c03403d 118 message). The local net may not be in 10.250.0/16 as that is used by the suite.
151b83f8 119
33191679 120(9) Exim must be built with TRUSTED_CONFIG_LIST support, so that the test
1a2a87af 121 configs can be placed into it. A suitable file location is .../exim/test/trusted_configs
52a7b5f3
JH
122 with content .../exim/test/test-config [fill out the ... to make full
123 paths]. This file should be owner/group matching CONFIGURE_OWNER/GROUP,
755c0d87 124 or root/root, and it has to be accessible for the login, under which
7a601efb 125 you run the tests. The config files in .../exim/test/confs/ should be
755c0d87 126 owner/group the same. DISABLE_D_OPTION must not be used. If ALT_CONFIG_PREFIX is used, it
1a2a87af 127 must contain the directory of the test-suite. WHITELIST_D_MACROS should contain:
7a601efb 128
fbbd45ff 129 DIR:EXIM_PATH:AA:ACL:ACLRCPT:ACL_MAIL:ACL_PREDATA:ACL_RCPT:AFFIX:ALLOW:ARG1:ARG2:AUTHF:AUTHS:AUTH_ID_DOMAIN:BAD:BANNER:BB:BR:BRB:CERT:COM:COMMAND_USER:CONNECTCOND:CONTROL:CREQCIP:CREQMAC:CRL:CSS:D6:DATA:DCF:DDF:DEFAULTDWC:DELAY:DETAILS:DRATELIMIT:DYNAMIC_OPTION:ELI:ERROR_DETAILS:ERT:FAKE:FALLBACK:FILTER:FILTER_PREPEND_HOME:FORBID:FORBID_SMTP_CODE:FUSER:HAI:HAP:HARDLIMIT:HEADER_LINE_MAXSIZE:HEADER_MAXSIZE:HELO_MSG:HL:HOSTS:HOSTS_AVOID_TLS:HOSTS_MAX_TRY:HVH:IFACE:IGNORE_QUOTA:INC:INSERT:IP1:IP2:LAST:LDAPSERVERS:LENCHECK:LIMIT:LIST:LOG_SELECTOR:MAXNM:MESSAGE_LOGS:MSIZE:NOTDAEMON:ONCE:ONLY:OPT:OPTION:ORDER:PAH:PEX:PORT:PTBC:QDG:QOLL:QUOTA:QUOTA_FILECOUNT:QWM:RCPT_MSG:REMEMBER:REQUIRE:RETRY:RETRY1:RETRY2:RETURN:RETURN_ERROR_DETAILS:REWRITE:ROUTE_DATA:RRATELIMIT:SELECTOR:SELF:SERVER:SERVERS:SREQCIP:SREQMAC:SRV:STRICT:SUB:SUBMISSION_OPTIONS:TIMEOUTDEFER:TIMES:TRUSTED:TRYCLEAR:UL:USE_SENDER:UTF8:VALUE:WMF
33191679 130
4b9529fc
PP
131(10) Exim must *not* be built with USE_READLINE, as the test-suite's automation
132 assumes the simpler I/O model.
daea6332 133 Exim must *not* be built with HEADERS_CHARSET set to UTF-8.
4b9529fc 134
33191679 135
151b83f8
PH
136
137OPTIONAL EXTRAS
138---------------
139
140If the Exim binary that is being tested contains extra functionality in
141addition to the minimum specified above, additional tests are run to exercise
142the extra functionality, except for a few special cases such as the databases
143(MySQL, PostgreSQL, LDAP) where special data is needed for the tests.
144
145
146RUNNING THE TEST SUITE
147----------------------
148
544b76dc
JH
149(1) Clone the git tree for Exim. This include both the Exim source and the
150 testsuite.
151b83f8 151
544b76dc 152(2) cd into the test/ subdirectory (where this README lives).
151b83f8 153
8c4f17b3
JH
154(3) Run "./configure" and then "make". This builds a few auxiliary programs that
155 are written in C.
151b83f8 156
8121f028 157(4) echo $PWD/test-config >> your_TRUSTED_CONFIG_LIST_filename
1a2a87af 158 Typically that is .../exim/test/trusted_configs
33191679
PP
159
160(5) Run "./runtest" (a Perl script) as described below.
151b83f8 161
33191679 162(6) If you want to see what tests are available, run "./listtests".
151b83f8
PH
163
164
165BREAKING OUT OF THE TEST SCRIPT
166-------------------------------
167
168If you abandon the test run by typing ^C, the interrupt may be passed to a
169program that the script is running, or it may be passed to the script itself.
170In the former case, the script should detect that the program has ended
171abnormally. In both cases, the script tries to clean up everything, including
172killing any Exim daemons that it has started. However, there may be race
173conditions in which the clean up does not happen. If, after breaking out of a
174run, you see strange errors in the next run, look for any left-over Exim
175daemons, and kill them by hand.
176
177
178THE LISTTESTS SCRIPT
179--------------------
180
181The individual test scripts are in subdirectories of the "scripts" directory.
182If you do not supply any arguments to ./listtests, it scans all the scripts in
183all the directories, and outputs the heading line from each script. The output
184is piped through "less", and begins like this:
185
186=== 0000-Basic ===
187Basic/0001 Basic configuration setting
188Basic/0002 Common string expansions
189Basic/0003 Caseless address blocking
190...
191
192Lines that start === give the name of the subdirectory containing the test
193scripts that follow. If you supply an argument to ./listtests, it is used as a
194Perl pattern to match case-independently against the names of the
195subdirectories. Only those that match are scanned. For example, "./listtests
196ipv6" outputs this:
197
198=== 1000-Basic-ipv6 ===
199=== Requires: support IPv6
200Basic-ipv6/1000 -bh and non-canonical IPv6 addresses
201Basic-ipv6/1001 recognizing IPv6 address in HELO/EHLO
202
203=== 2250-dnsdb-ipv6 ===
204=== Requires: support IPv6
205 lookup dnsdb
206dnsdb-ipv6/2250 dnsdb ipv6 lookup in string expansions
207
208If you supply a second argument to ./listtests, it is used as a Perl pattern to
209match case-independently against the individual script titles. For example,
210"./listtests . mx" lists all tests whose titles contain "mx", because "."
211matches all the subdirectory names.
212
213
214THE RUNTEST SCRIPT
215------------------
216
217If you do not supply any arguments to ./runtest, it searches for an Exim
1b1fefe3
HSHR
218source tree at the same level as the test suite directory. A source tree
219is a source tree, if it contains a build-* directory.
220
221It then looks for an Exim binary in a "build" directory of that source
222tree. If there are several Exim source trees, it chooses the latest
223version of Exim. Consider the following example:
151b83f8
PH
224
225 $ ls -F /source/exim
1b781f48 226 exim-4.60/ exim-4.62/ exim-testsuite-x.xx/
151b83f8 227
1b781f48
PH
228A simple ./runtest from within the test suite will use a 4.62 binary if it
229finds one, otherwise a 4.60 binary. If a binary cannot be found, the script
151b83f8
PH
230prompts for one. Alternatively, you can supply the binary on the command line:
231
232 ./runtest /usr/exim/bin/exim
233
1b781f48
PH
234A matching test suite is released with each Exim release; if you use a test
235suite that does not match the binary, some tests may fail.
236
237The test suite uses some of the Exim utilities (such as exim_dbmbuild), and it
238expects to find them in the same directory as Exim itself. If they are not
239found, the tests that use them are omitted. A suitable comment is output.
151b83f8
PH
240
241On the ./runtest command line, following the name of the binary, if present,
242there may be a number of options and then one or two numbers. The full syntax
243is as follows:
244
245 ./runtest [binary name] [runtest options] [exim options] \
246 [first test] [last test]
247
248There are some options for the ./runtest script itself:
249
c1c469db
TL
250 -CONTINUE This will allow the script to move past some failing tests. It will
251 write a simple failure line with the test number in a temporary
252 logfile test/failed-summary.log. Unexpected exit codes will still
253 stall the test execution and require interaction.
254
151b83f8
PH
255 -DEBUG This option is for debugging the test script. It causes some
256 tracing information to be output.
257
258 -DIFF By default, file comparisons are done using a private compare
259 command called "cf", which is built from source that is provided in
260 the src directory. This is a command I've had for nearly 20 years -
261 look at the source comments for its history - whose output I
262 prefer. However, if you want to use "diff" instead, give -DIFF as a
263 runtest option. In that case, "diff -u" is used for comparisons.
264 (If it turns out that most people prefer to use diff, I'll change
265 the default.)
266
1b1fefe3
HSHR
267 -FLAVOR <flavor>
268 -FLAVOUR <flavour>
269 This allows "overrides" for the test results. It's intended
270 use is to deal with distro specific differences in the test
6336058c 271 output. The default flavour is "FOO" if autodetection fails.
4c04137d 272 (Autodetection is possible for known flavours only. Known
6336058c
HSHR
273 flavours are computed after file name extensions in stdout/*
274 and stderr/*.)
275
276 If during the test run differences between the current and
277 the expected output are found and no flavour file exists already,
278 you may update the "common" expected output or you may create a
279 flavour file. If a flavour file already exists, any updates will go
1b1fefe3
HSHR
280 into that flavour file!
281
151b83f8
PH
282 -KEEP Normally, after a successful run, the test output files are
283 deleted. This option prevents this. It is useful when running a
284 single test, in order to look at the actual output before it is
285 modified for comparison with saved output.
286
287 -NOIPV4 Pretend that an IPv4 interface was not found. This is useful for
288 testing that the test suite correctly skips tests that require
289 a running IPv4 interface.
290
291 -NOIPV6 Pretend that an IPv6 interface was not found. This is useful for
292 testing that the test suite correctly skips tests that require
293 a running IPv6 interface.
294
295 -UPDATE If this option is set, any detected changes in test output are
296 automatically accepted and used to update the stored copies of the
297 output. It is a dangerous option, but it useful for the test suite
298 maintainer after making a change to the code that affects a lot of
299 tests (for example, the wording of a message).
300
1a13c13c
JH
301 -SLOW For very slow hosts that appear to have Heisenbugs, delay before
302 comparing output files from a testcase
303
151b83f8
PH
304The options for ./runtest must be given first (but after the name of the
305binary, if present). Any further options, that is, items on the command line
306that start with a hyphen, are passed to the Exim binary when it is run as part
307of a test. The only sensible use of this is to pass "-d" in order to run a test
308with debugging enabled. Any other options are likely to conflict with options
309that are set in the tests. Some tests are already set up to run with debugging.
310In these cases, -d on the command line overrides their own debug settings.
311
312The final two arguments specify the range of tests to be run. Test numbers lie
313in the range 1 to 9999. If no numbers are given, the defaults are 1 and 8999
314(sic). Tests with higher numbers (9000 upwards) are not run automatically
315because they require specific data (such as a particular MySQL table) that is
316unlikely to be generally available.
317
318Tests that require certain optional features of Exim are grouped by number, so
319in any given range, not all the tests will exist. Non-existent tests are just
320skipped, but if there are no tests at all in the given range, a message is
321output.
322
323If you give only one number, just that test is run (if it exists). Instead of a
324second number, you can give the character "+", which is interpreted as "to the
325end". Normally this is 8999; if the starting number is 9000 or higher, "+" is
326interpreted as 9999. Examples:
327
328 ./runtest 1300
329 ./runtest 1400 1699
330 ./runtest /usr/sbin/exim 5000 +
331 ./runtest -DIFF -d 81
332
333When the script starts up, the first thing it does is to check that you have
334sudo access to root. Then it outputs the version number of the Exim binary that
335it is testing, and also information about the optional facilities that are
336present (obtained from "exim -bV"). This is followed by some environmental
337information, including the current login id and the hosts's IP address. The
338script checks that the current user is in the Exim group, and that the Exim
339user has access to the test suite directory.
340
341The script outputs the list of tests requested, and a list of tests that will
342be omitted because the relevant optional facilities are not in the binary. You
343are then invited to press Return to start the tests running.
344
345
346TEST OUTPUT
347-----------
348
349When all goes well, the only permanent output is the identity of the tests as
350they are run, and "Script completed" for each test script, for example:
351
352 Basic/0001 Basic configuration setting
353 Script completed
354 Basic/0002 Basic string expansions
355 Script completed
356 Basic/0003 Caseless address blocking
357 Script completed
358 Basic/0004 Caseful address blocking
359 Script completed
360 Basic/0005 -bs to simple local delivery
361 ...
362
363While a script is running, it shows "Test n" on the screen, for each of the
364Exim tests within the script. There may also be comments from some tests when a
365delay is expected, for example, if there is a "sleep" while testing a timeout.
366
367Before each set of optional tests, an extra identifying line is output. For
368example:
369
370 >>> The following tests require: authenticator cram_md5
371 CRAM-MD5/2500 CRAM-MD5 server tests
372 Script completed
373 CRAM-MD5/2501 CRAM-MD5 client tests
374 Script completed
375
376If a test fails, you are shown the output of the text comparison that failed,
377and prompted as to what to do next. The output is shown using the "less"
1b781f48
PH
378command, or "more" if "less" is not available. The options for "less" are set
379to that it automatically exits if there is less that a screenful of output. By
380default, the output is from the "cf" program, and might look like this:
151b83f8
PH
381
382 DBM/1300 DBM files and exim_dbmbuild
383 ===============
384 Lines 7-9 of "test-stdout-munged" do not match lines 7-11 of "stdout/1300".
385 ----------
386 exim_dbmbuild exit code = 1
387 Continued set of lines is too long: max permitted length is 99999
388 exim_dbmbuild exit code = 1
389 ----------
390 dbmbuild abandoned
391 exim_dbmbuild exit code = 2
392 Continued set of lines is too long: max permitted length is 99999
393 dbmbuild abandoned
394 exim_dbmbuild exit code = 2
395 ===============
396 1 difference found.
397 "test-stdout-munged" contains 16 lines; "stdout/1300" contains 18 lines.
398
cc442294 399 Continue, Retry, Update & retry, Quit? [Q]
151b83f8
PH
400
401This example was generated by running the test with a version of Exim
402that had a bug in the exim_dbmbuild utility (the bug was fixed at release
4034.53). See "How the tests work" below for a description of the files that are
404used. In this case, the standard output differed from what was expected.
405
406The reply to the prompt must either be empty, in which case it takes the
407default that is given in brackets (in this case Q), or a single letter, in
cc442294 408upper or lower case (in this case, one of C, R, U, or Q). If you type anything
151b83f8
PH
409else, the prompt is repeated.
410
411"Continue" carries on as if the files had matched; that is, it ignores the
412mismatch. Any other output files for the same test will be compared before
413moving on to the next test.
414
415"Update & retry" copies the new file to the saved file, and reruns the test
416after doing any further comparisons that may be necessary.
417
cc442294
JH
418"Retry" does the same apart from the file copy.
419
151b83f8
PH
420Other circumstances give rise to other prompts. If a test generates output for
421which there is no saved data, the prompt (after a message stating which file is
4c04137d 422unexpectedly not empty) is:
151b83f8
PH
423
424 Continue, Show, or Quit? [Q]
425
426"Show" displays the data on the screen, and then you get the "Continue..."
427prompt. If a test ends with an unexpected return code, the prompt is:
428
429 show stdErr, show stdOut, Continue (without file comparison), or Quit? [Q]
430
431Typically in these cases there will be something interesting in the stderr
432or stdout output. There is a similar prompt after the "server" auxiliary
433program fails.
434
435
436OPENSSL AND GNUTLS ERROR MESSAGES
437---------------------------------
438
439Some of the TLS tests deliberately cause errors to check how Exim handles them.
440It has been observed that different releases of the OpenSSL and GnuTLS
441libraries generate different error messages. This may cause the comparison with
442the saved output to fail. Such errors can be ignored.
443
444
1b781f48
PH
445OTHER ISSUES
446------------
447
448. Some of the tests are time-sensitive (e.g. when testing timeouts, as in test
449 461). These may fail if run on a host that is also running a lot of other
450 processes.
451
452. Some versions of "ls" use a different format for times and dates. This can
453 cause test 345 to fail.
454
455. Test 0142 tests open file descriptors; on some hosts the output may vary.
456
05e0ef26
TL
457. Some tests may fail, for example 0022, because it says it uses cached data
458 when the expected output thinks it should not be in cache. Item #5 in the
459 Requirements section has:
460 "Exim must be built with its user and group specified at build time"
461 This means that you cannot use the "ref:username" in your Local/Makefile
462 when building the exim binary, in any of the following fields:
463 EXIM_USER EXIM_GROUP CONFIGURE_OWNER CONFIGURE_GROUP
464
465. If the runtest script warns that the hostname is not a Fully Qualified
466 Domain Name (FQDN), expect that some tests will fail, for example 0036,
467 with an extra log line saying the hostname doesn't resolve. You must use a
468 FQDN for the hostname for proper test functionality.
469
6822b909
TL
470. If you change your hostname to a FQDN, you must delete the test/dnszones
471 subdirectory. When you next run the runtest script, it will rebuild the
472 content to use the new hostname.
473
05e0ef26
TL
474. If your hostname has an uppercase characters in it, expect that some tests
475 will fail, for example, 0036, because some log lines will have the hostname
476 in all lowercase. The regex which extracts the hostname from the log lines
477 will not match the lowercased version.
478
479. Some tests may fail, for example 0015, with a cryptic error message:
480 Server return code 99
481 Due to security concerns, some specific files MUST have the group write bit
482 off. For the purposes of the test suite, some test/aux-fixed/* files MUST
483 have the group write bit off, so it's easier to just remove the group write
484 bit for all of them. If your umask is set to 002, the group write bit will
485 be on by default and you'll see this problem, so make sure your umask is
486 022 and re-checkout the test/ subdirectory.
487
3d41333d
TL
488. Some tests will fail if the username and group name are different. It does
489 not have to be the primary group, a secondary group is sufficient.
490
1b781f48 491
151b83f8
PH
492OTHER SCRIPTS AND PROGRAMS
493--------------------------
494
495There is a freestanding Perl script called "listtests" that scans the test
496scripts and outputs a list of all the tests, with a short descriptive comment
497for each one. Special requirements for groups of tests are also noted.
498
499The main runtest script makes use of a second Perl script and some compiled C
500programs. These are:
501
502patchexim A Perl script that makes a patched version of Exim (see the
503 next section for details).
504
505bin/cf A text comparison program (see above).
506
507bin/checkaccess A program that is run as root; it changes uid/gid to the
508 Exim user and group, and then checks that it can access
509 files in the test suite's directory.
510
511bin/client A script-driven SMTP client simulation.
512
513bin/client-gnutls A script-driven SMTP client simulation with GnuTLS support.
514 This is built only if GnuTLS support is detected on the host.
515
516bin/client-ssl A script-driven SMTP client simulation with OpenSSL support.
517 This is built only if OpenSSL support is detected on the
518 host.
519
effbc122
HSHR
520bin/client-anytls A symlink to either client-ssl or client-gnutls, if
521 any is built.
522
151b83f8
PH
523bin/fakens A fake "nameserver" for DNS tests (see below for details).
524
525bin/fd A program that outputs details of open file descriptors.
526
527bin/iefbr14 A program that does nothing, and returns 0. It's just like
528 the "true" command, but it is in a known place.
529
530bin/loaded Some dynamically loaded functions for testing dlfunc support.
531
bbe15da8
PH
532bin/mtpscript A script-driven SMTP/LMTP server simulation, on std{in,out}.
533
534bin/server A script-driven SMTP server simulation, over a socket.
535
536bin/showids Output the current uid, gid, euid, egid.
151b83f8
PH
537
538The runtest script also makes use of a number of ordinary commands such as
539"cp", "kill", "more", and "rm", via the system() call. In some cases these are
540run as root by means of sudo.
541
542
543STANDARD SUBSTITUTIONS
544----------------------
545
546In the following sections, there are several references to the "standard
547substitutions". These make changes to some of the stored files when they are
548used in a test. To save repetition, the substitutions themselves are documented
549here:
550
551 CALLER is replaced by the login name of the user running the tests
1b781f48 552 CALLERGROUP is replaced by the caller's group id
151b83f8
PH
553 CALLER_GID is replaced by the caller's group id
554 CALLER_UID is replaced by the caller's user id
555 DIR is replaced by the name of the test-suite directory
556 EXIMGROUP is replaced by the name of the Exim group
557 EXIMUSER is replaced by the name of the Exim user
558 HOSTIPV4 is replaced by the local host's IPv4 address
559 HOSTIPV6 is replaced by the local host's IPv6 address
560 HOSTNAME is replaced by the local host's name
561 PORT_D is replaced by a port number for normal daemon use
562 PORT_N is replaced by a port number that should never respond
563 PORT_S is replaced by a port number for normal bin/server use
7a601efb 564 PORT_DYNAMIC is replaced by a port number allocated dynamically
151b83f8
PH
565 TESTNUM is replaced by the current test number
566 V4NET is replaced by an IPv4 network number for testing
567 V6NET is replaced by an IPv6 network number for testing
568
569PORT_D is currently hard-wired to 1225, PORT_N to 1223, and PORT_S to 1224.
570V4NET is hardwired to 224 and V6NET to ff00. These networks are used for DNS
571testing purposes, and for testing Exim with -bh. The only requirement is that
572they are networks that can never be used for an IP address of a real host. I've
573chosen two multicast networks for the moment.
574
7a601efb
HSHR
575PORT_DYNAMIC is allocated by hunting for a free port (starting at port
5761024) a listener can bind to. This is done by runtest, for simulating
577inetd operations.
578
151b83f8
PH
579If the host has no IPv6 address, "<no IPv6 address found>" is substituted but
580that does not matter because no IPv6 tests will be run. A similar substitution
581is made if there is no IPv4 address, and again, tests that actually require a
582running IPv4 interface should be skipped.
583
584If the host has more than one IPv4 or IPv6 address, the first one that
585"ifconfig" lists is used. If the only available address is 127.0.0.1 (or ::1
1b781f48 586for IPv6) it is used, but another value is preferred if available.
151b83f8
PH
587
588In situations where a specific test is not being run (for example, when setting
589up dynamic data files), TESTNUM is replaced by an empty string, but should not
590in fact occur in such files.
591
592
593HOW THE TESTS WORK
594------------------
595
596Each numbered script runs Exim (sometimes several times) with its own Exim
597configuration file. The configurations are stored in the "confs" directory,
598and before running each test, a copy of the appropriate configuration, with the
599standard substitutions, is made in the file test-config. The -C command line
600option is used to tell Exim to use this configuration.
601
602The -D option is used to pass the path of the Exim binary to the configuration.
603This is not standardly substituted, because there are two possible binaries
604that might be used in the same test (one setuid to root, the other to the exim
605user). Some tests also make use of -D to vary the configuration for different
606calls to the Exim binary.
607
608Normally, of course, Exim gives up root privilege when -C and -D are used by
609unprivileged users. We do not want this to happen when running the tests,
610because we want to be able to test all aspects of Exim, including receiving
611mail from unprivileged users. The way this is handled is as follows:
612
613At the start of the runtest script, the patchexim script is run as root. This
614script makes a copy of the Exim binary that is to be tested, patching it as it
615does so. (This is a binary patch, not a source patch.) The patch causes the
616binary, when run, to "know" that it is running in the test harness. It does not
617give up root privilege when -C and -D are used, and in a few places it takes
618other special actions, such as delaying when starting a subprocess to allow
619debug output from the parent to be written first. If you want to know more,
620grep the Exim source files for "running_in_test_harness".
621
622The patched binary is placed in the directory eximdir/exim and given the normal
623setuid root privilege. This is, of course, a dangerous binary to have lying
624around, especially if there are unprivileged users on the system. To protect
625it, the eximdir directory is created with the current user as owner, exim as
626the group owner, and with access drwx--x---. Thus, only the user who is running
627the tests (who is known to have access to root) and the exim user have access
628to the modified Exim binary. When runtest terminates, the patched binary is
629removed.
630
631Each set of tests proceeds by interpreting its controlling script. The scripts
632are in subdirectories of the "scripts" directory. They are split up according
633to the requirements of the tests they contain, with the 0000-Basic directory
634containing tests that can always be run. Run the "listtests" script to obtain a
635list of tests.
636
637
638TEST OUTPUT
639-----------
640
641Output from script runs is written to the files test-stdout and test-stderr.
642When an Exim server is involved, test-stdout-server and test-stderr-server are
643used for its output. Before being compared with the saved output, the
644non-server and server files are concatenated, so a single saved file contains
645both.
646
647A directory called spool is used for Exim's spool files, and for Exim logs.
648These locations are specified in every test's configuration file.
649
650When messages are delivered to files, the files are put in the test-mail
651directory. Output from comparisons is written to test-cf.
652
653Before comparisons are done, output texts are modified ("munged") to change or
654remove parts that are expected to vary from run to run. The modified files all
655end with the suffix "-munged". Thus, you will see test-stdout-munged,
656test-mainlog-munged, test-mail-munged, and so on. Other files whose names start
657with "test-" are created and used by some of the tests.
658
659At the end of a successful test run, the spool directory and all the files
660whose names begin with "test-" are removed. If the run ends unsuccessfully
661(typically after a "Q" response to a prompt), the spool and test files are left
662in existence so that the problem can be investigated.
663
664
665TEST COMMANDS
666-------------
667
668Each test script consists of a list of commands, each optionally preceded by
669comments (lines starting with #) and (also optionally) a line containing an
670expected return code. Some of the commands are followed by data lines
671terminated by a line of four asterisks.
672
673The first line of each script must be a comment that briefly describes the
674script. For example:
675
676 # -bS Use of HELO/RSET
677
678A line consisting just of digits is interpreted as the expected return code
679for the command that follows. The default expectation when no such line exists
680is a zero return code. For example, here is a complete test script, containing
681just one command:
682
683 # -bS Unexpected EOF in headers
684 1
685 exim -bS -odi
686 mail from:<someone@some.where>
687 rcpt to:<blackhole@HOSTNAME>
688 data
689 from: me
690 ****
691
692The expected return code in this case is 1, and the data lines are passed to
693Exim on its standard input. Both the command line and the data lines have the
4c04137d 694standard substitutions applied to them. Thus, HOSTNAME in the example above will
151b83f8
PH
695be replaced by the local host's name. Long commands can be continued over
696several lines by using \ as a continuation character. This does *not* apply to
697data lines.
698
1b781f48 699Here follows a list of supported commands. They can be divided into two groups:
151b83f8
PH
700
701
702Commands with no input
703----------------------
704
705These commands are not followed by any input data, or by a line of asterisks.
706
cfc54830 707
a338ca2b
JH
708 ### This is a verbose comment
709
710A line starting with three hashmarks and some space copies the following text to
711both stdout and stderr file being written by the test.
712
151b83f8
PH
713 dbmbuild <file1> <file1>
714
715This command runs the exim_dbmbuild utility to build a DBM file. It is used
716only when DBM support is available in Exim, and typically follows the use of a
717"write" command (see below) that creates the input file.
718
719
d0e31199 720 dump <dbname>
cfc54830
PH
721
722This command runs the exim_dumpdb utility on the testing spool directory, using
723the database name given, for example: "dumpdb retry".
724
725
151b83f8
PH
726 echo <text>
727
728The text is written to the screen; this is used to output comments from
729scripts.
730
731
cfc54830
PH
732 exim_lock [options] <file name>
733
734This command runs the exim_lock utility with the given options and file name.
e2ae28ed
JH
735The file remains locked for following commands until a non-daemon "exim"
736completes.
cfc54830
PH
737
738
739 exinext <data>
740
741This command runs the exinext utility with the given argument data.
742
743
f3f065bb
PH
744 exigrep <data>
745
746This command runs the exigrep utility with the given data (the search pattern)
747on the current mainlog file.
748
749
151b83f8
PH
750 gnutls
751
752This command is present at the start of all but one of the tests that use
753GnuTLS. It copies a pre-existing parameter file into the spool directory, so
754that Exim does not have to re-create the file each time. The first GnuTLS test
cfc54830 755does not do this, in order to test that Exim can create the file.
151b83f8
PH
756
757
758 killdaemon
759
760This command must be given in any script that starts an Exim daemon, normally
761at the end. It searches for the PID file in the spool directory, and sends a
762SIGINT signal to the Exim daemon process whose PID it finds. See below for
763comments about starting Exim daemons.
764
765
766 millisleep <m>
767
768This command causes the script to sleep for m milliseconds. Nothing is output
769to the screen.
770
771
c9a55f6a
JH
772 munge <name>
773
774This command requests custom munging of the test outputs. The munge names
d0e31199 775used are coded in the runtest script (look for 'name of munge').
c9a55f6a
JH
776
777
151b83f8
PH
778 need_ipv4
779
780This command must be at the head of a script. If no IPv4 interface has been
781found, the entire script is skipped, and a comment is output.
782
783
784 need_ipv6
785
786This command must be at the head of a script. If no IPv6 interface has been
787found, the entire script is skipped, and a comment is output.
788
789
21c28500
PH
790 need_largefiles
791
792This command must be at the head of a script. If the Exim binary does not
4c04137d 793support large files (off_t is <= 4), the entire script is skipped, and a
21c28500
PH
794comment is output.
795
796
151b83f8
PH
797 need_move_frozen_messages
798
799This command must be at the head of a script. If the Exim binary does not have
800support for moving frozen messages (which is an optional feature), the entire
801script is skipped, and a comment is output.
802
803
804 no_message_check
805
806If this command is encountered anywhere in the script, messages that are
807delivered when the script runs are not compared with saved versions.
808
809
810 no_msglog_check
811
812If this command is encountered anywhere in the script, message log files that
813are still in existence at the end of the run (for messages that were not
814delivered) are not compared with saved versions.
815
cfc54830 816
151b83f8
PH
817 no_stderr_check
818
819If this command is encountered anywhere in the script, the stderr output from
820the run is not compared with a saved version.
821
822
823 no_stdout_check
824
825If this command is encountered anywhere in the script, the stdout output from
826the run is not compared with a saved version.
827
828
829 rmfiltertest
830
831This command indicates that the script is for a certain type of filter test, in
832which there are a lot of repetitive stdout lines that get in the way, because
833filter tests output data about the sender and recipient. Such lines are removed
834from the stdout output before comparing, for ease of human perusal.
835
836
837 sleep <n>
838
839This command causes the script to sleep for n seconds. If n is greater than
840one, "sleep <n>" is output to the screen, followed by a dot for every second
841that passes.
842
843
844 sortlog
845
846This command causes special sorting to occur on the mainlog file before
847comparison. Every sequence of contiguous delivery lines (lines containing the
848=> -> or *> flags) is sorted. This is necessary in some tests that use parallel
849deliveries because on different systems the processes may terminate in a
850different order.
851
852
cfc54830
PH
853A number of standard file management commands are also recognized. These are
854cat, chmod, chown, cp, du, ln, ls, du, mkdir, mkfifo, rm, rmdir, and touch.
855Some are run as root using "sudo".
151b83f8
PH
856
857
858Commands with input
859-------------------
860
861The remaining commands are followed by data lines for their standard input,
862terminated by four asterisks. Even if no data is required for the particular
863usage, the asterisks must be given.
864
865
bdf36f7c
JH
866 background
867
868This command takes one script line and runs it in the background,
869in parallel with following commands. For external daemons, eg. redis-server.
870
871
151b83f8
PH
872 catwrite <file name> [nxm[=start-of-line-text]]*
873
874This command operates like the "write" command, which is described below,
1b781f48 875except that the data it generates is copied to the end of the test-stdout file
151b83f8
PH
876as well as to the named file.
877
878
879
880 client [<options>] <ip address> <port> [<outgoing interface>]
881
882This command runs the auxiliary "client" program that simulates an SMTP client.
883It is controlled by a script read from its standard input, details of which are
a14e5636
PH
884given below. There are two options. One is -t, which must be followed directly
885by a number, to specify the command timeout in seconds (e.g. -t5). The default
41fdef91 886timeout is 5 seconds. The other option is -tls-on-connect, which causes the
a14e5636
PH
887client to try to start up a TLS session as soon as it has connected, without
888using the STARTTLS command. The client program connects to the given IP address
889and port, using the specified interface, if one is given.
151b83f8
PH
890
891
892 client-ssl [<options>] <ip address> <port> [<outgoing interface>] \
893 [<cert file>] [<key file>]
894
895When OpenSSL is available on the host, an alternative version of the client
896program is compiled, one that supports TLS using OpenSSL. The additional
f5d78688
JH
897arguments specify a certificate and key file when required for the connection.
898There are two additional options: -tls-on-connect, that causes the client to
4c04137d 899initiate TLS negotiation immediately on connection; -ocsp that causes the TLS
f5d78688
JH
900negotiation to include a certificate-status request. The latter takes a
901filename argument, the CA info for verifying the stapled response.
151b83f8
PH
902
903
904 client-gnutls [<options>] <ip address> <port> [<outgoing interface>] \
905 [<cert file>] [<key file>]
906
907When GnuTLS is available on the host, an alternative version of the client
908program is compiled, one that supports TLS using GnuTLS. The additional
909arguments specify a certificate and key file when required. There is one
910additional option, -tls-on-connect, that causes the client to initiate TLS
911negotiation immediately on connection.
912
913
914 exim [<options>] [<arguments>]
915
916This command runs the testing version of Exim. Any occurrence of "$msg1" in the
917command line is replaced by the ID of the first (oldest) message in Exim's
918(testing) spool. "$msg2" refers to the second, and so on. The name "exim" can
919be preceded by an environment setting as in this example:
920
921 LDAPTLS_REQCERT=never exim -be
922
923It can also be preceded by a number; this specifies a number of seconds to wait
924before closing the stdout pipe to Exim, and is used for some timeout tests. For
925example:
926
927 3 exim -bs
928
929Finally, "exim" can be preceded by "sudo", to run Exim as root. If more than
930one of these prefixes is present, they must be in the above order.
931
209ae7d1
JH
932If the options include "-DSERVER" but not "-DNOTDAEMON", the script waits for
933Exim to start but then continues without waiting for it to terminate. Typically
934this will be for a daemon-mode "-bd" operation. The daemon should be later
935terminated using "killdaemon".
936
151b83f8
PH
937
938 exim_exim [<options>] [<arguments>]
939
940This runs an alternative version of Exim that is setuid to exim rather than to
941root.
942
943
944 server [<options>] <port or socket> [<connection count>]
945
946This command runs the auxiliary "server" program that simulates an SMTP (or
947other) server. It is controlled by a script that is read from its standard
948input, details of which are given below. A number of options are implemented:
949
950 -d causes the server to output debugging information
951
8a512ed5 952 -t <sec> sets a timeout (default 5) for when the server is
59eaad2b
JH
953 awaiting an incoming connection. If negative, the
954 absolute value is used and a timeout results in a
955 nonfailure exit code
151b83f8
PH
956
957 -noipv4 causes the server not to set up an IPv4 socket
958
959 -noipv6 causes the server not to set up an IPv6 socket
960
8a512ed5
JH
961 -i <sec> sets an initial pause, to delay before creating the listen sockets
962
151b83f8
PH
963By default, in an IPv6 environment, both kinds of socket are set up. However,
964the test script knows which interfaces actually exist on the host, and it adds
965-noipv4 or -noipv6 to the server command as required. An error occurs if both
966these options are given.
967
968The only required argument is either a port number or the path name of a Unix
969domain socket. The port is normally PORT_S, which is changed to an actual
970number by the standard substitutions. The optional final argument specifies the
971number of different connections to expect (default 1). These must happen
972serially (one at a time). There is no support for multiple simultaneous
973connections. Here are some example commands:
974
975 server PORT_S
976 server -t 10 PORT_S 3
977 server /tmp/somesocket
978
979The following lines, up to a line of four asterisks, are the server's
980controlling standard input (described below). These lines are read and
e2ae28ed
JH
981remembered; during the following commands, until a non-deamon "exim" command
982is reached, the server is run in parallel. Then the server termination
983is waited for.
151b83f8
PH
984
985
986 write <file name> [nxm[=start-of-line-text]]*
987
988The "write" command is a way of creating files of specific sizes for buffering
989tests, or containing specific data lines. Being able to do this from within the
990script saves holding lots of little test files. The optional argument specifies
991n lines of length m. The lines consist of the letter "a". If start of line text
992is supplied, it replaces "a"s at the start of each line. Underscores in the
993start of line text are turned into spaces. The optional argument may be
994repeated. The data lines that follow a "write" command are split into two by a
995line of four plus signs. Any above the split are written before the
996fixed-length lines, and any below the split are written after. For example:
997
998 write test-data 3x30=AB_ 1x50
999 Pre-data
1000 lines
1001 ++++
1002 Post-data
1003 lines
1004 ****
1005
1006This command generates a file containing:
1007
1008 Pre-data
1009 lines
1010 AB aaaaaaaaaaaaaaaaaaaaaaaaaaa
1011 AB aaaaaaaaaaaaaaaaaaaaaaaaaaa
1012 AB aaaaaaaaaaaaaaaaaaaaaaaaaaa
1013 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
1014 Post-data
1015 lines
1016
1017If there are no fixed-length line specifiers, there is no need to split the
1018data, and a line of plusses is not needed.
1019
1020
1021 [sudo] perl
1022
1023This command runs Perl, with the data as its standard input, to allow arbitrary
1024one-off things to be done.
1025
1026
1027CLIENT SCRIPTS
1028--------------
1029
11a5264b 1030Lines in client scripts are of several kinds:
151b83f8
PH
1031
1032(1) If a line begins with three question marks and a space, the rest of the
1033 line defines the start of expected output from the server. If what is
1034 received does not match, the client bombs out with an error message.
1035
ce80533b
JH
1036(2) If a line begins with three question marks and an asterisk, the server
1037 is expected to close the connection.
1038
1039(3) If a line begins with four question marks, the rest of the line defines
1040 the start of one or more possible output lines from the server. When it
1041 matches, the client silently repeats the comparison using the next server
1042 line. When the match fails, the client silently proceeds to the next script
1043 line with the then-current server output unconsumed.
1044
1045(4) If a line starts with three plus signs followed by a space, the rest of the
151b83f8
PH
1046 line specifies a number of seconds to sleep for before proceeding.
1047
ce80533b 1048(5) If a line begins with three '>' characters and a space, the rest of the
5d036699
JH
1049 line is input to be sent to the server. Backslash escaping is done as
1050 described below, but no trailing "\r\n" is sent.
1051
ce80533b 1052(6) If a line begin with three '<' characters and a space, the rest of the
7bbb3621
JH
1053 line is a filename; the content of the file is inserted intto the script
1054 at this point.
1055
ce80533b 1056(7) Otherwise, the line is an input line line that is sent to the server. Any
151b83f8
PH
1057 occurrences of \r and \n in the line are turned into carriage return and
1058 linefeed, respectively. This is used for testing PIPELINING.
aded2255 1059 Any sequences of \x followed by two hex digits are converted to the equivalent
5d036699 1060 byte value. Any other character following a \ is sent verbatim.
7bbb3621 1061 The line is sent with a trailing "\r\n".
151b83f8
PH
1062
1063Here is a simple example:
1064
1065 client 127.0.0.1 PORT_D
3cc3f762 1066 ??? 220
151b83f8
PH
1067 EHLO xxx
1068 ??? 250-
1069 ??? 250
1070 AUTH PLAIN AbdXi0AdnD2CVy
1071 ??? 535
1072 quit
1073 ??? 221
1074 ****
1075
1076In the case of client-gnutls and client-ssl, if a command is "starttls", this
1077is remembered, and after a subsequent OK response, an attempt to move into TLS
1078mode occurs. If a command is "starttls_wait", the client sends "starttls" but
1079does not start up TLS; this is for testing timeouts. If a command is "stoptls",
1080an existing TLS connection is shut down, but nothing is sent.
1081
1082
1083SERVER SCRIPTS
1084--------------
1085
1086The server program sleeps till a connection occurs or its timeout is reached,
1087in which case it bombs out. The next set of command lines are interpreted. They
1088are of the following kinds:
1089
1090(1) A line that starts with '>' or with a digit is an output line that is sent
1091 to the client. In the case of '>':
1092
1093 (a) If the line starts with ">>", no terminating CRLF is sent.
1094 (b) If the line starts with ">CR>", just CR is sent at the end.
1095 (c) If the line starts with ">LF>", just LF is sent at the end.
1096 (d) If the line starts with ">*eof", nothing is sent and the connection
1097 is closed.
1098
7eb6c37c
JH
1099 The data that is sent starts after the initial '>' sequence. Within
1100 each line the sequence '\x' followed by two hex digits can be used
1101 to specify an arbitrary byte value. The sequence '\\' specifies a
1102 single backslash.
151b83f8
PH
1103
1104(2) A line that starts with "*sleep" specifies a number of seconds to wait
1105 before proceeding.
1106
1107(3) A line containing "*eof" specifies that the client is expected to close
1108 the connection at this point.
1109
1110(4) A line containing just '.' specifies that the client is expected to send
1111 many lines, terminated by one that contains just a dot.
1112
1113(5) Otherwise, the line defines the start of an input line that the client
1114 is expected to send. To allow for lines that start with digits, the line
1115 may start with '<', which is not taken as part of the input data. If the
7eb6c37c
JH
1116 lines starts with '<<' then only the characters are expected; no return-
1117 linefeed terminator. If the input does not match, the server bombs out
1118 with an error message. Backslash-escape sequences may be used in the
1119 line content as for output lines.
151b83f8 1120
bbe15da8 1121Here is a simple example of server use in a test script:
151b83f8
PH
1122
1123 server PORT_S
1124 220 Greetings
1125 EHLO
1126 250 Hello there
1127 MAIL FROM
1128 250 OK
1129 RCPT TO
1130 250 OK
1131 DATA
1132 354 Send it!
1133 .
1134 250 OK
1135 QUIT
1136 225 OK
1137 ****
1138
1139After a "server" command in a test script, the server runs in parallel until an
1140"exim" command is reached. The "exim" command attempts to deliver one or more
1141messages to port PORT_S on the local host. When it has finished, the test
1142script waits for the "server" process to finish.
1143
bbe15da8
PH
1144The "mtpscript" program is like "server", except that it uses stdin/stdout for
1145its input and output instead of a script. However, it is not called from test
1146scripts; instead it is used as the command for pipe transports in some
1147configurations, to simulate non-socket LMTP servers.
1148
151b83f8
PH
1149
1150AUXILIARY DATA FILES
1151--------------------
1152
1153Many of the tests make use of auxiliary data files. There are two types; those
1154whose content is fixed, and those whose content needs to be varied according to
1155the current environment. The former are kept in the directory aux-fixed. The
1156latter are distributed in the directory aux-var-src, and copied with the
1157standard substitutions into the directory aux-var at the start of each test
1158run.
1159
1160Most of the auxiliary files have names that start with a test number,
1161indicating that they are specific to that one test. A few fixed files (for
1162example, some TLS certificates) are used by more than one test, and so their
1163names are not of this form.
1164
4c04137d 1165There are also some auxiliary DNS zone files, which are described in the next
151b83f8
PH
1166section.
1167
1168
1169DNS LOOKUPS AND GETHOSTBYNAME
1170-----------------------------
1171
1172The original test suite required special testing zones to be loaded into a
1173local nameserver. This is no longer a requirement for the new suite. Instead, a
1174program called fakens is used to simulate a nameserver. When Exim is running in
1175the test harness, instead of calling res_search() - the normal call to the DNS
1176resolver - it calls a testing function. This handles a few special names itself
1177(for compatibility with the old test suite), but otherwise passes the query to
1178the fakens program.
1179
1180The fakens program consults "zone files" in the directory called dnszones, and
1181returns data in the standard resource record format for Exim to process as if
1182it came from the DNS. However, if the requested domain is not in any of the
1183zones that fakens knows about, it returns a special code that causes Exim to
1184pass the query on to res_search(). The zone files are:
1185
1186 db.test.ex A zone for the domain test.ex.
1187 db.ip4.10 A zone for one special case in 10.250.0.0/16 (see below)
1188 db.ip4.V4NET A zone for the domain V4NET.in-addr.arpa.
1189 db.ip4.127 A zone for the domain 127.in-addr.arpa.
1190 db.ip6.V6NET A zone for the domain inverted(V6NET).ip6.arpa.
1191 db.ip6.0 A zone for the domain 0.ip6.arpa.
1192
1193V4NET and V6NET are substituted with the current testing networks (see above).
1194In the case of V6NET, the network is four hex digits, and it is split and
1195inverted appropriately when setting up the zone.
1196
1197These fake zone files are built dynamically from sources in the dnszones-src
1198directory by applying the standard substitutions. The test suite also builds
1199dynamic zone files for the name of the current host and its IP address(es). The
1200idea is that there should not be any need to rely on an external DNS.
1201
1fb7660f
JH
1202The fakens program handles some names programmatically rather than using the
1203fake zone files. These are:
1204
1205 manyhome.test.ex This name is used for testing hosts with ridiculously large
1206 numbers of IP addresses; 2048 IP addresses are generated
1207 and returned. Doing it this way saves having to make the
1208 interface to fakens handle more records that can fit in the
1209 data block. The addresses that are generated are in the
1210 10.250.0.0/16 network.
1211
151b83f8
PH
1212 test.again.dns This always provokes a TRY_AGAIN response, for testing the
1213 handling of temporary DNS error. If the full domain name
1214 starts with digits, a delay of that many seconds occurs.
1215
1216 test.fail.dns This always provokes a NO_RECOVERY response, for testing
1217 DNS server failures.
1218
151b83f8
PH
1219The use of gethostbyname() and its IPv6 friends is also subverted when Exim is
1220running in the test harness. The test code handles a few special names
1221directly; for all the others it uses DNS lookups, which are then handled as
1222just described. Thus, the use of /etc/hosts is completely bypassed. The names
1223that are specially handled are:
1224
151b83f8
PH
1225 localhost Always returns 127.0.0.1 or ::1, for IPv4 and IPv6 lookups,
1226 respectively.
1227
1228 <an IP address> If the IP address is of the correct form for the lookup
1229 type (IPv4 or IPv6), it is returned. Otherwise a panic-die
1230 error occurs.
1231
1232The reverse zone db.ip4.10 is provided just for the manyhome.test.ex case. It
1233contains a single wildcard resource record. It also contains the line
1234
1235 PASS ON NOT FOUND
1236
1237Whenever fakens finds this line in a zone file, it returns PASS_ON instead of
1238HOST_NOT_FOUND. This causes Exim to pass the query to res_search().
1239
1240****