Merge branch 'dbmjz'
[exim.git] / doc / doc-txt / test-harness.txt
CommitLineData
75e0e026
PH
1EXIM'S BEHAVIOUR CHANGES WHEN RUNNING IN THE TEST HARNESS
2---------------------------------------------------------
3
4When Exim is running in its test harness, via the scripts in the exim-testsuite
5distribution, its behaviour is altered in a few ways, in order to make the
6regression testing work properly. The documentation for the test suite
7describes how a copy of the Exim binary is taken and patched in order to get it
8to run in the test harness. This document briefly lists the behavioural changes
9that result. They come into play when the Boolean variable running_in_test_
10harness is true.
11
12
13Privilege
14---------
15
16Exim does not give up its root privilege when called with -C or -D, nor does it
17insist on the caller being an admin user when starting a daemon, a queue
18runner, or requesting debug output.
19
20
21Small Pauses
22------------
23
24In a number of places, typically when a subprocess has been forked, there are
25short pauses of half or one second in one of the processes. This allows the
26other process to "go first"; it ensures that debugging or logging output always
27appears in the same order.
28
29
30Daemon
31------
32
33The daemon always writes a pid file when running in the test harness.
34
35
36CRAM-MD5
37--------
38
39The cram_md5 authenticator always uses the same challenge string.
40
41
42Appendfile
43----------
44
45After a quota error, the "time since last read" for the file is forced to 10s,
46for repeatability.
47
48
49Memory management
50-----------------
51
52Memory management debugging output contains only the store pool and the size
53(other information is too variable). New memory is initialized to contain F0 in
54all bytes.
55
56
57Queue running
58-------------
59
60There's a facility (-Tqt) for fudging queue times for testing retry logic.
61
62
63Syslog
64------
65
66Exim never writes to syslog in the test harness. Attempts to do so are silently
67ignored. None of the tests actually specify syslog logging for any actual log
68lines, but there is one that tests the inability to open the main and panic
69logs, which by default then tries to write to syslog.
70
71
72SMTP connection timeout
73-----------------------
74
75In order to be able to test timeout handling, a "connection refused" error is
76converted into a timeout if the timeout value is set to 999999s.
77
78
79Random numbers
80--------------
81
82The seed for the pseudo-random number generator is set to a fixed value in the
83test harness, to ensure repeatability.
84
85
86Bounce messages
87---------------
88
89When Exim is submitting a bounce message to itself, unless the configuration
90has set queue_only, it uses -odi so that the bounce is delivered before the
91subprocess returns. This avoids a race that might put log lines in an arbitrary
92order.
93
94
95DNS lookups
96-----------
97
98The real DNS resolver is never called. Instead, a fake resolver, which runs as
99a separate program, is used. It is part of the test suite and is documented
100there. This ensures complete control over the exact results of any DNS lookups.
101
102An attempt to look up a PTR record for 99.99.99.99 or an IP address for a host
103whose name ends with .test.again.dns always yields a "try again" error.
104
105A fake function is called instead of gethostbyname(). It recognizes the name
106"manyhome.test.ex" and generates a humungous number of IP addresses. It also
107recognizes an unqualified "localhost" and forces it to the appropriate loopback
108address (IPv4 or IPv6, as required). IP addresses are treated as literals. For
109other names, it does a DNS lookup (which of course actually calls the fake
110resolver) to find the host name.
111
112
113User names
114----------
115
116If unknown_login is set, it forces the login name, thus overriding the actual
117login for the test suite caller. When this happens, unknown_username provides a
118user name if it is set; otherwise an empty string is used.
119
120
121Ident
122-----
123
124If -bh is used and both the sending host port and the incoming interface port
125are supplied, an ident (RFC 1413) call is made for testing purposes.
126
127
128Debug output
129------------
130
131Debugging output from the function that waits for the clock to tick at an
132appropriate resolution (before completing the arrival of a message, for
133example) is suppressed because the fractions of seconds that it contains will
134never be repeatable.
135
136
137Philip Hazel
13815 February 2006