Code tidies to remove stuff that was needed only for the old test suite.
[exim.git] / doc / doc-txt / test-harness.txt
CommitLineData
75e0e026
PH
1$Cambridge: exim/doc/doc-txt/test-harness.txt,v 1.1 2006/02/16 10:05:33 ph10 Exp $
2
3EXIM'S BEHAVIOUR CHANGES WHEN RUNNING IN THE TEST HARNESS
4---------------------------------------------------------
5
6When Exim is running in its test harness, via the scripts in the exim-testsuite
7distribution, its behaviour is altered in a few ways, in order to make the
8regression testing work properly. The documentation for the test suite
9describes how a copy of the Exim binary is taken and patched in order to get it
10to run in the test harness. This document briefly lists the behavioural changes
11that result. They come into play when the Boolean variable running_in_test_
12harness is true.
13
14
15Privilege
16---------
17
18Exim does not give up its root privilege when called with -C or -D, nor does it
19insist on the caller being an admin user when starting a daemon, a queue
20runner, or requesting debug output.
21
22
23Small Pauses
24------------
25
26In a number of places, typically when a subprocess has been forked, there are
27short pauses of half or one second in one of the processes. This allows the
28other process to "go first"; it ensures that debugging or logging output always
29appears in the same order.
30
31
32Daemon
33------
34
35The daemon always writes a pid file when running in the test harness.
36
37
38CRAM-MD5
39--------
40
41The cram_md5 authenticator always uses the same challenge string.
42
43
44Appendfile
45----------
46
47After a quota error, the "time since last read" for the file is forced to 10s,
48for repeatability.
49
50
51Memory management
52-----------------
53
54Memory management debugging output contains only the store pool and the size
55(other information is too variable). New memory is initialized to contain F0 in
56all bytes.
57
58
59Queue running
60-------------
61
62There's a facility (-Tqt) for fudging queue times for testing retry logic.
63
64
65Syslog
66------
67
68Exim never writes to syslog in the test harness. Attempts to do so are silently
69ignored. None of the tests actually specify syslog logging for any actual log
70lines, but there is one that tests the inability to open the main and panic
71logs, which by default then tries to write to syslog.
72
73
74SMTP connection timeout
75-----------------------
76
77In order to be able to test timeout handling, a "connection refused" error is
78converted into a timeout if the timeout value is set to 999999s.
79
80
81Random numbers
82--------------
83
84The seed for the pseudo-random number generator is set to a fixed value in the
85test harness, to ensure repeatability.
86
87
88Bounce messages
89---------------
90
91When Exim is submitting a bounce message to itself, unless the configuration
92has set queue_only, it uses -odi so that the bounce is delivered before the
93subprocess returns. This avoids a race that might put log lines in an arbitrary
94order.
95
96
97DNS lookups
98-----------
99
100The real DNS resolver is never called. Instead, a fake resolver, which runs as
101a separate program, is used. It is part of the test suite and is documented
102there. This ensures complete control over the exact results of any DNS lookups.
103
104An attempt to look up a PTR record for 99.99.99.99 or an IP address for a host
105whose name ends with .test.again.dns always yields a "try again" error.
106
107A fake function is called instead of gethostbyname(). It recognizes the name
108"manyhome.test.ex" and generates a humungous number of IP addresses. It also
109recognizes an unqualified "localhost" and forces it to the appropriate loopback
110address (IPv4 or IPv6, as required). IP addresses are treated as literals. For
111other names, it does a DNS lookup (which of course actually calls the fake
112resolver) to find the host name.
113
114
115User names
116----------
117
118If unknown_login is set, it forces the login name, thus overriding the actual
119login for the test suite caller. When this happens, unknown_username provides a
120user name if it is set; otherwise an empty string is used.
121
122
123Ident
124-----
125
126If -bh is used and both the sending host port and the incoming interface port
127are supplied, an ident (RFC 1413) call is made for testing purposes.
128
129
130Debug output
131------------
132
133Debugging output from the function that waits for the clock to tick at an
134appropriate resolution (before completing the arrival of a message, for
135example) is suppressed because the fractions of seconds that it contains will
136never be repeatable.
137
138
139Philip Hazel
14015 February 2006