doc/doc-txt/test-harness.txt

   1 $Cambridge: exim/doc/doc-txt/test-harness.txt,v 1.1 2006/02/16 10:05:33 ph10 Exp $
   2
   3 EXIM'S BEHAVIOUR CHANGES WHEN RUNNING IN THE TEST HARNESS
   4 ---------------------------------------------------------
   5
   6 When Exim is running in its test harness, via the scripts in the exim-testsuite
   7 distribution, its behaviour is altered in a few ways, in order to make the
   8 regression testing work properly. The documentation for the test suite
   9 describes how a copy of the Exim binary is taken and patched in order to get it
  10 to run in the test harness. This document briefly lists the behavioural changes
  11 that result. They come into play when the Boolean variable running_in_test_
  12 harness is true.
  13
  14
  15 Privilege
  16 ---------
  17
  18 Exim does not give up its root privilege when called with -C or -D, nor does it
  19 insist on the caller being an admin user when starting a daemon, a queue
  20 runner, or requesting debug output.
  21
  22
  23 Small Pauses
  24 ------------
  25
  26 In a number of places, typically when a subprocess has been forked, there are
  27 short pauses of half or one second in one of the processes. This allows the
  28 other process to "go first"; it ensures that debugging or logging output always
  29 appears in the same order.
  30
  31
  32 Daemon
  33 ------
  34
  35 The daemon always writes a pid file when running in the test harness.
  36
  37
  38 CRAM-MD5
  39 --------
  40
  41 The cram_md5 authenticator always uses the same challenge string.
  42
  43
  44 Appendfile
  45 ----------
  46
  47 After a quota error, the "time since last read" for the file is forced to 10s,
  48 for repeatability.
  49
  50
  51 Memory management
  52 -----------------
  53
  54 Memory management debugging output contains only the store pool and the size
  55 (other information is too variable). New memory is initialized to contain F0 in
  56 all bytes.
  57
  58
  59 Queue running
  60 -------------
  61
  62 There's a facility (-Tqt) for fudging queue times for testing retry logic.
  63
  64
  65 Syslog
  66 ------
  67
  68 Exim never writes to syslog in the test harness. Attempts to do so are silently
  69 ignored. None of the tests actually specify syslog logging for any actual log
  70 lines, but there is one that tests the inability to open the main and panic
  71 logs, which by default then tries to write to syslog.
  72
  73
  74 SMTP connection timeout
  75 -----------------------
  76
  77 In order to be able to test timeout handling, a "connection refused" error is
  78 converted into a timeout if the timeout value is set to 999999s.
  79
  80
  81 Random numbers
  82 --------------
  83
  84 The seed for the pseudo-random number generator is set to a fixed value in the
  85 test harness, to ensure repeatability.
  86
  87
  88 Bounce messages
  89 ---------------
  90
  91 When Exim is submitting a bounce message to itself, unless the configuration
  92 has set queue_only, it uses -odi so that the bounce is delivered before the
  93 subprocess returns. This avoids a race that might put log lines in an arbitrary
  94 order.
  95
  96
  97 DNS lookups
  98 -----------
  99
 100 The real DNS resolver is never called. Instead, a fake resolver, which runs as
 101 a separate program, is used. It is part of the test suite and is documented
 102 there. This ensures complete control over the exact results of any DNS lookups.
 103
 104 An attempt to look up a PTR record for 99.99.99.99 or an IP address for a host
 105 whose name ends with .test.again.dns always yields a "try again" error.
 106
 107 A fake function is called instead of gethostbyname(). It recognizes the name
 108 "manyhome.test.ex" and generates a humungous number of IP addresses. It also
 109 recognizes an unqualified "localhost" and forces it to the appropriate loopback
 110 address (IPv4 or IPv6, as required). IP addresses are treated as literals. For
 111 other names, it does a DNS lookup (which of course actually calls the fake
 112 resolver) to find the host name.
 113
 114
 115 User names
 116 ----------
 117
 118 If unknown_login is set, it forces the login name, thus overriding the actual
 119 login for the test suite caller. When this happens, unknown_username provides a
 120 user name if it is set; otherwise an empty string is used.
 121
 122
 123 Ident
 124 -----
 125
 126 If -bh is used and both the sending host port and the incoming interface port
 127 are supplied, an ident (RFC 1413) call is made for testing purposes.
 128
 129
 130 Debug output
 131 ------------
 132
 133 Debugging output from the function that waits for the clock to tick at an
 134 appropriate resolution (before completing the arrival of a message, for
 135 example) is suppressed because the fractions of seconds that it contains will
 136 never be repeatable.
 137
 138
 139 Philip Hazel
 140 15 February 2006