Commit Graph

8330 Commits

Author SHA1 Message Date
Mark Andrews
dc445b7d8a Merge branch '68-rpz-system-test-fails-intermittently' into 'master'
Prevent rpz system test from wiping out forensic data

See merge request isc-projects/bind9!51

(cherry picked from commit 14388a4ad6)

034450ba add test numbers as use them in file names
05f9deba use symbolic test modes native/dnsrps
b98a9207 perform rpz testing with a loop rather that recursion
f7172cf2 add CHANGES note
2018-02-21 23:10:12 -05:00
Mark Andrews
d7cd9fe95b Merge branch '82-cacheclean-test-is-not-robust' into 'master'
Resolve "cacheclean test is not robust."

Closes #82

See merge request isc-projects/bind9!50

(cherry picked from commit aa7964dd4b)

146c16c0 check that sed selects a non zero range of lines
8c8ad74a add CHANGES entry
2018-02-21 20:17:21 -05:00
Stephen Morris
1d1b37d838 [rt46602] Changes to get tests running on Windows
1. Track changes to conf.sh.in in conf.sh.win32
2. Modifications to prevent Windows "Configure" script replacing
   the sed "@PORT@" substitution tokens in conf.sh.win32.
3. runall.sh now runs Windows tests sequentially

(cherry picked from commit 7bb9a97904)
2018-02-21 15:44:40 +00:00
Stephen Morris
a9e25e9686 [rt46602] Miscellaneous changes to fix rebase issues
(cherry picked from commit 7222c9fc76)
2018-02-21 15:44:40 +00:00
Stephen Morris
2b0b035571 [rt46602] Change name of shell variable holding current test directory
It was TESTNAME, but this is an obvious name and was used in one of
the system tests, something that interfered with the content of
progress messages.  It is now SYSTESTDIR.

(cherry picked from commit 30cd931a0d)
2018-02-21 15:44:40 +00:00
Michał Kępień
8fc6d51b0d [rt46602] Export environment variables in a portable manner
(cherry picked from commit 2c519531a7)
2018-02-21 15:44:40 +00:00
Michał Kępień
a8b10e9b3a [rt46602] Sanitize test names to fix "make check" on Solaris
(cherry picked from commit e005fb948d)
2018-02-21 15:44:40 +00:00
Stephen Morris
9078c24900 [rt46602] Add .gitignore file for catz
(cherry picked from commit 3dfb7c17eb)
2018-02-21 15:44:40 +00:00
Stephen Morris
121be65138 [rt46602] Pass noclean option to Makefile via exported variable
(cherry picked from commit 59d88ee192)
2018-02-21 15:44:40 +00:00
Stephen Morris
b98e011dc6 [rt46602] Mention conf.sh.win32 in instructions for adding a test.
(cherry picked from commit 23dfa27a84)
2018-02-21 15:44:40 +00:00
Michał Kępień
c6b66a406d [rt46602] Fix various typos, formatting issues and stylistic nits
(cherry picked from commit f0721de624)
2018-02-21 15:44:39 +00:00
Michał Kępień
18b0ad81ec [rt46602] Remove redundant variable assignment
(cherry picked from commit 42cf18d239)
2018-02-21 15:44:39 +00:00
Michał Kępień
63bbbb936a [rt46602] named.port is no longer created by setup.sh for allow_query
(cherry picked from commit 1ac4c3e4e5)
2018-02-21 15:44:39 +00:00
Michał Kępień
933bba9d73 [rt46602] Consistently print usage for runall.sh to stderr
(cherry picked from commit fe1ccafc34)
2018-02-21 15:44:39 +00:00
Michał Kępień
9ff064089b [rt46602] Clean up getopts usage
Ensure case clauses are in sync with the string passed to getopts.
Remove catch-all clauses as they will never be evaluated.

(cherry picked from commit 0dcee1cad3)
2018-02-21 15:44:39 +00:00
Michał Kępień
815820b3a9 [rt46602] Assign default values to variables set by command line options and consistently treat them as integers
(cherry picked from commit fe51c34df3)
2018-02-21 15:44:39 +00:00
Michał Kępień
3c7784b745 [rt46602] named.port cannot be ignored by git because it is required to be present by notify/ns4
(cherry picked from commit 1960de5530)
2018-02-21 15:44:39 +00:00
Michał Kępień
5b0a99e291 [rt46602] Re-add clean.sh invocation to catz/setup.sh
Commit 57aa7b60fd caused catz/setup.sh to no longer call clean.sh, which
results in the catz system test failing on subsequent runs if the first
run is interrupted or fails.

(cherry picked from commit 344d05063f)
2018-02-21 15:44:39 +00:00
Stephen Morris
6f341ed6cd [rt46602] Remove option to format the printed start time of a test
There was too much incompatibility between operating systems as to
the allowed options for "date".

(cherry picked from commit 451b7309ca)
2018-02-21 15:44:39 +00:00
Stephen Morris
d59a3f1c82 [rt46602] Escape line ends in shell script variable assignment
Escape the line ends in a multi-line variable assignment.  Under some
circumstances, substituting the variable caused syntax errors when
used as the list of values in a shell script "for" statement.

(cherry picked from commit 462766cd76)
2018-02-21 15:44:39 +00:00
Stephen Morris
c18241451b [rt46602] Fix EXTRAPORT substitutions in copy_setport
Fix problems where all EXTRAPORT macros were being replaced by
the EXTRAPORT1 value. (Copy and paste issue.)

(cherry picked from commit fb78f82ba1)
2018-02-21 15:44:39 +00:00
Stephen Morris
33394a55e4 [rt46602] Minor documentation changes
(cherry picked from commit cdf18a5009)
2018-02-21 15:44:39 +00:00
Stephen Morris
3906b1778f [rt46602] Correct way allow_query test produces RNDC output
(cherry picked from commit 7f2d41d4c4)
2018-02-21 15:44:39 +00:00
Stephen Morris
7cb5722c49 [rt46602] Simplify the cleanup code
(cherry picked from commit 0855a8a6cc)
2018-02-21 15:44:39 +00:00
Stephen Morris
269858ac02 [rt46602] Extensions to README
Added descriptions of how nameservers are started and stopped
during the tests, and how the framework cleans up the files created.

(cherry picked from commit f606b17dfb)
2018-02-21 15:44:39 +00:00
Stephen Morris
34a8e43ac1 [rt46602] Miscellenous small changes
(cherry picked from commit b88b508b22)
2018-02-21 15:44:39 +00:00
Stephen Morris
820c47b137 [rt46602] Handle echoing of multi-line output from tests
(cherry picked from commit 1f07c69f25)
2018-02-21 15:44:38 +00:00
Stephen Morris
f6d8f9cdcb [rt46602] Rename "test" variable to "systest"
... to avoid confusion between the variable and the command of the
same name.

(cherry picked from commit c12e06755f)
2018-02-21 15:44:38 +00:00
Stephen Morris
ec063cdc99 [rt46602] Add ability to prevent deletion of output files when running all tests
Some tests may need to examine all output files from all system tests.
Allow the deletion of these files to be inhibited if required.

(cherry picked from commit 00bc29640b)
2018-02-21 15:44:38 +00:00
Stephen Morris
d150fe27ed [rt46602] Error messages before start of test are output to stderr
Error messages concerning the invocation of run.sh are output to
stderr.  Messages after the test has started are output to stdout.

(cherry picked from commit 244d1c30e2)
2018-02-21 15:44:38 +00:00
Stephen Morris
3859c55031 [rt46602] Ensure that output sent to stderr is routed to the output file
(cherry picked from commit a5cf6bd12a)
2018-02-21 15:44:38 +00:00
Stephen Morris
626bd48214 [rt46602] Minor tidying up of documentation
(cherry picked from commit ce66752adc)
2018-02-21 15:44:38 +00:00
Stephen Morris
4dc368a837 [rt46602] Check prerequsities before network interfaces when running test.
(cherry picked from commit 549a6c6bc8)
2018-02-21 15:44:38 +00:00
Stephen Morris
d737040fcf [rt46602] Minor formatting changes to Perl files
(cherry picked from commit f0f5b5f045)
2018-02-21 15:44:38 +00:00
Stephen Morris
9e176a0a34 [rt46603] Provide echostart() and echoend() to output test start/end messages.
(cherry picked from commit c344e1d768)
2018-02-21 15:44:38 +00:00
Stephen Morris
62652988ed [rt46602] Ensure test output in systests.output is not mixed up
When running all the system tests, output from a test is sent to a
test.output file in the test directory.  These are combined in to
systests.output when the run finishes.

(cherry picked from commit 055e5be9fd)
2018-02-21 15:44:38 +00:00
Stephen Morris
0a372b07a9 [rt46602] Miscellaneous minor fixes
... for misspellings, typos, trailing spaces, duplicated lines etc.

(cherry picked from commit 3c25e5fa02)
2018-02-21 15:44:38 +00:00
Stephen Morris
0d9b1190ce [rt46602] Allocate 100 ports to each test
(cherry picked from commit edc7a18d84)
2018-02-21 15:44:38 +00:00
Stephen Morris
21e48340b3 [rt46602] Pass port numbers to tests via environment variables
(cherry picked from commit f5d8f07900)
2018-02-21 15:44:38 +00:00
Stephen Morris
32b4e0ce4d [rt46602] Ensure parallel tests are in alphabetical order
(cherry picked from commit b2d90da0c2)
2018-02-21 15:44:38 +00:00
Stephen Morris
353ab4dc69 [rt46602] Expanded system tests README
Add more information on running the tests, together with a section
on how the tests are organised, aimed at new developers.

(cherry picked from commit af005cdbcf)
2018-02-21 15:44:38 +00:00
Stephen Morris
a807f8a77a [rt46602] More updates to the test running framework
Tidy up the stop/start files and make switch usage consistent. Also
tidy up the various "clean" targets in the Makefile.

(cherry picked from commit b24c2e11d8)
2018-02-21 15:44:37 +00:00
Stephen Morris
cbe20bcc1f [rt46602] Modify some test files to cope with the "-p" flag
(cherry picked from commit 517d990bd4)
2018-02-21 15:44:37 +00:00
Stephen Morris
499e768d07 [rt46602] Tidy up run management
Miscellaneous tidying up of run management.  The most significant
change is that "runall.sh" now runs _all_ the tests, even the
ones that can run in parallel.  runsequential.sh is the script
to run tests that have not been converted to parallel running.

(cherry picked from commit 32fe6f7682)
2018-02-21 15:44:37 +00:00
Stephen Morris
b2f2c67fd9 [rt46602] Assign block of ports for each test
Some tests use more ports than just the query and control ports.
Each test that can run in parallel with other tests is now assigned
a unique block of 10 ports.

(cherry picked from commit e0ff77f9d3)
2018-02-21 15:44:37 +00:00
Stephen Morris
60945507d8 [rt46602] Allow catz system test to run in parallel with other tests
(cherry picked from commit 07341693e1)
2018-02-21 15:44:37 +00:00
Stephen Morris
2ed6a072c9 [rt46602] Miscellaneous changes to existing parallelised tests
Currently these tests are allow_query, rpzrecurse and serve-stale

1. Function to copy files and set port numbers renamed from copy_config
to copy_setports, as this is used to change the ports in Perl and Python
test scripts as well.
2. Changes to rpzrecurse/tests.sh to handle two calls to getopts (one to
parse port numbers, the other to parse rpzrecurse-specific options). Also
fixed various commands to use correct ports.
3. Updates to "clean.sh" scripts to ensure that all files created in the
test are removed.

(cherry picked from commit 78f2b9ca01)
2018-02-21 15:44:37 +00:00
Stephen Morris
6fbf96c112 [rt46602] Ensure that tests running in parallel use unique ports
Via an intermediate make file, tests that have been modified to be able
to run in parallel are assigned unique query and control port numbers
(other than 5300 and 9953 respectively).  Tests that have not yet been
modified all use ports 5300 and 9953, so must be run sequentially.

(cherry picked from commit e7429b124b)
2018-02-21 15:44:37 +00:00
Stephen Morris
f396d62285 [rt46602] Miscellaneous tidying up of code cherry-picked from experimental branch
(cherry picked from commit e1a69e5d1f)
2018-02-21 15:44:37 +00:00
Ondřej Surý
e34bd38f69 [rt46602] Allow parallel run of new dynamic port tests and the old ones
(cherry picked from commit 3b14d54fd7)
2018-02-21 15:44:37 +00:00