Commit Graph

3580 Commits

Author SHA1 Message Date
Stephen Morris
ec063cdc99 [rt46602] Add ability to prevent deletion of output files when running all tests
Some tests may need to examine all output files from all system tests.
Allow the deletion of these files to be inhibited if required.

(cherry picked from commit 00bc29640b)
2018-02-21 15:44:38 +00:00
Stephen Morris
d150fe27ed [rt46602] Error messages before start of test are output to stderr
Error messages concerning the invocation of run.sh are output to
stderr.  Messages after the test has started are output to stdout.

(cherry picked from commit 244d1c30e2)
2018-02-21 15:44:38 +00:00
Stephen Morris
3859c55031 [rt46602] Ensure that output sent to stderr is routed to the output file
(cherry picked from commit a5cf6bd12a)
2018-02-21 15:44:38 +00:00
Stephen Morris
626bd48214 [rt46602] Minor tidying up of documentation
(cherry picked from commit ce66752adc)
2018-02-21 15:44:38 +00:00
Stephen Morris
4dc368a837 [rt46602] Check prerequsities before network interfaces when running test.
(cherry picked from commit 549a6c6bc8)
2018-02-21 15:44:38 +00:00
Stephen Morris
d737040fcf [rt46602] Minor formatting changes to Perl files
(cherry picked from commit f0f5b5f045)
2018-02-21 15:44:38 +00:00
Stephen Morris
9e176a0a34 [rt46603] Provide echostart() and echoend() to output test start/end messages.
(cherry picked from commit c344e1d768)
2018-02-21 15:44:38 +00:00
Stephen Morris
62652988ed [rt46602] Ensure test output in systests.output is not mixed up
When running all the system tests, output from a test is sent to a
test.output file in the test directory.  These are combined in to
systests.output when the run finishes.

(cherry picked from commit 055e5be9fd)
2018-02-21 15:44:38 +00:00
Stephen Morris
0a372b07a9 [rt46602] Miscellaneous minor fixes
... for misspellings, typos, trailing spaces, duplicated lines etc.

(cherry picked from commit 3c25e5fa02)
2018-02-21 15:44:38 +00:00
Stephen Morris
0d9b1190ce [rt46602] Allocate 100 ports to each test
(cherry picked from commit edc7a18d84)
2018-02-21 15:44:38 +00:00
Stephen Morris
21e48340b3 [rt46602] Pass port numbers to tests via environment variables
(cherry picked from commit f5d8f07900)
2018-02-21 15:44:38 +00:00
Stephen Morris
32b4e0ce4d [rt46602] Ensure parallel tests are in alphabetical order
(cherry picked from commit b2d90da0c2)
2018-02-21 15:44:38 +00:00
Stephen Morris
353ab4dc69 [rt46602] Expanded system tests README
Add more information on running the tests, together with a section
on how the tests are organised, aimed at new developers.

(cherry picked from commit af005cdbcf)
2018-02-21 15:44:38 +00:00
Stephen Morris
a807f8a77a [rt46602] More updates to the test running framework
Tidy up the stop/start files and make switch usage consistent. Also
tidy up the various "clean" targets in the Makefile.

(cherry picked from commit b24c2e11d8)
2018-02-21 15:44:37 +00:00
Stephen Morris
cbe20bcc1f [rt46602] Modify some test files to cope with the "-p" flag
(cherry picked from commit 517d990bd4)
2018-02-21 15:44:37 +00:00
Stephen Morris
499e768d07 [rt46602] Tidy up run management
Miscellaneous tidying up of run management.  The most significant
change is that "runall.sh" now runs _all_ the tests, even the
ones that can run in parallel.  runsequential.sh is the script
to run tests that have not been converted to parallel running.

(cherry picked from commit 32fe6f7682)
2018-02-21 15:44:37 +00:00
Stephen Morris
b2f2c67fd9 [rt46602] Assign block of ports for each test
Some tests use more ports than just the query and control ports.
Each test that can run in parallel with other tests is now assigned
a unique block of 10 ports.

(cherry picked from commit e0ff77f9d3)
2018-02-21 15:44:37 +00:00
Stephen Morris
60945507d8 [rt46602] Allow catz system test to run in parallel with other tests
(cherry picked from commit 07341693e1)
2018-02-21 15:44:37 +00:00
Stephen Morris
2ed6a072c9 [rt46602] Miscellaneous changes to existing parallelised tests
Currently these tests are allow_query, rpzrecurse and serve-stale

1. Function to copy files and set port numbers renamed from copy_config
to copy_setports, as this is used to change the ports in Perl and Python
test scripts as well.
2. Changes to rpzrecurse/tests.sh to handle two calls to getopts (one to
parse port numbers, the other to parse rpzrecurse-specific options). Also
fixed various commands to use correct ports.
3. Updates to "clean.sh" scripts to ensure that all files created in the
test are removed.

(cherry picked from commit 78f2b9ca01)
2018-02-21 15:44:37 +00:00
Stephen Morris
6fbf96c112 [rt46602] Ensure that tests running in parallel use unique ports
Via an intermediate make file, tests that have been modified to be able
to run in parallel are assigned unique query and control port numbers
(other than 5300 and 9953 respectively).  Tests that have not yet been
modified all use ports 5300 and 9953, so must be run sequentially.

(cherry picked from commit e7429b124b)
2018-02-21 15:44:37 +00:00
Stephen Morris
f396d62285 [rt46602] Miscellaneous tidying up of code cherry-picked from experimental branch
(cherry picked from commit e1a69e5d1f)
2018-02-21 15:44:37 +00:00
Ondřej Surý
e34bd38f69 [rt46602] Allow parallel run of new dynamic port tests and the old ones
(cherry picked from commit 3b14d54fd7)
2018-02-21 15:44:37 +00:00
Ondřej Surý
79fe35af4a [rt46602] Put back original .gitignore files in the system tests
(cherry picked from commit caaec1ef6d)
2018-02-21 15:44:37 +00:00
Ondřej Surý
4465749695 [rt46602] Adjust rpzrecurse for dynamic port usage
Resolve conflict in run.sh during cherry-pick from an experimental branch.

(cherry picked from commit 2a435708e4)
2018-02-21 15:44:37 +00:00
Ondřej Surý
6890a3b63e [rt46602] Add $SEDPORTS variable to help with @PORT@ and @CONTROLPORT@ replacement
(cherry picked from commit e6802badc0)
2018-02-21 15:44:37 +00:00
Ondřej Surý
e00c4c1b66 [rt46602] Move most of ignored files to top of system tests directory
(cherry picked from commit d2c8286163)
2018-02-21 15:44:37 +00:00
Ondřej Surý
2d41a16ba9 [rt46602] Fix output from rndc reload
(cherry picked from commit 53bade5ece)
2018-02-21 15:44:37 +00:00
Ondřej Surý
5dc741cd3e [rt46602] Add .gitignore to allow_query
(cherry picked from commit bc276f6c05)
2018-02-21 15:44:37 +00:00
Ondřej Surý
c707a93f6d [rt46602] Utilize echo_i to allow mixed test output
(cherry picked from commit 7e3c96c867)
2018-02-21 15:44:36 +00:00
Ondřej Surý
2c99184849 [rt46602] Prefix R:<result> as R:<test>:<result>
Resolve conflict in run.sh during cherry-pick from experimental branch.

(cherry picked from commit 278c72fef6)
2018-02-21 15:44:36 +00:00
Ondřej Surý
3be9b6078f [rt46602] Add echo_i function to print I:<testname>:string
(cherry picked from commit f7ac41757a)
2018-02-21 15:44:36 +00:00
Ondřej Surý
9974e247ff [rt46602] Add serve-stale working files to .gitignore
(cherry picked from commit cba162e70e)
2018-02-21 15:44:36 +00:00
Ondřej Surý
0b174d71d3 [rt46602] Convert allow_query to dynamic port
(cherry picked from commit 992a2bafec)
2018-02-21 15:44:36 +00:00
Ondřej Surý
8a107c58f2 [rt46602] Update server-stale test to run on port passed from run.sh script
(cherry picked from commit f83ebd34b9)
2018-02-21 15:44:36 +00:00
Ondřej Surý
0b1a5a586f [rt46602] Change the system tests script to support running individual tests on different ports
Resolved conflict in run.sh.

(cherry picked from commit fb00f18005)
2018-02-21 15:44:36 +00:00
Mark Andrews
e5fcbe5f26 Merge branch '67-stabilize-cacheclean-system-test' into 'master'
Stabilize cacheclean system test

Closes #67

See merge request isc-projects/bind9!43

(cherry picked from commit 643c8c27ff)

01de79b3 Assign an index to each check in the cacheclean system test
aeea1faf Do not overwrite cache dumps
2bbff06d Wait until a cache dump completes instead of waiting for a fixed amount of time
ca1049b2 Improve the way cache contents are searched for "ns.flushtest.example"
2018-02-20 16:02:34 -05:00
Mark Andrews
57fbc03b11 return failure count 2018-02-16 14:08:26 +11:00
Mark Andrews
5aba593545 add brackets 2018-02-16 14:08:26 +11:00
Mark Andrews
e416b6f57c remove event which is only used for sizeof 2018-02-16 14:08:26 +11:00
Mark Andrews
d76f6a67dd use %u instead of %d 2018-02-16 14:08:26 +11:00
Mark Andrews
059ff0206b use %u instead of %d 2018-02-16 14:07:34 +11:00
Mark Andrews
90f7a76891 use %u instead of %d 2018-02-16 14:07:34 +11:00
Mark Andrews
7aab54de6f use %u instead of %d 2018-02-16 14:07:34 +11:00
Mark Andrews
2e0129aba4 use %u instead of %d 2018-02-16 14:07:34 +11:00
Mark Andrews
018cb006e1 use %u instead of %d 2018-02-16 14:07:34 +11:00
Mark Andrews
4230e366b4 use %u instead of %d 2018-02-16 14:07:34 +11:00
Mark Andrews
a6c47e456e use %u instead of %d 2018-02-16 14:07:34 +11:00
Mark Andrews
aaac93da3c make declarations consistent 2018-02-16 14:07:34 +11:00
Mark Andrews
51e274b187 report argument error in human form 2018-02-16 14:07:34 +11:00
Mark Andrews
191ebf1076 use %u instead of %d 2018-02-16 14:07:34 +11:00