Commit Graph
3596 Commits
Author SHA1 Message Date
Michał KępieńandStephen Morris 344d05063f [rt46602] Re-add clean.sh invocation to catz/setup.sh
Commit 57aa7b60fd caused catz/setup.sh to no longer call clean.sh, which
results in the catz system test failing on subsequent runs if the first
run is interrupted or fails.
2018-02-16 16:09:49 +00:00
Stephen Morris 451b7309ca [rt46602] Remove option to format the printed start time of a test
There was too much incompatibility between operating systems as to
the allowed options for "date".
2018-02-16 16:09:49 +00:00
Stephen Morris 462766cd76 [rt46602] Escape line ends in shell script variable assignment
Escape the line ends in a multi-line variable assignment.  Under some
circumstances, substituting the variable caused syntax errors when
used as the list of values in a shell script "for" statement.
2018-02-16 16:09:49 +00:00
Stephen Morris fb78f82ba1 [rt46602] Fix EXTRAPORT substitutions in copy_setport
Fix problems where all EXTRAPORT macros were being replaced by
the EXTRAPORT1 value. (Copy and paste issue.)
2018-02-16 16:09:49 +00:00
Stephen Morris cdf18a5009 [rt46602] Minor documentation changes 2018-02-16 16:09:49 +00:00
Stephen Morris 7f2d41d4c4 [rt46602] Correct way allow_query test produces RNDC output 2018-02-16 16:09:49 +00:00
Stephen Morris 0855a8a6cc [rt46602] Simplify the cleanup code 2018-02-16 16:09:48 +00:00
Stephen Morris f606b17dfb [rt46602] Extensions to README
Added descriptions of how nameservers are started and stopped
during the tests, and how the framework cleans up the files created.
2018-02-16 16:09:31 +00:00
Stephen Morris b88b508b22 [rt46602] Miscellenous small changes 2018-02-16 15:19:44 +00:00
Stephen Morris 1f07c69f25 [rt46602] Handle echoing of multi-line output from tests 2018-02-16 15:19:44 +00:00
Stephen Morris c12e06755f [rt46602] Rename "test" variable to "systest"
... to avoid confusion between the variable and the command of the
same name.
2018-02-16 15:19:44 +00:00
Stephen Morris 00bc29640b [rt46602] Add ability to prevent deletion of output files when running all tests
Some tests may need to examine all output files from all system tests.
Allow the deletion of these files to be inhibited if required.
2018-02-16 15:19:44 +00:00
Stephen Morris 244d1c30e2 [rt46602] Error messages before start of test are output to stderr
Error messages concerning the invocation of run.sh are output to
stderr.  Messages after the test has started are output to stdout.
2018-02-16 15:19:44 +00:00
Stephen Morris a5cf6bd12a [rt46602] Ensure that output sent to stderr is routed to the output file 2018-02-16 15:19:44 +00:00
Stephen Morris ce66752adc [rt46602] Minor tidying up of documentation 2018-02-16 15:19:44 +00:00
Stephen Morris 549a6c6bc8 [rt46602] Check prerequsities before network interfaces when running test. 2018-02-16 15:19:44 +00:00
Stephen Morris f0f5b5f045 [rt46602] Minor formatting changes to Perl files 2018-02-16 15:19:44 +00:00
Stephen Morris c344e1d768 [rt46603] Provide echostart() and echoend() to output test start/end messages. 2018-02-16 15:19:44 +00:00
Stephen Morris 055e5be9fd [rt46602] Ensure test output in systests.output is not mixed up
When running all the system tests, output from a test is sent to a
test.output file in the test directory.  These are combined in to
systests.output when the run finishes.
2018-02-16 15:19:44 +00:00
Stephen Morris 3c25e5fa02 [rt46602] Miscellaneous minor fixes
... for misspellings, typos, trailing spaces, duplicated lines etc.
2018-02-16 15:19:43 +00:00
Stephen Morris edc7a18d84 [rt46602] Allocate 100 ports to each test 2018-02-16 15:19:43 +00:00
Stephen Morris f5d8f07900 [rt46602] Pass port numbers to tests via environment variables 2018-02-16 15:19:43 +00:00
Stephen Morris b2d90da0c2 [rt46602] Ensure parallel tests are in alphabetical order 2018-02-16 15:19:43 +00:00
Stephen Morris af005cdbcf [rt46602] Expanded system tests README
Add more information on running the tests, together with a section
on how the tests are organised, aimed at new developers.
2018-02-16 15:19:43 +00:00
Stephen Morris b24c2e11d8 [rt46602] More updates to the test running framework
Tidy up the stop/start files and make switch usage consistent. Also
tidy up the various "clean" targets in the Makefile.
2018-02-16 14:57:52 +00:00
Stephen Morris 517d990bd4 [rt46602] Modify some test files to cope with the "-p" flag 2018-02-16 14:57:51 +00:00
Stephen Morris 32fe6f7682 [rt46602] Tidy up run management
Miscellaneous tidying up of run management.  The most significant
change is that "runall.sh" now runs _all_ the tests, even the
ones that can run in parallel.  runsequential.sh is the script
to run tests that have not been converted to parallel running.
2018-02-16 14:57:51 +00:00
Stephen Morris e0ff77f9d3 [rt46602] Assign block of ports for each test
Some tests use more ports than just the query and control ports.
Each test that can run in parallel with other tests is now assigned
a unique block of 10 ports.
2018-02-16 14:57:51 +00:00
Stephen Morris 07341693e1 [rt46602] Allow catz system test to run in parallel with other tests 2018-02-16 14:57:51 +00:00
Stephen Morris 78f2b9ca01 [rt46602] Miscellaneous changes to existing parallelised tests
Currently these tests are allow_query, rpzrecurse and serve-stale

1. Function to copy files and set port numbers renamed from copy_config
to copy_setports, as this is used to change the ports in Perl and Python
test scripts as well.
2. Changes to rpzrecurse/tests.sh to handle two calls to getopts (one to
parse port numbers, the other to parse rpzrecurse-specific options). Also
fixed various commands to use correct ports.
3. Updates to "clean.sh" scripts to ensure that all files created in the
test are removed.
2018-02-16 14:57:51 +00:00
Stephen Morris e7429b124b [rt46602] Ensure that tests running in parallel use unique ports
Via an intermediate make file, tests that have been modified to be able
to run in parallel are assigned unique query and control port numbers
(other than 5300 and 9953 respectively).  Tests that have not yet been
modified all use ports 5300 and 9953, so must be run sequentially.
2018-02-16 14:57:51 +00:00
Stephen Morris e1a69e5d1f [rt46602] Miscellaneous tidying up of code cherry-picked from experimental branch 2018-02-16 14:57:51 +00:00
Ondřej SurýandStephen Morris 3b14d54fd7 [rt46602] Allow parallel run of new dynamic port tests and the old ones 2018-02-16 14:57:51 +00:00
Ondřej SurýandStephen Morris caaec1ef6d [rt46602] Put back original .gitignore files in the system tests 2018-02-16 14:57:51 +00:00
Ondřej SurýandStephen Morris 2a435708e4 [rt46602] Adjust rpzrecurse for dynamic port usage
Resolve conflict in run.sh during cherry-pick from an experimental branch.
2018-02-16 14:57:51 +00:00
Ondřej SurýandStephen Morris e6802badc0 [rt46602] Add $SEDPORTS variable to help with @PORT@ and @CONTROLPORT@ replacement 2018-02-16 14:57:51 +00:00
Ondřej SurýandStephen Morris d2c8286163 [rt46602] Move most of ignored files to top of system tests directory 2018-02-16 14:57:51 +00:00
Ondřej SurýandStephen Morris 53bade5ece [rt46602] Fix output from rndc reload 2018-02-16 14:57:51 +00:00
Ondřej SurýandStephen Morris bc276f6c05 [rt46602] Add .gitignore to allow_query 2018-02-16 14:57:51 +00:00
Ondřej SurýandStephen Morris 7e3c96c867 [rt46602] Utilize echo_i to allow mixed test output 2018-02-16 14:57:51 +00:00
Ondřej SurýandStephen Morris 278c72fef6 [rt46602] Prefix R:<result> as R:<test>:<result>
Resolve conflict in run.sh during cherry-pick from experimental branch.
2018-02-16 14:57:50 +00:00
Ondřej SurýandStephen Morris f7ac41757a [rt46602] Add echo_i function to print I:<testname>:string 2018-02-16 14:57:50 +00:00
Ondřej SurýandStephen Morris cba162e70e [rt46602] Add serve-stale working files to .gitignore 2018-02-16 14:57:50 +00:00
Ondřej SurýandStephen Morris 992a2bafec [rt46602] Convert allow_query to dynamic port 2018-02-16 14:57:50 +00:00
Ondřej SurýandStephen Morris f83ebd34b9 [rt46602] Update server-stale test to run on port passed from run.sh script 2018-02-16 14:57:50 +00:00
Ondřej SurýandStephen Morris fb00f18005 [rt46602] Change the system tests script to support running individual tests on different ports
Resolved conflict in run.sh.
2018-02-16 14:57:50 +00:00
Mark Andrews a0720c9e56 return failure count 2018-02-16 10:20:39 +11:00
Mark Andrews 99af4e7d52 add brackets 2018-02-16 10:20:39 +11:00
Mark Andrews ca6faad2b6 remove event which is only used for sizeof 2018-02-16 10:20:39 +11:00
Mark Andrews 4745daafe0 use %u instead of %d 2018-02-16 10:20:38 +11:00