Commit Graph
8342 Commits
Author SHA1 Message Date
Mark Andrews 146c16c06e check that sed selects a non zero range of lines 2018-02-22 11:30:07 +11:00
Mark Andrews 643c8c27ff Merge branch '67-stabilize-cacheclean-system-test' into 'master'
Stabilize cacheclean system test

Closes #67

See merge request isc-projects/bind9!43
2018-02-20 16:01:00 -05:00
Stephen Morris 9f4c778f69 Merge branch 'rt46602' into 'master'
Allow parallel run of system tests

See merge request isc-projects/bind9!7
2018-02-20 09:00:59 -05:00
Michał Kępień ca1049b2f3 Improve the way cache contents are searched for "ns.flushtest.example"
During the "check flushtree clears adb correctly" check, expecting
"ns.flushtest.example" to always be the first name in the ADB dump is
fragile, because in a certain corner case "a.root-servers.nil" will be
the first name instead.

As the purpose of the relevant check is to ensure "ns.flushtest.example"
is removed from ADB by "rndc flushtree flushtest.example", search the
entire list of names present in ADB instead of just the first entry when
looking for "ns.flushtest.example".
2018-02-20 13:59:29 +01:00
Michał Kępień 2bbff06d36 Wait until a cache dump completes instead of waiting for a fixed amount of time
Dumping the cache is an asynchronous operation, so sleeping for a fixed
amount of time after running "rndc dumpdb" is imperfect as dumping cache
contents may take longer than expected on slower machines.  Instead of
always sleeping for 1 second, wait until the "; Dump complete" line
appears in the dump or 10 seconds pass, whichever comes first.
2018-02-20 13:59:28 +01:00
Michał Kępień aeea1faf01 Do not overwrite cache dumps
Unless configured otherwise in named.conf, "rndc dumpdb" causes a cache
dump to be written to a file called "named_dump.db" in the working
directory of the given named instance.  Repeatedly using this command
throughout different checks in the cacheclean system test causes cache
dumps for older checks to be overwritten, which hinders failure
diagnosis.  Prevent this by moving each cache dump to a check-specific
location after running "rndc dumpdb".

Furthermore, during the "check flushtree clears adb correctly" check,
dump_cache() is called twice without renaming the resulting files.
Prevent the first cache dump from being overwritten by moving it to a
different file before calling "rndc dumpdb" for the second time.
2018-02-20 13:59:27 +01:00
Michał Kępień 01de79b332 Assign an index to each check in the cacheclean system test
This will enable storing per-check forensic data, which facilitates
troubleshooting test failures.
2018-02-20 13:59:25 +01:00
Tinderbox User 72326f7701 update copyright notice / whitespace 2018-02-18 23:47:45 +00:00
Stephen Morris 7bb9a97904 [rt46602] Changes to get tests running on Windows
1. Track changes to conf.sh.in in conf.sh.win32
2. Modifications to prevent Windows "Configure" script replacing
   the sed "@PORT@" substitution tokens in conf.sh.win32.
3. runall.sh now runs Windows tests sequentially
2018-02-16 16:14:41 +00:00
Stephen Morris 7222c9fc76 [rt46602] Miscellaneous changes to fix rebase issues 2018-02-16 16:14:41 +00:00
Stephen Morris 30cd931a0d [rt46602] Change name of shell variable holding current test directory
It was TESTNAME, but this is an obvious name and was used in one of
the system tests, something that interfered with the content of
progress messages.  It is now SYSTESTDIR.
2018-02-16 16:14:40 +00:00
Michał KępieńandStephen Morris 2c519531a7 [rt46602] Export environment variables in a portable manner 2018-02-16 16:14:40 +00:00
Michał KępieńandStephen Morris e005fb948d [rt46602] Sanitize test names to fix "make check" on Solaris 2018-02-16 16:14:40 +00:00
Stephen Morris 3dfb7c17eb [rt46602] Add .gitignore file for catz 2018-02-16 16:14:40 +00:00
Stephen Morris 59d88ee192 [rt46602] Pass noclean option to Makefile via exported variable 2018-02-16 16:09:49 +00:00
Stephen Morris 23dfa27a84 [rt46602] Mention conf.sh.win32 in instructions for adding a test. 2018-02-16 16:09:49 +00:00
Michał KępieńandStephen Morris f0721de624 [rt46602] Fix various typos, formatting issues and stylistic nits 2018-02-16 16:09:49 +00:00
Michał KępieńandStephen Morris 42cf18d239 [rt46602] Remove redundant variable assignment 2018-02-16 16:09:49 +00:00
Michał KępieńandStephen Morris 1ac4c3e4e5 [rt46602] named.port is no longer created by setup.sh for allow_query 2018-02-16 16:09:49 +00:00
Michał KępieńandStephen Morris fe1ccafc34 [rt46602] Consistently print usage for runall.sh to stderr 2018-02-16 16:09:49 +00:00
Michał KępieńandStephen Morris 0dcee1cad3 [rt46602] Clean up getopts usage
Ensure case clauses are in sync with the string passed to getopts.
Remove catch-all clauses as they will never be evaluated.
2018-02-16 16:09:49 +00:00
Michał KępieńandStephen Morris fe51c34df3 [rt46602] Assign default values to variables set by command line options and consistently treat them as integers 2018-02-16 16:09:49 +00:00
Michał KępieńandStephen Morris 1960de5530 [rt46602] named.port cannot be ignored by git because it is required to be present by notify/ns4 2018-02-16 16:09:49 +00:00
Michał KępieńandStephen Morris 344d05063f [rt46602] Re-add clean.sh invocation to catz/setup.sh
Commit 57aa7b60fd caused catz/setup.sh to no longer call clean.sh, which
results in the catz system test failing on subsequent runs if the first
run is interrupted or fails.
2018-02-16 16:09:49 +00:00
Stephen Morris 451b7309ca [rt46602] Remove option to format the printed start time of a test
There was too much incompatibility between operating systems as to
the allowed options for "date".
2018-02-16 16:09:49 +00:00
Stephen Morris 462766cd76 [rt46602] Escape line ends in shell script variable assignment
Escape the line ends in a multi-line variable assignment.  Under some
circumstances, substituting the variable caused syntax errors when
used as the list of values in a shell script "for" statement.
2018-02-16 16:09:49 +00:00
Stephen Morris fb78f82ba1 [rt46602] Fix EXTRAPORT substitutions in copy_setport
Fix problems where all EXTRAPORT macros were being replaced by
the EXTRAPORT1 value. (Copy and paste issue.)
2018-02-16 16:09:49 +00:00
Stephen Morris cdf18a5009 [rt46602] Minor documentation changes 2018-02-16 16:09:49 +00:00
Stephen Morris 7f2d41d4c4 [rt46602] Correct way allow_query test produces RNDC output 2018-02-16 16:09:49 +00:00
Stephen Morris 0855a8a6cc [rt46602] Simplify the cleanup code 2018-02-16 16:09:48 +00:00
Stephen Morris f606b17dfb [rt46602] Extensions to README
Added descriptions of how nameservers are started and stopped
during the tests, and how the framework cleans up the files created.
2018-02-16 16:09:31 +00:00
Stephen Morris b88b508b22 [rt46602] Miscellenous small changes 2018-02-16 15:19:44 +00:00
Stephen Morris 1f07c69f25 [rt46602] Handle echoing of multi-line output from tests 2018-02-16 15:19:44 +00:00
Stephen Morris c12e06755f [rt46602] Rename "test" variable to "systest"
... to avoid confusion between the variable and the command of the
same name.
2018-02-16 15:19:44 +00:00
Stephen Morris 00bc29640b [rt46602] Add ability to prevent deletion of output files when running all tests
Some tests may need to examine all output files from all system tests.
Allow the deletion of these files to be inhibited if required.
2018-02-16 15:19:44 +00:00
Stephen Morris 244d1c30e2 [rt46602] Error messages before start of test are output to stderr
Error messages concerning the invocation of run.sh are output to
stderr.  Messages after the test has started are output to stdout.
2018-02-16 15:19:44 +00:00
Stephen Morris a5cf6bd12a [rt46602] Ensure that output sent to stderr is routed to the output file 2018-02-16 15:19:44 +00:00
Stephen Morris ce66752adc [rt46602] Minor tidying up of documentation 2018-02-16 15:19:44 +00:00
Stephen Morris 549a6c6bc8 [rt46602] Check prerequsities before network interfaces when running test. 2018-02-16 15:19:44 +00:00
Stephen Morris f0f5b5f045 [rt46602] Minor formatting changes to Perl files 2018-02-16 15:19:44 +00:00
Stephen Morris c344e1d768 [rt46603] Provide echostart() and echoend() to output test start/end messages. 2018-02-16 15:19:44 +00:00
Stephen Morris 055e5be9fd [rt46602] Ensure test output in systests.output is not mixed up
When running all the system tests, output from a test is sent to a
test.output file in the test directory.  These are combined in to
systests.output when the run finishes.
2018-02-16 15:19:44 +00:00
Stephen Morris 3c25e5fa02 [rt46602] Miscellaneous minor fixes
... for misspellings, typos, trailing spaces, duplicated lines etc.
2018-02-16 15:19:43 +00:00
Stephen Morris edc7a18d84 [rt46602] Allocate 100 ports to each test 2018-02-16 15:19:43 +00:00
Stephen Morris f5d8f07900 [rt46602] Pass port numbers to tests via environment variables 2018-02-16 15:19:43 +00:00
Stephen Morris b2d90da0c2 [rt46602] Ensure parallel tests are in alphabetical order 2018-02-16 15:19:43 +00:00
Stephen Morris af005cdbcf [rt46602] Expanded system tests README
Add more information on running the tests, together with a section
on how the tests are organised, aimed at new developers.
2018-02-16 15:19:43 +00:00
Stephen Morris b24c2e11d8 [rt46602] More updates to the test running framework
Tidy up the stop/start files and make switch usage consistent. Also
tidy up the various "clean" targets in the Makefile.
2018-02-16 14:57:52 +00:00
Stephen Morris 517d990bd4 [rt46602] Modify some test files to cope with the "-p" flag 2018-02-16 14:57:51 +00:00
Stephen Morris 32fe6f7682 [rt46602] Tidy up run management
Miscellaneous tidying up of run management.  The most significant
change is that "runall.sh" now runs _all_ the tests, even the
ones that can run in parallel.  runsequential.sh is the script
to run tests that have not been converted to parallel running.
2018-02-16 14:57:51 +00:00