Prevent runall.sh and "make test" from even attempting to run system
tests when "ifconfig.sh up" has not been run beforehand. This ensures
the user is not flooded with error messages in such a case.
- turn off some of the build tests
- increase test threads to 8
- build with --disable-static
- suppress the non-system tests under bin/tests (timers, task, etc)
- no longer grep for specific line numbers when checking
parameter logging, as those can change
- report the failure immediatey if parameter check fails
- add CHANGES note
- update copyrights and license headers
- add -j to the make commands in .gitlab-ci.yml to take
advantage of parallelization in the gitlab CI process
This enables the environment variables controlling run.sh behavior to be
permanently set in a working environment (e.g. to automatically force
colored output without using "-c" in each runall.sh invocation).
Relevant runall.sh command line arguments still have a higher priority.
As parallel.mk and runsequential.sh both pipe system test output through
"tee" (for the purpose of creating test.output), run.sh invoked from
these two files detects it is not writing to a terminal, which prevents
colored output from being generated. Allow forcing colored output using
a new command line argument for runall.sh, "-c", which sets an
environment variable (SYSTEMTEST_FORCE_COLOR) causing conf.sh to
unconditionally enable colored output.
The same environment variable can also be used directly to force colored
output when using "make test" instead of runall.sh.
Instead of exporting an environment variable containing a command line
argument (NOCLEAN="-n"), extend run.sh to handle a "boolean" environment
variable (SYSTEMTEST_NO_CLEAN) itself. The former method is buggy
because the value of NOCLEAN is set in parallel.mk when that file is
first created, but it is not subsequently updated upon each test run
(because make considers parallel.mk to be up to date).
To retain backward compatibility, the "-n" command line argument for
run.sh is still supported (and has a higher priority than the relevant
environment variable).
The SYSTEMTEST_NO_CLEAN environment variable can also be used directly
to prevent cleanup when using "make test" instead of runall.sh.
Apart from fixing a bug, this simplifies the way runall.sh controls
run.sh behavior due to the Makefile being bypassed. Direct processing
of environment variables in run.sh is more scalable in the long run,
given that the previously utilized technique, even with its
implementation fixed, would still require Makefile.in to be modified in
two places each time a new flag needed to be passed from runall.sh to
run.sh.
During the "check flushtree clears adb correctly" check, expecting
"ns.flushtest.example" to always be the first name in the ADB dump is
fragile, because in a certain corner case "a.root-servers.nil" will be
the first name instead.
As the purpose of the relevant check is to ensure "ns.flushtest.example"
is removed from ADB by "rndc flushtree flushtest.example", search the
entire list of names present in ADB instead of just the first entry when
looking for "ns.flushtest.example".
Dumping the cache is an asynchronous operation, so sleeping for a fixed
amount of time after running "rndc dumpdb" is imperfect as dumping cache
contents may take longer than expected on slower machines. Instead of
always sleeping for 1 second, wait until the "; Dump complete" line
appears in the dump or 10 seconds pass, whichever comes first.
Unless configured otherwise in named.conf, "rndc dumpdb" causes a cache
dump to be written to a file called "named_dump.db" in the working
directory of the given named instance. Repeatedly using this command
throughout different checks in the cacheclean system test causes cache
dumps for older checks to be overwritten, which hinders failure
diagnosis. Prevent this by moving each cache dump to a check-specific
location after running "rndc dumpdb".
Furthermore, during the "check flushtree clears adb correctly" check,
dump_cache() is called twice without renaming the resulting files.
Prevent the first cache dump from being overwritten by moving it to a
different file before calling "rndc dumpdb" for the second time.
1. Track changes to conf.sh.in in conf.sh.win32
2. Modifications to prevent Windows "Configure" script replacing
the sed "@PORT@" substitution tokens in conf.sh.win32.
3. runall.sh now runs Windows tests sequentially
It was TESTNAME, but this is an obvious name and was used in one of
the system tests, something that interfered with the content of
progress messages. It is now SYSTESTDIR.
