Commit Graph

1845 Commits

Author SHA1 Message Date
Evan Hunt
bf769f68a4 migrate tests from bin/tests/dnssec-signzone to bin/tests/system/dnssec
- added tests to the dnssec system test that duplicate the ones
  from bin/tests/dnssec-signzone
- changed cleanall.sh so it doesn't automatically remove all
  key files, because there are now some of those that are part of the
  distribution

(cherry picked from commit ccfe778c01)
(cherry picked from commit d8f8eee381)
(cherry picked from commit dbb225d83e)
(cherry picked from commit 161f525942)
2018-03-09 22:59:52 -08:00
Evan Hunt
e44ce6a1ab restore long options for run.sh
(cherry picked from commit 4a714bffbc)
2018-03-07 14:35:45 -08:00
Evan Hunt
ac91515aa8 eliminate unnecessary waiting in allow-query test
- grep in the log to see if configuration is finished rather
  than sleeping for 5 seconds

(cherry picked from commit a9b6bed6af)
(cherry picked from commit 37a6bb004c)
(cherry picked from commit 2e48caa476)
2018-03-07 14:19:55 -08:00
Evan Hunt
0ed2df43b0 rename allow_query to allow-query
(cherry picked from commit ffc3e73c36)
(cherry picked from commit 5d9e3f9b02)
(cherry picked from commit de6fe7dd83)
2018-03-07 14:19:52 -08:00
Evan Hunt
fc1b254b5e address a possible notify test failure
- wait for the transfer completion message to apear in the log instead
  of the notify message. this ensures we don't check for the presense of
  transfered records during the time between the notify and the
  transfer.

(cherry picked from commit ad32553c1d)
(cherry picked from commit dff94add3e)
(cherry picked from commit 5a5e476d00)
2018-03-06 11:48:16 -08:00
Evan Hunt
8c0d2127f9 revise soft limit test
- don't bail out of the loop if clients are exceeded, just count incidents
- verbosely describe expectations and results

(cherry picked from commit 86838b2a02)
2018-03-01 15:41:01 +01:00
Mark Andrews
a845283012 remove unsupported command line argument (-4) from nsupdate call
(cherry picked from commit ac3d2a0ed9)
2018-03-01 13:56:16 +01:00
Mark Andrews
83990e9bdc wait for 'all zones loaded' rather than 'reloading zones succeeded' which just indicates that everything kicked off ok
(cherry picked from commit 4cb0873a0a)
2018-02-28 14:57:19 +01:00
Evan Hunt
be98afaecf Do not destroy systests.output if there are no test.output files
(cherry picked from commit 38940619c6)
2018-02-27 10:04:37 -08:00
Michał Kępień
d241bf9d4c Use echoinfo for printing test summary
This ensures testsummary.sh is consistent with other scripts belonging
to the test framework.

(cherry picked from commit 642e20db8a)
2018-02-27 10:04:37 -08:00
Michał Kępień
b18292a208 Make testsummary.sh print a list of failed system tests
Print a list of failed system tests, if any, below the system test
summary to avoid the need to manually search through the test log.

(cherry picked from commit 28068857e7)
2018-02-27 10:04:36 -08:00
Michał Kępień
1709486103 Tweak the regular expression used for extracting system test results
The current regular expression used for extracting system test results
from systests.output, "^R:", is anchored at the start of a line, which
prevents colored system test output from being properly processed.  As
just "R:" would arguably be too general, extend the pattern a bit to
ensure it will only match lines containing system test results.

(cherry picked from commit 9006d6dbbd)
2018-02-27 10:04:36 -08:00
Evan Hunt
92c9e7c045 clean up test output
- removed a few remaing places where output wasn't being passed
  through echo_i or cat_i
- added a "digcomp" function to conf.sh.in to send digcomp.pl output
  through cat_i and return the correct exit value
- set SYSTESTDIR when calling echo_i from nsX directories, so that
  the test name will always be printed correctly
- fixed a test name typo in conf.sh.in

(cherry picked from commit 0e52fbd0b3)
(cherry picked from commit d57e8f6df3)
(cherry picked from commit 053c06d5e1)
2018-02-27 10:03:21 -08:00
Evan Hunt
4cb037a6e2 Merge branch '112-fix-mx-checks-for-dynamic-updates-v9_9' into 'v9_9'
Fix MX checks for dynamic updates

Closes #112

See merge request isc-projects/bind9!79
2018-02-26 15:20:49 -08:00
Michał Kępień
d86a7dbe69 Fix MX checks for dynamic updates
The check_mx() function in lib/ns/update.c incorrectly tests whether the
DNS_RDATA_CHECKMX/DNS_RDATA_CHECKMXFAIL flags are set for each applied
MX record update as these flags are never set in code paths related to
dynamic updates; they can only be set when loading a zone from a master
file (DNS_ZONEOPT_CHECKMX -> DNS_MASTER_CHECKMX -> DNS_RDATA_CHECKMX).
This flaw allows MX records containing IP addresses to be added to a
zone even when "check-mx fail;" is used.

Ensure correct behavior by modifying the relevant tests in check_mx() so
that they use DNS_ZONEOPT_CHECKMX/DNS_ZONEOPT_CHECKMXFAIL instead.

(cherry picked from commit 857a40c87b)
2018-02-26 15:20:06 -08:00
Evan Hunt
cdf3f823bb Merge branch 'check-for-test-interfaces-before-running-system-tests-v9_9' into 'v9_9'
Check for test interfaces before running system tests

See merge request isc-projects/bind9!19
2018-02-26 15:10:40 -08:00
Evan Hunt
699fffc34f test interfaces directly in runall.sh and parallel.mk
(cherry picked from commit a8b583e78b)
2018-02-26 15:10:40 -08:00
Michał Kępień
98b3b5bba7 Check for test interfaces before running system tests
Prevent runall.sh and "make test" from even attempting to run system
tests when "ifconfig.sh up" has not been run beforehand.  This ensures
the user is not flooded with error messages in such a case.

(cherry picked from commit e0221f2d25)
2018-02-26 15:10:40 -08:00
Evan Hunt
8c6e4b205c wait for synchronization; re-notify slave if the transfer doesn't happen
(cherry picked from commit 67a8b5ec17)
2018-02-26 15:03:03 -08:00
Evan Hunt
7124e34835 Fix merge error 2018-02-25 23:07:38 -08:00
Evan Hunt
8f4610f773 Merge branch 'tidy-runall.sh-environment-variables-v9_11-v9_10-v9_9' into 'v9_9'
Tidy runall.sh environment variables

See merge request isc-projects/bind9!18
2018-02-25 13:34:35 -08:00
Michał Kępień
1f42e865d2 Preserve values of environment variables which are already set
This enables the environment variables controlling run.sh behavior to be
permanently set in a working environment (e.g. to automatically force
colored output without using "-c" in each runall.sh invocation).

Relevant runall.sh command line arguments still have a higher priority.

(cherry picked from commit d989d20fe5)
(cherry picked from commit 2cd174dda9)
(cherry picked from commit fdcac29f63)
2018-02-25 13:34:35 -08:00
Michał Kępień
a7ae215925 Allow forcing colored system test output
As parallel.mk and runsequential.sh both pipe system test output through
"tee" (for the purpose of creating test.output), run.sh invoked from
these two files detects it is not writing to a terminal, which prevents
colored output from being generated.  Allow forcing colored output using
a new command line argument for runall.sh, "-c", which sets an
environment variable (SYSTEMTEST_FORCE_COLOR) causing conf.sh to
unconditionally enable colored output.

The same environment variable can also be used directly to force colored
output when using "make test" instead of runall.sh.

(cherry picked from commit a324031a82)
(cherry picked from commit cbebf7ef59)
(cherry picked from commit 2f3b3b7aff)
2018-02-25 13:34:35 -08:00
Michał Kępień
fcefc77d35 Allow retaining system test output using an environment variable
Instead of exporting an environment variable containing a command line
argument (NOCLEAN="-n"), extend run.sh to handle a "boolean" environment
variable (SYSTEMTEST_NO_CLEAN) itself.  The former method is buggy
because the value of NOCLEAN is set in parallel.mk when that file is
first created, but it is not subsequently updated upon each test run
(because make considers parallel.mk to be up to date).

To retain backward compatibility, the "-n" command line argument for
run.sh is still supported (and has a higher priority than the relevant
environment variable).

The SYSTEMTEST_NO_CLEAN environment variable can also be used directly
to prevent cleanup when using "make test" instead of runall.sh.

Apart from fixing a bug, this simplifies the way runall.sh controls
run.sh behavior due to the Makefile being bypassed.  Direct processing
of environment variables in run.sh is more scalable in the long run,
given that the previously utilized technique, even with its
implementation fixed, would still require Makefile.in to be modified in
two places each time a new flag needed to be passed from runall.sh to
run.sh.

(cherry picked from commit 3862043879)
(cherry picked from commit e9c4dbe361)
(cherry picked from commit 0e3840e4b4)
2018-02-25 13:34:35 -08:00
Evan Hunt
8315e76561 improve reliability of xfer test
(cherry picked from commit 5d7d6881a1)
(cherry picked from commit 09a1a261dc)
(cherry picked from commit 948bd83ce5)
(cherry picked from commit 4160936c92)
2018-02-25 13:34:23 -08:00
Evan Hunt
99db49b439 Merge branch 'parallel-v9_9' into v9_9
chg: dev: Convert most of the remaining system tests to be able to run in parallel [GL #91]

Closes #91

See merge request isc-projects/bind9!59
2018-02-25 13:29:36 -08:00
Evan Hunt
7677cb277a final cleanup
- add CHANGES note
- update copyrights and license headers
2018-02-25 13:13:26 -08:00
Evan Hunt
ae6c141f7b tidying up after backport
(cherry picked from commit 854f7846d8)
(cherry picked from commit 6653dcab3d)
2018-02-25 13:10:00 -08:00
Evan Hunt
b65aa505e6 improve reliability of the xfer test on slow systems
- lengthened the wait loop for the transfers to complete

(cherry picked from commit b9246ce773)
(cherry picked from commit a5015d7791)
(cherry picked from commit c369fc06ca)
(cherry picked from commit 6a5092f537)
2018-02-25 11:19:26 -08:00
Evan Hunt
37d6f7b624 remove v6synth
- this was a test for the allow-v6-synthesis option, which was
  deprecated and no longer works. the test was removed
  from conf.sh.in long ago.

(cherry picked from commit 474b10a133)
(cherry picked from commit 7971873639)
(cherry picked from commit 626566c72d)
(cherry picked from commit 62f239f5cf)
2018-02-25 11:19:07 -08:00
Evan Hunt
87c7455b4c removed some unnecessary greps
(cherry picked from commit 731ad28be2)
(cherry picked from commit 17245b86d5)
(cherry picked from commit 350f18b0d1)
(cherry picked from commit 539be337d9)
2018-02-25 11:19:05 -08:00
Evan Hunt
a236a1e2b4 reorder tests to get better use of parallelism
(cherry picked from commit 56e440f65f)
(cherry picked from commit 4429f8b291)
(cherry picked from commit fa8740feda)
(cherry picked from commit 438dcd9f9c)
2018-02-25 11:18:48 -08:00
Evan Hunt
f8fe59f279 parallelize most system tests
(cherry picked from commit c032c54dda)
(cherry picked from commit 2b81f322cb)
(cherry picked from commit 167fa161d1)
(cherry picked from commit 1f297d33a0)
2018-02-25 11:17:55 -08:00
Stephen Morris
58c2522b85 [rt46602] Change name of shell variable holding current test directory
It was TESTNAME, but this is an obvious name and was used in one of
the system tests, something that interfered with the content of
progress messages.  It is now SYSTESTDIR.

(cherry picked from commit 30cd931a0d)
(cherry picked from commit 78f2976087)
(cherry picked from commit 4a29e3ba65)
2018-02-25 09:28:46 -08:00
Michał Kępień
1240f51426 [rt46602] Export environment variables in a portable manner
(cherry picked from commit 2c519531a7)
(cherry picked from commit 747df7d66a)
(cherry picked from commit ddb84ca0af)
2018-02-25 09:28:46 -08:00
Michał Kępień
8945f29fb1 [rt46602] Sanitize test names to fix "make check" on Solaris
(cherry picked from commit e005fb948d)
(cherry picked from commit cfaeb55e8e)
(cherry picked from commit fac15bf973)
2018-02-25 09:28:46 -08:00
Stephen Morris
2ee63a76c0 [rt46602] Pass noclean option to Makefile via exported variable
(cherry picked from commit 59d88ee192)
(cherry picked from commit 0ae7e64829)
(cherry picked from commit 4fbaafa480)
2018-02-25 09:28:46 -08:00
Stephen Morris
2a24aac12e [rt46602] Mention conf.sh.win32 in instructions for adding a test.
(cherry picked from commit 23dfa27a84)
(cherry picked from commit 06a9a37d3b)
(cherry picked from commit d9740c20da)
2018-02-25 09:28:45 -08:00
Michał Kępień
d634594e56 [rt46602] Fix various typos, formatting issues and stylistic nits
(cherry picked from commit f0721de624)
(cherry picked from commit b40e75a75c)
(cherry picked from commit 85f03bb30f)
2018-02-25 09:28:45 -08:00
Michał Kępień
2a31548089 [rt46602] Remove redundant variable assignment
(cherry picked from commit 42cf18d239)
(cherry picked from commit 4436930712)
(cherry picked from commit 90654248f6)
2018-02-25 09:28:45 -08:00
Michał Kępień
df5de60665 [rt46602] named.port is no longer created by setup.sh for allow_query
(cherry picked from commit 1ac4c3e4e5)
(cherry picked from commit 77e6ff0fae)
(cherry picked from commit 2e3d58ad30)
2018-02-25 09:28:45 -08:00
Michał Kępień
653dcebec8 [rt46602] Consistently print usage for runall.sh to stderr
(cherry picked from commit fe1ccafc34)
(cherry picked from commit 47e278b7c9)
(cherry picked from commit 66985fad62)
2018-02-25 09:28:45 -08:00
Michał Kępień
4ef8f1c774 [rt46602] Clean up getopts usage
Ensure case clauses are in sync with the string passed to getopts.
Remove catch-all clauses as they will never be evaluated.

(cherry picked from commit 0dcee1cad3)
(cherry picked from commit 0c8e56568f)
(cherry picked from commit 6d9fb2b429)
2018-02-25 09:28:45 -08:00
Michał Kępień
17fce84a63 [rt46602] Assign default values to variables set by command line options and consistently treat them as integers
(cherry picked from commit fe51c34df3)
(cherry picked from commit 3752edb28e)
(cherry picked from commit 1adbfeb3ed)
2018-02-25 09:28:45 -08:00
Michał Kępień
6d8fde8a92 [rt46602] named.port cannot be ignored by git because it is required to be present by notify/ns4
(cherry picked from commit 1960de5530)
(cherry picked from commit f47ed5ceb0)
(cherry picked from commit 01e21e3ee2)
2018-02-25 09:28:45 -08:00
Stephen Morris
98ddc2c422 [rt46602] Remove option to format the printed start time of a test
There was too much incompatibility between operating systems as to
the allowed options for "date".

(cherry picked from commit 451b7309ca)
(cherry picked from commit 8376949ea8)
(cherry picked from commit 52f784315e)
2018-02-25 09:28:45 -08:00
Stephen Morris
90dccece92 [rt46602] Escape line ends in shell script variable assignment
Escape the line ends in a multi-line variable assignment.  Under some
circumstances, substituting the variable caused syntax errors when
used as the list of values in a shell script "for" statement.

(cherry picked from commit 462766cd76)
(cherry picked from commit 53ebdf225f)
(cherry picked from commit 82ffebc73a)
2018-02-25 09:28:42 -08:00
Stephen Morris
6f8c363547 [rt46602] Fix EXTRAPORT substitutions in copy_setport
Fix problems where all EXTRAPORT macros were being replaced by
the EXTRAPORT1 value. (Copy and paste issue.)

(cherry picked from commit fb78f82ba1)
(cherry picked from commit 8513c4256b)
(cherry picked from commit c8880ce14a)
2018-02-25 09:27:13 -08:00
Stephen Morris
0c59973a13 [rt46602] Minor documentation changes
(cherry picked from commit cdf18a5009)
(cherry picked from commit 4731a82eef)
(cherry picked from commit 04120c4223)
2018-02-25 09:27:13 -08:00
Stephen Morris
2dae3e430a [rt46602] Correct way allow_query test produces RNDC output
(cherry picked from commit 7f2d41d4c4)
(cherry picked from commit 36136dec55)
(cherry picked from commit 7125bddda8)
2018-02-25 09:27:13 -08:00