Commit Graph

26057 Commits

Author SHA1 Message Date
Evan Hunt
d965560e41 fixed an upforwd test error.
(cherry picked from commit 54e010d51e)
2018-02-25 13:56:04 -08:00
Evan Hunt
cd3602b6a8 Merge branch 'tidy-runall.sh-environment-variables-v9_11' into 'v9_11'
Tidy runall.sh environment variables

See merge request isc-projects/bind9!18
2018-02-24 17:37:44 -08:00
Michał Kępień
2cd174dda9 Preserve values of environment variables which are already set
This enables the environment variables controlling run.sh behavior to be
permanently set in a working environment (e.g. to automatically force
colored output without using "-c" in each runall.sh invocation).

Relevant runall.sh command line arguments still have a higher priority.

(cherry picked from commit d989d20fe5)
2018-02-24 17:37:44 -08:00
Michał Kępień
cbebf7ef59 Allow forcing colored system test output
As parallel.mk and runsequential.sh both pipe system test output through
"tee" (for the purpose of creating test.output), run.sh invoked from
these two files detects it is not writing to a terminal, which prevents
colored output from being generated.  Allow forcing colored output using
a new command line argument for runall.sh, "-c", which sets an
environment variable (SYSTEMTEST_FORCE_COLOR) causing conf.sh to
unconditionally enable colored output.

The same environment variable can also be used directly to force colored
output when using "make test" instead of runall.sh.

(cherry picked from commit a324031a82)
2018-02-24 17:37:44 -08:00
Michał Kępień
e9c4dbe361 Allow retaining system test output using an environment variable
Instead of exporting an environment variable containing a command line
argument (NOCLEAN="-n"), extend run.sh to handle a "boolean" environment
variable (SYSTEMTEST_NO_CLEAN) itself.  The former method is buggy
because the value of NOCLEAN is set in parallel.mk when that file is
first created, but it is not subsequently updated upon each test run
(because make considers parallel.mk to be up to date).

To retain backward compatibility, the "-n" command line argument for
run.sh is still supported (and has a higher priority than the relevant
environment variable).

The SYSTEMTEST_NO_CLEAN environment variable can also be used directly
to prevent cleanup when using "make test" instead of runall.sh.

Apart from fixing a bug, this simplifies the way runall.sh controls
run.sh behavior due to the Makefile being bypassed.  Direct processing
of environment variables in run.sh is more scalable in the long run,
given that the previously utilized technique, even with its
implementation fixed, would still require Makefile.in to be modified in
two places each time a new flag needed to be passed from runall.sh to
run.sh.

(cherry picked from commit 3862043879)
2018-02-24 17:37:44 -08:00
Evan Hunt
f09698659a Merge branch '71-xfer-system-test-fails-intermittently-v9_12-v9_11' into 'v9_11'
chg: dev: Improve the resilience of the xfer system test, which could be flaky under load.

Closes #71

See merge request isc-projects/bind9!61
2018-02-24 01:46:54 -08:00
Evan Hunt
948bd83ce5 improve reliability of xfer test
(cherry picked from commit 5d7d6881a1)
(cherry picked from commit 09a1a261dc)
2018-02-24 01:46:52 -08:00
Evan Hunt
df7b99e275 Merge branch 'kyua-oot-v9_11' into 'v9_11'
chg: dev: Unit tests were broken in out-of-tree builds.

See merge request isc-projects/bind9!57
2018-02-23 15:50:35 -08:00
Petr Menšík
6d9b38e9fe unit/unittest.sh is generated by configure. It will always be
generated into builddir. If out-of-tree build is used, make unit
will always fail. Kyuafiles and testdata still have to be copied
manually into the builddir.

(cherry picked from commit 95cde3608a)
2018-02-23 15:50:33 -08:00
Evan Hunt
53e26b7abc Merge branch 'master-v9_12-v9_11' into 'v9_11'
Code cleanups

See merge request isc-projects/bind9!36
2018-02-23 13:52:49 -08:00
Petr Menšík
1b0cf4fea6 Reduce repeated detection of mysql lib directory.
Use common part once a bit later.

(cherry picked from commit 2a70eae04f)
(cherry picked from commit d04af52222)
2018-02-23 13:52:49 -08:00
Petr Menšík
6876501605 Reuse new function from rt46864 for similar block elsewhere.
(cherry picked from commit e7a93321f0)
(cherry picked from commit cb98ce8e67)
2018-02-23 13:52:43 -08:00
Evan Hunt
e3d4c66704 Merge branch '95-fix-dyndb-v9_11' into 'v9_11'
Resolve "dyndb system test fails intermittently"

Closes #95

See merge request isc-projects/bind9!63
2018-02-23 13:44:57 -08:00
Evan Hunt
dcd2348c8e improve dyndb test resilience
- no longer grep for specific line numbers when checking
  parameter logging, as those can change
- report the failure immediatey if parameter check fails

(cherry picked from commit 749df056be)
2018-02-23 13:44:53 -08:00
Evan Hunt
c4ed7c3a41 Merge branch '91-parallel-tests-v9_12-v9_11' into 'v9_11'
chg: dev: Convert most of the remaining system tests to be able to run in parallel [GL #91]

Closes #91

See merge request isc-projects/bind9!59
2018-02-23 13:34:45 -08:00
Evan Hunt
854f7846d8 tidying up after backport 2018-02-23 13:28:47 -08:00
Evan Hunt
c369fc06ca improve reliability of the xfer test on slow systems
- lengthed the wait loop for the transfers to complete

(cherry picked from commit b9246ce773)
(cherry picked from commit a5015d7791)
2018-02-23 13:25:29 -08:00
Evan Hunt
626566c72d remove v6synth
- this was a test for the allow-v6-synthesis option, which was
  deprecated and no longer works. the test was removed
  from conf.sh.in long ago.

(cherry picked from commit 474b10a133)
(cherry picked from commit 7971873639)
2018-02-23 13:25:27 -08:00
Evan Hunt
350f18b0d1 removed some unnecessary greps
(cherry picked from commit 731ad28be2)
(cherry picked from commit 17245b86d5)
2018-02-23 13:24:53 -08:00
Evan Hunt
fa8740feda reorder tests to get better use of parallelism
(cherry picked from commit 56e440f65f)
(cherry picked from commit 4429f8b291)
2018-02-23 13:24:51 -08:00
Evan Hunt
167fa161d1 parallelize most system tests
(cherry picked from commit c032c54dda)
(cherry picked from commit 2b81f322cb)
2018-02-23 13:23:31 -08:00
Stephen Morris
71332991fc Fixups after merge of branch 'rt46602-v9_11' into 'v9_11' 2018-02-22 15:41:04 +00:00
Stephen Morris
310523e5de Merge branch 'rt46602-v9_11' into 'v9_11'
Allow parallel run of system tests

See merge request isc-projects/bind9!7
2018-02-22 11:18:39 +00:00
Stephen Morris
8aaa92693c [rt46602] Update CHANGES for parallel tests
(cherry picked from commit dbf753b52d)
2018-02-22 11:18:19 +00:00
Stephen Morris
c4f5d07c1c [rt46602] Changes to get tests running on Windows
1. Track changes to conf.sh.in in conf.sh.win32
2. Modifications to prevent Windows "Configure" script replacing
   the sed "@PORT@" substitution tokens in conf.sh.win32.
3. runall.sh now runs Windows tests sequentially

(cherry picked from commit 7bb9a97904)
2018-02-22 11:17:30 +00:00
Stephen Morris
3d4a425266 [rt46602] Miscellaneous changes to fix rebase issues
(cherry picked from commit 7222c9fc76)
2018-02-22 11:15:22 +00:00
Stephen Morris
78f2976087 [rt46602] Change name of shell variable holding current test directory
It was TESTNAME, but this is an obvious name and was used in one of
the system tests, something that interfered with the content of
progress messages.  It is now SYSTESTDIR.

(cherry picked from commit 30cd931a0d)
2018-02-22 11:14:41 +00:00
Michał Kępień
747df7d66a [rt46602] Export environment variables in a portable manner
(cherry picked from commit 2c519531a7)
2018-02-22 11:14:41 +00:00
Michał Kępień
cfaeb55e8e [rt46602] Sanitize test names to fix "make check" on Solaris
(cherry picked from commit e005fb948d)
2018-02-22 11:14:40 +00:00
Stephen Morris
6508d98a98 [rt46602] Add .gitignore file for catz
(cherry picked from commit 3dfb7c17eb)
2018-02-22 11:14:40 +00:00
Stephen Morris
0ae7e64829 [rt46602] Pass noclean option to Makefile via exported variable
(cherry picked from commit 59d88ee192)
2018-02-22 11:14:40 +00:00
Stephen Morris
06a9a37d3b [rt46602] Mention conf.sh.win32 in instructions for adding a test.
(cherry picked from commit 23dfa27a84)
2018-02-22 11:14:40 +00:00
Michał Kępień
b40e75a75c [rt46602] Fix various typos, formatting issues and stylistic nits
(cherry picked from commit f0721de624)
2018-02-22 11:14:40 +00:00
Michał Kępień
4436930712 [rt46602] Remove redundant variable assignment
(cherry picked from commit 42cf18d239)
2018-02-22 11:14:40 +00:00
Michał Kępień
77e6ff0fae [rt46602] named.port is no longer created by setup.sh for allow_query
(cherry picked from commit 1ac4c3e4e5)
2018-02-22 11:14:40 +00:00
Michał Kępień
47e278b7c9 [rt46602] Consistently print usage for runall.sh to stderr
(cherry picked from commit fe1ccafc34)
2018-02-22 11:14:39 +00:00
Michał Kępień
0c8e56568f [rt46602] Clean up getopts usage
Ensure case clauses are in sync with the string passed to getopts.
Remove catch-all clauses as they will never be evaluated.

(cherry picked from commit 0dcee1cad3)
2018-02-22 11:14:39 +00:00
Michał Kępień
3752edb28e [rt46602] Assign default values to variables set by command line options and consistently treat them as integers
(cherry picked from commit fe51c34df3)
2018-02-22 11:14:39 +00:00
Michał Kępień
f47ed5ceb0 [rt46602] named.port cannot be ignored by git because it is required to be present by notify/ns4
(cherry picked from commit 1960de5530)
2018-02-22 11:14:39 +00:00
Michał Kępień
8f4770facf [rt46602] Re-add clean.sh invocation to catz/setup.sh
Commit 57aa7b60fd caused catz/setup.sh to no longer call clean.sh, which
results in the catz system test failing on subsequent runs if the first
run is interrupted or fails.

(cherry picked from commit 344d05063f)
2018-02-22 11:14:39 +00:00
Stephen Morris
8376949ea8 [rt46602] Remove option to format the printed start time of a test
There was too much incompatibility between operating systems as to
the allowed options for "date".

(cherry picked from commit 451b7309ca)
2018-02-22 11:14:39 +00:00
Stephen Morris
53ebdf225f [rt46602] Escape line ends in shell script variable assignment
Escape the line ends in a multi-line variable assignment.  Under some
circumstances, substituting the variable caused syntax errors when
used as the list of values in a shell script "for" statement.

(cherry picked from commit 462766cd76)
2018-02-22 11:14:32 +00:00
Stephen Morris
8513c4256b [rt46602] Fix EXTRAPORT substitutions in copy_setport
Fix problems where all EXTRAPORT macros were being replaced by
the EXTRAPORT1 value. (Copy and paste issue.)

(cherry picked from commit fb78f82ba1)
2018-02-22 11:13:58 +00:00
Stephen Morris
4731a82eef [rt46602] Minor documentation changes
(cherry picked from commit cdf18a5009)
2018-02-22 11:13:58 +00:00
Stephen Morris
36136dec55 [rt46602] Correct way allow_query test produces RNDC output
(cherry picked from commit 7f2d41d4c4)
2018-02-22 11:13:58 +00:00
Stephen Morris
fb5974f902 [rt46602] Simplify the cleanup code
(cherry picked from commit 0855a8a6cc)
2018-02-22 11:13:58 +00:00
Stephen Morris
56ba2e822b [rt46602] Extensions to README
Added descriptions of how nameservers are started and stopped
during the tests, and how the framework cleans up the files created.

(cherry picked from commit f606b17dfb)
2018-02-22 11:13:58 +00:00
Stephen Morris
3329876ffd [rt46602] Miscellenous small changes
(cherry picked from commit b88b508b22)
2018-02-22 11:13:58 +00:00
Stephen Morris
6602a31229 [rt46602] Handle echoing of multi-line output from tests
(cherry picked from commit 1f07c69f25)
2018-02-22 11:13:58 +00:00
Stephen Morris
7a5476173a [rt46602] Rename "test" variable to "systest"
... to avoid confusion between the variable and the command of the
same name.

(cherry picked from commit c12e06755f)
2018-02-22 11:13:58 +00:00