Add a Visual Studio build job using the "Debug" build configuration to
GitLab CI without enabling it for every pipeline as it takes about twice
as long to complete as its "Release" counterpart.
(cherry picked from commit 12564928a7)
Add a set of jobs to GitLab CI that create a BIND source tarball and
then build and test its contents. Run those extra jobs only when a tag
is pushed to the Git repository as they are only meant to be sanity
checks of BIND source tarball contents.
(cherry picked from commit 8d56749046)
The util/prepare-softhsm2.sh script is useful for initializing a working
SoftHSM environment which can be used by unit tests and system tests.
However, since it is a test-specific script, it does not really belong
in the util/ subdirectory which is mostly pruned during the BIND source
tarball creation process. Move the prepare-softhsm2.sh script to
bin/tests/ so that its location is more appropriate for its purpose and
also so that it does not get removed during the BIND source tarball
creation process, allowing it to be used for setting up test
environments for tarball-based builds.
(cherry picked from commit c0be772ebc)
Convert the logic (currently present in the form of "rm -rf" calls in
util/kit.sh) for removing files and directories which are tracked by Git
but redundant in release tarballs into a set of .gitattributes rules
which allow the same effect to be achieved using "git archive".
(cherry picked from commit 925ecb0aae)
In bin/tests/system/conf.sh.win32, the "lwresd" system test is
erroneously listed both in SEQUENTIALDIRS and PARALLELDIRS. Keep it
only in the former to prevent GitLab CI pipelines from consistently
failing on Windows due to SUBDIRS containing a different number of tests
than found in bin/tests/system/systests.output.
The autosign test has a test case where a DNSSEC maintaiend zone
has a set of DNSSEC keys without any timing metadata set. It
tests if named picks up the key for publication and signing if a
delayed dnssec-settime/loadkeys event has occured.
The test failed intermittently despite the fact it sleeps for 5
seconds but the triggered key reconfigure action should happen after
3 seconds.
However, the test output showed that the test query came in before
the key reconfigure action was complete (see excerpts below).
The loadkeys command is received:
15:38:36 received control channel command 'loadkeys delay.example.'
The reconfiguring zone keys action is triggered after 3 seconds:
15:38:39 zone delay.example/IN: reconfiguring zone keys
15:38:39 DNSKEY delay.example/NSEC3RSASHA1/7484 (ZSK) is now published
15:38:39 DNSKEY delay.example/NSEC3RSASHA1/7455 (KSK) is now published
15:38:39 writing to journal
Two seconds later the test query comes in:
15:38:41 client @0x7f1b8c0562b0 10.53.0.1#44177: query
15:38:41 client @0x7f1b8c0562b0 10.53.0.1#44177: endrequest
And 6 more seconds later the reconfigure keys action is complete:
15:38:47 zone delay.example/IN: next key event: 05-Dec-2019 15:48:39
This commit fixes the test by checking the "next key event" log has
been seen before executing the test query, making sure that the
reconfigure keys action has been complete.
This commit however does not fix, nor explain why it took such a long
time (8 seconds) to reconfigure the keys.
(cherry picked from commit 2e4273b55a)
(cherry picked from commit fb0ddd5bfe)
When libidn2 2.3.0+ is used, the "idna" system test fails as dig does
not fail as expected any more due to libidn2 2.3.0 using Unicode 11.0.0
IDNA mapping tables [1] instead of Unicode 6.3.0 ones which were used
until libidn2 2.2.0. Specifically, the SOCKS character (🧦) used in the
invalid input U-label check in the "idna" system test is a valid IDNA
character as per the IDNA mapping tables for Unicode 11.0.0. Fix the
relevant checks by using a different character (🟥) which is an invalid
IDNA character for all versions of Unicode released to date.
[1] https://gitlab.com/libidn/libidn2/commit/5eaafda64e733bfe15f66c8a39fd9bc2753ba8f2
The LC_ALL=C assignments in the "idna" system test, which were only
meant to affect a certain subset of checks, in fact persist throughout
all the subsequent checks in that system test. That affects the test's
behavior and is misleading.
When the "VARIABLE=value command ..." syntax is used in a shell script,
in order for the variable assignment to only apply to "command", the
latter must be an external binary; otherwise, the VARIABLE=value
assignment persists for all subsequent commands in a script:
$ cat foo.sh
#!/bin/sh
foo() {
/bin/sh bar.sh
}
BAR="baz0"
BAR="baz1" /bin/sh bar.sh
echo "foo: BAR=${BAR}"
BAR="baz2" foo
echo "foo: BAR=${BAR}"
$ cat bar.sh
#!/bin/sh
echo "bar: BAR=${BAR}"
$ /bin/sh foo.sh
bar: BAR=baz1
foo: BAR=baz0
bar: BAR=baz2
foo: BAR=baz2
$
Fix by saving the value of LC_ALL before the relevant set of checks in
the "idna" system test, restoring it afterwards, and dropping the
"LC_ALL=C command ..." syntax.
(cherry picked from commit 2ee7ff23ce)
The first step in all existing setup.sh scripts is to call clean.sh. To
reduce code duplication and ensure all system tests added in the future
behave consistently with existing ones, invoke clean.sh from run.sh
before calling setup.sh.
(cherry picked from commit d8905b7a9c)
Since the role of the bin/tests/system/clean.sh script has now been
reduced to calling a given system test's clean.sh script, remove the
former altogether and replace its only use with a direct invocation of
the latter.
(cherry picked from commit bf3eeac067)
Since files containing system test output are no longer stored in test
subdirectories, bin/tests/system/clean.sh no longer needs to take care
of removing the test.output file for a given test as testsummary.sh
already takes care of that and even if a test suite terminates
abnormally and another one is started, tee invoked without the -a
command line switch overwrites the destination file if it exists, so
leftover test.output.* files from previous test suite runs are not a
concern. Remove the -r command line switch and the code associated with
it from the relevant scripts.
(cherry picked from commit b4d37878f6)
Some clean.sh scripts contain overly broad file deletion wildcards which
cause the test.output file (used by the system test framework for
collecting output) in a given system test's directory to be erroneously
removed immediately after the test is started (due to setup.sh scripts
calling clean.sh at the beginning). This prevents the test's output
from being placed in bin/tests/system/systests.output at the end of a
test suite run and thus can lead to test failures being ignored. Fix by
storing each test's output in a test.output.<test-name> file in
bin/tests/system/, which prevents clean.sh scripts from removing it (as
they should only ever affect files contained in a given system test's
directory).
(cherry picked from commit b0916bba41)
At the end of each system test suite run, the system test framework
collects all existing test.output files from system test subdirectories
and produces bin/tests/system/systests.output from those files.
However, it does not check whether a test.output file was found for
every executed test. Thus, if the test.output file is accidentally
deleted by the system test itself (e.g. due to an overly broad file
removal wildcard present in clean.sh), its output will not be included
in bin/tests/system/systests.output. Since the result of each system
test suite run is determined by bin/tests/system/testsummary.sh, which
only operates on the contents of bin/tests/system/systests.output, this
can lead to test failures being ignored. Fix by ensuring the number of
test results found in bin/tests/system/systests.output is equal to the
number of tests run and triggering a system test suite failure in case
of a discrepancy between these two values.
(cherry picked from commit 3c3085be3c)
The previous code had some errors that would be triggered on platforms
without stdatomics but with support for xadd assembly instruction.
The major error was combining two uint32_t values from the
multifield atomic structure using a logical AND '&&' instead of a
bitwise OR '|'.
Some preprocessor rules were redundant and thus were simplified,
regarding the definition of ISC_STATS_USEMULTIFIELDS macro.
Correctly changed rwlock type to read on isc_stats_get_counter.
This prevents races on fctx->client whenever a new fetch joins a existing
fetch (by calling fctx_join) as it is now invariant for the active life of
fctx.
(cherry picked from commit 9ca6ad6311)
xmlInitThreads() and xmlCleanupThreads() are called from within
ns_statschannels_configure() and ns_statschannels_shutdown(),
respectively. Both of these functions are executed by worker threads,
not the main named thread. This causes ASAN to report memory leaks like
the following one upon shutdown (as long as named is asked to produce
any XML output over its configured statistics channels during its
lifetime):
Direct leak of 968 byte(s) in 1 object(s) allocated from:
#0 0x7f677c249cd8 in __interceptor_calloc /build/gcc/src/gcc/libsanitizer/asan/asan_malloc_linux.cc:153
#1 0x7f677bc1838f in xmlGetGlobalState (/usr/lib/libxml2.so.2+0xa838f)
The data mentioned in the above report is a libxml2 state structure
stored as thread-specific data. Such chunks of memory are automatically
released (by a destructor passed to pthread_key_create() by libxml2)
whenever a thread that allocated a given chunk exits. However, if
xmlCleanupThreads() is called by a given thread before it exits, the
destructor will not be invoked (due to xmlCleanupThreads() calling
pthread_key_delete()) and ASAN will report a memory leak. Thus,
xmlInitThreads() and xmlCleanupThreads() must not be called from worker
threads. Since xmlInitThreads() must be called on Windows in order for
libxml2 to work at all, move xmlInitThreads() and xmlCleanupThreads()
calls to the main named thread (which does not produce any XML output
itself) in order to prevent the memory leak from being reported by ASAN.
(cherry picked from commit b425b5d56e)
Loaded GeoIP2 databases are only released when named is shut down, but
not during server reconfiguration. This causes memory to be leaked
every time "rndc reconfig" or "rndc reload" is used, as long as any
GeoIP2 database is in use. Fix by releasing any loaded GeoIP2 databases
before reloading them. Do not call dns_geoip_shutdown() until server
shutdown as that function releases the memory context used for caching
GeoIP2 lookup results.
(cherry picked from commit 670afbe84a)
Michał Kępień
Mark Andrews
Matthijs Mekking
Diego dos Santos Fronza