Commit Graph
37783 Commits
Author SHA1 Message Date
Michal Nowak 46e98810d7 Add Alpine Linux 3.18
(cherry picked from commit ddb846454d)
2023-05-31 12:03:52 +02:00
Michal Nowak b751e2b4be Merge branch 'mnowak/look-for-core-files-in-TOP_BUILDDIR-9.18' into 'bind-9.18'
[9.18] Look for core files in $TOP_BUILDDIR

See merge request isc-projects/bind9!7986
2023-05-30 20:27:33 +00:00
Michal Nowak 2476d43acf Look for core files in $TOP_BUILDDIR
The get_core_dumps.sh script couldn't find and process core files of
out-of-tree configurations because it looked for them in the source
instead of the build directory.

(cherry picked from commit a13448a769)
2023-05-30 21:31:41 +02:00
Michal Nowak 99e910e4b9 Merge branch 'mnowak/custom-userspace-rcu-library-9.18' into 'bind-9.18'
[9.18] Change images for TSAN jobs

See merge request isc-projects/bind9!7987
2023-05-30 19:30:02 +00:00
Michal Nowak 44fff18b68 Change images for TSAN jobs
Fedora 38 and Debian "bullseye" images were "forked" to images used only
for TSAN CI jobs. The new images contain TSAN-aware liburcu that does
not fit well with ASAN CI jobs for which original images were also used.

liburcu is not used in this branch, but images are shared among
branches, and their use needs to be consistent in all maintained
branches.

(cherry picked from commit 04dda8661f)
2023-05-30 20:35:12 +02:00
Tom Krizek 4f3cfba6c0 Merge branch 'tkrizek-fix-pytest-base-port-9.18' into 'bind-9.18'
[9.18] Fix base_port calculation in pytest runner

See merge request isc-projects/bind9!7983
2023-05-30 15:37:37 +00:00
Tom Krizek 1b8f0711f2 Fix base_port calculation in pytest runner
The selected base port should be in the range <port_min, port_max), the
formula was incorrect.

Credit for discovering this fault goes to Ondrej Sury.

(cherry picked from commit e8ea6b610b)
2023-05-30 15:37:29 +02:00
Matthijs Mekking 076b8363fc Merge branch '3950-serve-stale-strikes-again-9.18' into 'bind-9.18'
[9.18] Fix serve-stale bug when cache has no data

See merge request isc-projects/bind9!7909
2023-05-30 13:15:13 +00:00
Matthijs Mekking cbe0cddcd4 Add release note and changes for #3950
Fixing another serve-stale bug is still news.

(cherry picked from commit 23dbb6ba72)
2023-05-30 13:46:34 +02:00
Matthijs Mekking b90bad93cb Fix serve-stale bug when cache has no data
We recently fixed a bug where in some cases (when following an
expired CNAME for example), named could return SERVFAIL if the target
record is still valid (see isc-projects/bind9#3678, and
isc-projects/bind9!7096). We fixed this by considering non-stale
RRsets as well during the stale lookup.

However, this triggered a new bug because despite the answer from
cache not being stale, the lookup may be triggered by serve-stale.
If the answer from database is not stale, the fix in
isc-projects/bind9!7096 erroneously skips the serve-stale logic.

Add 'answer_found' checks to the serve-stale logic to fix this issue.

(cherry picked from commit bbd163acf6)
2023-05-30 13:46:00 +02:00
Matthijs Mekking ad5d447348 Add serve-stale test case for GL #3950
Add a test case where when priming the cache with a slow authoritative
resolver, the stale-answer-client-timeout option should not return
a delegation to the client (it should wait until an applicable answer
is found, if no entry is found in the cache).

(cherry picked from commit c3d4fd3449)
2023-05-30 13:45:54 +02:00
Ondřej Surý 2a498d944a Merge branch '4098-remove-cruft-epoll-kqueue-configure-options-9.18' into 'bind-9.18'
[9.18] Remove obsolete epoll/kqueue/devpoll configure options

See merge request isc-projects/bind9!7975
2023-05-29 06:07:16 +00:00
Ondřej Surý 4fb2c9568d Add CHANGES note for [GL #4098]
(cherry picked from commit 0266760fdd)
2023-05-29 07:58:51 +02:00
Ondřej Surý 6b6076c882 Remove obsolete epoll/kqueue/devpoll configure options
Since we don't use networking directly but rather via libuv, these
configure options were no-op.  Remove the configure checks for epoll
(Linux), kqueue (BSDs) and /dev/poll (Solaris).

(cherry picked from commit 051f3d612f)
2023-05-29 07:58:03 +02:00
Mark Andrews aca974dc29 Merge branch '4090-corrected-bad-insist-logic-in-isc_radix_remove-bind-9.18' into 'bind-9.18'
[9.18] Resolve "Corrected bad INSIST logic in isc_radix_remove()"

See merge request isc-projects/bind9!7974
2023-05-29 04:42:09 +00:00
Mark Andrews eb52c30524 Add regression test for [GL # 4090]
These insertions are added to produce a radix tree that will trigger
the INSIST reported in [GL #4090].  Due to fixes added since BIND 9.9
an extra insert in needed to ensure node->parent is non NULL.

(cherry picked from commit 03ebe96110)
2023-05-29 13:27:51 +10:00
Mark Andrews 27eb8ed20f Move isc_mem_put to after node is checked for equality
isc_mem_put NULL's the pointer to the memory being freed.  The
equality test 'parent->r == node' was accidentally being turned
into a test against NULL.

(cherry picked from commit ac2e0bc3ff)
2023-05-29 13:27:51 +10:00
Evan Hunt 9a1d565f07 Merge branch '3905-deprecate-tkey-dhkey-v9_18' into 'bind-9.18'
mark 'tkey-dhkey' as deprecated

See merge request isc-projects/bind9!7956
2023-05-28 08:07:25 +00:00
Evan Hunt 9a8f8d6046 CHANGES and release note for [GL #3905] 2023-05-28 00:55:55 -07:00
Evan Hunt 88383aa158 mark 'tkey-dhkey' as deprecated
Diffie-Hellman TKEY mode has been removed for 9.20.
2023-05-28 00:55:34 -07:00
Artem Boldariev 97d672d368 Merge branch '4091-syncrhonise-access-to-the-client-tlsctx-cache-9.18' into 'bind-9.18'
[9.18] ZMGR: TLS contexts cache - properly synchronise access

See merge request isc-projects/bind9!7972
2023-05-26 14:11:40 +00:00
Artem Boldariev cec8947bc1 ZMGR: TLS contexts cache - properly synchronise access
This commit ensures that access to the TLS context cache within zone
manager is properly synchronised.

Previously there was a possibility for it to get unexpectedly
NULLified for a brief moment by a call to
dns_zonemgr_set_tlsctx_cache() from one thread, while being accessed
from another (e.g. from got_transfer_quota()). This behaviour could
lead to server abort()ing on configuration reload (under very rare
circumstances).

That behaviour has been fixed.

(cherry picked from commit 0b95cf74ff)
2023-05-26 15:24:51 +03:00
Michal Nowak 207f9bb1c3 Merge branch 'mnowak/gitlab-runner-autoscaling-9.18' into 'bind-9.18'
[9.18] Run most Docker CI jobs in AWS with autoscaler

See merge request isc-projects/bind9!7969
2023-05-26 09:54:12 +00:00
Michal Nowak 94d83b7960 Run most Docker CI jobs in AWS with autoscaler
All but the "respdiff-long" job, for which our AWS instances do not have
enough memory, are now being spawned in the AWS by the autoscaler
executor.

(cherry picked from commit f09cf69594)
2023-05-26 11:47:20 +02:00
Evan Hunt 59827b21d1 Merge branch '4072-tcp-dispatch-timeout-bind-9.18' into 'bind-9.18'
[9.18] fix handling of TCP timeouts

See merge request isc-projects/bind9!7968
2023-05-26 09:32:22 +00:00
Evan Hunt e9b6991357 fix handling of TCP timeouts
when a TCP dispatch times out, we call tcp_recv() with a result
value of ISC_R_TIMEDOUT; this cancels the oldest dispatch
entry in the dispatch's active queue, plus any additional entries
that have waited longer than their configured timeouts. if, at
that point, there were more dispatch entries still on the active
queue, it resumes reading, but until now it failed to restart
the timer.

this has been corrected: we now calculate a new timeout
based on the oldest dispatch entry still remaining.  this
requires us to initialize the start time of each dispatch entry
when it's first added to the queue.

in order to ensure that the handling of timed-out requests is
consistent, we now calculate the runtime of each dispatch
entry based on the same value for 'now'.

incidentally also fixed a compile error that turned up when
DNS_DISPATCH_TRACE was turned on.

(cherry picked from commit 0e800467ee)
2023-05-26 02:07:02 -07:00
Evan Hunt 7045f33d45 Merge branch '4079-multiple-keyrings-bind-9.18' into 'bind-9.18'
[9.18] prevent TSIG keys from being added to multiple rings

See merge request isc-projects/bind9!7965
2023-05-25 23:43:36 +00:00
Evan Hunt 913fce98bf CHANGES for [GL #4079] 2023-05-25 15:08:59 -07:00
Evan Hunt afae1b65e8 prevent TSIG keys from being added to multiple rings
it was possible to add a TSIG key to more than one TSIG
keyring at a time, and this was in fact happening with the
session key, which was generated once and then added to the
keyrings for each view as it was configured.

this has been corrected and a REQUIRE added to dns_tsigkeyring_add()
to prevent it from happening again.
2023-05-25 15:08:48 -07:00
Arаm Sаrgsyаn 911f3af2fb Merge branch '3765-interfacemgr-use-after-afree-on-shutdown-9.18' into 'bind-9.18'
[9.18] Fix an interfacemgr use-after-free error in zoneconf.c:isself()

See merge request isc-projects/bind9!7964
2023-05-25 11:42:18 +00:00
Aram Sargsyan 685d9ece61 Add a CHANGES note for [GL #3765]
(cherry picked from commit b409db08cf)
2023-05-25 08:39:20 +00:00
Aram Sargsyan 1d736cf451 Fix an interfacemgr use-after-free error in zoneconf.c:isself()
The 'named_g_server->interfacemgr' pointer is saved in the zone
structure using dns_zone_setisself(), as a void* argument to be
passed to the isself() callback, so there is no attach/detach,
and when shutting down, the interface manager can be destroyed
by the shutdown_server(), running in exclusive mode, and causing
isself() to crash when trying to use the pointer.

Instead of keeping the interface manager pointer in the zone
structure, just check and use the 'named_g_server->interfacemgr'
itself, as it was implemented originally in the
3aca8e5bf3 commit. Later, in the
8eb88aafee commit, the code was
changed to pass the interface manager pointer using the additional
void* argument, but the commit message doesn't mention if there
was any practical reason for that.

Additionally, don't pass the interfacemgr pointer to the
ns_interfacemgr_getaclenv() function before it is checked
against NULL.

(cherry picked from commit 64d69e4adc)
2023-05-25 08:39:20 +00:00
Tom Krizek 66254cf56d Merge branch 'tkrizek/system-tests-pytest-runner-9.18' into 'bind-9.18'
[9.18] System tests: support using pytest instead of run.sh

See merge request isc-projects/bind9!7959
2023-05-23 16:02:15 +00:00
Tom Krizek 81b7682e24 Reorder dead primary checks in upforwd test
The check which attempts to forward dynamic update to a dead primary may
trigger a timing issue #4080. For some reason, this has manifested under
the pytest runner, while the test still passes with the legacy runner.

Move the dead primary check closer to the end of the test to avoid
hitting this issue before we have a proper fix.

(cherry picked from commit edaa5f5d2a)
2023-05-23 17:41:35 +02:00
Tom Krizek b9b3b53e2c Add CHANGES and release note for [GL #3978]
(cherry picked from commit fd889bf0ad)
2023-05-23 17:41:35 +02:00
Tom Krizek f3c0005257 Tear down module logger handler in system tests
The module-level logger has a handler that writes into a temporary
directory. Ensure the logging output is flushed and the handler is
closed before attempting to remove this temporary directory.

(cherry picked from commit 0f8a2b07a4)
2023-05-23 17:41:35 +02:00
Tom Krizek 6811f9704a Rewrite run.sh to invoke pytest in a system test directory
Previously, run.sh tried to use pytest's -k option for test selection.
The downside was that this filter expression matched any test case with
the given substring, rather than executing a system test suite with the
given name.

The run.sh has been rewritten to invoke pytest from a system test
directory instead. This behaves more consistently with the run.sh from
legacy system test framework.

run.sh is now also a shell script to avoid confusion regarding its
file extension.

(cherry picked from commit 1aaefc9cf4)
2023-05-23 17:41:34 +02:00
Tom Krizek de206d6436 Add .log.txt to gitignore
It can be useful to append the .txt extension to logs. When this
extension is used, GitLab is able to set the proper content type on such
artifacts in CI. This makes it possible to display those files directly
in the browser rather than having to download them.

(cherry picked from commit 8d40156bb2)
2023-05-23 17:41:34 +02:00
Tom Krizek e2f7dbcbd1 Remove "which" declaration from env vars in EL8+ tests
EL8+ systems declare "which" function using environment variables in the
/etc/profile.d/which2.sh file. Because of our suboptimal environment
variable detection, which is required in order to support the legacy
runner, these variables are picked up by the pytest runner.

If subprocesses are spawned with these environment variables set, it
will cause the following issue when they spawn yet another subprocess:

/bin/sh: which: line 1: syntax error: unexpected end of file
/bin/sh: error importing function definition for `which'

(cherry picked from commit 68b1c6877b)
2023-05-23 17:41:34 +02:00
Tom Krizek 5b5210854d Capture log output during pytest runner setup
Instantiate a new logger that is used during pytest initialization /
configuration. This logging isn't handled by pytest itself, since it
happens outside of any tests or fixtures.

Root logger can't be reused for this purpose, because that would
duplicate the logs. Instead, create a conftest-specific logger for this
purpose.

Unfortunately, this introduces another log file,
pytest.conftest.log.txt, which contains only the logging from pytest
initialization. However, unless one is debugging the runner /
environment, there should be no need to investigate this file.

(cherry picked from commit 12c724ee07)
2023-05-23 17:41:34 +02:00
Tom Krizek 16eeb36432 Execute long running system tests first
In order to take the most advantage of parallel execution of tests,
ensure certain long running tests are scheduled first.

The list of tests considered long-running was created empirically. In
addition to the test run time, its position in the default
(alphabetical) ordering was also taken into account.

(cherry picked from commit 99e2e50c0e)
2023-05-23 17:41:34 +02:00
Tom Krizek b9596d2107 Disable pylint check for too-few-public-methods
This check is overly aggressive and not really useful, especially for
non-Python codebase, where the primary use of Python is for testing.

(cherry picked from commit 6f134283eb)
2023-05-23 17:41:34 +02:00
Tom Krizek 1acb8a3f39 Add test specific logger for pytests
The logger fixture is provided as a test-level logging facility which
can be easily passed to tests to enable capturing and/or displaying
messages from tests written in Python.

While this works optimally with the pytest runner, messages on INFO
level or above will also be visible when using the legacy runner.

(cherry picked from commit 952776b61f)
2023-05-23 17:41:34 +02:00
Tom Krizek 862f6e043a Mark selected statschannel tests as xfail
The test_zone_timers_secondary_json() and
test_zone_timers_secondary_xml() tests are affected by issue #3983. Due
to the way tests are run, they are only affected when executing them
with the pytest runner.

Strict mode is set for pytest runner, as it always fails there. The
strict mode ensures we'll catch the change when the it starts passing
once the underlying issue is fixed. It can't be set for the legacy
runner, since the test (incorrectly) passes there.

Related #3983

(cherry picked from commit 087a9b3c97)
2023-05-23 17:41:34 +02:00
Tom Krizek f0db1c27ba Ensure assertions and exceptions end up in system test log
If a test fails with an assertion failure or exception, its content
along with traceback is displayed in pytest output. This information
should be preserved in the test-specific logger for a given system test
to make it easier to debug test failures.

(cherry picked from commit 3a20e8d990)
2023-05-23 17:41:34 +02:00
Petr ŠpačekandTom Krizek 0277c5098c Use raw byte format of env variables in pytest
In order to avoid issues with decoding/encoding env variables due to
different encodings on different systems, deal with the environment
variables directly as bytes.

(cherry picked from commit 37ed9ad2f8)
2023-05-23 17:41:33 +02:00
Tom Krizek 23b2777067 Display pytest failures for system:gcc:tarball
Since the tarball build&tests happen in a subdirectory, ensure the
after_script switches to it, so artifacts can be found properly.

(cherry picked from commit a6559176f1)
2023-05-23 17:41:33 +02:00
Tom Krizek 445b30b883 Handle missing test_results due to pytest runner interrupt
If pytest execution is interrupted, the hook that exposes test_results
to the pytest session is never called so the results can't be
interpreted.

(cherry picked from commit 0a063f51d3)
2023-05-23 17:41:33 +02:00
Tom Krizek 6103ee988e Run system tests sequentially if xdist is not available
(cherry picked from commit 1cc55d01c7)
2023-05-23 17:41:33 +02:00
Petr ŠpačekandTom Krizek 8627d41d4d Include logs from failing tests in JUnit output
(cherry picked from commit 8952618262)
2023-05-23 17:41:33 +02:00