Commit Graph

37899 Commits

Author SHA1 Message Date
Tom Krizek
6f07ba6b7d Use arithmetic expansion in system tests
Change the way arithmetic operations are performed in system test shell
scripts from using `expr` to `$(())`. This ensures that updating the
variable won't end up with a non-zero exit code, which would case the
script to exit prematurely when `set -e` is in effect.

The following replacements were performed using sed in all text files
(git grep -Il '' | xargs sed -i):

s/status=`expr $status + $ret`/status=$((status + ret))/g
s/n=`expr $n + 1`/n=$((n + 1))/g
s/t=`expr $t + 1`/t=$((t + 1))/g
s/status=`expr $status + 1`/status=$((status + 1))/g
s/try=`expr $try + 1`/try=$((try + 1))/g

(manually picked from commit 4d42bdc245)
2023-07-17 14:12:59 +02:00
Tom Krizek
d715bae429 Run system tests with set -e
Ensure all shell system tests are executed with the errexit option set.
This prevents unchecked return codes from commands in the test from
interfering with the tests, since any failures need to be handled
explicitly.

(cherry picked from commit 01bc805f89)
2023-07-17 13:46:19 +02:00
Michal Nowak
7abd0723fc Merge branch 'mnowak/detect-core-dumps-after-server-failed-to-stop-9.18' into 'bind-9.18'
[9.18] Detect core dumps after BIND failed to stop cleanly

See merge request isc-projects/bind9!8100
2023-07-13 13:52:34 +00:00
Michal Nowak
577f0723e0 Detect core dumps after BIND failed to stop cleanly
With the pytest runner, when BIND crashed during test runtime, the
get_core_dumps.sh script hasn't been run, and core dumps were not
detected.

(cherry picked from commit 89c77daddb)
2023-07-13 15:14:47 +02:00
Tom Krizek
80d4b602a7 Merge branch 'tkrizek/set-up-version-and-release-notes-for-bind-9.18.18' into 'bind-9.18'
Set up version and release notes for BIND 9.18.18

See merge request isc-projects/bind9!8093
2023-07-10 13:45:28 +00:00
Tom Krizek
e472710cac Set up release notes for BIND 9.18.18 2023-07-10 15:39:38 +02:00
Tom Krizek
77271086a2 Update BIND version to 9.18.18-dev 2023-07-10 15:39:37 +02:00
Arаm Sаrgsyаn
7428653012 Merge branch '4171-add-shutdown-check-in-dns_catz_dbupdate_callback-9.18' into 'bind-9.18'
[9.18] Add shutdown checks in dns_catz_dbupdate_callback()

See merge request isc-projects/bind9!8089
2023-07-06 12:32:11 +00:00
Aram Sargsyan
d753174392 Add a CHANGES note for [GL #4171]
(cherry picked from commit 8887da60ad)
2023-07-06 11:27:50 +00:00
Aram Sargsyan
4fdb57a1f3 Add shutdown checks in dns_catz_dbupdate_callback()
When a zone database update callback is called, the 'catzs' object,
extracted from the callback argument, might be already shutting down,
in which case the 'catzs->zones' can be NULL and cause an assertion
failure when calling isc_ht_find().

Add an early return from the callback if 'catzs->shuttingdown' is true.

Also check the validity of 'catzs->zones' after locking 'catzs' in
case there is a race with dns_catz_shutdown_catzs() running in another
thread.

(cherry picked from commit 28bb419edc)
2023-07-06 11:27:45 +00:00
Arаm Sаrgsyаn
e1529a6a01 Merge branch '4132-catz-data-race-in-db.c-9.18' into 'bind-9.18'
[9.18] Fix a data race between the dns_zone and dns_catz modules

See merge request isc-projects/bind9!8036
2023-07-06 11:25:37 +00:00
Ondřej Surý
26bb402b44 Run RPZ and catalog zones tasks in exclusive mode
All the heavy RPZ and CATZ work is already running with offloaded
threads, and running the remaining small functions in exclusive mode
offers more synchronization guaranties.

Move the update notify registration code from the offloaded
dns__catz_update_cb() function into dns__catz_done_cb().

After this change, it should be safe to remove the lock/unlock code
from the dns_catz_dbupdate_register() and dns_catz_dbupdate_unregister()
functions, as they were causing a benign TSAN lock-order-inversion
report.
2023-07-06 10:44:03 +00:00
Aram Sargsyan
471a2c0dd5 Add a CHANGES note for [GL #4132]
(cherry picked from commit f73007afe7)
2023-07-06 10:44:03 +00:00
Aram Sargsyan
c67ce97045 Fix a data race between the dns_zone and dns_catz modules
The dns_zone_catz_enable_db() and dns_zone_catz_disable_db()
functions can race with similar operations in the catz module
because there is no synchronization between the threads.

Add catz functions which use the view's catalog zones' lock
when registering/unregistering the database update notify callback,
and use those functions in the dns_zone module, instead of doing it
directly.

(cherry picked from commit 6f1f5fc307)
2023-07-06 10:44:03 +00:00
Matthijs Mekking
b78a555078 Merge branch '2471-test-three-is-a-crowd-v9_18' into 'bind-9.18'
[9.18] Add "three is a crowd" test case

See merge request isc-projects/bind9!8088
2023-07-06 10:22:40 +00:00
Matthijs Mekking
9cb0a2731d Add test for "three is a crowd" bug (GL #2375)
Add this test scenario for a bug fixed a while ago. When a third key is
introduced while the previous rollover hasn't finished yet, the keymgr
could decide to remove the first two keys, because it was not checking
for an indirect dependency on the keys.

In other words, the previous bug behavior was that the first two keys
were removed from the zone too soon.

This test case checks that all three keys stay in the zone, and no keys
are removed premature after another new key has been introduced.

(cherry picked from commit 9c40cf0566)
2023-07-06 10:57:54 +02:00
Matthijs Mekking
6859294c26 Check all keys despite early failure
In the kasp script, if one expected key is not found, continue checking
the other key ids, even if there is no match for the first one.  This
provides a bit more information which keys mismatch and makes for
easier debugging test failures.

(cherry picked from commit 674249f66a)
2023-07-06 10:57:09 +02:00
Evan Hunt
3955457c95 Merge branch 'each-refactor-fetchopt-bind-9.18' into 'bind-9.18'
[9.18] clean up numbering of FETCHOPT and ADDRINFO flags

See merge request isc-projects/bind9!8084
2023-07-04 19:36:50 +00:00
Evan Hunt
995b78ea4e clean up numbering of FETCHOPT and ADDRINFO flags
in the past there was overlap between the fields used
as resolver fetch options and ADB addrinfo flags. this has
mostly been eliminated; now we can clean up the rest of
it and remove some confusing comments.

(cherry picked from commit 0955cf1af5)
2023-07-04 11:58:09 -07:00
Tom Krizek
6db34b6ebb Merge branch '4126-statschannel-connection-close-keepalive-9.18' into 'bind-9.18'
[9.18] Improve statschannel HTTP Connection: header logic

See merge request isc-projects/bind9!8082
2023-07-04 13:02:57 +00:00
Tony Finch
1ddf2b87f5 Improve statschannel HTTP Connection: header protocol conformance
In HTTP/1.0 and HTTP/1.1, RFC 9112 section 9.6 says the last response
in a connection should include a `Connection: close` header, but the
statschannel server omitted it.

In an HTTP/1.0 response, the statschannel server can sometimes send a
`Connection: keep-alive` header when it is about to close the
connection. There are two ways:

If the first request on a connection is keep-alive and the second
request is not, then _both_ responses have `Connection: keep-alive`
but the connection is (correctly) closed after the second response.

If a single request contains

	Connection: close
	Connection: keep-alive

then RFC 9112 section 9.3 says the keep-alive header is ignored, but
the statschannel sends a spurious keep-alive in its response, though
it correctly closes the connection.

To fix these bugs, make it more clear that the `httpd->flags` are part
of the per-request-response state. The Connection: flags are now
described in terms of the effect they have instead of what causes them
to be set.

(manually picked from commit e18ca83a3b)
2023-07-04 14:53:08 +02:00
Tom Krizek
56546ecf20 Merge branch 'tkrizek-shutdown-rndc-hang-9.18' into 'bind-9.18'
[9.18] Split shutdown test into separate test cases

See merge request isc-projects/bind9!8083
2023-07-04 12:43:45 +00:00
Tom Krizek
184eddb595 Use timeout for rndc status in shutdown test
Pass 5 second timeout to the rndc status command(s) to avoid hitting the
hard 10 second timeout from subprocess.call, which would result in an
unwanted exception that would only mask the real issue: if the rndc
status times out in this test, it is likely due to the server not
stopping as it should.

(cherry picked from commit ceed694659)
2023-07-04 14:36:05 +02:00
Tom Krizek
c7b0ac0367 Split shutdown test into separate test cases
The shutdown test attempts to shut down the server using two different
methods - rndc and sigterm. Use pytest.mark.parametrize to run these as
separate test cases for easier identification of failures.

(cherry picked from commit 603c58ee28)
2023-07-04 14:36:03 +02:00
Michał Kępień
0c23b13291 Merge branch '4169-add-google-site-verification-tag-to-the-toc-page-9.18' into 'bind-9.18'
[9.18] Add Google Site Verification tag to the TOC page

See merge request isc-projects/bind9!8077
2023-06-29 16:49:44 +00:00
Michał Kępień
83e811f518 Add Google Site Verification tag to the TOC page
This should allow tracking HTTP 404 errors for Read the Docs using the
Google Search Console.

(cherry picked from commit 13c35ab0b3)
2023-06-29 18:48:31 +02:00
Tom Krizek
5cd3051793 Merge branch 'tkrizek-disable-delv-in-tsan-9.18' into 'bind-9.18'
[9.18] Disable delv tests under TSAN

See merge request isc-projects/bind9!8071
2023-06-29 12:33:35 +00:00
Tom Krizek
4dcb0c67d3 Check for unset variables only after conf.sh is loaded
Make the cds/setup.sh compatible with the workaround which relies on
testing the TSAN_OPTIONS variable which may not be set.

(cherry picked from commit 76d9873ef6)
2023-06-29 13:14:33 +02:00
Tom Krizek
b94f5146a1 Fix checking for executables in shell conditions in tests
Surround the variables which are checked whether they're executable in
double quotes. Without them, empty paths won't be properly interpreted
as not executable.

(manually picked from commit 06056c44a7)
2023-06-29 13:14:26 +02:00
Tom Krizek
cfada0f62e Only use delv if available in mkeys test
Check that $DELV is an executable before using it in a test.

(cherry picked from commit 384339dbba)
2023-06-29 13:00:26 +02:00
Tom Krizek
6eeec896ef Disable delv tests under TSAN
Since delv can occasionally hang in system tests when running with TSAN
(see GL#4119), disable these tests as a workaround. Otherwise, the hung
delv process will just waste CI resources and prevent any meaningful
output from the rest of the test suite.

(cherry picked from commit fbcf37f914)
2023-06-29 13:00:24 +02:00
Mark Andrews
1eed6e28c8 Merge branch '4154-restore-the-ability-to-read-old-hmac-md5-key-pair-files-bind-9.18' into 'bind-9.18'
[9.18] Resolve "Restore the ability to read old HMAC-MD5 key pair files."

See merge request isc-projects/bind9!8069
2023-06-29 01:38:11 +00:00
Mark Andrews
c73876fa90 Emit deprecated warning for K* file pairs
We try reading the same file using different methods so only
emit a warning if we successfully read the file.

(cherry picked from commit e3e20ed76e)
2023-06-29 10:52:48 +10:00
Mark Andrews
3db353c646 Add release note entry for [GL #4154]
(cherry picked from commit 91cb94dcbb)
2023-06-29 10:52:48 +10:00
Mark Andrews
16d6b10c5b Test legacy HMAC key files with dig
tsig-keygen is now used to generate key files for TSIG.  These have
a different format to those that were generated by dnssec-keygen.
Test that dig can still read these files.

(cherry picked from commit c91a208ef5)
2023-06-29 10:52:46 +10:00
Mark Andrews
a16e10ad8a Test support with legacy HMAC K files with nsupdate
tsig-keygen generates key files that are different to those that
where generated by dnssec-keygen.  Check that nsupdate can still
read those old format files.

(cherry picked from commit e1fb17e72c)
2023-06-29 10:41:45 +10:00
Mark Andrews
2376abc18e Restore the ability to read legacy K*+157+* files
The ability to read legacy HMAC-MD5 K* keyfile pairs using algorithm
number 157 was accidentally lost when the algorithm numbers were
consolidated into a single block, in commit
09f7e0607a.

The assumption was that these algorithm numbers were only known
internally, but they were also used in key files. But since HMAC-MD5
got renumbered from 157 to 160, legacy HMAC-MD5 key files no longer
work.

Move HMAC-MD5 back to 157 and GSSAPI back to 160.  Add exception for
GSSAPI to list_hmac_algorithms.

(cherry picked from commit 3f93d3f757)
2023-06-29 10:32:10 +10:00
Ondřej Surý
068d14a963 Merge branch '3325-strict-qname-minimisation-should-check-for-cached-nodata-for-ns-name-9.18' into 'bind-9.18'
[9.18] Use NS rather than A records for qname-minimization relaxed

See merge request isc-projects/bind9!8066
2023-06-28 11:18:41 +00:00
Mark Andrews
ae853dd2b6 Add release note for [GL #3325]
(cherry picked from commit 2a71ed47a7)
2023-06-28 12:33:18 +02:00
Mark Andrews
1e7f5eb161 Add CHANGES note for [GL #3325]
(cherry picked from commit f378e02423)
2023-06-28 12:32:50 +02:00
Mark Andrews
5739b4817a In rctx_answer return DNS_R_DELEGATION on NOFOLLOW
When DNS_FETCHOPT_NOFOLLOW is set DNS_R_DELEGATION needs to be
returned to restart the resolution process rather than converting
it to ISC_R_SUCCESS.

(cherry picked from commit ea11650376)
2023-06-28 12:32:26 +02:00
Mark Andrews
7f2eeb60ee Skip some QNAME mininisation queries if possible
If we know that the NS RRset for an intermediate label doesn't exist
on cache contents don't query using that name when looking for a
referral.

(cherry picked from commit 80bc0ee075)
2023-06-28 12:32:23 +02:00
Mark Andrews
b3a97da7a7 Use NS rather than A records for qname-minimization relaxed
Remove all references to DNS_FETCHOPT_QMIN_USE_A and adjust
the expected tests results in the qmin system test.

(cherry picked from commit dd00b3c50b)
2023-06-28 12:31:49 +02:00
Mark Andrews
0eb73bb6a3 Merge branch '4047-assertion-failure-in-dns_resolver_attach-at-resolver-c-10599' into 'bind-9.18'
Resolve "Assertion failure in dns_resolver_attach() at resolver.c:10599"

See merge request isc-projects/bind9!8045
2023-06-27 06:53:41 +00:00
Mark Andrews
0d3693f08f Remove unnecessary REQUIRE in dns_resolver_attach
There is no harm in aquiring an additional reference to the resolver
after it has started shutting down.  All the REQUIRE was doing was
introducing a point of failure when shutting down the server.
2023-06-27 05:19:56 +00:00
Evan Hunt
adf199b309 Merge branch '950-test-validation-bind-9.18' into 'bind-9.18'
[9.18] explicitly set dnssec-validation in system tests

See merge request isc-projects/bind9!8063
2023-06-27 00:18:29 +00:00
Evan Hunt
a31dd61b0f explicitly set dnssec-validation in system tests
the default value of dnssec-validation is 'auto', which causes
a server to send a key refresh query to the root zone when starting
up. this is undesirable behavior in system tests, so this commit
sets dnssec-validation to either 'yes' or 'no' in all tests where
it had not previously been set.

this change had the mostly-harmless side effect of changing the cached
trust level of unvalidated answer data from 'answer' to 'authanswer',
which caused a few test cases in which dumped cache data was examined in
the serve-stale system test to fail. those test cases have now been
updated to expect 'authanswer'.

(cherry picked from commit 0b09ee8cdc)
2023-06-26 15:03:06 -07:00
Ondřej Surý
f6ee78c61a Merge branch '4049-detect-formerr-with-an-echoed-dns-cookie-client-cookie-and-retry-without-dns-cookie-9.18' into 'bind-9.18'
[9.18] Handle FORMERR on unknown EDNS option that are echoed

See merge request isc-projects/bind9!8061
2023-06-26 17:05:44 +00:00
Michał Kępień
74ef08a7b5 Add release note for #4049
(cherry picked from commit 466530c73b)
2023-06-26 18:52:37 +02:00
Mark Andrews
20db7497b1 Add CHANGES note for [GL #4049]
(cherry picked from commit 97e1bb0e98)
2023-06-26 16:36:36 +02:00