The changelog job is supposed to test that the text from GitLab MR
title&description is valid rst syntax and can be built with sphinx. In
49128fc1, the way gitchangelog generates entries was changed - it no
longer writes to the changelog file, but generates output on stdout
instead. Ensure the generated notes is actually written to (some)
rendered file which is part of the docs so that the subsequent sphinx
build attempts to render the note.
(cherry picked from commit 380a30ba8d)
Add support for EDE codes 1 & 2 which might occurs during DNSSEC validation in case of unsupported RRSIG algorithm or DNSKEY digest.
See #2715
Backport of MR !9948
Merge branch 'backport-2715-ede-unsupported-digest-alg-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9996
A DNSSEC validation can fail in the case where multiple DNSKEY are
available for a zone and none of them are supported, but for different
reasons: one has a DS record in the parent zone using an unsupported
digest while the other one uses an unsupported encryption algorithm.
Add a specific test case covering this flow and making sure that two
extended DNS error are provided: code 1 and 2, each of them highlighting
unsupported algorithm and digest.
(cherry picked from commit 244923b9dc)
Add support for EDE codes 1 (Unsupported DNSKEY Algorithm) and 2
(Unsupported DS Digest Type) which might occurs during DNSSEC
validation in case of unsupported DNSKEY algorithm or DS digest type.
Because DNSSEC internally kicks off various fetches, we need to copy
all encountered extended errors from fetch responses to the fetch
context. Upon an event, the errors from the fetch context are copied
to the client response.
(cherry picked from commit 46a58acdf5)
The minimal dnspython version to run this test is 2.5.0.
Backport of MR !8662
Merge branch 'backport-mnowak/pytest_rewrite_cipher-suites-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9992
Marks starting with "with" or "without" make more sense linguistically
than those starting with "have" or "have_not".
(cherry picked from commit df7e9f4ac3)
Ensure the zone transfers have completed (successfully or not) before
running the test cases, because they assume zone transfers have been
done.
(cherry picked from commit 23fb615963)
The servers are setup and torn down once per each test module. All the
logs and server state persists between individual tests within the same
module. The servers fixture representing these servers should be
module-wide as well.
(cherry picked from commit a72ff9fd57)
The minimal required dnspython version is 2.5.0 because of the need for
the "verify" argument in dns.query.tls().
(cherry picked from commit 100b759863)
The base image tends to have a rather old dnspython version and when
used with pylint and mypy it produces errors about newer dnspython
features the old version does not know about.
$ mypy "bin/tests/system/isctest/"
bin/tests/system/isctest/query.py:55: error: Unexpected keyword argument "verify" for "tls" [call-arg]
/usr/lib/python3/dist-packages/dns/query.py:958: note: "tls" defined here
$ pylint --rcfile $CI_PROJECT_DIR/.pylintrc --disable=wrong-import-position $(git ls-files 'bin/tests/system/*.py' | grep -vE 'ans\.py')
************* Module isctest.query
bin/tests/system/isctest/query.py:55:11: E1123: Unexpected keyword argument 'verify' in function call (unexpected-keyword-arg)
(cherry picked from commit b2964cc922)
When explicitly set to True, the "verify" argument lets dnspython verify
certificates used for the connection. As most certificates in the system
test will inevitably be self-signed, the "verify" argument defaults to
False.
The "verify" argument is present in dnspython since the version 2.5.0.
(cherry picked from commit df8c419058)
The "without_fips" mark disables test function when BIND 9 was built
with the FIPS mode enabled as not everything works in FIPS-enabled
builds.
(cherry picked from commit feecbd8e77)
Adjust the limit of maximum disagreements in respdiff results based on
recent pipeline results.
The respdiff and respdiff:asan seem to have almost identical results,
typically around 0.07 % of differences with ocassional spikes up to
around 0.11 %. Similar results are for respdiff:tsan, perhaps with more
common spikes with values up to around 0.12 %. Set the limit to 0.15 %
to allow for some tolerance due to network conditions, time of day etc.
The respdiff:third-party has a slightly higher disagreements average,
with typical values being around 0.12 %. Set the limit to 0.2 %.
Exceeding either of those values should be quite clear indication that
some resolution behaviour has changed, since the values appear to be
very stable within the newly configured limits.
Backport of MR !9950
Merge branch 'backport-nicki/ci-respdiff-limits-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9989
Adjust the limit of maximum disagreements in respdiff results based on
recent pipeline results.
The respdiff and respdiff:asan seem to have almost identical results,
typically around 0.07 % of differences with ocassional spikes up to
around 0.11 %. Similar results are for respdiff:tsan, perhaps with more
common spikes with values up to around 0.12 %. Set the limit to 0.15 %
to allow for some tolerance due to network conditions, time of day etc.
The respdiff:third-party has a slightly higher disagreements average,
with typical values being around 0.12 %. Set the limit to 0.2 %.
Exceeding either of those values should be quite clear indication that
some resolution behaviour has changed, since the values appear to be
very stable within the newly configured limits.
(cherry picked from commit 0584d3f65f)
Closes#5123
Backport of MR !9966
Merge branch 'backport-5123-document-refreshing-a-secondary-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9986
In the case when `dnssec-signzone` is called on an already signed zone, and the private key file is unavailable, a signature that needs to be refreshed may be dropped without being able to generate a replacement. This has been fixed.
Closes#5126
Backport of MR !9951
Merge branch 'backport-5126-dnssec-signzone-retain-rrsig-if-key-is-offline-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9982
Track inside the dns_dnsseckey structure whether we have seen the
private key, or if this key only has a public key file.
If the key only has a public key file, or a DNSKEY reference in the
zone, mark the key 'pubkey'. In dnssec-signzone, if the key only
has a public key available, consider the key to be offline. Any
signatures that should be refreshed for which the key is not available,
retain the signature.
So in the code, 'expired' becomes 'refresh', and the new 'expired'
is only used to determine whether we need to keep the signature if
the corresponding key is not available (retaining the signature if
it is not expired).
In the 'keysthatsigned' function, we can remove:
- key->force_publish = false;
- key->force_sign = false;
because they are redundant ('dns_dnsseckey_create' already sets these
values to false).
(cherry picked from commit 5e3aef364f)
Add a test case for the scenario below.
There is a case when signing a zone with dnssec-signzone where the
private key file is moved outside the key directory (for offline
ksk purposes), and then the zone is resigned. The signature of the
DNSKEY needs refreshing, but is not expired.
Rather than removing the signature without having a valid replacement,
leave the signature in the zone (despite it needs to be refreshed).
(cherry picked from commit 0a91321d78)
Extended DNS error mechanism (EDE) may have several errors raised during a DNS resolution. `named` is now able to add up to three EDE codes in a DNS response. In the case of duplicate error codes, only the first one will be part of the DNS response.
Closes#5085
Backport of MR !9952
Merge branch 'backport-5085-multiple-ede-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9978
Extended DNS error mechanism (EDE) enables to have several EDE raised
during a DNS resolution (typically, a DNSSEC query will do multiple
fetches which each of them can have an error). Add support to up to 3
EDE errors in an DNS response. If duplicates occur (two EDEs with the
same code, the extra text is not compared), only the first one will be
part of the DNS answer.
Because the maximum number of EDE is statically fixed, `ns_client_t`
object own a static vector of `DNS_DE_MAX_ERRORS` (instead of a linked
list, for instance). The array can be fully filled (all slots point to
an allocated `dns_ednsopt_t` object) or partially filled (or
empty). In such case, the first NULL slot means there is no more EDE
objects.
(cherry picked from commit 4096f27130)
There was confusion about whether the interval was calculated from
the validity period provided on the command line (with -s and -e),
or from the signature being replaced.
Add text to clarify that the interval is calculated from the new
validity period.
Closes#5128
Backport of MR !9955
Merge branch 'backport-5128-clarify-dnssec-signzone-interval-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9983
There was confusion about whether the interval was calculated from
the validity period provided on the command line (with -s and -e),
or from the signature being replaced.
Add text to clarify that the interval is calculated from the new
validity period.
(cherry picked from commit ae42fa69fa)
If the generated status output exceeds 4096 it was silently truncated, now we output that the status was truncated.
Closes#4180
Backport of MR !9905
Merge branch 'backport-4180-possible-truncation-in-dns_keymgr_status-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9981
If the generated status output exceeds 4096 it was silently truncated,
now we output that the status was truncated.
(cherry picked from commit 7ae7851173)
Closes#5098
Backport of MR !9922
Merge branch 'backport-5098-missing-yaml-string-termination-delv-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9979
Shutdown the fetch context immediately after the last fetch has been canceled from
that particular fetch context.
Backport of MR !9958
Merge branch 'backport-ondrej/shutdown-the-fetch-context-early-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9977
Currently, the fetch context will continue running even when the last
fetch (response) has been removed from the context, so named can process
and cache the answer. This can lead to a situation where the number of
outgoing recursing clients exceeds the the configured number for
recursive-clients.
Be more stringent about the recursive-clients limit and shutdown the
fetch context immediately after the last fetch has been canceled from
that particular fetch context.
(cherry picked from commit 9f945c8b67)
Resolver under heavy-load could exhaust the memory available for storing
the information in the Address Database (ADB) effectively evicting already
stored information in the ADB. The memory used to retrieve and provide
information from the ADB is now not a subject of the same memory limits
that are applied for storing the information in the Address Database.
Closes#5127
Backport of MR !9954
Merge branch 'backport-5127-change-ADB-memory-split-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9975
Address Database (ADB) shares the memory for the short lived ADB
objects (finds, fetches, addrinfo) and the long lived ADB
objects (names, entries, namehooks). This could lead to a situation
where the resolver-heavy load would force evict ADB objects from the
database to point where ADB is completely empty, leading to even more
resolver-heavy load.
Make the short lived ADB objects use the other memory context that we
already created for the hashmaps. This makes the ADB overmem condition
to not be triggered by the ongoing resolver fetches.
(cherry picked from commit 05faff6d53)
Prevent lock contention among many worker threads referring to the same database node at the same time. This would improve zone and cache database performance for the heavily contended database nodes.
Backport of !9963Closes#5130
Merge branch '5130-reduce-lock-contention-in-decrement-reference-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9964
Previously, this function always acquires a node write lock if it
might need node cleanup in case the reference decrements to 0. In
fact, the lock is unnecessary if the reference is larger than 1 and it
can be optimized as an "easy" case. This optimization could even be
"necessary". In some extreme cases, many worker threads could repeat
acquring and releasing the reference on the same node, resulting in
severe lock contention for nothing (as the ref wouldn't decrement to 0
in most cases). This change would prevent noticeable performance
drop like query timeout for such cases.
Co-authored-by: JINMEI Tatuya <jtatuya@infoblox.com>
Co-authored-by: Ondřej Surý <ondrej@isc.org>
(cherry picked from commit 7f4471594d)
TAGS file are generated from `make tags` using etags. Other index tags
are already ignored (GTAGS, GPATH, etc.). Also ignoring `TAGS`.
(cherry picked from commit 2164ea8abd)
Revert the fix for #4950 for 9.20.
This reverts MR !9438.
History: A performance improvement for NSEC3 closest encloser lookups (#4460) was introduced (in MR !9436) and backported to 9.20 (MR !9438) and to 9.18 in (MR !9439). It was released in 9.18.30 (and 9.20.2 and 9.21.1).
There was a bug in the code (#4950), so we reverted the change in !9611, !9613 and !9614 (not released).
Then a new attempt was merged in main (MR !9610) and backported to 9.20 (MR !9631) and 9.18 (MR !9632). The latter should not have been backported and was reverted in !9689.
We now also revert the fix for 9.20
Related #5108
Merge branch 'revert-4950-bind-logs-expected-covering-nsec3-got-an-exact-match-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9947
Closes#5115
Backport of MR !9934
Merge branch 'backport-5115-fix-default-iana-root-zone-mirror-configuration-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9944
Commit 4555a31934 renamed the top-level
"primaries" block in bin/named/config.c to "remote-servers". This
configuration block lists the primary servers used for an IANA root zone
mirror when no primary servers are explicitly specified for it in the
configuration. However, the relevant part of the named_zone_configure()
function only looks for a top-level "primaries" block and not for any of
its synonyms. As a result, configuring an IANA root zone mirror with
just:
zone "." {
type mirror;
};
now results in a cryptic fatal error on startup:
loading configuration: not found
exiting (due to fatal error)
Fix by using the correct top-level block name in named_zone_configure().
(cherry picked from commit 010d2eb436)
Response policy zones (RPZ) and catalog zones were not working correctly if they had an $INCLUDE statement defined. This has been fixed.
Closes#5111
Backport of MR !9930
Merge branch 'backport-5111-includes-disable-rpz-and-catz-fix-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9941
The code in zone_startload() disables RPZ and CATZ for a zone if
dns_master_loadfile() returns anything other than ISC_R_SUCCESS,
which makes sense, but it's an error because zone_startload() can
also return DNS_R_SEENINCLUDE upon success when the zone had an
$INCLUDE statement.
(cherry picked from commit 3d7a9fba3b)
Add performance tests of DoH using the GET protocol to nightly pipelines.
Backport of MR !9926
Merge branch 'backport-nicki/ci-shotgun-doh-get-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9939
