The lame-ttl processing was overriden to be disabled in the config,
but the code related to the lame-ttl was still kept in the resolver
code. More importantly, the DNS_RESOLVER_BADCACHETTL() macro would
cause the entries in the resolver badcache to be always cached for at
least 30 seconds even if the lame-ttl would be set to 0.
Remove the dns_badcache code from the dns_resolver unit, so we save some
processing time and memory in the resolver code.
Instead of cleaning the dns_badcache opportunistically, add per-loop
LRU, so each thread-loop can clean the expired entries. This also
allows removal of the atomic operations as the badcache entries are now
immutable, instead of updating the badcache entry in place, the old
entry is now deleted from the hashtable and the LRU list, and the new
entry is inserted in the LRU.
Re-split format strings that had been poorly split by multiple
clang-format runs using different versions of clang-format.
Closes#5043
Backport of MR !9752
Merge branch 'backport-5043-re-split-format-strings-lib-ns-update-c-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9755
Re-split format strings that had been poorly split by multiple
clang-format runs using different versions of clang-format.
(cherry picked from commit a24d6e1654)
* Add new clang-format option to remove redundant semicolons
* Add new clang-format option to remove redundant parentheses
Backport of MR !9749
Merge branch 'backport-ondrej/code-style-2024-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9750
The new clang-format options (since clang-format 16) removes semicolons
after the closing braces of functions and constructors/destructors.
(cherry picked from commit c7420eccd9)
The new clang-format option (since 17) can remove redundant parentheses
in the return statement (with configuration value ReturnStatement).
(cherry picked from commit 3873b0c279)
There is a data race between the statistics channel, which uses
`dns_zone_getxfr()` to get a reference to `zone->xfr`, and the creation
of `zone->xfr`, because the latter happens outside of a zone lock.
Split the `dns_xfrin_create()` function into two parts to separate the
zone transfer starting part from the zone transfer object creation part.
This allows us to attach the new object to a local variable first, then
attach it to `zone->xfr` under a lock, and only then start the transfer.
Closes#5011
Backport of MR !9716
Merge branch 'backport-5011-dns_zone_getxfr-race-fix-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9728
There is a data race between the statistics channel, which uses
`dns_zone_getxfr()` to get a reference to `zone->xfr`, and the creation
of `zone->xfr`, because the latter happens outside of a zone lock.
Split the `dns_xfrin_create()` function into two parts to separate the
zone tranfer startring part from the zone transfer object creation part.
This allows us to attach the new object to a local variable first, then
attach it to `zone->xfr` under a lock, and only then start the transfer.
(cherry picked from commit dbf230650f)
Add an 'initial-ds' entry to bind.keys for the new root key, ID
38696, which is scheduled for publication in January 2025.
Closes#4896
Backport of MR !9422
Merge branch 'backport-4896-update-bind-keys-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9746
Add an 'initial-ds' entry to bind.keys for the new root key, ID
38696, scheduled for publication in January 2025.
(cherry picked from commit 609bf35075)
The isc/crypto.h now directly includes the OpenSSL headers (evp.h) and
any application that includes that header also needs to have
OPENSSL_CFLAGS in the Makefile.am. Adjust the required automake files
as needed.
Backport of MR !9713
Merge branch 'backport-ondrej/add-missing-OPENSSL_CFLAGS-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9715
The isc/crypto.h now directly includes the OpenSSL headers (evp.h) and
any application that includes that header also needs to have
OPENSSL_CFLAGS in the Makefile.am. Adjust the required automake files
as needed.
(cherry picked from commit 88103e72d5)
Keeping the Known Issues as part of the rendered docs has the issue that
the list can't be updated on the official docs website until the next
release. This is unpractical is a high-priority issue is discovered
shortly after a release. Keep the Known Issues in wiki and simply link
to the list from the rendered docs. The wiki article can be updated at
any time as needed.
When canceling the ADB find, the lock on the find gets released for
a brief period of time to be locked again inside adbname lock. During
the brief period that the ADB find is unlocked, it can get canceled by
other means removing it from the adbname list which in turn causes
assertion failure due to a double removal from the adbname list.
This has been fixed.
Closes#5024
Backport of MR !9722
Merge branch 'backport-5024-fix-crash-in-dns_adb_cancelfind-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9744
When canceling the ADB find, the lock on the find gets released for
a brief period of time to be locked again inside adbname lock. During
the brief period that the ADB find is unlocked, it can get canceled by
other means removing it from the adbname list which in turn causes
assertion failure due to a double removal from the adbname list.
Recheck if the find->adbname is still valid after acquiring the lock
again and if not just skip the double removal. Additionally, attach to
the adbname as in the worst case, the adbname might also cease to exist
if the scheduler would block this particular thread for a longer period
of time invalidating the lock we are going to acquire and release.
(cherry picked from commit 128e50e1ff)
Some omissions of !9426 discovered during the backports
Backport of MR !9739
Merge branch 'backport-nicki/extra-artifacts-fixups-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9741
When a test is re-run by the flaky plugin, the TestReport outcomes
collected in the pytest_runtest_makereport() hook should be overriden.
Each of the setup/call/teardown phases is reported again and since we
care about the overall outcome, their respective results should be
overriden so that only the outcome from the final test (re)run gets
reported.
Prior to this change, it lead to a situation where an extra_artifact
generated during the test might be ignored. This was caused because the
check was skipped, since the test was incorrectly considered as "failed"
in the case where the test would fail on the first run, but pass on a
subsequent flaky rerun.
(cherry picked from commit b66fb31dcb)
``clean.sh`` scripts have been replaced by lists of expected artifacts for each system test module. The list is defined using the custom ``pytest.mark.extra_artifacts`` mark, which can use both filenames and globs.
Closes#4261
Backport of MR !9426
Merge branch 'backport-4261-add-pytest-fixture-checking-test-artifacts-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9734
The artifact lists in clean.sh and extra_artifacts might be slightly
different. The list was updated for each test to reflect the current
state.
(cherry picked from commit 7c259fe254)
Prior to introducing the pytest runner, clean.sh files were used as a
list of files that the test is expected to leave around as artifacts and
check that no extra files were created.
With the pytest runner, those scripts are no longer used, but the
ability to detect extraneous files is still useful. Add a new
"extra_artifacts" mark which can be used for the same purpose.
(cherry picked from commit 3a9f4edddc)
Remove named_g_sessionkey and named_g_sessionkeyname as they are declared and cleaned up but otherwise are unused.
Closes#5023
Backport of MR !9720
Merge branch 'backport-5023-remove-named_g_sessionkey-as-it-is-unused-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9725
The zone transfer summary will now print the expire option value in the zone transfer summary.
Closes#5013
Backport of MR !9694
Merge branch 'backport-5013-print-expire-option-in-transfer-summary-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9714
The tool 'dnssec-ksr' now allows for KSK generation, as well as planned KSK rollovers. When signing a bundle from a Key Signing Request (KSR), only the key that is active in that time frame is being used for signing. Also, the CDS and CDNSKEY records are now added and removed at the correct time.
Closes#4697Closes#4705
Backport of MR !9452
Merge branch 'backport-4705-dnssec-ksr-only-sign-with-active-ksks-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9711
dnssec-ksr can now sign KSR files with multiple KSKs. A planned KSK
rollover is supported, meaning the KSR will first be signed with
one KSK and later with another. The timing metadata for CDS and
CDNSKEY records are also taken into account, so these records are
only published when the time is between "SyncPublish" and "SyncDelete".
(cherry picked from commit d7f2a2f437)
Add a test case for Offline KSK where during the lifespan of the Signed
Key Response a KSK rollover happens. Ensure that the correct DNSKEY,
CDNSKEY, and CDS records are published at the right times.
(cherry picked from commit 8cf5f972f4)
When the zone is initially signed, the CDNSKEY/CDS RRset is not
immediately published. The DNSKEY and signatures must propagate first.
Adjust the test to allow for this case.
(cherry picked from commit 708927e03d)
Add an option to dnssec-ksr keygen, -o, to create KSKs instead of ZSKs.
This way, we can create a set of KSKS for a given period too.
For KSKs we also need to set timing metadata, including "SyncPublish"
and "SyncDelete". This functionality already exists in keymgr.c so
let's make the function accessible.
Replace dnssec-keygen calls with dnssec-ksr keygen for KSK in the
ksr system test and check keys for created KSKs as well. This requires
a slight modification of the check_keys function to take into account
KSK timings and metadata.
(cherry picked from commit 680aedb595)
Add missing checks for `$FEATURETEST --have-fips-dh` in notify system test to match those in setup.sh.
Closes#5015
Backport of MR !9707
Merge branch 'backport-5015-tls-notify-checks-fail-on-ol-8-fips-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9712
In the ksr system test, the 'test_ksr_twotone' case may fail if there are two keys with the same keytag (but different algorithms), because one key is expected to be signing and the other is not.
Switch to regular expression matching and include the algorithm in the search string.
Closes#5017
Backport of MR !9701
Merge branch 'backport-5017-unexpected-match-ksr-twotone-again-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9710
In the ksr system test, the test_ksr_twotone case may fail if there
are two keys with the same keytag (but different algorithms), because
one key is expected to be signing and the other is not.
Switch to regular expression matching and include the algorithm in the
search string.
(cherry picked from commit 795fcc9f80)
With the introduction of the generated changelog, the CHANGES file
became a symlink to doc/arm/changelog.rst. After the changes made in
!9549, the changelog file transitioned from being a wholly generated
file to one that includes versioned changelog files, which are
themselves generated. However, while implementing !9549, we overlooked
that the CHANGES file is copied to a release directory on an FTP server
and contains just "include" directives, not the changelog itself.
Therefore, in the same fashion as the "RELEASE-NOTES*.html" file, create
a "CHANGELOG*.html" file that redirects to the Changelog appendix of the
ARM.
Closes#5000
Backport of MR !9690
Merge branch 'backport-5000-provide-correct-changelog-on-ftp-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9703
With the introduction of the generated changelog, the CHANGES file
became a symlink to doc/arm/changelog.rst. After the changes made in
!9549, the changelog file transitioned from being a wholly generated
file to one that includes versioned changelog files, which are
themselves generated. However, while implementing !9549, we overlooked
that the CHANGES file is copied to a release directory on an FTP server
and contains just "include" directives, not the changelog itself.
Therefore, in the same fashion as the "RELEASE-NOTES*.html" file, create
a "CHANGELOG*.html" file that redirects to the Changelog appendix of the
ARM.
(cherry picked from commit e40bd273e4)
Configuration files in system tests which require some variables (e.g.
port numbers) filled in during test setup, can now use jinja2 templates
when `jinja2` python package is available.
Any `*.j2` file found within the system test directory will be
automatically rendered with the environment variables into a file
without the `.j2` extension by the pytest runner. E.g.
`ns1/named.conf.j2` will become `ns1/named.conf` during test setup. To
avoid automatic rendering, use `.j2.manual` extension and render the
files manually at test time.
New `templates` pytest fixture has been added. Its `render()` function
can be used to render a template with custom test variables. This can be
useful to fill in different config options during the test. With
advanced jinja2 template syntax, it can also be used to include/omit
entire sections of the config file rather than using `named1.conf.in`,
`named2.conf.in` etc.
Closes#4938
Backport of MR !9587
Merge branch 'backport-4938-use-jinja2-templates-in-system-tests-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9699
Configuration files in system tests which require some variables (e.g.
port numbers) filled in during test setup, can now use jinja2 templates
when `jinja2` python package is available.
Any `*.j2` file found within the system test directory will be
automatically rendered with the environment variables into a file
without the `.j2` extension by the pytest runner. E.g.
`ns1/named.conf.j2` will become `ns1/named.conf` during test setup. To
avoid automatic rendering, use `.j2.manual` extension and render the
files manually at test time.
New `templates` pytest fixture has been added. Its `render()` function
can be used to render a template with custom test variables. This can be
useful to fill in different config options during the test. With
advanced jinja2 template syntax, it can also be used to include/omit
entire sections of the config file rather than using `named1.conf.in`,
`named2.conf.in` etc.
(cherry picked from commit 60e118c4fb)
Emphasize more that the `inline-signing` default value has changed in 9.20.0.
Merge branch 'matthijs-improve-release-notes-wrt-inline-signing-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9647
Use a stricter hazard check which ensures the audience tag is present in
the MR title and is one of the known values. This prevents siuations
where incorrect audience is accidentally used, resulting in a missing
changelog entry or a release note.
(cherry picked from commit cdb93bcbd4)
Coverity Scan reported a new issue for the ksr system test. There is allegedly a null pointer dereference (FORWARD_NULL) in check_keys().
This popped up because previously we set 'retired' to 0 in case of unlimited lifetime, but we changed it to None.
It is actually a false positive, because if lifetime is unlimited there will be only one key in 'keys'.
However, the code would be better if we always initialized 'active' and if it is not the first key and retired is set, set the successor key's active time to the retire time of the predecessor key.
Closes#5004
Backport of MR !9687
Merge branch 'backport-5004-cid-510858-ksr-check-keys-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9693
Coverity Scan reported a new issue for the ksr system test. There
is allegedly a null pointer dereference (FORWARD_NULL) in check_keys().
This popped up because previously we set 'retired' to 0 in case of
unlimited lifetime, but we changed it to None.
It is actually a false positive, because if lifetime is unlimited
there will be only one key in 'keys'.
However, the code would be better if we always initialized 'active'
and if it is not the first key and retired is set, set the successor
key's active time to the retire time of the predecessor key.
(cherry picked from commit e777efb576)
A test may fail if the key id is shorter than 5 digits. Add a leading space to the expected strings which start with the key tag to avoid the issue.
Closes#5002
Backport of MR !9688
Merge branch 'backport-5002-unexpected-match-ksr-twotwone-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9692
The test_ksr_twotwone may fail if the key id is shorter than 5 digits.
Add a leading space to the expected strings which start with the key
tag to avoid the issue.
(cherry picked from commit d5f32f6990)
Make system tests symlinks and logged test names consistent across pytest versions.
Backport of MR !9071
Merge branch 'backport-nicki/pytest-v8-compat-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9685
While this file is no longer created / used in the main branch, it may
linger around when switching from maintenance branches.
(cherry picked from commit 6262d002bf)
The pytest collection mechanism has been overhauled in pytest 8.0.0,
resulting in a different node tree when collecting the tests. Ensure the
paths / names we're using that are derived from the node tree are
consistent across different pytest versions.
Particularly, this has affected the convenience symlink name (which is
supposed to be in the form of e.g. dns64_sh_dns64 for the dns64 module
and tests_sh_dns64.py module) and the test name that's logged at the
start of the test, which is supposed to include the system test
directory relative to the root system test directory as well as the
module name (e.g. dns64/tests_sh_dns64.py).
Related https://github.com/pytest-dev/pytest/issues/7777
(cherry picked from commit 7118cbed98)
Notifies configured to use TLS will now be sent over TLS, instead of plaintext UDP or TCP.
Also, failing to load the TLS configuration for notify now also results in an error.
Closes#4821
Backport of MR !9407
Merge branch 'backport-4821-notify-over-tls-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9684
Use tls-forward-secrecy instead of tls-expired for tls-x2 and regenerate
the expired certificate for tls-x6 to reflect the swap of ns2 and ns3.
(cherry picked from commit bbdc6b26aa)
This allows for dispatch to use existing TCP/HTTPS/TLS etc. streams without accidentally using an unexpected transport.
Closes#4989
Backport of MR !9633
Merge branch 'backport-4989-fix-transport-use-with-dispatch-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9682
Dispatch needs to know the transport that is being used over the
TCP connection to correctly allow for it to be reused. Add a
transport parameter to dns_dispatch_createtcp and dns_dispatch_gettcp
and use it when selecting a TCP socket for reuse.
(cherry picked from commit baab8a5d75)
When working with key timestamps, ensure we correctly set the UTC
timezone in order for the tests to work consistently regardless of the
local time setting.
Closes#4999
Backport of MR !9673
Merge branch 'backport-4999-pytest-kasp-use-utc-timezone-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9679
Use a different timezone via the TZ variable in at least one of the
system test jobs in order to detect possible issues with timezone
handling in python.
(cherry picked from commit 46810be809)
When working with key timestamps, ensure we correctly set the UTC
timezone in order for the tests to work consistently regardless of the
local time setting.
(cherry picked from commit f840deba33)
Backport of MR !9626
Merge branch 'backport-nicki/pylint-disable-too-few-too-many-checks-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9656
Enforcing pylint standards and default for our test code seems
counter-productive. Since most of the newly added code are tests or is
test-related, encountering these checks rarely make us refactor the code
in other ways and we just disable these checks individually. Code that
is too complex or convoluted will be pointed out in reviews anyways.
(cherry picked from commit 7639c58c48)
It is possible that the zone is not yet fully signed because it is
signed in batches. Retry the AXFR and verify command a couple of times.
(cherry picked from commit b8b3df0676)
If a function is expected to assert / raise on failure (rather than
return boolean), its name should start with "check_".
(cherry picked from commit 67957d1f54)
Move all test cases from tests.sh to tests_ksr.py. The only test that
is not moved is the check that key id's match expected keys. The
shell-based system test checks two earlier set environment variables
against each other that has become redundant in the pytest variant,
because we now check the signed key response against a list of keys
and for each key we take into account the timing metadata. So we
already ensure that each published key is in the correct key bundle.
(cherry picked from commit a15bf6704b)
Write initial pytest kasp library. This contains everything that is
required for testing Offline KSK functionality with pytest.
This includes:
- addtime: adding a value to a timing metadata
- get_timing_metdata: retrieve timing metadata from keyfile
- get_metadata/get_keystate: retrieve metadata from statefile
- get_keytag: retrieve keytag from base keyfile string
- get_keyrole: get key role from statefile
- dnskey_equals: compare DNSKEY record from file against a string
- cds_equals: compare CDS derived from file against a string
- zone_is_signed: wait until a zone is completely signed
- dnssec_verify: verify a DNSSEC signed zone with dnssec-verify
- check_dnssecstatus: check rndc dnssec -status output
- check_signatures: check that signatures for a given RRset are correct
- check_dnskeys: check that the published DNSKEY RRset is correct
- check_cds: check that the published CDS RRset is correct
- check_apex: check SOA, DNSKEY, CDNSKEY, and CDS RRset
- check_subdomain: check an RRset below the apex
(cherry picked from commit a3829990fd)
When DSCP support was removed, parsing of hostnames in rndc.conf was accidentally broken, resulting in an assertion failure. This has been fixed.
Closes#4991
Backport of MR !9669
Merge branch 'backport-4991-rndc-fix-parsing-hostnames-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9676
When DSCP was removed the parsing of hostnames was accidentally
broken resulting in an assertion failure. Call cfg_parse_tuple
rather than using custom code in parse_sockaddrnameport.
(cherry picked from commit 6c095f89f5)
See isc-projects/bind9#4996
Backport of MR !9662
Merge branch 'backport-mnowak/mark-upforwd-xfail-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9672
With Sphinx 8.1.0, footnotes can't stand on their own and have to be
referenced from somewhere, otherwise build fails, e.g.:
doc/dnssec-guide/signing.rst:1470: WARNING: Footnote [#] is not referenced. [ref.footnote]
Backport of MR !9663
Merge branch 'backport-mnowak/fix-dnssec-guide-footnote-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9670
With Sphinx 8.1.0, footnotes can't stand on their own and have to be
referenced from somewhere, otherwise build fails, e.g.:
doc/dnssec-guide/signing.rst:1470: WARNING: Footnote [#] is not referenced. [ref.footnote]
(cherry picked from commit bdf8859e2d)
Options of the form `[+-]option=<value>` failed to display the value on the printed command line. This has been fixed.
Closes#4993
Backport of MR !9653
Merge branch 'backport-4993-dig-restore-command-line-values-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9666
BIND 9.20.0 broke `recursive-clients 0;`. This has now been fixed.
Closes#4987
Backport of MR !9621
Merge branch 'backport-4987-fix-recursive-clients-0-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9654
HTTP URI of doc/arm/requirements.txt on the "main" branch is the source
of PyPI packages in CI base image. To make it consistent in all
maintained CI branches on RTD, use the HTTP URI as well.
This change is non-material on the "main" branch, but ensures the right
packages on stable branches where for RTD we currently leverage outdated
packages because we failed to update doc/arm/requirements.txt there.
Backport of MR !9630
Merge branch 'backport-mnowak/rtd-always-resource-requirements-txt-from-main-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9660
HTTP URI of doc/arm/requirements.txt on the "main" branch is the source
of PyPI packages in CI base image. To make it consistent in all
maintained CI branches on RTD, use the HTTP URI as well.
This change is non-material on the "main" branch, but ensures the right
packages on stable branches where for RTD we currently leverage outdated
packages because we failed to update doc/arm/requirements.txt there.
(cherry picked from commit 8fb6115492)
With Sphinx 8.1.0, footnotes can't stand on their own and have
referenced from somewhere:
/builds/isc-projects/bind9/doc/arm/general.rst:439: WARNING: Footnote [#] is not referenced. [ref.footnote]
/builds/isc-projects/bind9/doc/arm/general.rst:441: WARNING: Footnote [#] is not referenced. [ref.footnote]
/builds/isc-projects/bind9/doc/arm/general.rst:445: WARNING: Footnote [#] is not referenced. [ref.footnote]
/builds/isc-projects/bind9/doc/arm/general.rst:457: WARNING: Footnote [#] is not referenced. [ref.footnote]
(cherry picked from commit 54410e034f)
The unit test doh_test tends do fail quite often due to exceeding run
time limit in the unit:clang:freebsd14:amd64 job. Use a retry on gitlab
level to alleviate the issue until a better fix is available.
Related #4924
Backport of MR !9578
Merge branch 'backport-4924-retry-doh_test-freebsd14-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9637
The unit test doh_test tends do fail quite often due to exceeding run
time limit in the unit:clang:freebsd14:amd64 job. Use a retry on gitlab
level to alleviate the issue until a better fix is available.
(cherry picked from commit 1636864ddb)
This script is only maintained on the "main" branch.
Merge branch 'mnowak/drop-util-release-tarball-comparison-sh-script' into 'bind-9.20'
See merge request isc-projects/bind9!9629
CookieOption with new .server/.client attributes (rather than .data) was
added to dnspython. Adjust the code to use the new attributes if
available and fall back to the old code for dnspython<2.7.0
compatibility.
(cherry picked from commit 0d90b13646)
The performance improvement for finding the NSEC3 closest encloser when generating authoritative responses could cause servers to return incorrect NSEC3 records in some cases. This has been fixed.
Closes#4950
Backport of MR !9610
Merge branch 'backport-4950-bind-logs-expected-covering-nsec3-got-an-exact-match-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9631
maxlabels is the suffix length that corresponds to the latest
NXDOMAIN response. minlabels is the suffix length that corresponds
to longest found existing name.
(cherry picked from commit 67f31c5046)
When running shotgun tests on tagged releases, the increased number of
jobs may cause the shotgun pipeline to take longer than 50 minutes to
finish.
Backport of MR !9599
Merge branch 'backport-nicki/increase-shotgun-pipeline-timeout-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9624
When running shotgun tests on tagged releases, the increased number of
jobs may cause the shotgun pipeline to take longer than 50 minutes to
finish.
(cherry picked from commit 3b227e1161)
Prior to doing key management, BIND 9 will check if the key files on disk match the expected keys. If key files for previously observed keys have become unavailable, this will prevent the internal key manager from running.
Backport of MR !9337
Merge branch 'backport-4763-do-not-roll-if-key-files-are-missing-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9622
In a multi-signer setup, removing DNSKEY records from the zone should
not be treated as a key that previously exists in the keyring, thus
blocking the keymgr. Add a test case to make sure.
(cherry picked from commit 5f552293d7)
Test that if a key to be purged is in the keyring, it does not
prevent the keymgr from running. Normally a key that is in the keyring
should be available again on the next run, but that is not true for
a key that can be purged.
In addition, fix some wait_for_log calls, by adding the missing
'|| ret=1' parts.
(cherry picked from commit a2317425bc)
Some test cases were working but for the wrong reasons. These started
to fail when I implemented the first approach for #4763, where the
existence of a DNSKEY together with an empty keyring is suspicious and
would prevent the keymgr from running.
These are:
1. kasp: The multisigner-model2.kasp zone has ZSKs from other providers
in the zone, but not yet its own keys. Pregenerate signing keys and
add them to the unsigned zone as well.
2. kasp: The dynamic-signed-inline-signing.kasp zone has a key generated
and added in the raw version of the zone. But the key file is stored
outside the key-directory for the given zone. Add '-K keys' to the
dnssec-keygen command.
(cherry picked from commit d1e263ef13)
Prior to running the keymgr, first make sure that existing keys
are present in the new keylist. If not, treat this as an operational
error where the keys are made offline (temporarily), possibly unwanted.
(cherry picked from commit 5fdad05a8a)
In this specific case the key files are temporary unavailable, for
example because of an operator error, or a mount failure). In such
cases, BIND should not try to roll over these keys.
(cherry picked from commit a3afbd9d6f)
This change ensures that the port is set before attempting a UDP query. Before that a situation could appear when previous query have completed over a different transport (that uses a dedicated port) and then a UDP query will be attempted over the port of the previous transport.
Backport of !9618.
Merge branch 'artem-debian-bug-1059582-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9619
This commit ensures that the port is set before attempting a UDP
query. Before that a situation could appear when previous query have
completed over a different transport (that uses a dedicated port) and
then a UDP query will be attempted over the port of the previous
transport.
(cherry picked from commit e390ed4421)
This release note was missing due to a malformed Merge Request title.
The text is not copied verbatim, but changed to something more release
note-like.
Backport of MR !9598
Merge branch 'backport-4460-add-missing-release-note-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9615
This release note was missing due to a malformed Merge Request title.
The text is not copied verbatim, but changed to something more release
note-like.
(cherry picked from commit 5860bafc60)
Revert "fix: chg: Improve performance when looking for the closest encloser when returning NSEC3 proofs"
This reverts merge request !9436Closes#4950
Backport of MR !9611
Merge branch 'backport-revert-78d48f7a-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9613
In two places, after linking the client to the manager's
"recursing-clients" list using the check_recursionquota()
function, the query.c module fails to unlink it on error
paths. Fix the bugs by unlinking the client from the list.
Backport of MR !9586
Merge branch 'backport-aram/unlink-recursing-clients-on-error-paths-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9604
Rename check_recursionquota() to acquire_recursionquota(), and
implement a new function called release_recursionquota() to
reverse the action. It helps with decreasing code duplication.
(cherry picked from commit 7bd44a4182)
In two places, after linking the client to the manager's
"recursing-clients" list using the check_recursionquota()
function, the query.c module fails to unlink it on error
paths. Fix the bugs by unlinking the client from the list.
Also make sure that unlinking happens before detaching the
client's handle, as it is the logically correct order, e.g.
in case if it's the last handle and ns__client_reset_cb()
can be called because of the detachment.
(cherry picked from commit 36c4808903)
The dns_zone_getxfrintime() function fails to lock the zone before
accessing its 'xfrintime' structure member, which can cause a data
race between soa_query() and the statistics channel. Add the missing
locking/unlocking pair, like it's done in numerous other similar
functions.
Closes#4976
Backport of MR !9591
Merge branch 'backport-4976-zone-xfrintime-data-race-fix-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9601
The dns_zone_getxfrintime() function fails to lock the zone before
accessing its 'xfrintime' structure member, which can cause a data
race between soa_query() and the statistics channel. Add the missing
locking/unlocking pair, like it's done in numerous other similar
functions.
(cherry picked from commit ab07803465)
The 'nodetach' member is a leftover from the times when non-zero
'stale-answer-client-timeout' values were supported, and currently
is always 'false'. Clean up the member and its usage.
Backport of MR !9592
Merge branch 'backport-aram/cleanup-ns-client-nodetach-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9600
The 'nodetach' member is a leftover from the times when non-zero
'stale-answer-client-timeout' values were supported, and currently
is always 'false'. Clean up the member and its usage.
(cherry picked from commit b8c068835e)
The outgoing UDP sockets enabled `SO_REUSEADDR` that allows sharing of the UDP sockets, but with one big caveat - the socket that was opened the last would get all traffic. The dispatch code would ignore the invalid responses in the dns_dispatch, but this could lead to unexpected results.
Backport of MR !9569
Merge branch 'backport-ondrej/fix-outgoing-UDP-port-selection-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9583
Currently, the outgoing UDP sockets have enabled
SO_REUSEADDR (SO_REUSEPORT on BSDs) which allows multiple UDP sockets to
bind to the same address+port. There's one caveat though - only a
single (the last one) socket is going to receive all the incoming
traffic. This in turn could lead to incoming DNS message matching to
invalid dns_dispatch and getting dropped.
Disable setting the SO_REUSEADDR on the outgoing UDP sockets. This
needs to be done explicitly because `uv_udp_open()` silently enables the
option on the socket.
(cherry picked from commit eec30c33c2)
When matching the received Query ID in the TCP connection,
an invalid received Query ID can very rarely cause assertion
failure.
Closes#4952
Backport of MR !9580
Merge branch 'backport-4952-check-whether-QID-is-not-deleted-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9582
When matching the TCP dispatch responses, we should skip the responses
that do not belong to our TCP connection. This can happen with faulty
upstream server that sends invalid QID back to us.
(cherry picked from commit 4ef316e21e)
The dns_dispatch_add() function registers the 'resp' entry in
'disp->mgr->qids' hash table with 'resp->port' being 0, but in
tcp_recv_success(), when looking up an entry in the hash table
after a successfully received data the port is used, so if the
local port was set (i.e. it was not 0) it fails to find the
entry and results in an unexpected error.
Set the 'resp->port' to the given local port value extracted from
'disp->local'.
Closes#4969
Backport of MR !9576
Merge branch 'backport-4969-dispatch-tcp-source-port-bug-fix-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9581
The dns_dispatch_add() function registers the 'resp' entry in
'disp->mgr->qids' hash table with 'resp->port' being 0, but in
tcp_recv_success(), when looking up an entry in the hash table
after a successfully received data the port is used, so if the
local port was set (i.e. it was not 0) it fails to find the
entry and results in an unexpected error.
Set the 'resp->port' to the given local port value extracted from
'disp->local'.
(cherry picked from commit d49a8f518a)
As the relaxed memory ordering doesn't ensure any memory
synchronization, it is possible that the increment will succeed even
in the case when it should not - there is a race between
atomic_fetch_sub(..., acq_rel) and atomic_fetch_add(..., relaxed).
Only the result is consistent, but the previous value for both calls
could be same when both calls are executed at the same time.
Backport of MR !9460
Merge branch 'backport-ondrej/use-release-memory-ordering-for-reference-counting-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9567
As the relaxed memory ordering doesn't ensure any memory
synchronization, it is possible that the increment will succeed even
in the case when it should not - there is a race between
atomic_fetch_sub(..., acq_rel) and atomic_fetch_add(..., relaxed).
Only the result is consistent, but the previous value for both calls
could be same when both calls are executed at the same time.
(cherry picked from commit 88227ea665)
When `sig-validity-interval` was obsoleted, the text that the signature validity interval should be multiples of the SOA expire interval was removed. Restore this text to the description of the `signatures-validity` option.
Closes#4951
Backport of MR !9566
Merge branch 'backport-4951-document-signatures-validity-soa-expire-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9573
The example.com zone file given in the "Configurations and Zone Files"
chapter has an SOA expire of 3 weeks, which is not a multiple of
the default signatures-validity value. Adjust the SOA expire so that
it is much lower than the signatures-validity default.
(cherry picked from commit 8aa94931c3)
When `sig-validity-interval` was obsoleted, the text that the signature
validity interval should be multiples of the SOA expire interval was
removed. Restore this text to the description of the
`signatures-validity` option.
(cherry picked from commit b09230004b)
The `dns.query.udp` and `dns.query.tcp` methods are [prone to timeouts](https://gitlab.isc.org/isc-projects/bind9/-/jobs/4785053); their `isctest.query` equivalents should be used in system tests instead.
Backport of MR !9560
Merge branch 'backport-mnowak/convert-dns-query-udp-and-tcp-to-isctest-query-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9574
Static-stub addresses and addresses from other sources were being
mixed together, resulting in static-stub queries going to addresses
not specified in the configuration, or alternatively, static-stub
addresses being used instead of the correct server addresses.
Closes#4850
Backport of MR !9314
Merge branch 'backport-4850-add-an-additional-class-of-names-to-adb-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9571
Static-stub address and addresses from other sources where being
mixed together resulting in static-stub queries going to addresses
not specified in the configuration or alternatively static-stub
addresses being used instead of the real addresses.
(cherry picked from commit b3a2c790f3)
Log a query response summary using the new category `responses`. Logging can be controlled by the option `responselog` and `rndc responselog`.
Closes#459
Backport of MR !9449
Merge branch 'backport-459-log-query-response-status-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9526
Extends the resolver system test to to check whether responses are
logged to the query log in addition to the queries.
(cherry picked from commit 61415432ec)
Query and response log shares the same flags. Move flags logging out of
log_query to share it with log_response. Use buffer instead of snprintf
to fill flags a bit faster.
Signed-off-by: Petr Menšík <pemensik@redhat.com>
(cherry picked from commit e6b19af2dd)
Remove answer flag from log, log instead count of records for each
message section. Include EDNS version and few flags of response. Add
also status of result.
Still does not include body of responses rrset.
(cherry picked from commit 6f879aba65)
Log to the querylog the rcode of a previous query using
the identifier 'response:' to diffenciate queries from
responses.
(cherry picked from commit 5fad79c92f)
An exit path in the dns_dispatch_add() function fails to get out of
the RCU critical section when returning early. Add the missing
rcu_read_unlock() call.
Backport of MR !9561
Merge branch 'backport-aram/add-missing-rcu_read_unlock-in-dns_dispatch_add-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9564
An exit path in the dns_dispatch_add() function fails to get out of
the RCU critical section when returning early. Add the missing
rcu_read_unlock() call.
(cherry picked from commit 4123d59fbc)
Add the new record type WALLET (262). This provides a mapping from a domain name to a cryptographic currency wallet. Multiple mappings can exist if multiple records exist.
Closes#4947
Backport of MR !9521
Merge branch 'backport-4947-add-wallet-type-to-named-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9554
The 'rndc dumpdb' command wasn't reporting errors which
occurred when starting up the database dump process by named,
like, for example, a permission denied error for the
'dump-file' file. This has been fixed. Note, however, that
'rndc dumpdb' performs asynchronous writes, so errors can
also occur during the dumping process, which will not be
reported back to 'rndc', but which will still be logged by
named.
Closes#4944
Backport of MR !9547
Merge branch 'backport-4944-rndc-dumpdb-do-not-ignore-errors-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9553
Check that with a preexisting dump file without a write permission
no actual dump happens, and rndc returns an error result.
Also add a couple of missing entries in the cleaning script.
(cherry picked from commit 63bddd7192)
The named_server_dumpdb() function, which is called when a 'rndc dumpdb'
command is issued, returns a 'isc_result_t' result code and it has been
always ignored since its introduction in
eb8713ed94, where it was still called
ns_server_dumpdb(). The orignal reasoning is not preserved, but it could
have been also a simple copy-paste mistake, as there are commands, which
return 'void' and require manually setting 'result = ISC_R_SUCCESS;', as
it was done here. Anyway, named will now return the actual result, and
'rndc' will report an error, when the 'dumpdb' command fails.
(cherry picked from commit 8b866ba8a0)
Keep the changelog for each version in a separate source file to avoid
rebase conflicts when handling the -S version.
---
Related !9549Closes#4946
Merge branch 'nicki/reorganize-changelog-docfiles-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9550
Since the changes aren't tracked in the single changelog.rst file,
generate the changelog to stdout instead, so it can be easily redirected
to the proper file.
Due to the recent improvements to the TCP processing, much higher loads
can be handled by BIND9 without causing client timeouts. The updated
parameters give us useful data for both cold and hot cache testing.
Backport of MR !9494
Merge branch 'backport-nicki/increase-tcp-dot-shotgun-load-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9548
Due to the recent improvements to the TCP processing, much higher loads
can be handled by BIND9 without causing client timeouts. The updated
parameters give us useful data for both cold and hot cache testing.
(cherry picked from commit e89afd3349)
The ans4 server wasn't shutting down cleanly sometimes. Check that read returns the
expected value for the message length or exit read loop.
Closes#4301
Backport of MR !9537
Merge branch 'backport-4301-check-read-value-in-ans4-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9545
The ans4 server wasn't shutting down cleanly sometimes. Check that
read returns the expected value for the message length or exit read
loop.
(cherry picked from commit 21baad0a8f)
The shutdown system test sends queries when named is shutting down, not
in an attempt to get answers but to destabilize the server into a crash.
With isctest.query.udp() defaulting to try up to ten times with a
ten-second timeout to get a response we don't care about from a likely
terminated server, we make the test run much longer than needed because
of retries and long timeouts.
Also, see isc-projects/bind9#4943.
Backport of MR !9507
Merge branch 'backport-mnowak/shutdown-downgrade-timeout-and-attempts-arguments-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9541
The shutdown system test sends queries when named is shutting down, not
in an attempt to get answers but to destabilize the server into a crash.
With isctest.query.udp() defaulting to try up to ten times with a
ten-second timeout to get a response we don't care about from a likely
terminated server, we make the test run much longer than needed because
of retries and long timeouts.
(cherry picked from commit 463ab2f3f5)
Incoming transfers that took longer than 30 seconds would stop reading from the TCP stream and the incoming transfer would be indefinitely stuck causing BIND 9 to hang during shutdown.
This has been fixed and the `max-transfer-time-in` and `max-transfer-idle-in` timeouts are now honoured.
Closes#4949
Backport of MR !9531
Merge branch 'backport-4949-fix-ignored-and-invalid-dispatch-timeout-in-dns_xfrin-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9536
The dns_dispatch_add() call in the dns_xfrin unit had hardcoded 30
second limit. This meant that any incoming transfer would be stopped in
it didn't finish within 30 seconds limit. Additionally, dns_xfrin
callback was ignoring the return value from dns_dispatch_getnext() when
restarting the reading from the TCP stream; this could cause transfers
to get stuck waiting for a callback that would never come due to the
dns_dispatch having already been shut down.
Call the dns_dispatch_add() without a timeout and properly handle the
result code from the dns_dispatch_getnext().
(cherry picked from commit 96ef98558c)
The dns_dispatch_add() has timeout parameter that could not be 0 (for
not timeout). Modify the dns_dispatch implementation to accept a zero
timeout for cases where the timeouts are undesirable because they are
managed externally.
(cherry picked from commit 0f810b3144)
The issue of long-running operations potentially blocking query resolution has been fixed. Revert this temporary workaround and restore the number of threadpool threads.
Related #4898
Backport of MR !9530
Merge branch 'backport-4898-remove-workaround-and-note-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9532
when the QP cache was adapted from the RBTDB, some variable names weren't changed, and still incorrectly reference the RBT.
Backport of MR !9515
Merge branch 'backport-each-qpcache-nits-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9525
when the QP cache was adapted from the RBT database, some names
weren't changed. this could be confusing, so let's change them now.
also, we no longer need to include rbt.h.
(cherry picked from commit 5a444838db)
Some 'notify' and 'xfer-in' related log messages were logged at the
'general' category instead of their own category. This has been fixed.
Closes#2730
Backport of MR !9451
Merge branch 'backport-2730-logging-category-for-notify-and-xfer-related-messages-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9514
Some notify/xfer related log messages are logged at the general
category. Set a more suitable caterogry for those messages.
(cherry picked from commit 7c45caa8a5)
If the operating system UDP queue gets full and the outgoing UDP sending
starts to be delayed, BIND 9 could exhibit memory spikes as it tries to
enqueue all the outgoing UDP messages. Try a bit harder to deliver the
outgoing UDP messages synchronously and if that fails, drop the outgoing
DNS message that would get queued up and then timeout on the client side.
Closes#4930
Backport of MR !9506
Merge branch 'backport-4930-limit-the-UDP-send-queue-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9511
If the operating system UDP queue gets full and the outgoing UDP sending
starts to be delayed, BIND 9 could exhibit memory spikes as it tries to
enqueue all the outgoing UDP messages. As those are not going to be
delivered anyway (as we argued when we stopped enlarging the operating
system send and receive buffers), try to send the UDP messages directly
using `uv_udp_try_send()` and if that fails, drop the outgoing UDP
message.
(cherry picked from commit b576c4c977)
We currently set SO_INCOMING_CPU incorrectly, and testing by Ondrej
shows that fixing the issue by setting affinities is worse than letting
the kernel schedule threads without constraints. So we should not set
SO_INCOMING_CPU anymore.
Closes#4936
Backport of MR !9497
Merge branch 'backport-4936-remove-so-incoming-cpu-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9504
We currently set SO_INCOMING_CPU incorrectly, and testing by Ondrej
shows that fixing the issue and setting affinities is worse than letting
the kernel schedule threads without constraints. So we should not set
SO_INCOMING_CPU anymore.
(cherry picked from commit 8b8149cdd2)
When resolving a zone with a 'forward only' policy, and
finding out that all the forwarders are marked as "bad",
the 'ServerQuota' counter of the statistics channel was
incorrectly increased. This has been fixed.
Closes#1793
Backport of MR !9493
Merge branch 'backport-1793-serverquota-counter-bug-with-forward-only-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9502
Check that the fix in the previous commit works and that the
'ServerQuota' counter in the statistics channel is still unset
after a SERVFAIL result in a 'forward only' zone.
(cherry picked from commit 81b3c5d908)
The 'all_spilled' local variable in resolver.c:fctx_getaddresses()
is 'true' by default, and only becomes false when there is at least
one successfully found NS address. However, when a 'forward only;'
configuration is used, the code jumps over the part where it looks
for NS addresses and doesn't reset the 'all_spilled' to false, which
results in incorretly increased 'serverquota' statistics variable,
and also in invalid return error code from the function. The result
code error didn't make any differences, because all codes other than
'ISC_R_SUCCESS' or 'DNS_R_WAIT' were treated in the same way, and
the result code was never logged anywhere.
Set the default value of 'all_spilled' to 'false', and only make it
'true' before actually starting to look up NS addresses.
(cherry picked from commit e430ce7039)
Silence Coverity CID 468757 and 468767 (DATA RACE read not locked) by converting dnssec-signzone to use atomics for statistics counters rather than using a lock.
Closes#4939
Backport of MR !9496
Merge branch 'backport-4939-remove-stats-lock-from-dnssec-signzone-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9500
Silence Coverity CID 468757 and 468767 (DATA RACE read not locked)
by converting dnssec-signzone to use atomics for statistics counters
rather than using a lock. This should be marginally faster than
using the lock as well when statistics are requested.
(cherry picked from commit 473cbd4e87)
As part of the KeyTrap \[CVE-2023-50387\] mitigation, the DNSSEC CPU-intensive operations were offloaded to a separate threadpool that we use to run other tasks that could affect the networking latency.
If that threadpool is running some long-running tasks like RPZ, catalog zone processing, or zone file operations, it would delay DNSSEC validations to a point where the resolving signed DNS records would fail.
Split the CPU-intensive and long-running tasks into separate threadpools in a way that the long-running tasks don't block the CPU-intensive operations.
Closes#4898
Backport of MR !9473
Merge branch 'backport-4898-move-offloaded-DNSSEC-to-own-threads-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9495
Currently, the isc_work API is overloaded. It runs both the
CPU-intensive operations like DNSSEC validations and long-term tasks
like RPZ processing, CATZ processing, zone file loading/dumping and few
others.
Under specific circumstances, when many large zones are being loaded, or
RPZ zones processed, this stops the CPU-intensive tasks and the DNSSEC
validation is practically stopped until the long-running tasks are
finished.
As this is undesireable, this commit moves the CPU-intensive operations
from the isc_work API to the isc_helper API that only runs fast memory
cleanups now.
(cherry picked from commit 8a96a3af6a)
Add an extra thread that can be used to offload operations that would
affect latency, but are not long-running tasks; those are handled by
isc_work API.
Each isc_loop now has matching isc_helper thread that also built on top
of uv_loop. In fact, it matches most of the isc_loop functionality, but
only the `isc_helper_run()` asynchronous call is exposed.
(cherry picked from commit 6370e9b311)
Closes#4634
Backport of MR !9150
Merge branch 'backport-4634-drop-dns.resolver-module-from-system-tests-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9491
When verifying a message in an offloaded thread there is a race with
the worker thread which writes to the same buffer. Clone the message
buffer before offloading.
Closes#4929
Backport of MR !9481
Merge branch 'backport-4929-data-race-in-dns_dnssec_verifymessage-memmove-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9490
When verifying a message in an offloaded thread there is a race with
the worker thread which writes to the same buffer. Clone the message
buffer before offloading.
(cherry picked from commit 35ef25e5ea)
When the libxml2 and libjson-c libraries are not supported, the statistics channel can't return anything useful, so it is now disabled. Use of `statistics-channel` in `named.conf` is a fatal error.
Closes#4895
Backport of MR !9423
Merge branch 'backport-4895-link-style-sheet-to-libxml2-support-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9486
If neither libxml2 nor libjson_c are available have named-checkconf
fail if a statistics-channels block is specified.
(cherry picked from commit b9246418e8)
The `statschannel` system test failed if only one of `libxml2` or `json-c` is
available / configured as checks were being run against the non available
statistics page.
Closes#4919
Backport of MR !9454
Merge branch 'backport-4919-fix-statschannel-system-test-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9483
Forward-port of !9375Closes#4832
Merge branch '4832-bind-arm-review-from-2022-through-9-18-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9480
clang 19 was updated in the base image.
Backport of MR !9475
Merge branch 'backport-mnowak/fix-clang-format-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9477
This change allows fallback from an IXFR failure to AXFR when the reason is `DNS_R_TOOMANYRECORDS`. This is because this error condition could be temporary only in an intermediate version of IXFR transactions and it's possible that the latest version of the zone doesn't have that condition. In such a case, the secondary would never be able to update the zone (even if it could) without this fallback.
This fallback behavior is particularly useful with the recently introduced `max-records-per-type` and `max-types-per-name` options: the primary may not have these limitations and may temporarily introduce "too many" records, breaking IXFR. If the primary side subsequently deletes these records, this fallback will help recover the zone transfer failure automatically; without it, the secondary side would first need to increase the limit, which requires more operational overhead and has its own adverse effect.
Closes#4928
Backport of MR !9333
Merge branch 'backport-fallback-ixfr-to-axfr-on-toomanyrecords-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9471
This change allows fallback from an IXFR failure to AXFR when the
reason is DNS_R_TOOMANYRECORDS. This is because this error condition
could be temporary only in an intermediate version of IXFR
transactions and it's possible that the latest version of the zone
doesn't have that condition. In such a case, the secondary would never
be able to update the zone (even if it could) without this fallback.
This fallback behavior is particularly useful with the recently
introduced max-records-per-type and max-types-per-name options:
the primary may not have these limitations and may temporarily
introduce "too many" records, breaking IXFR. If the primary side
subsequently deletes these records, this fallback will help recover
the zone transfer failure automatically; without it, the secondary
side would first need to increase the limit, which requires more
operational overhead and has its own adverse effect.
This change also fixes a minor glitch that DNS_R_TOOMANYRECORDS wasn't
logged in xfrin_fail.
(cherry picked from commit 7289090683)
The named process could terminate unexpectedly when processing access
control lists (ACLs). This has been fixed.
Closes#4908
Backport of MR !9458
Merge branch 'backport-4908-acl-assertion-failure-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9466
The rcu_xchg_pointer() function can be used outside of a critical
section, and usually must be followed by a synchronize_rcu() or
call_rcu() call to detach from the resource, unless if there are
some guarantees in place because of our own reference counting.
(cherry picked from commit 0367c60759)
This file was initially created for unit testing, but later code was added to generate the file. The static file should have been removed from the git repo.
Closes#4916
Backport of MR !9450
Merge branch 'backport-4916-skr-unit-test-rm-test-file-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9465
This file was initially created for unit testing, but later code was
added to generate the file. The static file should have been removed
from the git repo.
(cherry picked from commit 903534c9a9)
If the ZSK has unlimited lifetime, the timing metadata "Inactive" and "Delete" cannot be found and is treated as an error, preventing the zone to be signed. This has been fixed.
Closes#4914
Backport of MR !9447
Merge branch 'backport-4914-offline-ksk-zsk-lifetime-unlimited-bug-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9453
If the ZSK has lifetime unlimited, the timing metadata "Inactive" and
"Delete" cannot be found and is treated as an error. Fix by allowing
these metadata to not exist.
(cherry picked from commit 5af53a329f)
The statements that already exist in the grammar can't be created with
the namedconf:statement. Use a plain definition list for these
statements and add a manual anchor for each one so links to them can be
created.
Avoid using the :any: syntax in the definition lists, as that just
creates a link to the duplicate and completely unrelated statement,
which just makes the documentation more confusing.
Merge branch 'nicki/prepare-documentation-for-bind-9.20.2' into 'v9.20.2-release'
See merge request isc-private/bind9!738
The statements that already exist in the grammar can't be created with
the namedconf:statement. Use a plain definition list for these
statements and add a manual anchor for each one so links to them can be
created.
Avoid using the :any: syntax in the definition lists, as that just
creates a link to the duplicate and completely unrelated statement,
which just makes the documentation more confusing.
Introduce this temporary workaround to reduce the impact of long-running
tasks in offload threads which can block the resolution of queries.
Related isc-projects/bind9#4898
Backport of isc-private/bind9!740
Merge branch 'backport-4898-workaround-double-threadpool-threads-9.20' into 'v9.20.2-release'
See merge request isc-private/bind9!741
Introduce this temporary workaround to reduce the impact of long-running
tasks in offload threads which can block the resolution of queries.
(cherry picked from commit 6857df20a4)
Under rare circumstances, named could terminate unexpectedly
when validating a DNSKEY resource record if the validation
was canceled in the meantime. This has been fixed.
Closesisc-projects/bind9#4911
Backport of isc-private/bind9!731
Merge branch 'backport-4911-assertion-failure-in-validate_dnskey_dsset_done-9.20' into 'v9.20.2-release'
See merge request isc-private/bind9!734
When a validator is already shut down, val->name becomes NULL. We
need to process and keep the ISC_R_CANCELED or ISC_R_SHUTTINGDOWN
result code before calling validate_async_done(), otherwise, when it
is called with the hardcoded DNS_R_NOVALIDSIG result code, it can
cause an assetion failure when val->name (being NULL) is used in
proveunsecure().
(cherry picked from commit d85918aebf)
A very rare assertion failure can be triggered when the incoming transfer is either forcefully shut down or it is finished during printing the details about the statistics channel. This has been fixed.
Closes#4860
Backport of MR !9336
Merge branch 'backport-4860-destroy-xfrin-timers-on-the-loop-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9377
There are few places where we attach/detach from the dns_xfrin object
while running on a different thread than the zone's assigned thread -
xfrin_xmlrender() in the statschannel and dns_zone_stopxfr() to name the
two places where it happens now. In the rare case, when the incoming
transfer completes (or shuts down) in the brief period between the other
thread attaches and detaches from the dns_xfrin, the isc_timer_destroy()
calls would be called by the last thread calling the xfrin_detach().
In the worst case, it would be this other thread causing assertion
failure. Move the isc_timer_destroy() call to xfrin_end() function
which is always called on the right thread and to match this move
isc_timer_create() to xfrin_start() - although this other change makes
no difference.
(cherry picked from commit 3bca3cb5cf)
Administrators may wish to constrain the set of cores that BIND 9 runs on via the 'taskset', 'cpuset' or 'numactl' programs (or equivalent on other O/S).
If the admin has used taskset, the `named` will now follow to automatically use the given number of CPUs rather than the system wide count.
Closes#4884
Backport of MR !9398
Merge branch 'backport-4884-use-cpuset-to-get-number-of-cpus-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9442
Administrators may wish to constrain the set of cores that BIND 9 runs
on via the 'taskset', 'cpuset' or 'numactl' programs (or equivalent on
other O/S), for example to achieve higher (or more stable) performance
by more closely associating threads with individual NIC rx queues. If
the admin has used taskset, it follows that BIND ought to
automatically use the given number of CPUs rather than the system wide
count.
Co-Authored-By: Ray Bellis <ray@isc.org>
(cherry picked from commit 5a2df8caf5)
Instead of keeping the whole array of test_node_t objects, just keep the
hashvalues and generated the rest of the test_node_t on the fly. The
test still works this way and the file size has been reduced from 2M to
90k.
Closes#4851
Backport of MR !9318
Merge branch 'backport-4851-generate-problematic-isc_hashmap-test-data-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9321
Instead of keeping the whole array of test_node_t objects, just keep the
hashvalues and generated the rest of the test_node_t on the fly. The
test still works this way and the file size has been reduced from 2M to
90k.
(cherry picked from commit 2310c322c0)
find->status is a private field of dns_adbfind_t so it now has an accessor function and has been made atomic.
Closes#4802
Backport of MR !9137
Merge branch 'backport-4802-address-potential-tsan-issue-with-find-status-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9433
Use the fact that the database returns the longest matching part of the requested name to find the required NSEC3 record. If there are multiple versions present in the database we may have to search further.
Closes#4460
Backport of MR !9436
Merge branch 'backport-4460-auth-nsec3-many-labels-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9438
Return partial match from dns_db_find/dns_db_find when requested
to short circuit the closest encloser discover process. Most of the
time this will be the actual closest encloser but may not be when
there yet to be committed / cleaned up versions of the zone with
names below the actual closest encloser.
(cherry picked from commit d42ea08f16)
Delay relinquishing root privileges until the control channel has been configured, for the benefit of systems that require root to use privileged port numbers. This mostly affects systems without fine-grained privilege systems (i.e., other than Linux).
Closes#4793
Backport of MR !9123
Merge branch 'backport-4793-bind-9-19-24-not-listening-to-rndc-port-953-on-localhost-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9444
On systems where root access is needed to configure privileged
ports, we don't want to fully relinquish root privileges until
after the control channel (which typically runs on port 953) has
been established.
named_os_changeuser() now takes a boolean argument 'permanent'.
This allows us to switch the effective userid temporarily with
named_os_changeuser(false) and restore it with named_os_restoreuser(),
before permanently dropping privileges with named_os_changeuser(true).
(cherry picked from commit d57fa148af)
With max-recursion-queries set to 50 the resolver system test was
unstable in the "checking query resolution for a domain with a valid
glueless delegation chain" check as ns1 replied with SERVFAIL.
Closes#4897
Backport of MR !9435
Merge branch 'backport-4897-resolver-ns1-max-recursion-queries-100-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9440
With max-recursion-queries set to 50 the resolver system test was
unstable in the "checking query resolution for a domain with a valid
glueless delegation chain" check as ns1 replied with SERVFAIL.
(cherry picked from commit 8e0244d300)
Move lock earlier in the call sequence to address access without lock report.
```
1559 /*
1560 * Caller must be holding the fctx lock.
1561 */
CID 468796: (#1 of 1): Data race condition (MISSING_LOCK)
1. missing_lock: Accessing fctx->state without holding lock fetchctx.lock. Elsewhere, fetchctx.state is written to with fetchctx.lock held 2 out of 2 times.
1562 REQUIRE(fctx->state == fetchstate_done);
1563
1564 FCTXTRACE("sendevents");
1565
1566 LOCK(&fctx->lock);
1567
```
Closes#4902
Backport of MR !9427
Merge branch 'backport-4902-accessing-fctx-state-without-holding-lock-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9437
fctx->state should be read with the lock held.
1559 /*
1560 * Caller must be holding the fctx lock.
1561 */
CID 468796: (#1 of 1): Data race condition (MISSING_LOCK)
1. missing_lock: Accessing fctx->state without holding lock fetchctx.lock.
Elsewhere, fetchctx.state is written to with fetchctx.lock held 2 out of 2 times.
1562 REQUIRE(fctx->state == fetchstate_done);
1563
1564 FCTXTRACE("sendevents");
1565
1566 LOCK(&fctx->lock);
1567
(cherry picked from commit 43f0b0e8eb)
Backport of MR !9424
Merge branch 'backport-mnowak/avoid-some-artifacts-in-stress-tests-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9428
The `malloc_usable_size()` can return size larger than originally allocated and when these sizes disagree the fortifier enabled by `_FORTIFY_SOURCE=3` detects overflow and stops the `named` execution abruptly. Stop using these convenience functions as they are primary used for introspection-only.
Closes#4880
Backport of MR !9400
Merge branch 'backport-4880-dont-use-malloc_usable_size-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9418
Although the nanual page of malloc_usable_size says:
Although the excess bytes can be over‐written by the application
without ill effects, this is not good programming practice: the
number of excess bytes in an allocation depends on the underlying
implementation.
it looks like the premise is broken with _FORTIFY_SOURCE=3 on newer
systems and it might return a value that causes program to stop with
"buffer overflow" detected from the _FORTIFY_SOURCE. As we do have own
implementation that tracks the allocation size that we can use to track
the allocation size, we can stop relying on this introspection function.
Also the newer manual page for malloc_usable_size changed the NOTES to:
The value returned by malloc_usable_size() may be greater than the
requested size of the allocation because of various internal
implementation details, none of which the programmer should rely on.
This function is intended to only be used for diagnostics and
statistics; writing to the excess memory without first calling
realloc(3) to resize the allocation is not supported. The returned
value is only valid at the time of the call.
Remove usage of both malloc_usable_size() and malloc_size() to be on the
safe size and only use the internal size tracking mechanism when
jemalloc is not available.
(cherry picked from commit d61712d14e)
Fetches generated automatically as a result of 'prefetch' are now
exempt from the 'fetches-per-zone' and 'fetches-per-server' quotas.
This should help in maintaining the cache from which query responses
can be given.
Closes#4219
Backport of MR !9095
Merge branch 'backport-4219-exempt-good-queries-from-fetch-limits-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9420
Add notes to the 'fetches-per-zone' and 'fetches-per-server' clauses
documentation to document that prefetches are exempted.
(cherry picked from commit ad329ddcaa)
Give prefetches a free pass through the quota so that the cache
entries for popular zones could be updated successfully even if the
quota for is already reached.
(cherry picked from commit c7e8b7cf63)
Give prefetches a free pass through the quota so that the cache entry
for a popular zone could be updated successfully even if the quota for
it is already reached.
(cherry picked from commit cada2de31f)
The cross-version-config-tests job fails when a system test is removed
from the upcoming release. To avoid this, remove the system test also
from the $BIND_BASELINE_VERSION.
See the failure mode at https://gitlab.isc.org/isc-projects/bind9/-/jobs/4668947.
Backport of MR !9413
Merge branch 'backport-mnowak/remove-dialup-from-cross-version-config-tests-job-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9415
The cross-version-config-tests job fails when a system test is removed
from the upcoming release. To avoid this, remove the system test also
from the $BIND_BASELINE_VERSION.
(cherry picked from commit 60f5f2a9d9)
This supports bit-for-bit reproducibility of built documentation.
Closes#4886
Backport of MR !9399
Merge branch 'backport-issue-4886/order-preserving-documentation-tags-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9408
The 'set' datatype in Python does not provide iteration-order
guarantees related to insertion-order. That means that its
usage in the 'split_csv' helper function during documentation
build can produce nondeterministic results.
That is non-desirable for two reasons: it means that the
documentation output may appear to vary unnecessarily between
builds, and secondly there could be loss-of-information in cases
where tag order in the source documentation is significant.
This patch implements order-preserving de-duplication of tags,
allowing authors to specify tags using intentional priority
ordering, while also removing tags that appear more than once.
(cherry picked from commit 5a79b36f56)
Milestone is automatically set by 'merged-metadata' CI job to 'Not
released yet' so it does not matter what the original value was.
(cherry picked from commit 73b950dc29)
1. Set milestone to 'Not released yet' after merge
We will set milestone to actual version number when we actually tag a
particular version. This will get rid of mass MR reassignment when we
do last minute changes to a release plan etc.
2. Adjust No CHANGES and Release Notes MR labels to match gitchangelog
workflow.
(cherry picked from commit d1c64d588b)
Previously CI job for the autobackport bot inherited "interruptible:
true" global configuration. This caused premature termination of the job
when another merge was finished before the autobackport job ran to
completion.
(cherry picked from commit 3165261ecd)
It is useful when multiple signers are being used
to sign a zone to able to specify a restricted
range of range of key tags that will be used by an
operator to sign the zone. This adds controls to
named (dnssec-policy), dnssec-signzone, dnssec-keyfromlabel and
dnssec-ksr (dnssec-policy) to specify such ranges.
Closes#4830
Backport of MR !9258
Merge branch 'backport-4830-support-restricted-key-tag-range-when-generating-new-keys-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9396
A new argument has been added to dnssec-keygen and dnssec-keyfromlabel
to restrict the tag value of key generated / imported to a particular
range. This is intended to be used by multi-signers.
Co-authored-by: Suzanne Goldlust <sgoldlust@isc.org>
(cherry picked from commit 0c347fb321)
If there is an algorithm rollover and two keys of different algorithm share the same keytags, then there is a possibility that if we check that a key matches a specific state, we are checking against the wrong key. This has been fixed by not only checking for matching key tag but also key algorithm.
Closes#4878
Backport of MR !9381
Merge branch 'backport-4878-fix-algorithm-rollover-keytag-conflict-bug-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9393
If there is an algorithm rollover and two keys of different algorithm
share the same keytags, then there is a possibility that if we check
that a key matches a specific state, we are checking against the wrong
key.
Fix this by not only checking for matching key id but also key
algorithm.
(cherry picked from commit f37eb33f29)
If there is a keytag conflict between keys with different algorithms,
we need to supply what key algorithm is used so we can get the right
public key.
For clarity, print the algorithm on the found keys after 'check_keys'.
(cherry picked from commit 7bb6d82505)
With the release of BIND 9.19.25 this workaround is not needed anymore.
This reverts commit c58dd6e78f.
Backport of MR !9065
Merge branch 'backport-mnowak/revert-workaround-temporary-cross-version-config-tests-limitation-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9395
Add a new configuration option `offline-ksk` to enable Offline KSK key management. Signed Key Response (SKR) files created with `dnssec-ksr` (or other program) can now be imported into `named` with the new `rndc skr -import` command. Rather than creating new DNSKEY, CDS and CDNSKEY records and generating signatures covering these types, these records are loaded from the currently active bundle from the imported SKR.
The implementation is loosely based on: https://www.iana.org/dnssec/archive/files/draft-icann-dnssec-keymgmt-01.txtCloses#1128
Backport of MR !9119
Merge branch 'backport-1128-offline-ksk-rndc-import-skr-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9389
Some things we no longer want to do when we are in offline-ksk mode.
1. Don't check for inactive and private keys if the key is a KSK.
2. Don't update the TTL of DNSKEY, CDS and CDNSKEY RRset, these come
from the SKR.
(cherry picked from commit 2e3068ed60)
When it is time to generate a new signature (dns_dnssec_sign), rather
than create a new one, retrieve it from the SKR.
(cherry picked from commit 61cf599fbf)
When we are appending contents of a DNSKEY rdataset to a keylist,
don't attempt to read the private key file of a KSK when we are in
offline-ksk mode.
(cherry picked from commit 30d20b110e)
With offline-ksk enabled, we don't run the keymgr because the key
timings are determined by the SKR. We do update the key states but
we derive them from the timing metadata.
Then, we can skip a other tasks in offline-ksk mode, like DS checking
at the parent and CDS synchronization, because the CDS and CDNSKEY
RRsets also come from the SKR.
(cherry picked from commit 2190aa904f)
When a zone has a skr structure, lookup the currently active bundle
that contains the right key and signature material.
(cherry picked from commit 63e058c29e)
This added source code stores SKR data. It is loosely based on:
https://www.iana.org/dnssec/archive/files/draft-icann-dnssec-keymgmt-01.txt
A SKR contains a list of signed DNSKEY RRsets. Each change in data
should be stored in a separate bundle. So if the RRSIG is refreshed that
means it is stored in the next bundle. Likewise, if there is a new ZSK
pre-published, it is in the next bundle.
In addition (not mentioned in the draft), each bundle may contain
signed CDS and CDNSKEY RRsets.
Each bundle has an inception time. These will determine when we need
to re-sign or re-key the zone.
(cherry picked from commit 445722d2bf)
Test importing a Signed Key Response. Files should be loaded and once
loaded the correct bundle should be used. Alsoe test cases where the
bundle is not the first bundle in the SKR.
(cherry picked from commit afe093258c)
Add the code and documentation required to provide KSR import using
rndc. This is just the command, and the feature is at this point in
time still not implemented.
(cherry picked from commit edbb219fda)
Prepare the system test for tests that require a server to import
created SKR files. This means the test script needs adjustments to
take into account the directory test files are located.
In addition, the check_keys function is renamed to ksr_check_keys
because the name clashes with check_keys from kasp.sh. It also has an
extra parameter added, offset, that can be used to check ksr files that
are created in the past or future.
(cherry picked from commit 367154c1de)
Add a new configuration option to enable Offline KSK key management.
Offline KSK cannot work with CSK because it splits how keys with the
KSK and ZSK role operate. Therefore, one key cannot have both roles.
Add a configuration check to ensure this.
(cherry picked from commit 0598381236)
This adds the command line arguments: `-d` (debug), `-l` (list tests) and `-t test` (run this test) to the unit tests.
e.g.
```
% ./rdata_test -t zonemd
[==========] selected: Running 1 test(s).
[ RUN ] zonemd
[ OK ] zonemd
[==========] selected: 1 test(s) run.
[ PASSED ] 1 test(s).
%
```
Closes#4579
Backport of MR !9384
Merge branch 'backport-4579-restore-the-ability-to-select-individual-unit-tests-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9385
named now prints its initial working directory during startup and the
changed working directory when loading or reloading its configuration
file if it has a valid 'directory' option defined.
Closes#4731
Backport of MR !9362
Merge branch 'backport-4731-log-workdir-full-path-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9372
named now prints its initial working directory during startup and the
changed working directory when loading or reloading its configuration
file if it has a valid 'directory' option defined.
(cherry picked from commit fd8e1d161f)
It looks like that all supported systems now have support for
_POSIX_SAVED_IDS, so it's safe to use setegid() and setegid() because
those will not change saved used/group IDs.
Closes#4862
Backport of MR !9363
Merge branch 'backport-4862-just-use-seteuid-and-setegid-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9371
It looks like that all supported systems now have support for
_POSIX_SAVED_IDS, so it's safe to use setegid() and setegid() because
those will not change saved used/group IDs.
(cherry picked from commit ab517fc6e4)
If the machine running the job is extra loaded (or extra slow), the
print test in the badcache unit test would fail because the TTL would be
59 (possibly even lower) and the test expects the badcache.out to only
have TTL 60. Refactor the test to check for the expected strings and
check whether the TTL is in the expected range.
Closes#4861
Backport of MR !9338
Merge branch 'backport-4861-allow-TTL-skew-in-badcache_test-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9368
If the machine running the job is extra loaded (or extra slow), the
print test in the badcache unit test would fail because the TTL would be
59 (possibly even lower) and the test expects the badcache.out to only
have TTL 60. Refactor the test to check for the expected strings and
check whether the TTL is in the expected range.
(cherry picked from commit 7860b6519d)
On FreeBSD, the TCP connection callback could spuriously receive
ISC_R_CONNECTIONRESET even when connection to the loopback interface.
Skip the other checks in such case and graciously shutdown the TCP
connection.
Closes#4849
Backport of MR !9303
Merge branch 'backport-4849-dispatch_test-fails-in-connected_shutdown-on-freebsd-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9366
On FreeBSD, the TCP connection callback could spuriously receive
ISC_R_CONNECTIONRESET even when connection to the loopback interface.
Skip the other checks in such case and graciously shutdown the TCP
connection.
(cherry picked from commit e53cb61cf7)
Related: #4847
Backport of MR !9300
Merge branch 'backport-4847-changelog-sorting-and-tweaks-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9364
When manually handling the release notes (due to rst markup, fixups
etc.), the different MR number for backports causes needless friction.
Remove the reference from release notes and keep it only in changelog
which isn't manually redacted.
(cherry picked from commit 993ba7cc7f)
To reduce the friction when handling the release notes, it is preferable
to have the sections sorted by issue number, rather than merge order.
Fallback to commit subject line if unavailable (e.g. for changelog
entries).
(cherry picked from commit 759948fffe)
The shim implementation of setresuid() was wrong - there was a copy and
paste error and it was calling setresgid() instead. This only affects
NetBSD because Linux, FreeBSD and OpenBSD have setresuid() and
setresgid() implementation available from the system library.
Closes#4862
Backport of MR !9359
Merge branch 'backport-4862-fix-setresuid-shim-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9361
The shim implementation of setresuid() was wrong - there was a copy and
paste error and it was calling setresgid() instead. This only affects
NetBSD because Linux, FreeBSD and OpenBSD have setresuid() and
setresgid() implementation available from the system library.
(cherry picked from commit 5567407a82)
Since we've started the new changelog & release notes process, the file
for drafting release notes for the version-to-be is no longer needed.
(cherry picked from commit c795cfec54)
The test_traffic_json and test_traffic_xml occasionally fail when
running under TSAN. This happens in CI and is most likely a result of
some instability that doesn't seem to be easily reproduced.
Closes#4598
Backport of MR !9293
Merge branch 'backport-4598-mark-statschannel-test-flaky-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9350
The test_traffic_json and test_traffic_xml occasionally fail when
running under TSAN. This happens in CI and is most likely a result of
some instability that doesn't seem to be easily reproduced.
(cherry picked from commit ec2fc7680a)
Log canceled resolver queries (e.g. when shutting down a hung
fetch) in DEBUG3 level instead of DEBUG1 which is used for the
"unrecognized" result codes.
Closes#4797
Backport of MR !9148
Merge branch 'backport-4797-rpz_rewrite-add-ISC_R_CANCELED-processing-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9347
Log canceled queries (e.g. when shutting down a hung fetch)
in DEBUG3 level instead of DEBUG1 which is used for the
"unrecognized" result codes.
(cherry picked from commit 8bb9568467)
In rpz system tests, we could leak file if the applying the updates has
failed. Add the missing fclose() before returning.
Backport of MR !9317
Merge branch 'backport-ondrej/add-missing-flose-to-rpz-testlib-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9345
In rpz system tests, we could leak file if the applying the updates has
failed. Add the missing fclose() before returning.
(cherry picked from commit 2855ec8f5f)
The getifaddr() works fine for years, so we don't have to
keep the callback to parse /proc/net/if_inet6 anymore.
Closes#4852
Backport of MR !9315
Merge branch 'backport-4852-handle-errors-from-rewind-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9341
The getifaddr() works fine for years, so we don't have to
keep the callback to parse /proc/net/if_inet6 anymore.
(cherry picked from commit 2fbf9757b8)
The clang-scan 19 has reported that we are ignoring errno after the call
to rewind(). As we don't really care about the result, just silence the
error, the whole code will be removed in the development version anyway
as it is not needed.
(cherry picked from commit dda5ba53df)
Instead of directly using the result of dirfd() in the unlinkat() call,
check whether the returned file descriptor is actually valid. That
doesn't really change the logic as the unlinkat() would fail with
invalid descriptor anyway, but this is cleaner and will report the right
error returned directly by dirfd() instead of EBADF from unlinkat().
Closes#4853
Backport of MR !9316
Merge branch 'backport-4853-check-result-of-dirfd-in-isc_log-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9343
Instead of directly using the result of dirfd() in the unlinkat() call,
check whether the returned file descriptor is actually valid. That
doesn't really change the logic as the unlinkat() would fail with
invalid descriptor anyway, but this is cleaner and will report the right
error returned directly by dirfd() instead of EBADF from unlinkat().
(cherry picked from commit 59f4fdebc0)
New version of clang (19) has introduced a stricter checks when mixing
integer (and float types) with enums. In this case, we used enum {}
as C17 doesn't have constexpr yet. Change the time conversion constants
to be static const unsigned int instead of enum values.
Closes#4845
Backport of MR !9313
Merge branch 'backport-4845-change-NS_PER_SEC-type-from-enum-to-integer-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9339
The contexpr introduced in C23 standard makes perfect sense to be used
instead of preprocessor macros - the symbols are kept, etc. Define
ISC_CONSTEXPR to be `constexpr` for C23 and `static const` for the older
C standards. Use the newly introduced macro for the NS_PER_SEC and
friends time constants.
(cherry picked from commit 122a142241)
New version of clang (19) has introduced a stricter checks when mixing
integer (and float types) with enums. In this case, we used enum {}
as C17 doesn't have constexpr yet. Change the time conversion constants
to be static const unsigned int instead of enum values.
(cherry picked from commit b03e90e0d4)
The new TSAN images, the TSAN-enabled images install libraries to
opt/tsan, synchronize the configure options and CFLAGS between gcc:tsan
and clang:tsan images and set the PKG_CONFIG_PATH to /opt/tsan/lib.
Additionally, drop Debian bullseye that's EOL now.
Backport of MR !9324
Merge branch 'backport-ondrej/use-staging-tsan-images-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9325
The new TSAN images, the TSAN-enabled images install libraries to
/opt/tsan, synchronize the configure options and CFLAGS between gcc:tsan
and clang:tsan images and set the PKG_CONFIG_PATH to /opt/tsan/lib.
(cherry picked from commit 2a46396f29)
Check if 'lctx->logconfig' is NULL before using it in isc_log_doit(),
because it's possible that isc_log_destroy() was already called, e.g.
when a 'call_rcu' function wants to log a message during shutdown.
Closes#4842
Backport of MR !9297
Merge branch 'backport-4842-shutdown-crash-in-isc_log_doit-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9323
Check if 'lctx->logconfig' is NULL before using it in isc_log_doit(),
because it's possible that isc_log_destroy() was already called, e.g.
when a 'call_rcu' function wants to log a message during shutdown.
(cherry picked from commit 656e04f48a)
GCC 11.1+ emits a note during compilation when there are 64-bit
atomic fields in a structure, because it fixed a compiler bug
by changing the alignment of such fields, which caused ABI change.
Add -Wno-psabi to CFLAGS for such builds in order to silence the
warning. That shouldn't be a problem since we don't expose our
structures to the outside.
Closes#4841
Backport of MR !9319
Merge branch 'backport-4841-alignment-of-_Atomic-long-long-unsigned-int-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9322
GCC 11.1+ emits a note during compilation when there are 64-bit
atomic fields in a structure, because it fixed a compiler bug
by changing the alignment of such fields, which caused ABI change.
Add -Wno-psabi to CFLAGS for such builds in order to silence the
warning. That shouldn't be a problem since we don't expose our
structures to the outside.
(cherry picked from commit 867066aa53)
As we now setup the logging very early, parsing the default config would
always print warnings about experimental (and possibly deprecated)
options in the default config. This would even mess with commands like
`named -V` and it is also wrong to warn users about using experimental
options in the default config, because they can't do anything about
this. Add CFG_PCTX_NODEPRECATED and CFG_PCTX_NOEXPERIMENTAL options
that we can pass to cfg parser and silence the early warnings caused by
using experimental options in the default config.
Backport of MR !9304
Merge branch 'backport-ondrej/silence-warnings-from-default-config-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9305
As we now setup the logging very early, parsing the default config would
always print warnings about experimental (and possibly deprecated)
options in the default config. This would even mess with commands like
`named -V` and it is also wrong to warn users about using experimental
options in the default config, because they can't do anything about
this. Add CFG_PCTX_NODEPRECATED and CFG_PCTX_NOEXPERIMENTAL options
that we can pass to cfg parser and silence the early warnings caused by
using experimental options in the default config.
(cherry picked from commit 86f1ec34dc)
When the round robin hashing reorders the map entries on deletion, we
were adjusting the iterator table size only when the reordering was
happening at the internal table boundary. The iterator table size had
to be reduced by one to prevent seeing the entry that resized on
position [0] twice because it migrated to [iter->size - 1] position.
However, the same thing could happen when the same entry migrates a
second time from [iter->size - 1] to [iter->size - 2] position (and so
on) because the check that we are manipulating the entry just in the [0]
position was insufficient. Instead of checking the position [pos == 0],
we now check that the [pos % iter->size == 0], thus ignoring all the
entries that might have moved back to the end of the internal table.
Closes#4838
Backport of MR !9292
Merge branch 'backport-4838-fix-assertion-failure-in-hashmap-deletion-iterator-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9310
When iterating through the old internal hashmap table, skip all the
nodes that have been already migrated to the new table. We know that
all positions with index less than .hiter are NULL.
(cherry picked from commit 3e4d153453)
When the round robin hashing reorders the map entries on deletion, we
were adjusting the iterator table size only when the reordering was
happening at the internal table boundary. The iterator table size had
to be reduced by one to prevent seeing the entry that resized on
position [0] twice because it migrated to [iter->size - 1] position.
However, the same thing could happen when the same entry migrates a
second time from [iter->size - 1] to [iter->size - 2] position (and so
on) because the check that we are manipulating the entry just in the [0]
position was insufficient. Instead of checking the position [pos == 0],
we now check that the [pos % iter->size == 0], thus ignoring all the
entries that might have moved back to the end of the internal table.
(cherry picked from commit acdc57259f)
Add second iterator test with a well-known configuration of the nodes
array that causes the last element of the array to migrate two times to
the previous node.
(cherry picked from commit 482eed2e31)
Since changelog entries are now generated from MR title&description,
they aren't sanity checked during a regular docs build. If these contain
special sequences that will be interpreted by sphinx, it might result in
breakage that would have to be amended manually.
Add a CI check to test a doc build with changelog after the MR is merged
to ensure that the docs can be built when generating changelog from
pristine git contents.
Related #4847
Backport of MR !9294
Merge branch 'backport-nicki/add-changelog-entry-check-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9311
Some distributions (notably, debian bookworm) have deprecated the
`python` interpreter in favor of `python3`. Since our scripts are
python3 anyway, use the proper numbered version in shebang to make
scripts easily executable.
(cherry picked from commit 480dcdef9a)
Since changelog entries are now generated from MR title&description,
they aren't sanity checked during a regular docs build. If these contain
special sequences that will be interpreted by sphinx, it might result in
breakage that would have to be amended manually.
Add a CI check to test a doc build with changelog after the MR is merged
to ensure that the docs can be built when generating changelog from
pristine git contents.
(cherry picked from commit bf69e8f149)
Ensure the issue number in changelog isn't accidentally removed for backport MRs.
Backport of MR !9295
Merge branch 'backport-nicki/fix-gitchangelog-replacement-regexs-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9308
Prior to this change, the issue number could be accidentally removed by
the `Backport of` text, depending on the order of the MR description
contents. Ensure all the removals for text in MR descriptions happen
first, and only then run the replacement regex for issue number, which
appends it to the end of the last non-empty line (which will no longer
be removed).
The only removals that happen after the replacement are guaranteed to
always happen after the end of MR description, since they're
auto-generated by gitlab when the merge commit is created, thus won't
affect the line with the issue number.
Also remove the needless isc-private/bind9 replacement. References
to private MRs are already removed by the very first regex.
(cherry picked from commit 37274bebe1)
During a reconfiguration named wasn't reconfiguring catalog zones' member zones. This has been fixed.
Backport of MR !695Closesisc-projects/bind9#4733
Merge branch '4733-fix-catz-member-zones-reconfigure-9.20' into 'v9.20.1-release'
See merge request isc-private/bind9!726
This condition in catalog zone processing functions is checked only
for sanity, so there is no meaning for a log message that normally
shouldn't be ever logged.
(cherry picked from commit 0b3c8ee55e6cda1f2de35aa15f6a4d82d072c309)
During a reconfiguration named doesn't reconfigure catalog zones
member zones. Implement the necessary code to reconfigure catz
member zones.
(cherry picked from commit fd71ed5d73ae2dc968e8782ce652fb8ef257c25e)
Catalog zone member zones should be reconfigured as all the other
zones during a reconfiguration. Test it by checking whether the newly
added allow-query setting affects a member zone.
(cherry picked from commit cd04b89dba85781c194f22ce6fe358c972a14758)
The fcount_incr() was not increasing counter->count when force was set
to true, but fcount_decr() would try to decrease the counter leading to
underflow and assertion failure. Swap the order of the arguments in the
condition, so the !force is evaluated after incrementing the .count.
Closes#4846
Backport of MR !9298
Merge branch 'backport-4786-forced-fcount_incr-should-increment-count-and-allowed-fix-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9299
The fcount_incr() was not increasing counter->count when force was set
to true, but fcount_decr() would try to decrease the counter leading to
underflow and assertion failure. Swap the order of the arguments in the
condition, so the !force is evaluated after incrementing the .count.
(cherry picked from commit 8e86e55af1)
LaTeX in CI and on ReadTheDocs [fails][1] to render a PDF version of ARM if
the Changelog section is included. The running theory is that the
verbatim section of more than twenty thousand lines is too big to meet
LaTeX self-imposed constraints, and it fails with:
! TeX capacity exceeded, sorry [main memory size=5000000].
Or it just hangs if extra_mem_bot=30000000 is set in
/etc/texmf/texmf.d/01main_memory_bump.cnf:
! Dimension too large.
\fb@put@frame ...p \ifdim \dimen@ >\ht \@tempboxa
\fb@putboxa #1\fb@afterfra...
l.56913 \end{sphinxVerbatim}
Make each BIND 9 release a separate code block to work around the issue.
Further split up the sections for some exceptionally large releases, for
the same reason.
[1]: https://gitlab.isc.org/isc-projects/bind9/-/jobs/4584011
Backport of MR !9266
Merge branch 'backport-mnowak/fix-arm-changelog-section-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9289
LaTeX in CI and on ReadTheDocs fails to render a PDF version of ARM if
the Changelog section is included. The running theory is that the
verbatim section of more than twenty thousand lines is too big to meet
LaTeX self-imposed constraints, and it fails with:
! TeX capacity exceeded, sorry [main memory size=5000000].
Or it just hangs if extra_mem_bot=30000000 is set in
/etc/texmf/texmf.d/01main_memory_bump.cnf:
! Dimension too large.
\fb@put@frame ...p \ifdim \dimen@ >\ht \@tempboxa
\fb@putboxa #1\fb@afterfra...
l.56913 \end{sphinxVerbatim}
Make each BIND 9 release a separate code block to work around the issue.
Further split up the sections for some exceptionally large releases, for
the same reason.
(cherry picked from commit bc802359b0)
There were cases in resolver.c when the `max-recursion-queries` quota was ineffective. It was possible to craft zones that would cause a resolver to waste resources by sending excessive queries while attempting to resolve a name. This has been addressed by correcting errors in the implementation of `max-recursion-queries`, and by reducing the default value from 100 to 32.
In addition, a new `max-query-restarts` option has been added which limits the number of times a recursive server will follow CNAME or DNAME records before terminating resolution. This was previously a hard-coded limit of 16, and now defaults to 11.
Closes#4741
Backport of MR !9281
Merge branch 'backport-4741-reclimit-restarts-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9282
max-query-restarts and max-recursion-queries values can now be set
on the command line in delv for testing purposes.
(cherry picked from commit 0d010ddebe)
implement, document, and test the 'max-query-restarts' option
which specifies the query restart limit - the number of times
we can follow CNAMEs before terminating resolution.
(cherry picked from commit 104f3b82fb)
MAX_RESTARTS is no longer hard-coded; ns_server_setmaxrestarts()
and dns_client_setmaxrestarts() can now be used to modify the
max-restarts value at runtime. in both cases, the default is 11.
(cherry picked from commit c5588babaf)
the number of steps that can be followed in a CNAME chain
before terminating the lookup has been reduced from 16 to 11.
(this is a hard-coded value, but will be made configurable later.)
(cherry picked from commit 05d78671bb)
previously, validator queries for DNSKEY and DS records were
not counted toward the quota for max-recursion-queries; they
are now.
(cherry picked from commit af7db89513)
there were cases in resolver.c when queries for NS records were
started without passing a pointer to the parent fetch's query counter;
as a result, the max-recursion-queries quota for those queries started
counting from zero, instead of sharing the limit for the parent fetch,
making the quota ineffective in some cases.
(cherry picked from commit d3b7e92783)
The new Fedora 40 TSAN images use libuv, urcu and OpenSSL libraries compiled with ThreadSanitizer. This (in theory) should enable better detection of memory races in those (most important) libraries.
Backport of MR !9264
Merge branch 'backport-ondrej/test-new-tsan-images-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9276
The TSAN-enabled libraries are installed to /usr/local, pass the
PKG_CONFIG_PATH and few other options to CFLAGS to the configure
arguments.
(cherry picked from commit ed766efc15)
When the SSL object was destroyed, it would invalidate all SSL_SESSION
objects including the cached, but not yet used, TLS session objects.
Properly disassociate the SSL object from the SSL_SESSION before we
store it in the TLS session cache, so we can later destroy it without
invalidating the cached TLS sessions.
Closes#4834
Backport of MR !9271
Merge branch 'backport-4834-detach-SSL-from-cached-SSL_SESSION-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9274
When the SSL object was destroyed, it would invalidate all SSL_SESSION
objects including the cached, but not yet used, TLS session objects.
Properly disassociate the SSL object from the SSL_SESSION before we
store it in the TLS session cache, so we can later destroy it without
invalidating the cached TLS sessions.
Co-authored-by: Ondřej Surý <ondrej@isc.org>
Co-authored-by: Artem Boldariev <artem@isc.org>
Co-authored-by: Aram Sargsyan <aram@isc.org>
(cherry picked from commit c11b736e44)
When TLS connection (TLSstream) connection was accepted, the children
listening socket was not attached to sock->server and thus it could have
been freed before all the accepted connections were actually closed.
In turn, this would cause us to call isc_tls_free() too soon - causing
cascade errors in pending SSL_read_ex() in the accepted connections.
Properly attach and detach the children listening socket when accepting
and closing the server connections.
Closes#4833
Backport of MR !9270
Merge branch 'backport-4833-tlssock-needs-to-attach-to-child-tlslistener-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9273
When TLS connection (TLSstream) connection was accepted, the children
listening socket was not attached to sock->server and thus it could have
been freed before all the accepted connections were actually closed.
In turn, this would cause us to call isc_tls_free() too soon - causing
cascade errors in pending SSL_read_ex() in the accepted connections.
Properly attach and detach the children listening socket when accepting
and closing the server connections.
(cherry picked from commit 684f3eb8e6)
Missing file util/dtrace.sh prevented builds on system without dtrace utility.
This has been corrected.
Fixes: #4835
Backport of MR !9262
Merge branch 'backport-pspacek/gitattribute-fixes-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9272
Ensure that system tests can be executed without Python hypothesis
package.
Closes#4831
Backport of MR !9265
Merge branch 'backport-4831-isctest-make-hypothesis-optional-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9267
Query responses should contain the question section with some exceptions. Dig was not reporting this.
Closes#4808
Backport of MR !9233
Merge branch 'backport-4808-have-dig-report-missing-question-section-in-axfr-response-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9269
The question section should be present in the first AXFR/IXFR
response and in other QUERY responses unless no question was sent.
Issue a warning if the question section is not present.
(cherry picked from commit 327e890910)
Ensure that the selected algorithms remains stable throughout the entire test session. Crypto support detection was rewritten to python and simplified.
Closes#4202Closes#4422
Related #3810
Backport of MR !8803
Merge branch 'backport-4202-algorithm-detection-pytest-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9201
When attempting to run the system tests using v9.20.0 code, the test
setup will fail with ERROR due to setup.sh calling conf.sh which
attempts to call get_algorithms.py script which was deleted in this MR.
This should be reverted once v9.20.1 with the updated code is released.
Ensure all the variables are initialized when running the main function
of isctest module. This enables proper environment variables during test
script development when only conf.sh is sourced, rather than the script
being executed by the pytest runner.
(cherry picked from commit d7ace928b5)
Run the crypto support checks when initializing the isctest package and
save those results in environment variable. This removes the need to
repeatedly check for crypto operation support, as it's not something
that would change at test runtime.
(cherry picked from commit 25cb39b7fc)
Instead of invoking get_algorithms.py script repeatedly (which may yield
different results), move the algorithm configuration to an isctest
module. This ensures the variables are consistent across the entire test
run.
(cherry picked from commit 8302db407c)
Fix an assertion failure that could happen as a result of data race between free_gluetable() and addglue() on the same headers.
Closes#4691
Backport of MR !9126
Merge branch 'backport-4691-fix-data-race-between-free_gluetable-and-addglue-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9256
When adding glue to the header, we add header to the wait-free stack to
be cleaned up later which sets wfc_node->next to non-NULL value. When
the actual cleaning happens we would only cleanup the .glue_list, but
since the database isn't locked for the time being, the headers could be
reused while cleaning the existing glue entries, which creates a data
race between database versions.
Revert the code back to use per-database-version hashtable where keys
are the node pointers. This allows each database version to have
independent glue cache table that doesn't affect nodes or headers that
could already "belong" to the future database version.
(cherry picked from commit 5beae5faf9)
when searching the cache for a node so that we can delete an rdataset, it isn't necessary to set the 'create' flag. if the
node doesn't exist yet, we won't be able to delete anything from it anyway.
Backport of MR !9158
Merge branch 'backport-each-minor-findnode-refactor-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9253
when searching the cache for a node so that we can delete an
rdataset, it is not necessary to set the 'create' flag. if the
node doesn't exist yet, we then we won't be able to delete
anything from it anyway.
(cherry picked from commit 6b720bfe1a)
When a priming query is complete, it's currently logged at level ISC_LOG_DEBUG(1), regardless of success or failure. We are now raising it to ISC_LOG_NOTICE in the case of failure. [GL #3516]
Closes#3516
Backport of MR !9121
Merge branch 'backport-3516-log-priming-errors-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9250
when a priming query is complete, it's currently logged at
level ISC_LOG_DEBUG(1), regardless of success or failure. we
are now raising it to ISC_LOG_NOTICE in the case of failure.
(cherry picked from commit a84d54c6ff)
The previous work in this area was led by the belief that we might be
calling call_rcu() from within call_rcu() callbacks. After carefully
checking all the current callback, it became evident that this is not
the case and the problem isn't enough rcu_barrier() calls, but something
entirely else.
Call the rcu_barrier() just once as that's enough and the multiple
rcu_barrier() calls will not hide the real problem anymore, so we can
find it.
Backport of MR !9134
Merge branch 'backport-4607-call-a-single-rcu_barrier-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9247
The previous work in this area was led by the belief that we might be
calling call_rcu() from within call_rcu() callbacks. After carefully
checking all the current callback, it became evident that this is not
the case and the problem isn't enough rcu_barrier() calls, but something
entirely else.
Call the rcu_barrier() just once as that's enough and the multiple
rcu_barrier() calls will not hide the real problem anymore, so we can
find it.
(cherry picked from commit 13941c8ca7)
Checking the version of `named-checkconf` would end with assertion failure. This has been fixed.
Closes#4827
Backport of MR !9243
Merge branch 'backport-4827-cleanup-dst-only-if-initialized-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9246
The dst_lib_destroy() should be called only if dst_lib_init() was called
before. In named-checkconf, that is guarded by dst_cleanup variable
that was erroneously set to true by default. Set the dst_cleanup to
'false' by default.
(cherry picked from commit c54880e3fa)
An assertion failure triggers when the TSIG has valid cryptographic signature, but the time is invalid. This can happen when the times between the primary and secondary servers are not synchronised.
Closes#4811
Backport of MR !9234
Merge branch 'backport-4811-fix-isc_buffer_putuint48-buffer-size-requirement-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9245
Add a system test that sets TSIG fudge to 0, waits three seconds and
then sends signed message to the server. This tests the path where the
time difference between the client and the server is outside of the TSIG
fudge value.
(cherry picked from commit 8def0c3b12)
The tsig unit test was only testing if everything went ok, but it was
not testing whether the error paths work. Add two more unit tests - one
uses the time outside of the TSIG skew, and the second trashes the
signature with random data.
(cherry picked from commit 3835d75f00)
When putting the 48-bit number into a fixed-size buffer that's exactly 6
bytes, the assertion failure would occur as the 48-bit number is
internally represented as 64-bit number and the code was checking if
there is enough space for `sizeof(val)`. This causes assertion failure
when otherwise valid TSIG signature has a bad timing information.
Specify the size of the argument explicitly, so the 48-bit number
doesn't require 8-byte long buffer.
(cherry picked from commit 37dbd57c16)
When automatic-interface-scan is disabled, the route socket was still being opened. Add new API to connect / disconnect from the route socket only as needed.
Additionally, move the block that disables periodic interface rescans to a place where it actually have access to the configuration values. Previously, the values were being checked before the configuration was loaded.
Backport of https://gitlab.isc.org/isc-projects/bind9/-/merge_requests/9122
Merge branch '4757-dont-open-routing-socket-if-not-needed-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9239
When automatic-interface-scan is disabled, the route socket was still
being opened. Add new API to connect / disconnect from the route socket
only as needed.
Additionally, move the block that disables periodic interface rescans to
a place where it actually have access to the configuration values.
Previously, the values were being checked before the configuration was
loaded.
(cherry picked from commit b26079fdaf)
The fcount_incr() was incorrectly skipping the accounting for the fetches-per-zone if the force argument was set to true. We want to skip the accounting only when the fetches-per-zone is completely disabled, but for individual names we need to do the accounting even if we are forcing the result to be success.
Backport of https://gitlab.isc.org/isc-projects/bind9/-/merge_requests/9115
Merge branch 'backport-4786-forced-fcount_incr-should-still-increment-count-and-allowed-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9241
The fcount_incr() was incorrectly skipping the accounting for the
fetches-per-zone if the force argument was set to true. We want to skip
the accounting only when the fetches-per-zone is completely disabled,
but for individual names we need to do the accounting even if we are
forcing the result to be success.
(cherry picked from commit a513d4c07f)
The PTHREAD_MUTEX_ADAPTIVE_NP and PTHREAD_MUTEX_ERRORCHECK_NP are usually not defines, but enum values, so simple preprocessor check doesn't work.
Check for PTHREAD_MUTEX_ADAPTIVE_NP from the autoconf AS_COMPILE_IFELSE block and define HAVE_PTHREAD_MUTEX_ADAPTIVE_NP. This should enable adaptive mutex on Linux and FreeBSD.
As PTHREAD_MUTEX_ERRORCHECK actually comes from POSIX and Linux glibc does define it when compatibility macros are being set, we can just use PTHREAD_MUTEX_ERRORCHECK instead of PTHREAD_MUTEX_ERRORCHECK_NP.
Backport of https://gitlab.isc.org/isc-projects/bind9/-/merge_requests/9111
Merge branch 'backport-ondrej/fix-adaptive-mutex-use-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9240
The PTHREAD_MUTEX_ADAPTIVE_NP and PTHREAD_MUTEX_ERRORCHECK_NP are
usually not defines, but enum values, so simple preprocessor check
doesn't work.
Check for PTHREAD_MUTEX_ADAPTIVE_NP from the autoconf AS_COMPILE_IFELSE
block and define HAVE_PTHREAD_MUTEX_ADAPTIVE_NP. This should enable
adaptive mutex on Linux and FreeBSD.
As PTHREAD_MUTEX_ERRORCHECK actually comes from POSIX and Linux glibc
does define it when compatibility macros are being set, we can just use
PTHREAD_MUTEX_ERRORCHECK instead of PTHREAD_MUTEX_ERRORCHECK_NP.
(cherry picked from commit cc4f99bc6d)
The system tests were overriding the local locale by setting LANG to C.
This does not override the locale in case there are individual LC_<*>
variables like LC_CTYPE explicitly set.
Use LC_ALL=C instead which is the proper way of overriding all currently
set locales.
Backport of MR !9109
Merge branch 'backport-ondrej/use-LC_ALL-not-LANG-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9235
The system tests were overriding the local locale by setting LANG to C.
This does not override the locale in case there are individual LC_<*>
variables like LC_CTYPE explicitly set.
Use LC_ALL=C instead which is the proper way of overriding all currently
set locales.
(cherry picked from commit 10147efc87)
I split this into two commits, one for the actual newline removal, and one for issues I found, ruining the yaml output when some errors were outputted.
Closes: #4772
Backport of MR !9112
Merge branch 'backport-yaml-indent-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9231
Not all invocations had it, and this makes it more consistent with
dighost_warning. Also remove the conditional newline when not outputting
yaml
(cherry picked from commit 1dd76fe780)
Add INSIST to fail if the multiplication would cause the variables to overflow.
Closes#4798
Backport of MR !9131
Merge branch 'backport-4798-cid-498025-and-cid-498031-overflowed-constant-integer_overflow-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9229
If bufsize overflows we will have an infinite loop. In practice
this will not happen unless we have made a coding error. Add an
INSIST to detect this condition.
181retry:
182 isc_buffer_allocate(mctx, &b, bufsize);
183 result = dns_rdata_totext(rdata, NULL, b);
184 if (result == ISC_R_NOSPACE) {
185 isc_buffer_free(&b);
CID 498031: (#1 of 1): Overflowed constant (INTEGER_OVERFLOW)
overflow_const: Expression bufsize, which is equal to 0, overflows
the type that receives it, an unsigned integer 32 bits wide.
186 bufsize *= 2;
187 goto retry;
188 }
(cherry picked from commit 20ac13fb23)
If size overflows we will have an infinite loop. In practice
this will not happen unless we have made a coding error. Add
an INSIST to detect this condition.
181 while (!done) {
182 isc_buffer_allocate(mctx, &b, size);
183 result = dns_rdata_totext(rdata, NULL, b);
184 if (result == ISC_R_SUCCESS) {
185 printf("%.*s\n", (int)isc_buffer_usedlength(b),
186 (char *)isc_buffer_base(b));
187 done = true;
188 } else if (result != ISC_R_NOSPACE) {
189 check_result(result, "dns_rdata_totext");
190 }
191 isc_buffer_free(&b);
CID 498025: (#1 of 1): Overflowed constant (INTEGER_OVERFLOW)
overflow_const: Expression size, which is equal to 0, overflows the type that
receives it, an unsigned integer 32 bits wide.
192 size *= 2;
193 }
(cherry picked from commit e7ef0a60ab)
Decrementing optlen immediately before calling continue is unneccesary
and inconsistent with the rest of dns_message_pseudosectiontoyaml
and dns_message_pseudosectiontotext. Coverity was also reporting
an impossible false positive overflow of optlen (CID 499061).
4176 } else if (optcode == DNS_OPT_CLIENT_TAG) {
4177 uint16_t id;
4178 ADD_STRING(target, "; CLIENT-TAG:");
4179 if (optlen == 2U) {
4180 id = isc_buffer_getuint16(&optbuf);
4181 snprintf(buf, sizeof(buf), " %u
", id);
4182 ADD_STRING(target, buf);
CID 499061: (#1 of 1): Overflowed constant (INTEGER_OVERFLOW)
overflow_const: Expression optlen, which is equal to 65534, underflows
the type that receives it, an unsigned integer 16 bits wide.
4183 optlen -= 2;
4184 POST(optlen);
4185 continue;
4186 }
4187 } else if (optcode == DNS_OPT_SERVER_TAG) {
Backport of MR !9130
Merge branch 'backport-marka-remove-unnecessary-operations-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9223
Decrementing optlen immediately before calling continue is unneccesary
and inconsistent with the rest of dns_message_pseudosectiontoyaml
and dns_message_pseudosectiontotext. Coverity was also reporting
an impossible false positive overflow of optlen (CID 499061).
4176 } else if (optcode == DNS_OPT_CLIENT_TAG) {
4177 uint16_t id;
4178 ADD_STRING(target, "; CLIENT-TAG:");
4179 if (optlen == 2U) {
4180 id = isc_buffer_getuint16(&optbuf);
4181 snprintf(buf, sizeof(buf), " %u\n", id);
4182 ADD_STRING(target, buf);
CID 499061: (#1 of 1): Overflowed constant (INTEGER_OVERFLOW)
overflow_const: Expression optlen, which is equal to 65534, underflows
the type that receives it, an unsigned integer 16 bits wide.
4183 optlen -= 2;
4184 POST(optlen);
4185 continue;
4186 }
4187 } else if (optcode == DNS_OPT_SERVER_TAG) {
(cherry picked from commit 47338c2c87)
Closes#4770
Backport of MR !9104
Merge branch 'backport-4770-digdelv-system-test-can-report-more-errors-than-they-actually-are-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9221
Add a note that 'rndc reload' and 'rndc reconfig' can't change the
querylog option during the runtime of named.
Closes#4801
Backport of MR !9136
Merge branch 'backport-4801-arm-querylog-clarification-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9206
The minimum allowed value of 'resolver-query-timeout' was lowered to
301 milliseconds instead of the earlier 10000 milliseconds (which is the
default). As earlier, values less than or equal to 300 are converted to
seconds before applying the limit.
Closes#4320
Backport of MR !9091
Merge branch 'backport-4320-allow-shorter-resolver-query-timeout-configuration-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9220
Before there was a gap from 301 to 9999 which would be converted
to 10000 and now there is no such gap.
This settimeout_belowmin test was checking the behavior of a value
in the gap. As there is now no gap left, the minimum is 301 and
anything below that is converted to seconds as before. In order
for this check to still test the "below minimum" behavior, change
the value from 9000 to 300.
Update the settimeout_overmax value test too so it logically aligns
with the minimum value test.
(cherry picked from commit 63b787effe)
There are use cases for which shorter timeout values make sense.
For example if there is a load balancer which sets RD=1 and
forwards queries to a BIND resolver which is then configured to
talk to backend servers which are not visible in the public NS set.
WIth a shorter timeout value the frontend can give back SERVFAIL
early when backends are not available and the ultimate client will
not penalize the BIND-frontend for non-response.
(cherry picked from commit 5f47c2b567)
A new optional argument '-force' has been added to the command channel
command 'rndc retransfer'. When it is specified, named aborts the
ongoing zone transfer (if there is one), and starts a new transfer.
Closes#2299
Backport of MR !9102
Merge branch 'backport-2299-implement-rndc-force-retransfer-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9219
Use a big zone and the slow transfer mode. Initiate a retransfer, wait
several seconds, then initiate a retransfer using a '-force' argument,
which should cancel the previous transfer and start a new one.
(cherry picked from commit e48f4e8101)
With this new optional argument if there is an ongoing zone
transfer it will be aborted before a new zone transfer is scheduled.
(cherry picked from commit 402ca316ae)
The period between the most significant nibble of the encoded IPv4 address and the 2.0.0.2.IP6.ARPA suffix was missing resulting in the wrong name being checked. Add system test for 6to4-self implementation.
Closes#4766
Backport of MR !9099
Merge branch 'backport-4766-add-system-test-for-6to4-self-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9217
The period between the most significant nibble of the IPv4 address
and the 2.0.0.2.IP6.ARPA suffix was missing resulting in the wrong
name being checked.
(cherry picked from commit bca63437a1)
Remove the false positive "success resolving" log message when QNAME minimisation is in effect and the final result is NXDOMAIN.
Closes#4784
Backport of MR !9117
Merge branch 'backport-4784-false-qname-minimisation-error-being-reported-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9215
Closes#4796
Backport of MR !9127
Merge branch 'backport-4796-yaml-stringify-question-and-records-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9213
In yaml mode we emit a string for each question and record. Certain
names and data could result in invalid yaml being produced. Use single
quote string for all questions and records. This requires that single
quotes get converted to two quotes within the string.
(cherry picked from commit 393d7fa78e)
Closes#4774
Backport of MR !9105
Merge branch 'backport-4774-resolver-system-test-didn-t-fail-on-all-subtest-errors-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9211
Closes#4775
Backport of MR !9106
Merge branch 'backport-4775-reject-zero-length-alpn-in-alpn-fromtext-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9209
ALPN are defined as 1*255OCTET in RFC 9460. commatxt_fromtext was not
rejecting invalid inputs produces by missing a level of escaping
which where later caught be dns_rdata_fromwire on reception.
These inputs should have been rejected
svcb in svcb 1 1.svcb alpn=\,abc
svcb1 in svcb 1 1.svcb alpn=a\,\,abc
and generated 00 03 61 62 63 and 01 61 00 02 61 62 63 respectively.
The correct inputs to include commas in the alpn requires double
escaping.
svcb in svcb 1 1.svcb alpn=\\,abc
svcb1 in svcb 1 1.svcb alpn=a\\,\\,abc
and generate 04 2C 61 62 63 and 06 61 2C 2C 61 62 63 respectively.
(cherry picked from commit b51c9eb797)
Replace #define DNS_GETDB_ with struct of bools to make
it easier to pretty-print the attributes in a debugger.
Closes#4559
Backport of MR !9093
Merge branch 'backport-4559-convert-dns_getdb_x-defines-to-1-bit-long-bools-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9205
When cutting a long CNAME chain, named was returning NOERROR
instead of SERVFAIL (alongside with a partial answer). This
has been fixed.
Closes#4449
Backport of MR !9090
Merge branch 'backport-4449-return-servfail-for-a-long-cname-chain-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9203
Update the CNAME chain test to correspond to the changed behavior,
because now named returns SERVFAIL when hitting the maximum query
restarts limit (e.g. happening when following a long CNAME chain).
In the current test auth will hit the limit and return partial data
with a SERVFAIL code, while the resolver will return no data with
a SERVFAIL code after auth returns SERVFAIL to it.
(cherry picked from commit 7751c7eca6)
Due to the maximum query restart limitation a long CNAME chain
it is cut after 16 queries but named still returns NOERROR.
Return SERVFAIL instead and the partial answer.
(cherry picked from commit b621f1d88e)
On 32 bit machines isc_meminfo_totalphys could return an incorrect value.
Closes#4799
Backport of MR !9132
Merge branch 'backport-4799-cid-498034-overflowed-return-value-integer_overflow-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9199
On a 32 bit machine casting to size_t can still lead to an overflow.
Cast to uint64_t. Also detect all possible negative values for
pages and pagesize to silence warning about possible negative value.
39#if defined(_SC_PHYS_PAGES) && defined(_SC_PAGESIZE)
1. tainted_data_return: Called function sysconf(_SC_PHYS_PAGES),
and a possible return value may be less than zero.
2. assign: Assigning: pages = sysconf(_SC_PHYS_PAGES).
40 long pages = sysconf(_SC_PHYS_PAGES);
41 long pagesize = sysconf(_SC_PAGESIZE);
42
3. Condition pages == -1, taking false branch.
4. Condition pagesize == -1, taking false branch.
43 if (pages == -1 || pagesize == -1) {
44 return (0);
45 }
46
5. overflow: The expression (size_t)pages * pagesize might be negative,
but is used in a context that treats it as unsigned.
CID 498034: (#1 of 1): Overflowed return value (INTEGER_OVERFLOW)
6. return_overflow: (size_t)pages * pagesize, which might have underflowed,
is returned from the function.
47 return ((size_t)pages * pagesize);
48#endif /* if defined(_SC_PHYS_PAGES) && defined(_SC_PAGESIZE) */
(cherry picked from commit e8dbc5db92)
Stop updating `find.result_v4` and `find.result_v4` in `clean_finds_at_name`. The values are supposed to be static.
Closes#4118
Backport of MR !9108
Merge branch 'backport-4118-data-race-lib-dns-adb-c-1537-in-clean_finds_at_name-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9197
Don't verify the just signed zone as the RRSIGs could have expired before the signing process completes
Closes#4781#2476
Backport of MR !9114
Merge branch 'backport-4781-statschannel-setup-can-fail-due-to-short-validity-interval-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9195
The updatecheck-kskonly.secure zone is being used to test dynamic
updates while the KSK is offline. It ensures that the DNSKEY RRset
will retain the RRSIG record, while the updated data is being signed
with the currently active ZSK.
When walking through ZSK rollovers, ensure that the newest ZSK (ZSK3)
is published before doing the dynamic update, preventing timing
related test failures.
Also fix the test log line ($ZSK_ID3 was not yet created at the time
of logging).
(cherry picked from commit e874632488)
Adjust key state and timing metadata if dnssec-policy key lifetime configuration is updated, so that it also
affects existing keys.
Closes#4677
Backport of MR !9118
Merge branch 'backport-4677-dnssec-policy-key-lifetime-reconfigure-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9191
The key lifetime should no longer be adjusted if the key is being
retired earlier, for example because a manual rollover was started.
This would falsely be seen as a dnssec-policy lifetime reconfiguration,
and would adjust the retire/removed time again.
This also means we should update the status output, and the next
rollover scheduled is now calculated using (retire-active) instead of
key lifetime.
(cherry picked from commit 129973ebb0)
If dnssec-policy is reconfigured and the key lifetime has changed,
update existing keys with the new lifetime and adjust the retire
and removed timing metadata accordingly.
If the key has no lifetime yet, just initialize the lifetime. It
may be that the retire/removed timing metadata has already been set.
Skip keys which goal is not set to omnipresent. These keys are already
in the progress of retiring, or still unused.
(cherry picked from commit 1cec0b0448)
Check if the key lifetime is updated in the key files. Make sure the
inactive and removed timing metadata are adjusted accordingly.
(cherry picked from commit 2237895bb4)
Closes#4813
Backport of MR !9176
Merge branch 'backport-4813-fix-typo-in-reference-rst-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9189
Closes#4777
Backport of MR !9107
Merge branch '4777-retry-job-aws-spot-instance-interruption-event-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9184
Use a single source of truth, the git log, to generate the list of CHANGES. Use the .rst format and include it in the ARM for a quick reference with proper gitlab links to issues and merge requests.
Closes#75
Backport of MR !9152
Merge branch 'nicki/add-gitchangelog-9.20' into 'bind-9.20'
See merge request isc-projects/bind9!9180
The files in contrib/ directory shouldn't be subject to our pylint
check. They can come from external sources and we don't subject these to
the same standards as the rest of the BIND9 code / scripts.
(cherry picked from commit 7cbb052649)
The configuration has been crafted to cater for BIND9 needs:
- Define actions that have an equivalent section in existing Release
Notes
- Assume the commits that have the necessary changelog makrup are merge
commits from GitLab and transform them into messages and proper links
- Put the resulting changelog into the proper place in
doc/arm/changelog.rst
- Have a separate configuration for changelog and release notes. Both of
these should be generated from the `git log`, with release notes being
subject to more scrutiny and further editing
(cherry picked from commit 0ec8b99ea3)
Create new changelog and include it in the documentation. Include the
previous CHANGES as plain text without any markup.
(cherry picked from commit e9b6031e0c)
Add an option which can be used to put short commit sha at the end of
each commit subject line in the generated changelog.
(cherry picked from commit c2b23fa2de)
Given our workflow, this could easily lead to misattribution. It's also
not an actionable information and it can be found in the MR / git log
instead.
(cherry picked from commit a8258d1c53)
The project hasn't seen any new development/changes since 2018 and it
seems unlikely we'd be able to get any changes into the upstream. Since
it's isolated into a single file and its task is fairly straighforward,
pull the code into our own repository and maintain it here as needed.
This also makes it easier to make any changes that are specific to our
project.
(cherry picked from commit 63247d8a73)
- [ ] [:link:][step_backports] **(SwEng)** Prepare backports of the merge request addressing the problem for all affected (and still maintained) branches of a given product
- [ ] [:link:][step_finish_advisory] **(Support)** Finish preparing the Security Advisory
- [ ] [:link:][step_meta_issue] **(QA)** Create (or update) the private issue containing links to fixes & reproducers for all CVEs fixed in a given release cycle
- [ ] [:link:][step_changes] **(QA)** (BIND 9 only) Reserve a block of `CHANGES` placeholders once the complete set of vulnerabilities fixed in a given release cycle is determined
- [ ] [:link:][step_merge_fixes] **(QA)** Merge the CVE fixes in CVE identifier order
- [ ] [:link:][step_patches] **(QA)** Prepare a standalone patch for the last stable release of each affected (and still maintained) product branch
- [ ] [:link:][step_asn_releases] **(QA)** Prepare ASN releases (as outlined in the Release Checklist)
| [security] | Fix for a significant security flaw |
| [experimental] | Used for new features when the syntax or other aspects of the design are still in flux and may change |
| [port] | Portability enhancement |
| [maint] | Updates to built-in data such as root server addresses and keys |
| [tuning] | Changes to built-in configuration defaults and constants to improve performance |
| [performance] | Other changes to improve server performance |
| [protocol] | Updates to the DNS protocol such as new RR types |
| [test] | Changes to the automatic tests, not affecting server functionality |
| [cleanup] | Minor corrections and refactoring |
| [doc] | Documentation |
| [contrib] | Changes to the contributed tools and libraries in the 'contrib' subdirectory |
| [placeholder] | Used in the main development branch to reserve change numbers for use in other branches, e.g., when fixing a bug that only exists in older releases |
In general, [func] and [experimental] tags only appear in new-feature
releases (i.e., those with version numbers ending in zero). Some new
functionality may be backported to older releases on a case-by-case basis.
All other change types may be applied to all currently supported releases.
#### Bug report identifiers
Most notes in the CHANGES file include a reference to a bug report or
Most notes in the ARM Changelog appendix include a reference to a bug report or
issue number. Prior to 2018, these were usually of the form `[RT #NNN]`
and referred to entries in the "bind9-bugs" RT database, which was not open
to the public. More recent entries use the form `[GL #NNN]` or, less often,
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.