If the operating system UDP queue gets full and the outgoing UDP sending
starts to be delayed, BIND 9 could exhibit memory spikes as it tries to
enqueue all the outgoing UDP messages. As those are not going to be
delivered anyway (as we argued when we stopped enlarging the operating
system send and receive buffers), try to send the UDP messages directly
using `uv_udp_try_send()` and if that fails, drop the outgoing UDP
message.
We currently set SO_INCOMING_CPU incorrectly, and testing by Ondrej
shows that fixing the issue by setting affinities is worse than letting
the kernel schedule threads without constraints. So we should not set
SO_INCOMING_CPU anymore.
Closes#4936
Merge branch '4936-remove-so-incoming-cpu' into 'main'
See merge request isc-projects/bind9!9497
We currently set SO_INCOMING_CPU incorrectly, and testing by Ondrej
shows that fixing the issue and setting affinities is worse than letting
the kernel schedule threads without constraints. So we should not set
SO_INCOMING_CPU anymore.
The new ForwardOnlyFail statistics channel counter indicates the
number of queries failed due to bad forwarders for 'forward only'
zones.
Related to #1793
Merge branch 'aram/add-new-stats-channel-counter-forwardonlyfail' into 'main'
See merge request isc-projects/bind9!9498
When resolving a zone with a 'forward only' policy, and
finding out that all the forwarders are marked as "bad",
the 'ServerQuota' counter of the statistics channel was
incorrectly increased. This has been fixed.
Closes#1793
Merge branch '1793-serverquota-counter-bug-with-forward-only' into 'main'
See merge request isc-projects/bind9!9493
Check that the fix in the previous commit works and that the
'ServerQuota' counter in the statistics channel is still unset
after a SERVFAIL result in a 'forward only' zone.
The 'all_spilled' local variable in resolver.c:fctx_getaddresses()
is 'true' by default, and only becomes false when there is at least
one successfully found NS address. However, when a 'forward only;'
configuration is used, the code jumps over the part where it looks
for NS addresses and doesn't reset the 'all_spilled' to false, which
results in incorretly increased 'serverquota' statistics variable,
and also in invalid return error code from the function. The result
code error didn't make any differences, because all codes other than
'ISC_R_SUCCESS' or 'DNS_R_WAIT' were treated in the same way, and
the result code was never logged anywhere.
Set the default value of 'all_spilled' to 'false', and only make it
'true' before actually starting to look up NS addresses.
Silence Coverity CID 468757 and 468767 (DATA RACE read not locked) by converting dnssec-signzone to use atomics for statistics counters rather than using a lock.
Closes#4939
Merge branch '4939-remove-stats-lock-from-dnssec-signzone' into 'main'
See merge request isc-projects/bind9!9496
Silence Coverity CID 468757 and 468767 (DATA RACE read not locked)
by converting dnssec-signzone to use atomics for statistics counters
rather than using a lock. This should be marginally faster than
using the lock as well when statistics are requested.
As part of the KeyTrap \[CVE-2023-50387\] mitigation, the DNSSEC CPU-intensive operations were offloaded to a separate threadpool that we use to run other tasks that could affect the networking latency.
If that threadpool is running some long-running tasks like RPZ, catalog zone processing, or zone file operations, it would delay DNSSEC validations to a point where the resolving signed DNS records would fail.
Split the CPU-intensive and long-running tasks into separate threadpools in a way that the long-running tasks don't block the CPU-intensive operations.
Closes#4898
Merge branch '4898-move-offloaded-DNSSEC-to-own-threads' into 'main'
See merge request isc-projects/bind9!9473
Currently, the isc_work API is overloaded. It runs both the
CPU-intensive operations like DNSSEC validations and long-term tasks
like RPZ processing, CATZ processing, zone file loading/dumping and few
others.
Under specific circumstances, when many large zones are being loaded, or
RPZ zones processed, this stops the CPU-intensive tasks and the DNSSEC
validation is practically stopped until the long-running tasks are
finished.
As this is undesireable, this commit moves the CPU-intensive operations
from the isc_work API to the isc_helper API that only runs fast memory
cleanups now.
Add an extra thread that can be used to offload operations that would
affect latency, but are not long-running tasks; those are handled by
isc_work API.
Each isc_loop now has matching isc_helper thread that also built on top
of uv_loop. In fact, it matches most of the isc_loop functionality, but
only the `isc_helper_run()` asynchronous call is exposed.
When verifying a message in an offloaded thread there is a race with
the worker thread which writes to the same buffer. Clone the message
buffer before offloading.
Closes#4929
Merge branch '4929-data-race-in-dns_dnssec_verifymessage-memmove' into 'main'
See merge request isc-projects/bind9!9481
When verifying a message in an offloaded thread there is a race with
the worker thread which writes to the same buffer. Clone the message
buffer before offloading.
Remove the use of "port" when configuring query-source(-v6),
transfer-source(-v6), notify-source(-v6), parental-source(-v6),
etc. Remove the use of source ports for parental-agents.
Also remove the deprecated options use-{v4,v6}-udp-ports and
avoid-{v4,v6}udp-ports.
Closes#3843
Merge branch '3843-remove-deprecated-source-port-options' into 'main'
See merge request isc-projects/bind9!9469
Remove the use of "port" when configuring query-source(-v6),
transfer-source(-v6), notify-source(-v6), parental-source(-v6),
etc. Remove the use of source ports for parental-agents.
Also remove the deprecated options use-{v4,v6}-udp-ports and
avoid-{v4,v6}udp-ports.
When the libxml2 and libjson-c libraries are not supported, the statistics channel can't return anything useful, so it is now disabled. Use of `statistics-channel` in `named.conf` is a fatal error.
Closes#4895
Merge branch '4895-link-style-sheet-to-libxml2-support' into 'main'
See merge request isc-projects/bind9!9423
The `statschannel` system test failed if only one of `libxml2` or `json-c` is
available / configured as checks were being run against the non available
statistics page.
Closes#4919
Merge branch '4919-fix-statschannel-system-test' into 'main'
See merge request isc-projects/bind9!9454
Future port the missing TCP4Clients/TCP6Clients documentation entry.
Commit abc47f5ce4 added two new statistics
counters without documenting them. Add the missing counter descriptions
to the ARM.
(cherry picked from commit cb79240091)
Merge branch 'aydin/tcpclient-doc' into 'main'
See merge request isc-projects/bind9!9078
Commit abc47f5ce4 added two new statistics
counters without documenting them. Add the missing counter descriptions
to the ARM.
(cherry picked from commit cb79240091)
This change allows fallback from an IXFR failure to AXFR when the reason is `DNS_R_TOOMANYRECORDS`. This is because this error condition could be temporary only in an intermediate version of IXFR transactions and it's possible that the latest version of the zone doesn't have that condition. In such a case, the secondary would never be able to update the zone (even if it could) without this fallback.
This fallback behavior is particularly useful with the recently introduced `max-records-per-type` and `max-types-per-name` options: the primary may not have these limitations and may temporarily introduce "too many" records, breaking IXFR. If the primary side subsequently deletes these records, this fallback will help recover the zone transfer failure automatically; without it, the secondary side would first need to increase the limit, which requires more operational overhead and has its own adverse effect.
Closes#4928
Merge branch 'fallback-ixfr-to-axfr-on-toomanyrecords' into 'main'
See merge request isc-projects/bind9!9333
This change allows fallback from an IXFR failure to AXFR when the
reason is DNS_R_TOOMANYRECORDS. This is because this error condition
could be temporary only in an intermediate version of IXFR
transactions and it's possible that the latest version of the zone
doesn't have that condition. In such a case, the secondary would never
be able to update the zone (even if it could) without this fallback.
This fallback behavior is particularly useful with the recently
introduced max-records-per-type and max-types-per-name options:
the primary may not have these limitations and may temporarily
introduce "too many" records, breaking IXFR. If the primary side
subsequently deletes these records, this fallback will help recover
the zone transfer failure automatically; without it, the secondary
side would first need to increase the limit, which requires more
operational overhead and has its own adverse effect.
This change also fixes a minor glitch that DNS_R_TOOMANYRECORDS wasn't
logged in xfrin_fail.
The named process could terminate unexpectedly when processing ACL. This has been fixed.
Closes#4908
Merge branch '4908-acl-assertion-failure' into 'main'
See merge request isc-projects/bind9!9458
The rcu_xchg_pointer() function can be used outside of a critical
section, and usually must be followed by a synchronize_rcu() or
call_rcu() call to detach from the resource, unless if there are
some guarantees in place because of our own reference counting.
`named-checkconf` now takes "-n" to ignore "not configured" errors. This allows named-checkconf to check the syntax of configurations from other builds which have support for more options.
Merge branch '4913-add-option-to-named-checkconf-to-override-notconfigured-flag' into 'main'
See merge request isc-projects/bind9!9446
named-checkconf now takes "-n" to ignore "not configured" errors.
This allows named-checkconf to check the syntax of configurations
from other builds which have support for more options.
This file was initially created for unit testing, but later code was added to generate the file. The static file should have been removed from the git repo.
Closes#4916
Merge branch '4916-skr-unit-test-rm-test-file' into 'main'
See merge request isc-projects/bind9!9450
This file was initially created for unit testing, but later code was
added to generate the file. The static file should have been removed
from the git repo.
If the ZSK has unlimited lifetime, the timing metadata "Inactive" and "Delete" cannot be found and is treated as an error, preventing the zone to be signed. This has been fixed.
Closes#4914
Merge branch '4914-offline-ksk-zsk-lifetime-unlimited-bug' into 'main'
See merge request isc-projects/bind9!9447
If the ZSK has lifetime unlimited, the timing metadata "Inactive" and
"Delete" cannot be found and is treated as an error. Fix by allowing
these metadata to not exist.
Delay relinquishing root privileges until the control channel has been configured, for the benefit of systems that require root to use privileged port numbers. This mostly affects systems without fine-grained privilege systems (i.e., other than Linux).
Closes#4793
Merge branch '4793-bind-9-19-24-not-listening-to-rndc-port-953-on-localhost' into 'main'
Closes#4793
See merge request isc-projects/bind9!9123
On systems where root access is needed to configure privileged
ports, we don't want to fully relinquish root privileges until
after the control channel (which typically runs on port 953) has
been established.
named_os_changeuser() now takes a boolean argument 'permanent'.
This allows us to switch the effective userid temporarily with
named_os_changeuser(false) and restore it with named_os_restoreuser(),
before permanently dropping privileges with named_os_changeuser(true).
Administrators may wish to constrain the set of cores that BIND 9 runs on via the 'taskset', 'cpuset' or 'numactl' programs (or equivalent on other O/S).
If the admin has used taskset, the `named` will now follow to automatically use the given number of CPUs rather than the system wide count.
Closes#4884
Merge branch '4884-use-cpuset-to-get-number-of-cpus' into 'main'
Closes#4884
See merge request isc-projects/bind9!9398
Administrators may wish to constrain the set of cores that BIND 9 runs
on via the 'taskset', 'cpuset' or 'numactl' programs (or equivalent on
other O/S), for example to achieve higher (or more stable) performance
by more closely associating threads with individual NIC rx queues. If
the admin has used taskset, it follows that BIND ought to
automatically use the given number of CPUs rather than the system wide
count.
Co-Authored-By: Ray Bellis <ray@isc.org>
With max-recursion-queries set to 50 the resolver system test was
unstable in the "checking query resolution for a domain with a valid
glueless delegation chain" check as ns1 replied with SERVFAIL.
Closes#4897
Merge branch '4897-resolver-ns1-max-recursion-queries-100' into 'main'
Closes#4897
See merge request isc-projects/bind9!9435
With max-recursion-queries set to 50 the resolver system test was
unstable in the "checking query resolution for a domain with a valid
glueless delegation chain" check as ns1 replied with SERVFAIL.
Use the fact that the database returns the longest matching part of the requested name to find the required NSEC3 record. If there are multiple versions present in the database we may have to search further.
Closes#4460
Merge branch '4460-auth-nsec3-many-labels' into 'main'
Closes#4460
See merge request isc-projects/bind9!9436
Return partial match from dns_db_find/dns_db_find when requested
to short circuit the closest encloser discover process. Most of the
time this will be the actual closest encloser but may not be when
there yet to be committed / cleaned up versions of the zone with
names below the actual closest encloser.
Move lock earlier in the call sequence to address access without lock report.
```
1559 /*
1560 * Caller must be holding the fctx lock.
1561 */
CID 468796: (#1 of 1): Data race condition (MISSING_LOCK)
1. missing_lock: Accessing fctx->state without holding lock fetchctx.lock. Elsewhere, fetchctx.state is written to with fetchctx.lock held 2 out of 2 times.
1562 REQUIRE(fctx->state == fetchstate_done);
1563
1564 FCTXTRACE("sendevents");
1565
1566 LOCK(&fctx->lock);
1567
```
Closes#4902
Merge branch '4902-accessing-fctx-state-without-holding-lock' into 'main'
Closes#4902
See merge request isc-projects/bind9!9427
fctx->state should be read with the lock held.
1559 /*
1560 * Caller must be holding the fctx lock.
1561 */
CID 468796: (#1 of 1): Data race condition (MISSING_LOCK)
1. missing_lock: Accessing fctx->state without holding lock fetchctx.lock.
Elsewhere, fetchctx.state is written to with fetchctx.lock held 2 out of 2 times.
1562 REQUIRE(fctx->state == fetchstate_done);
1563
1564 FCTXTRACE("sendevents");
1565
1566 LOCK(&fctx->lock);
1567
find->status is a private field of dns_adbfind_t so it now has an accessor function and has been made atomic.
Closes#4802
Merge branch '4802-address-potential-tsan-issue-with-find-status' into 'main'
Closes#4802
See merge request isc-projects/bind9!9137
Fetches generated automatically as a result of 'prefetch' are now
exempt from the 'fetches-per-zone' and 'fetches-per-server' quotas.
This should help in maintaining the cache from which query responses
can be given.
Closes#4219
Merge branch '4219-exempt-good-queries-from-fetch-limits' into 'main'
Closes#4219
See merge request isc-projects/bind9!9095
Give prefetches a free pass through the quota so that the cache
entries for popular zones could be updated successfully even if the
quota for is already reached.
Give prefetches a free pass through the quota so that the cache entry
for a popular zone could be updated successfully even if the quota for
it is already reached.
The `malloc_usable_size()` can return size larger than originally allocated and when these sizes disagree the fortifier enabled by `_FORTIFY_SOURCE=3` detects overflow and stops the `named` execution abruptly. Stop using these convenience functions as they are primary used for introspection-only.
Closes#4880
Merge branch '4880-dont-use-malloc_usable_size' into 'main'
Closes#4880
See merge request isc-projects/bind9!9400
Although the nanual page of malloc_usable_size says:
Although the excess bytes can be over‐written by the application
without ill effects, this is not good programming practice: the
number of excess bytes in an allocation depends on the underlying
implementation.
it looks like the premise is broken with _FORTIFY_SOURCE=3 on newer
systems and it might return a value that causes program to stop with
"buffer overflow" detected from the _FORTIFY_SOURCE. As we do have own
implementation that tracks the allocation size that we can use to track
the allocation size, we can stop relying on this introspection function.
Also the newer manual page for malloc_usable_size changed the NOTES to:
The value returned by malloc_usable_size() may be greater than the
requested size of the allocation because of various internal
implementation details, none of which the programmer should rely on.
This function is intended to only be used for diagnostics and
statistics; writing to the excess memory without first calling
realloc(3) to resize the allocation is not supported. The returned
value is only valid at the time of the call.
Remove usage of both malloc_usable_size() and malloc_size() to be on the
safe size and only use the internal size tracking mechanism when
jemalloc is not available.
The cross-version-config-tests job fails when a system test is removed
from the upcoming release. To avoid this, remove the system test also
from the $BIND_BASELINE_VERSION.
See the failure mode at https://gitlab.isc.org/isc-projects/bind9/-/jobs/4668947.
Merge branch 'mnowak/remove-dialup-from-cross-version-config-tests-job' into 'main'
See merge request isc-projects/bind9!9413
The cross-version-config-tests job fails when a system test is removed
from the upcoming release. To avoid this, remove the system test also
from the $BIND_BASELINE_VERSION.
This supports bit-for-bit reproducibility of built documentation.
Closes#4886
Merge branch 'issue-4886/order-preserving-documentation-tags' into 'main'
Closes#4886
See merge request isc-projects/bind9!9399
The 'set' datatype in Python does not provide iteration-order
guarantees related to insertion-order. That means that its
usage in the 'split_csv' helper function during documentation
build can produce nondeterministic results.
That is non-desirable for two reasons: it means that the
documentation output may appear to vary unnecessarily between
builds, and secondly there could be loss-of-information in cases
where tag order in the source documentation is significant.
This patch implements order-preserving de-duplication of tags,
allowing authors to specify tags using intentional priority
ordering, while also removing tags that appear more than once.
1. Set milestone to 'Not released yet' after merge
We will set milestone to actual version number when we actually tag a
particular version. This will get rid of mass MR reassignment when we
do last minute changes to a release plan etc.
2. Adjust No CHANGES and Release Notes MR labels to match gitchangelog
workflow.
Previously CI job for the autobackport bot inherited "interruptible:
true" global configuration. This caused premature termination of the job
when another merge was finished before the autobackport job ran to
completion.
The `dialup` and `heartbeat-interval` options have been removed, along with all code implementing them. Using these options is now a fatal error.
Closes#4237
Merge branch '4237-remove-dialup' into 'main'
Closes#4237
See merge request isc-projects/bind9!8160
The new 'request-ixfr-max-diffs' configuration option sets the
maximum number of incoming incremental zone transfer (IXFR) differences,
exceeding which triggers a full zone transfer (AXFR).
Closes#4389
Merge branch '4389-request-ixfr-max-diffs' into 'main'
Closes#4389
See merge request isc-projects/bind9!9094
Configure a maximum of 3 allowed differences and add 5 new records.
Check that named detected that the differences exceed the allowed
limit and successfully retries with AXFR.
This limits the maximum number of received incremental zone
transfer differences for a secondary server. Upon reaching the
confgiured limit, the secondary aborts IXFR and initiates a full
zone transfer (AXFR).
It is useful when multiple signers are being used
to sign a zone to able to specify a restricted
range of range of key tags that will be used by an
operator to sign the zone. This adds controls to
named (dnssec-policy), dnssec-signzone, dnssec-keyfromlabel and
dnssec-ksr (dnssec-policy) to specify such ranges.
Closes#4830
Merge branch '4830-support-restricted-key-tag-range-when-generating-new-keys' into 'main'
Closes#4830
See merge request isc-projects/bind9!9258
A new argument has been added to dnssec-keygen and dnssec-keyfromlabel
to restrict the tag value of key generated / imported to a particular
range. This is intended to be used by multi-signers.
Co-authored-by: Suzanne Goldlust <sgoldlust@isc.org>
With the release of BIND 9.19.25 this workaround is not needed anymore.
This reverts commit c58dd6e78f.
Merge branch 'mnowak/revert-workaround-temporary-cross-version-config-tests-limitation' into 'main'
See merge request isc-projects/bind9!9065
If there is an algorithm rollover and two keys of different algorithm share the same keytags, then there is a possibility that if we check that a key matches a specific state, we are checking against the wrong key. This has been fixed by not only checking for matching key tag but also key algorithm.
Closes#4878
Merge branch '4878-fix-algorithm-rollover-keytag-conflict-bug' into 'main'
Closes#4878
See merge request isc-projects/bind9!9381
If there is an algorithm rollover and two keys of different algorithm
share the same keytags, then there is a possibility that if we check
that a key matches a specific state, we are checking against the wrong
key.
Fix this by not only checking for matching key id but also key
algorithm.
If there is a keytag conflict between keys with different algorithms,
we need to supply what key algorithm is used so we can get the right
public key.
For clarity, print the algorithm on the found keys after 'check_keys'.
Add a new configuration option `offline-ksk` to enable Offline KSK key management. Signed Key Response (SKR) files created with `dnssec-ksr` (or other program) can now be imported into `named` with the new `rndc skr -import` command. Rather than creating new DNSKEY, CDS and CDNSKEY records and generating signatures covering these types, these records are loaded from the currently active bundle from the imported SKR.
The implementation is loosely based on: https://www.iana.org/dnssec/archive/files/draft-icann-dnssec-keymgmt-01.txtCloses#1128
Merge branch '1128-offline-ksk-rndc-import-skr' into 'main'
Closes#1128
See merge request isc-projects/bind9!9119
Some things we no longer want to do when we are in offline-ksk mode.
1. Don't check for inactive and private keys if the key is a KSK.
2. Don't update the TTL of DNSKEY, CDS and CDNSKEY RRset, these come
from the SKR.
With offline-ksk enabled, we don't run the keymgr because the key
timings are determined by the SKR. We do update the key states but
we derive them from the timing metadata.
Then, we can skip a other tasks in offline-ksk mode, like DS checking
at the parent and CDS synchronization, because the CDS and CDNSKEY
RRsets also come from the SKR.
This added source code stores SKR data. It is loosely based on:
https://www.iana.org/dnssec/archive/files/draft-icann-dnssec-keymgmt-01.txt
A SKR contains a list of signed DNSKEY RRsets. Each change in data
should be stored in a separate bundle. So if the RRSIG is refreshed that
means it is stored in the next bundle. Likewise, if there is a new ZSK
pre-published, it is in the next bundle.
In addition (not mentioned in the draft), each bundle may contain
signed CDS and CDNSKEY RRsets.
Each bundle has an inception time. These will determine when we need
to re-sign or re-key the zone.
Test importing a Signed Key Response. Files should be loaded and once
loaded the correct bundle should be used. Alsoe test cases where the
bundle is not the first bundle in the SKR.
Add the code and documentation required to provide KSR import using
rndc. This is just the command, and the feature is at this point in
time still not implemented.
Prepare the system test for tests that require a server to import
created SKR files. This means the test script needs adjustments to
take into account the directory test files are located.
In addition, the check_keys function is renamed to ksr_check_keys
because the name clashes with check_keys from kasp.sh. It also has an
extra parameter added, offset, that can be used to check ksr files that
are created in the past or future.
Add a new configuration option to enable Offline KSK key management.
Offline KSK cannot work with CSK because it splits how keys with the
KSK and ZSK role operate. Therefore, one key cannot have both roles.
Add a configuration check to ensure this.
This add the command line arguments: `-d` (debug), `-l` (list tests) and `-t test` (run this test) to the unit tests.
Closes#4579
Merge branch '4579-restore-the-ability-to-select-individual-unit-tests' into 'main'
Closes#4579
See merge request isc-projects/bind9!9384
A very rare assertion failure can be triggered when the incoming transfer is either forcefully shut down or it is finished during printing the details about the statistics channel. This has been fixed.
Closes#4860
Merge branch '4860-destroy-xfrin-timers-on-the-loop' into 'main'
Closes#4860
See merge request isc-projects/bind9!9336
There are few places where we attach/detach from the dns_xfrin object
while running on a different thread than the zone's assigned thread -
xfrin_xmlrender() in the statschannel and dns_zone_stopxfr() to name the
two places where it happens now. In the rare case, when the incoming
transfer completes (or shuts down) in the brief period between the other
thread attaches and detaches from the dns_xfrin, the isc_timer_destroy()
calls would be called by the last thread calling the xfrin_detach().
In the worst case, it would be this other thread causing assertion
failure. Move the isc_timer_destroy() call to xfrin_end() function
which is always called on the right thread and to match this move
isc_timer_create() to xfrin_start() - although this other change makes
no difference.
named now prints its initial working directory during startup and the
changed working directory when loading or reloading its configuration
file if it has a valid 'directory' option defined.
Closes#4731
Merge branch '4731-log-workdir-full-path' into 'main'
Closes#4731
See merge request isc-projects/bind9!9362
named now prints its initial working directory during startup and the
changed working directory when loading or reloading its configuration
file if it has a valid 'directory' option defined.
It looks like that all supported systems now have support for
_POSIX_SAVED_IDS, so it's safe to use setegid() and setegid() because
those will not change saved used/group IDs.
Closes#4862
Merge branch '4862-just-use-seteuid-and-setegid' into 'main'
Closes#4862
See merge request isc-projects/bind9!9363
It looks like that all supported systems now have support for
_POSIX_SAVED_IDS, so it's safe to use setegid() and setegid() because
those will not change saved used/group IDs.
If the machine running the job is extra loaded (or extra slow), the
print test in the badcache unit test would fail because the TTL would be
59 (possibly even lower) and the test expects the badcache.out to only
have TTL 60. Refactor the test to check for the expected strings and
check whether the TTL is in the expected range.
Closes#4861
Merge branch '4861-allow-TTL-skew-in-badcache_test' into 'main'
Closes#4861
See merge request isc-projects/bind9!9338
If the machine running the job is extra loaded (or extra slow), the
print test in the badcache unit test would fail because the TTL would be
59 (possibly even lower) and the test expects the badcache.out to only
have TTL 60. Refactor the test to check for the expected strings and
check whether the TTL is in the expected range.
Instead of juggling different logging context, use one single logging
context that gets initialized in the libisc constructor and destroyed in
the libisc destructor.
Closes#4848
Merge branch '4848-refactor-isc_log-api-to-use-single-global-logging-context' into 'main'
Closes#4848
See merge request isc-projects/bind9!9301
The new
isc_log_createandusechannel() function combines following calls:
isc_log_createchannel()
isc_log_usechannel()
calls into a single call that cannot fail and therefore can be used in
places where we know this cannot fail thus simplifying the error
handling.
Remove the complicated mechanism that could be (in theory) used by
external libraries to register new categories and modules with
statically defined lists in <isc/log.h>. This is similar to what we
have done for <isc/result.h> result codes. All the libraries are now
internal to BIND 9, so we don't need to provide a mechanism to register
extra categories and modules.
The isc_log_write1() and isc_log_vwrite1() functions were meant to
de-duplicate the messages sent to the isc_log subsystem. However, they
were never used in an entire code base and the whole mechanism around it
was complicated and very inefficient. Just remove those, there are
better ways to deduplicate syslog messages inside syslog daemons now.
Add isc_logconfig_get() function to get the current logconfig and use
the getter to replace most of the little dancing around setting up
logging in the tools. Thus:
isc_log_create(mctx, &lctx, &logconfig);
isc_log_setcontext(lctx);
dns_log_setcontext(lctx);
...
...use lcfg...
...
isc_log_destroy();
is now only:
logconfig = isc_logconfig_get(lctx);
...use lcfg...
For thread-safety, isc_logconfig_get() should be surrounded by RCU read
lock, but since we never use isc_logconfig_get() in threaded context,
the only place where it is actually used (but not really needed) is
named_log_init().
Instead of juggling different logging context, use one single logging
context that gets initialized in the libisc constructor and destroyed in
the libisc destructor.
The application is still responsible for creating the logging
configuration before using the isc_log API.
This patch is first in the series in a way that it is transparent for
the users of the isc_log API as the isc_log_create() and
isc_log_destroy() are now thin shims that emulate the previous
functionality, but it isc_log_create() will always return internal
isc__lctx pointer and isc_log_destroy() will actually not destroy the
internal isc__lctx context.
Signed-off-by: Ondřej Surý <ondrej@isc.org>
On FreeBSD, the TCP connection callback could spuriously receive
ISC_R_CONNECTIONRESET even when connection to the loopback interface.
Skip the other checks in such case and graciously shutdown the TCP
connection.
Closes#4849
Merge branch '4849-dispatch_test-fails-in-connected_shutdown-on-freebsd' into 'main'
Closes#4849
See merge request isc-projects/bind9!9303
On FreeBSD, the TCP connection callback could spuriously receive
ISC_R_CONNECTIONRESET even when connection to the loopback interface.
Skip the other checks in such case and graciously shutdown the TCP
connection.
Update Internal_use_only-CVE.md checklist changing some tasks and responsibilities reflecting the new ASN process with RT vs SF. Support will now be sending ASNs to customers, although Marketing will announce the new releases (both publicly and to support customers).
Merge branch 'vicky-main-patch-57381' into 'main'
See merge request isc-projects/bind9!9307
When manually handling the release notes (due to rst markup, fixups
etc.), the different MR number for backports causes needless friction.
Remove the reference from release notes and keep it only in changelog
which isn't manually redacted.
To reduce the friction when handling the release notes, it is preferable
to have the sections sorted by issue number, rather than merge order.
Fallback to commit subject line if unavailable (e.g. for changelog
entries).
The shim implementation of setresuid() was wrong - there was a copy and
paste error and it was calling setresgid() instead. This only affects
NetBSD because Linux, FreeBSD and OpenBSD have setresuid() and
setresgid() implementation available from the system library.
Closes#4862
Merge branch '4862-fix-setresuid-shim' into 'main'
Closes#4862
See merge request isc-projects/bind9!9359
The shim implementation of setresuid() was wrong - there was a copy and
paste error and it was calling setresgid() instead. This only affects
NetBSD because Linux, FreeBSD and OpenBSD have setresuid() and
setresgid() implementation available from the system library.
The test_traffic_json and test_traffic_xml occasionally fail when
running under TSAN. This happens in CI and is most likely a result of
some instability that doesn't seem to be easily reproduced.
Closes#4598
Merge branch '4598-mark-statschannel-test-flaky' into 'main'
Closes#4598
See merge request isc-projects/bind9!9293
The test_traffic_json and test_traffic_xml occasionally fail when
running under TSAN. This happens in CI and is most likely a result of
some instability that doesn't seem to be easily reproduced.
Log canceled resolver queries (e.g. when shutting down a hung
fetch) in DEBUG3 level instead of DEBUG1 which is used for the
"unrecognized" result codes.
Closes#4797
Merge branch '4797-rpz_rewrite-add-ISC_R_CANCELED-processing' into 'main'
Closes#4797
See merge request isc-projects/bind9!9148
In rpz system tests, we could leak file if the applying the updates has
failed. Add the missing fclose() before returning.
Merge branch 'ondrej/add-missing-flose-to-rpz-testlib' into 'main'
See merge request isc-projects/bind9!9317
Instead of directly using the result of dirfd() in the unlinkat() call,
check whether the returned file descriptor is actually valid. That
doesn't really change the logic as the unlinkat() would fail with
invalid descriptor anyway, but this is cleaner and will report the right
error returned directly by dirfd() instead of EBADF from unlinkat().
Closes#4853
Merge branch '4853-check-result-of-dirfd-in-isc_log' into 'main'
Closes#4853
See merge request isc-projects/bind9!9316
Instead of directly using the result of dirfd() in the unlinkat() call,
check whether the returned file descriptor is actually valid. That
doesn't really change the logic as the unlinkat() would fail with
invalid descriptor anyway, but this is cleaner and will report the right
error returned directly by dirfd() instead of EBADF from unlinkat().
The getifaddr() works fine for years, so we don't have to
keep the callback to parse /proc/net/if_inet6 anymore.
Closes#4852
Merge branch '4852-handle-errors-from-rewind' into 'main'
Closes#4852
See merge request isc-projects/bind9!9315
The clang-scan 19 has reported that we are ignoring errno after the call
to rewind(). As we don't really care about the result, just silence the
error, the whole code will be removed in the development version anyway
as it is not needed.
New version of clang (19) has introduced a stricter checks when mixing
integer (and float types) with enums. In this case, we used enum {}
as C17 doesn't have constexpr yet. Change the time conversion constants
to be static const unsigned int instead of enum values.
Closes#4845
Merge branch '4845-change-NS_PER_SEC-type-from-enum-to-integer' into 'main'
Closes#4845
See merge request isc-projects/bind9!9313
The contexpr introduced in C23 standard makes perfect sense to be used
instead of preprocessor macros - the symbols are kept, etc. Define
ISC_CONSTEXPR to be `constexpr` for C23 and `static const` for the older
C standards. Use the newly introduced macro for the NS_PER_SEC and
friends time constants.
New version of clang (19) has introduced a stricter checks when mixing
integer (and float types) with enums. In this case, we used enum {}
as C17 doesn't have constexpr yet. Change the time conversion constants
to be static const unsigned int instead of enum values.
The new TSAN images, the TSAN-enabled images install libraries to
opt/tsan, synchronize the configure options and CFLAGS between gcc:tsan
and clang:tsan images and set the PKG_CONFIG_PATH to /opt/tsan/lib.
Additionally, drop Debian bullseye that's EOL now.
Merge branch 'ondrej/use-staging-tsan-images' into 'main'
See merge request isc-projects/bind9!9324
The new TSAN images, the TSAN-enabled images install libraries to
/opt/tsan, synchronize the configure options and CFLAGS between gcc:tsan
and clang:tsan images and set the PKG_CONFIG_PATH to /opt/tsan/lib.
Check if 'lctx->logconfig' is NULL before using it in isc_log_doit(),
because it's possible that isc_log_destroy() was already called, e.g.
when a 'call_rcu' function wants to log a message during shutdown.
Closes#4842
Merge branch '4842-shutdown-crash-in-isc_log_doit' into 'main'
Closes#4842
See merge request isc-projects/bind9!9297
Check if 'lctx->logconfig' is NULL before using it in isc_log_doit(),
because it's possible that isc_log_destroy() was already called, e.g.
when a 'call_rcu' function wants to log a message during shutdown.
GCC 11.1+ emits a note during compilation when there are 64-bit
atomic fields in a structure, because it fixed a compiler bug
by changing the alignment of such fields, which caused ABI change.
Add -Wno-psabi to CFLAGS for such builds in order to silence the
warning. That shouldn't be a problem since we don't expose our
structures to the outside.
Closes#4841
Merge branch '4841-alignment-of-_Atomic-long-long-unsigned-int' into 'main'
Closes#4841
See merge request isc-projects/bind9!9319
GCC 11.1+ emits a note during compilation when there are 64-bit
atomic fields in a structure, because it fixed a compiler bug
by changing the alignment of such fields, which caused ABI change.
Add -Wno-psabi to CFLAGS for such builds in order to silence the
warning. That shouldn't be a problem since we don't expose our
structures to the outside.
The build system ensures it is always included for every source file.
Merge branch 'aydin/no-include-config-h' into 'main'
See merge request isc-projects/bind9!9320
Instead of keeping the whole array of test_node_t objects, just keep the
hashvalues and generated the rest of the test_node_t on the fly. The
test still works this way and the file size has been reduced from 2M to
90k.
Closes#4851
Merge branch '4851-generate-problematic-isc_hashmap-test-data' into 'main'
Closes#4851
See merge request isc-projects/bind9!9318
Instead of keeping the whole array of test_node_t objects, just keep the
hashvalues and generated the rest of the test_node_t on the fly. The
test still works this way and the file size has been reduced from 2M to
90k.
Change the placement of the attributes to match the existing usage in
other places (after the declaration).
Merge branch 'ondrej/ISC_CONSTRUCTOR-style' into 'main'
See merge request isc-projects/bind9!9291
Since changelog entries are now generated from MR title&description,
they aren't sanity checked during a regular docs build. If these contain
special sequences that will be interpreted by sphinx, it might result in
breakage that would have to be amended manually.
Add a CI check to test a doc build with changelog after the MR is merged
to ensure that the docs can be built when generating changelog from
pristine git contents.
Related #4847
Merge branch 'nicki/add-changelog-entry-check' into 'main'
See merge request isc-projects/bind9!9294
Some distributions (notably, debian bookworm) have deprecated the
`python` interpreter in favor of `python3`. Since our scripts are
python3 anyway, use the proper numbered version in shebang to make
scripts easily executable.
Since changelog entries are now generated from MR title&description,
they aren't sanity checked during a regular docs build. If these contain
special sequences that will be interpreted by sphinx, it might result in
breakage that would have to be amended manually.
Add a CI check to test a doc build with changelog after the MR is merged
to ensure that the docs can be built when generating changelog from
pristine git contents.
When the round robin hashing reorders the map entries on deletion, we
were adjusting the iterator table size only when the reordering was
happening at the internal table boundary. The iterator table size had
to be reduced by one to prevent seeing the entry that resized on
position [0] twice because it migrated to [iter->size - 1] position.
However, the same thing could happen when the same entry migrates a
second time from [iter->size - 1] to [iter->size - 2] position (and so
on) because the check that we are manipulating the entry just in the [0]
position was insufficient. Instead of checking the position [pos == 0],
we now check that the [pos % iter->size == 0], thus ignoring all the
entries that might have moved back to the end of the internal table.
Closes#4838
Merge branch '4838-fix-assertion-failure-in-hashmap-deletion-iterator' into 'main'
Closes#4838
See merge request isc-projects/bind9!9292
When iterating through the old internal hashmap table, skip all the
nodes that have been already migrated to the new table. We know that
all positions with index less than .hiter are NULL.
When the round robin hashing reorders the map entries on deletion, we
were adjusting the iterator table size only when the reordering was
happening at the internal table boundary. The iterator table size had
to be reduced by one to prevent seeing the entry that resized on
position [0] twice because it migrated to [iter->size - 1] position.
However, the same thing could happen when the same entry migrates a
second time from [iter->size - 1] to [iter->size - 2] position (and so
on) because the check that we are manipulating the entry just in the [0]
position was insufficient. Instead of checking the position [pos == 0],
we now check that the [pos % iter->size == 0], thus ignoring all the
entries that might have moved back to the end of the internal table.
Add second iterator test with a well-known configuration of the nodes
array that causes the last element of the array to migrate two times to
the previous node.
Ensure the issue number in changelog isn't accidentally removed for backport MRs.
Merge branch 'nicki/fix-gitchangelog-replacement-regexs' into 'main'
See merge request isc-projects/bind9!9295
Prior to this change, the issue number could be accidentally removed by
the `Backport of` text, depending on the order of the MR description
contents. Ensure all the removals for text in MR descriptions happen
first, and only then run the replacement regex for issue number, which
appends it to the end of the last non-empty line (which will no longer
be removed).
The only removals that happen after the replacement are guaranteed to
always happen after the end of MR description, since they're
auto-generated by gitlab when the merge commit is created, thus won't
affect the line with the issue number.
Also remove the needless isc-private/bind9 replacement. References
to private MRs are already removed by the very first regex.
As we now setup the logging very early, parsing the default config would
always print warnings about experimental (and possibly deprecated)
options in the default config. This would even mess with commands like
`named -V` and it is also wrong to warn users about using experimental
options in the default config, because they can't do anything about
this. Add CFG_PCTX_NODEPRECATED and CFG_PCTX_NOEXPERIMENTAL options
that we can pass to cfg parser and silence the early warnings caused by
using experimental options in the default config.
Merge branch 'ondrej/silence-warnings-from-default-config' into 'main'
See merge request isc-projects/bind9!9304
As we now setup the logging very early, parsing the default config would
always print warnings about experimental (and possibly deprecated)
options in the default config. This would even mess with commands like
`named -V` and it is also wrong to warn users about using experimental
options in the default config, because they can't do anything about
this. Add CFG_PCTX_NODEPRECATED and CFG_PCTX_NOEXPERIMENTAL options
that we can pass to cfg parser and silence the early warnings caused by
using experimental options in the default config.
The code in conftools/ directory hasn't been touched since 2000.
Nobody knows what it does and nobody even knows how to build it
or test it. Just remove the whole directory.
Merge branch 'ondrej/remove-outdated-perllib-integration' into 'main'
See merge request isc-projects/bind9!9302
The code in conftools/ directory hasn't been touched since 2000.
Nobody knows what it does and nobody even knows how to build it
or test it. Just remove the whole directory.
OpenSSL has added support for deterministic ECDSA (RFC 6979) with
version 3.2.
Use it by default as it removes arguably its most fragile side of ECDSA.
The derandomization doesn't pose a risk for DNS usecases and is allowed by FIPS 186-5.
Closes https://gitlab.isc.org/isc-projects/bind9/-/issues/299
Merge branch '299-change-ecdsa-to-deterministic-usage-elliptic-curve-digital-signature-algorithm-rfc-6979' into 'main'
Closes#299
See merge request isc-projects/bind9!9128
OpenSSL has added support for deterministic ECDSA (RFC 6979) with
version 3.2.
Use it by default as derandomization doesn't pose a risk for DNS
usecases and is allowed by FIPS 186-5.
During a reconfiguration named wasn't reconfiguring catalog zones'
member zones. This has been fixed.
Closesisc-projects/bind9#4733
Merge branch '4733-fix-catz-member-zones-reconfigure' into 'v9.21.0-release'
See merge request isc-private/bind9!695
This condition in catalog zone processing functions is checked only
for sanity, so there is no meaning for a log message that normally
shouldn't be ever logged.
Catalog zone member zones should be reconfigured as all the other
zones during a reconfiguration. Test it by checking whether the newly
added allow-query setting affects a member zone.
The fcount_incr() was not increasing counter->count when force was set
to true, but fcount_decr() would try to decrease the counter leading to
underflow and assertion failure. Swap the order of the arguments in the
condition, so the !force is evaluated after incrementing the .count.
Closes#4846
Merge branch '4786-forced-fcount_incr-should-still-increment-count-and-allowed-fix' into 'main'
Closes#4846
See merge request isc-projects/bind9!9298
The fcount_incr() was not increasing counter->count when force was set
to true, but fcount_decr() would try to decrease the counter leading to
underflow and assertion failure. Swap the order of the arguments in the
condition, so the !force is evaluated after incrementing the .count.
LaTeX in CI and on ReadTheDocs [fails][1] to render a PDF version of ARM if
the Changelog section is included. The running theory is that the
verbatim section of more than twenty thousand lines is too big to meet
LaTeX self-imposed constraints, and it fails with:
! TeX capacity exceeded, sorry [main memory size=5000000].
Or it just hangs if extra_mem_bot=30000000 is set in
/etc/texmf/texmf.d/01main_memory_bump.cnf:
! Dimension too large.
\fb@put@frame ...p \ifdim \dimen@ >\ht \@tempboxa
\fb@putboxa #1\fb@afterfra...
l.56913 \end{sphinxVerbatim}
Make each BIND 9 release a separate code block to work around the issue.
Further split up the sections for some exceptionally large releases, for
the same reason.
[1]: https://gitlab.isc.org/isc-projects/bind9/-/jobs/4584011
Merge branch 'mnowak/fix-arm-changelog-section' into 'main'
See merge request isc-projects/bind9!9266
LaTeX in CI and on ReadTheDocs fails to render a PDF version of ARM if
the Changelog section is included. The running theory is that the
verbatim section of more than twenty thousand lines is too big to meet
LaTeX self-imposed constraints, and it fails with:
! TeX capacity exceeded, sorry [main memory size=5000000].
Or it just hangs if extra_mem_bot=30000000 is set in
/etc/texmf/texmf.d/01main_memory_bump.cnf:
! Dimension too large.
\fb@put@frame ...p \ifdim \dimen@ >\ht \@tempboxa
\fb@putboxa #1\fb@afterfra...
l.56913 \end{sphinxVerbatim}
Make each BIND 9 release a separate code block to work around the issue.
Further split up the sections for some exceptionally large releases, for
the same reason.
Since the enable_fips_mode() now resides inside the isc_tls unit, BIND 9
would fail to compile when FIPS mode was enabled as the DST subsystem
logging functions were missing.
Move the crypto library logging functions from the openssl_link unit to
isc_tls unit and enhance it, so it can now be used from both places
keeping the old dst__openssl_toresult* macros alive.
Merge branch 'ondrej/move-openssl-logging-to-isc_tls-unit' into 'main'
See merge request isc-projects/bind9!9286
Since the enable_fips_mode() now resides inside the isc_tls unit, BIND 9
would fail to compile when FIPS mode was enabled as the DST subsystem
logging functions were missing.
Move the crypto library logging functions from the openssl_link unit to
isc_tls unit and enhance it, so it can now be used from both places
keeping the old dst__openssl_toresult* macros alive.
There were cases in resolver.c when the `max-recursion-queries` quota was ineffective. It was possible to craft zones that would cause a resolver to waste resources by sending excessive queries while attempting to resolve a name. This has been addressed by correcting errors in the implementation of `max-recursion-queries`, and by reducing the default value from 100 to 32.
In addition, a new `max-query-restarts` option has been added which limits the number of times a recursive server will follow CNAME or DNAME records before terminating resolution. This was previously a hard-coded limit of 16, and now defaults to 11.
Closes#4741
Merge branch '4741-reclimit-restarts' into 'main'
Closes#4741
See merge request isc-projects/bind9!9281
implement, document, and test the 'max-query-restarts' option
which specifies the query restart limit - the number of times
we can follow CNAMEs before terminating resolution.
MAX_RESTARTS is no longer hard-coded; ns_server_setmaxrestarts()
and dns_client_setmaxrestarts() can now be used to modify the
max-restarts value at runtime. in both cases, the default is 11.
the number of steps that can be followed in a CNAME chain
before terminating the lookup has been reduced from 16 to 11.
(this is a hard-coded value, but will be made configurable later.)
there were cases in resolver.c when queries for NS records were
started without passing a pointer to the parent fetch's query counter;
as a result, the max-recursion-queries quota for those queries started
counting from zero, instead of sharing the limit for the parent fetch,
making the quota ineffective in some cases.
Use `[[noreturn]]` when compiling with C23 or greater.
The attribute macro name has been capitalized as `NORETURN` as defining it as `noreturn` breaks external headers.
`#define noreturn __attribute__((noreturn))` wasn't used as C11's `stdnoreturn.h`/`_Noreturn` is required to build BIND9 in the first place.
Merge branch 'aydin/noreturn-c23' into 'main'
See merge request isc-projects/bind9!9149
Instead of calling dst_lib_init() and dst_lib_destroy() explicitly by
all the programs, create a separate memory context for the DST subsystem
and use the library constructor and destructor to initialize the DST
internals.
Merge branch 'ondrej/move-dst_lib_init-and-destroy-to-lib-ctor-dtor' into 'main'
See merge request isc-projects/bind9!9254
Instead of calling dst_lib_init() and dst_lib_destroy() explicitly by
all the programs, create a separate memory context for the DST subsystem
and use the library constructor and destructor to initialize the DST
internals.
The new Fedora 40 TSAN images use libuv, urcu and OpenSSL libraries compiled with ThreadSanitizer. This (in theory) should enable better detection of memory races in those (most important) libraries.
Merge branch 'ondrej/test-new-tsan-images' into 'main'
See merge request isc-projects/bind9!9264
When the SSL object was destroyed, it would invalidate all SSL_SESSION
objects including the cached, but not yet used, TLS session objects.
Properly disassociate the SSL object from the SSL_SESSION before we
store it in the TLS session cache, so we can later destroy it without
invalidating the cached TLS sessions.
Closes#4834
Merge branch '4834-detach-SSL-from-cached-SSL_SESSION' into 'main'
Closes#4834
See merge request isc-projects/bind9!9271
When the SSL object was destroyed, it would invalidate all SSL_SESSION
objects including the cached, but not yet used, TLS session objects.
Properly disassociate the SSL object from the SSL_SESSION before we
store it in the TLS session cache, so we can later destroy it without
invalidating the cached TLS sessions.
Co-authored-by: Ondřej Surý <ondrej@isc.org>
Co-authored-by: Artem Boldariev <artem@isc.org>
Co-authored-by: Aram Sargsyan <aram@isc.org>
When TLS connection (TLSstream) connection was accepted, the children
listening socket was not attached to sock->server and thus it could have
been freed before all the accepted connections were actually closed.
In turn, this would cause us to call isc_tls_free() too soon - causing
cascade errors in pending SSL_read_ex() in the accepted connections.
Properly attach and detach the children listening socket when accepting
and closing the server connections.
Closes#4833
Merge branch '4833-tlssock-needs-to-attach-to-child-tlslistener' into 'main'
Closes#4833
See merge request isc-projects/bind9!9270
When TLS connection (TLSstream) connection was accepted, the children
listening socket was not attached to sock->server and thus it could have
been freed before all the accepted connections were actually closed.
In turn, this would cause us to call isc_tls_free() too soon - causing
cascade errors in pending SSL_read_ex() in the accepted connections.
Properly attach and detach the children listening socket when accepting
and closing the server connections.
Missing file util/dtrace.sh prevented builds on system without dtrace utility.
This has been corrected.
Fixes: #4835
Merge branch 'pspacek/gitattribute-fixes' into 'main'
Closes#4835
See merge request isc-projects/bind9!9262
Ensure that system tests can be executed without Python hypothesis
package.
Closes#4831
Merge branch '4831-isctest-make-hypothesis-optional' into 'main'
Closes#4831
See merge request isc-projects/bind9!9265
Query responses should contain the question section with some exceptions. Dig was not reporting this.
Closes#4808
Merge branch '4808-have-dig-report-missing-question-section-in-axfr-response' into 'main'
Closes#4808
See merge request isc-projects/bind9!9233
The question section should be present in the first AXFR/IXFR
response and in other QUERY responses unless no question was sent.
Issue a warning if the question section is not present.
The OpenSSL 1.x Engines support has been deprecated in the OpenSSL 3.x
and is going to be removed from the upstream OpenSSL. Remove the OpenSSL
Engine support from BIND 9 in favor of OpenSSL 3.x Providers.
Closes#4828
Merge branch '4828-remove-OpenSSL-engine-support' into 'main'
Closes#4828
See merge request isc-projects/bind9!9252
OpenSSL_version() first appeared in OpenSSL 1.1.0 and have been
available since LibreSSL 2.7.1 and OpenBSD 6.3, thus we can remove the
compatibility shims with older versions that are not supported anymore.
Since the support for OpenSSL Engines has been removed, we can now also
remove the checks for OPENSSL_API_LEVEL; The OpenSSL 3.x APIs will be
used when compiling with OpenSSL 3.x, and OpenSSL 1.1.xx APIs will be
used only when OpenSSL 1.1.x is used.
The OpenSSL 1.x Engines support has been deprecated in the OpenSSL 3.x
and is going to be removed. Remove the OpenSSL Engine support in favor
of OpenSSL Providers.
The isc_mem_create() in printversion() was created outside of an #ifdef
HAVE_GEOIP, but destroyed inside the #ifdef; move it to the outside of
the #ifdef where it belongs.
This is really a nit as we immediately exit() after printing the
versions, but I found it and it would bug me for the rest of my life.
Merge branch 'ondrej/fix-missing-isc_mem_destroy-in-printversion' into 'main'
See merge request isc-projects/bind9!9255
The isc_mem_create() in printversion() was created outside of an #ifdef
HAVE_GEOIP, but destroyed inside the #ifdef; move it to the outside of
the #ifdef where it belongs.
This is really a nit as we immediately exit() after printing the
versions, but I found it and it would bug me for the rest of my life.
Fix an assertion failure that could happen as a result of data race between free_gluetable() and addglue() on the same headers.
Closes#4691
Merge branch '4691-fix-data-race-between-free_gluetable-and-addglue' into 'main'
Closes#4691
See merge request isc-projects/bind9!9126
when searching the cache for a node so that we can delete an rdataset, it isn't necessary to set the 'create' flag. if the
node doesn't exist yet, we won't be able to delete anything from it anyway.
Merge branch 'each-minor-findnode-refactor' into 'main'
See merge request isc-projects/bind9!9158
When adding glue to the header, we add header to the wait-free stack to
be cleaned up later which sets wfc_node->next to non-NULL value. When
the actual cleaning happens we would only cleanup the .glue_list, but
since the database isn't locked for the time being, the headers could be
reused while cleaning the existing glue entries, which creates a data
race between database versions.
Revert the code back to use per-database-version hashtable where keys
are the node pointers. This allows each database version to have
independent glue cache table that doesn't affect nodes or headers that
could already "belong" to the future database version.
when searching the cache for a node so that we can delete an
rdataset, it is not necessary to set the 'create' flag. if the
node doesn't exist yet, we then we won't be able to delete
anything from it anyway.
dns_difftuple_create() could only return success, so change
its type to void and clean up all the calls to it.
Merge branch 'each-difftuple-create-cannot-fail' into 'main'
See merge request isc-projects/bind9!9151
dns_difftuple_create() could only return success, so change
its type to void and clean up all the calls to it.
other functions that only returned a result value because of it
have been cleaned up in the same way.
When a priming query is complete, it's currently logged at level ISC_LOG_DEBUG(1), regardless of success or failure. We are now raising it to ISC_LOG_NOTICE in the case of failure. [GL #3516]
Closes#3516
Merge branch '3516-log-priming-errors' into 'main'
Closes#3516
See merge request isc-projects/bind9!9121
when a priming query is complete, it's currently logged at
level ISC_LOG_DEBUG(1), regardless of success or failure. we
are now raising it to ISC_LOG_NOTICE in the case of failure.
There isn't a realistic reason to ever use e = 4294967297. Fortunately
its codepath wasn't reachable to users and can be safetly removed.
Keep in mind the `dns_key_generate` header comment was outdated. e = 3
hasn't been used since 2006 so there isn't a reason to panic. The
toggle was the public exponents between 65537 and 4294967297.
Merge branch 'aydin/rsa-cleanup' into 'main'
See merge request isc-projects/bind9!9133
There isn't a realistic reason to ever use e = 4294967297. Fortunately
its codepath wasn't reachable to users and can be safetly removed.
Keep in mind the `dns_key_generate` header comment was outdated. e = 3
hasn't been used since 2006 so there isn't a reason to panic. The
toggle was the public exponents between 65537 and 4294967297.
CRC-64 has been added for map files. Now that the map file format has
been removed, there isn't a reason to keep the implementation.
Merge branch 'aydin/cleanup-crc' into 'main'
See merge request isc-projects/bind9!9135
The previous work in this area was led by the belief that we might be
calling call_rcu() from within call_rcu() callbacks. After carefully
checking all the current callback, it became evident that this is not
the case and the problem isn't enough rcu_barrier() calls, but something
entirely else.
Call the rcu_barrier() just once as that's enough and the multiple
rcu_barrier() calls will not hide the real problem anymore, so we can
find it.
Merge branch '4607-call-a-single-rcu_barrier' into 'main'
See merge request isc-projects/bind9!9134
The previous work in this area was led by the belief that we might be
calling call_rcu() from within call_rcu() callbacks. After carefully
checking all the current callback, it became evident that this is not
the case and the problem isn't enough rcu_barrier() calls, but something
entirely else.
Call the rcu_barrier() just once as that's enough and the multiple
rcu_barrier() calls will not hide the real problem anymore, so we can
find it.
OpenSSL 1.1.1 or better (or equivalent LibreSSL version) is now required to compile BIND 9.
Closes#2806
Merge branch '2806-remove-ax_check_openssl' into 'main'
Closes#2806
See merge request isc-projects/bind9!9110
Since the minimal OpenSSL version is now OpenSSL 1.1.1, remove all kind
of OpenSSL shims and checks for functions that are now always present in
the OpenSSL libraries.
Co-authored-by: Ondřej Surý <ondrej@isc.org>
Co-authored-by: Aydın Mercan <aydin@isc.org>
OpenSSL supports pkg-config method since the 0.9.8 version and we
already require pkg-config for other mandatory libraries. Also
the way the AX_CHECK_OPENSSL macro was integrated into the configure
script was confusing - the macro would be used only if the libcrypto.pc
and libssl.pc file are not usable, so calling ./configure
--with-openssl=/usr/local would have no effect when PKG_CHECK_MODULES
would be successful.
As BIND 9.20 does not support RHEL/CentOS 7 which just reach
end-of-life, we can safely bump the OpenSSL requirements to version
1.1.1, which in turn will allow us to simplify our OpenSSL integration.
Checking the version of `named-checkconf` would end with assertion failure. This has been fixed.
Closes#4827
Merge branch '4827-cleanup-dst-only-if-initialized' into 'main'
Closes#4827
See merge request isc-projects/bind9!9243
The dst_lib_destroy() should be called only if dst_lib_init() was called
before. In named-checkconf, that is guarded by dst_cleanup variable
that was erroneously set to true by default. Set the dst_cleanup to
'false' by default.
An assertion failure triggers when the TSIG has valid cryptographic signature, but the time is invalid. This can happen when the times between the primary and secondary servers are not synchronised.
Closes#4811
Merge branch '4811-fix-isc_buffer_putuint48-buffer-size-requirement' into 'main'
Closes#4811
See merge request isc-projects/bind9!9234
Add a system test that sets TSIG fudge to 0, waits three seconds and
then sends signed message to the server. This tests the path where the
time difference between the client and the server is outside of the TSIG
fudge value.
The tsig unit test was only testing if everything went ok, but it was
not testing whether the error paths work. Add two more unit tests - one
uses the time outside of the TSIG skew, and the second trashes the
signature with random data.
When putting the 48-bit number into a fixed-size buffer that's exactly 6
bytes, the assertion failure would occur as the 48-bit number is
internally represented as 64-bit number and the code was checking if
there is enough space for `sizeof(val)`. This causes assertion failure
when otherwise valid TSIG signature has a bad timing information.
Specify the size of the argument explicitly, so the 48-bit number
doesn't require 8-byte long buffer.
The fcount_incr() was incorrectly skipping the accounting for the
fetches-per-zone if the force argument was set to true. We want to skip
the accounting only when the fetches-per-zone is completely disabled,
but for individual names we need to do the accounting even if we are
forcing the result to be success.
Closes#4786
Merge branch '4786-forced-fcount_incr-should-still-increment-count-and-allowed' into 'main'
Closes#4786
See merge request isc-projects/bind9!9115
The fcount_incr() was incorrectly skipping the accounting for the
fetches-per-zone if the force argument was set to true. We want to skip
the accounting only when the fetches-per-zone is completely disabled,
but for individual names we need to do the accounting even if we are
forcing the result to be success.
The system tests were overriding the local locale by setting LANG to C.
This does not override the locale in case there are individual LC_<*>
variables like LC_CTYPE explicitly set.
Use LC_ALL=C instead which is the proper way of overriding all currently
set locales.
Merge branch 'ondrej/use-LC_ALL-not-LANG' into 'main'
See merge request isc-projects/bind9!9109
The system tests were overriding the local locale by setting LANG to C.
This does not override the locale in case there are individual LC_<*>
variables like LC_CTYPE explicitly set.
Use LC_ALL=C instead which is the proper way of overriding all currently
set locales.
The tlsstream part of the isc_nmsocket_t gets initialized via designater
initializer and doesn't need the extra memset() later; just remove it.
Merge branch 'ondrej/remove-superfluous-memset-in-isc_nmsocket_init' into 'main'
See merge request isc-projects/bind9!9120
The PTHREAD_MUTEX_ADAPTIVE_NP and PTHREAD_MUTEX_ERRORCHECK_NP are
usually not defines, but enum values, so simple preprocessor check
doesn't work.
Check for PTHREAD_MUTEX_ADAPTIVE_NP from the autoconf AS_COMPILE_IFELSE
block and define HAVE_PTHREAD_MUTEX_ADAPTIVE_NP. This should enable
adaptive mutex on Linux and FreeBSD.
As PTHREAD_MUTEX_ERRORCHECK actually comes from POSIX and Linux glibc
does define it when compatibility macros are being set, we can just use
PTHREAD_MUTEX_ERRORCHECK instead of PTHREAD_MUTEX_ERRORCHECK_NP.
Merge branch 'ondrej/fix-adaptive-mutex-use' into 'main'
See merge request isc-projects/bind9!9111
The PTHREAD_MUTEX_ADAPTIVE_NP and PTHREAD_MUTEX_ERRORCHECK_NP are
usually not defines, but enum values, so simple preprocessor check
doesn't work.
Check for PTHREAD_MUTEX_ADAPTIVE_NP from the autoconf AS_COMPILE_IFELSE
block and define HAVE_PTHREAD_MUTEX_ADAPTIVE_NP. This should enable
adaptive mutex on Linux and FreeBSD.
As PTHREAD_MUTEX_ERRORCHECK actually comes from POSIX and Linux glibc
does define it when compatibility macros are being set, we can just use
PTHREAD_MUTEX_ERRORCHECK instead of PTHREAD_MUTEX_ERRORCHECK_NP.
When automatic-interface-scan is disabled, the route socket was still
being opened. Add new API to connect / disconnect from the route socket
only as needed.
Additionally, move the block that disables periodic interface rescans to
a place where it actually have access to the configuration values.
Previously, the values were being checked before the configuration was
loaded.
Closes#4757
Merge branch '4757-dont-open-routing-socket-if-not-needed' into 'main'
Closes#4757
See merge request isc-projects/bind9!9122
When automatic-interface-scan is disabled, the route socket was still
being opened. Add new API to connect / disconnect from the route socket
only as needed.
Additionally, move the block that disables periodic interface rescans to
a place where it actually have access to the configuration values.
Previously, the values were being checked before the configuration was
loaded.
I split this into two commits, one for the actual newline removal, and one for issues I found, ruining the yaml output when some errors were outputted.
Closes: #4772
Merge branch 'yaml-indent' into 'main'
Closes#4772
See merge request isc-projects/bind9!9112
Add INSIST to fail if the multiplication would cause the variables to overflow.
Closes#4798
Merge branch '4798-cid-498025-and-cid-498031-overflowed-constant-integer_overflow' into 'main'
Closes#4798
See merge request isc-projects/bind9!9131
If bufsize overflows we will have an infinite loop. In practice
this will not happen unless we have made a coding error. Add an
INSIST to detect this condition.
181retry:
182 isc_buffer_allocate(mctx, &b, bufsize);
183 result = dns_rdata_totext(rdata, NULL, b);
184 if (result == ISC_R_NOSPACE) {
185 isc_buffer_free(&b);
CID 498031: (#1 of 1): Overflowed constant (INTEGER_OVERFLOW)
overflow_const: Expression bufsize, which is equal to 0, overflows
the type that receives it, an unsigned integer 32 bits wide.
186 bufsize *= 2;
187 goto retry;
188 }
If size overflows we will have an infinite loop. In practice
this will not happen unless we have made a coding error. Add
an INSIST to detect this condition.
181 while (!done) {
182 isc_buffer_allocate(mctx, &b, size);
183 result = dns_rdata_totext(rdata, NULL, b);
184 if (result == ISC_R_SUCCESS) {
185 printf("%.*s\n", (int)isc_buffer_usedlength(b),
186 (char *)isc_buffer_base(b));
187 done = true;
188 } else if (result != ISC_R_NOSPACE) {
189 check_result(result, "dns_rdata_totext");
190 }
191 isc_buffer_free(&b);
CID 498025: (#1 of 1): Overflowed constant (INTEGER_OVERFLOW)
overflow_const: Expression size, which is equal to 0, overflows the type that
receives it, an unsigned integer 32 bits wide.
192 size *= 2;
193 }
Decrementing optlen immediately before calling continue is unneccesary
and inconsistent with the rest of dns_message_pseudosectiontoyaml
and dns_message_pseudosectiontotext. Coverity was also reporting
an impossible false positive overflow of optlen (CID 499061).
4176 } else if (optcode == DNS_OPT_CLIENT_TAG) {
4177 uint16_t id;
4178 ADD_STRING(target, "; CLIENT-TAG:");
4179 if (optlen == 2U) {
4180 id = isc_buffer_getuint16(&optbuf);
4181 snprintf(buf, sizeof(buf), " %u\n", id);
4182 ADD_STRING(target, buf);
CID 499061: (#1 of 1): Overflowed constant (INTEGER_OVERFLOW)
overflow_const: Expression optlen, which is equal to 65534, underflows
the type that receives it, an unsigned integer 16 bits wide.
4183 optlen -= 2;
4184 POST(optlen);
4185 continue;
4186 }
4187 } else if (optcode == DNS_OPT_SERVER_TAG) {
Merge branch 'marka-remove-unnecessary-operations' into 'main'
See merge request isc-projects/bind9!9130
Decrementing optlen immediately before calling continue is unneccesary
and inconsistent with the rest of dns_message_pseudosectiontoyaml
and dns_message_pseudosectiontotext. Coverity was also reporting
an impossible false positive overflow of optlen (CID 499061).
4176 } else if (optcode == DNS_OPT_CLIENT_TAG) {
4177 uint16_t id;
4178 ADD_STRING(target, "; CLIENT-TAG:");
4179 if (optlen == 2U) {
4180 id = isc_buffer_getuint16(&optbuf);
4181 snprintf(buf, sizeof(buf), " %u\n", id);
4182 ADD_STRING(target, buf);
CID 499061: (#1 of 1): Overflowed constant (INTEGER_OVERFLOW)
overflow_const: Expression optlen, which is equal to 65534, underflows
the type that receives it, an unsigned integer 16 bits wide.
4183 optlen -= 2;
4184 POST(optlen);
4185 continue;
4186 }
4187 } else if (optcode == DNS_OPT_SERVER_TAG) {
The minimum allowed value of 'resolver-query-timeout' was lowered to
301 milliseconds instead of the earlier 10000 milliseconds (which is the
default). As earlier, values less than or equal to 300 are converted to
seconds before applying the limit.
Closes#4320
Merge branch '4320-allow-shorter-resolver-query-timeout-configuration' into 'main'
Closes#4320
See merge request isc-projects/bind9!9091
Before there was a gap from 301 to 9999 which would be converted
to 10000 and now there is no such gap.
This settimeout_belowmin test was checking the behavior of a value
in the gap. As there is now no gap left, the minimum is 301 and
anything below that is converted to seconds as before. In order
for this check to still test the "below minimum" behavior, change
the value from 9000 to 300.
Update the settimeout_overmax value test too so it logically aligns
with the minimum value test.
There are use cases for which shorter timeout values make sense.
For example if there is a load balancer which sets RD=1 and
forwards queries to a BIND resolver which is then configured to
talk to backend servers which are not visible in the public NS set.
WIth a shorter timeout value the frontend can give back SERVFAIL
early when backends are not available and the ultimate client will
not penalize the BIND-frontend for non-response.
A new optional argument '-force' has been added to the command channel
command 'rndc retransfer'. When it is specified, named aborts the
ongoing zone transfer (if there is one), and starts a new transfer.
Closes#2299
Merge branch '2299-implement-rndc-force-retransfer' into 'main'
Closes#2299
See merge request isc-projects/bind9!9102
Use a big zone and the slow transfer mode. Initiate a retransfer, wait
several seconds, then initiate a retransfer using a '-force' argument,
which should cancel the previous transfer and start a new one.
The period between the most significant nibble of the encoded IPv4 address and the 2.0.0.2.IP6.ARPA suffix was missing resulting in the wrong name being checked. Add system test for 6to4-self implementation.
Closes#4766
Merge branch '4766-add-system-test-for-6to4-self' into 'main'
Closes#4766
See merge request isc-projects/bind9!9099
The period between the most significant nibble of the IPv4 address
and the 2.0.0.2.IP6.ARPA suffix was missing resulting in the wrong
name being checked.
Remove the false positive "success resolving" log message when QNAME minimisation is in effect and the final result is NXDOMAIN.
Closes#4784
Merge branch '4784-false-qname-minimisation-error-being-reported' into 'main'
Closes#4784
See merge request isc-projects/bind9!9117
In yaml mode we emit a string for each question and record. Certain
names and data could result in invalid yaml being produced. Use single
quote string for all questions and records. This requires that single
quotes get converted to two quotes within the string.
ALPN are defined as 1*255OCTET in RFC 9460. commatxt_fromtext was not
rejecting invalid inputs produces by missing a level of escaping
which where later caught be dns_rdata_fromwire on reception.
These inputs should have been rejected
svcb in svcb 1 1.svcb alpn=\,abc
svcb1 in svcb 1 1.svcb alpn=a\,\,abc
and generated 00 03 61 62 63 and 01 61 00 02 61 62 63 respectively.
The correct inputs to include commas in the alpn requires double
escaping.
svcb in svcb 1 1.svcb alpn=\\,abc
svcb1 in svcb 1 1.svcb alpn=a\\,\\,abc
and generate 04 2C 61 62 63 and 06 61 2C 2C 61 62 63 respectively.
Add a note that 'rndc reload' and 'rndc reconfig' can't change the
querylog option during the runtime of named.
Closes#4801
Merge branch '4801-arm-querylog-clarification' into 'main'
Closes#4801
See merge request isc-projects/bind9!9136
Replace #define DNS_GETDB_ with struct of bools to make
it easier to pretty-print the attributes in a debugger.
Closes#4559
Merge branch '4559-convert-dns_getdb_x-defines-to-1-bit-long-bools' into 'main'
Closes#4559
See merge request isc-projects/bind9!9093
When cutting a long CNAME chain, named was returning NOERROR
instead of SERVFAIL (alongside with a partial answer). This
has been fixed.
Closes#4449
Merge branch '4449-return-servfail-for-a-long-cname-chain' into 'main'
Closes#4449
See merge request isc-projects/bind9!9090
Update the CNAME chain test to correspond to the changed behavior,
because now named returns SERVFAIL when hitting the maximum query
restarts limit (e.g. happening when following a long CNAME chain).
In the current test auth will hit the limit and return partial data
with a SERVFAIL code, while the resolver will return no data with
a SERVFAIL code after auth returns SERVFAIL to it.
Due to the maximum query restart limitation a long CNAME chain
it is cut after 16 queries but named still returns NOERROR.
Return SERVFAIL instead and the partial answer.
Ensure that the selected algorithms remains stable throughout the entire test session. Crypto support detection was rewritten to python and simplified.
Closes#4202Closes#4422
Related #3810
Merge branch '4202-algorithm-detection-pytest' into 'main'
Closes#4202 and #4422
See merge request isc-projects/bind9!8803
Ensure all the variables are initialized when running the main function
of isctest module. This enables proper environment variables during test
script development when only conf.sh is sourced, rather than the script
being executed by the pytest runner.
Run the crypto support checks when initializing the isctest package and
save those results in environment variable. This removes the need to
repeatedly check for crypto operation support, as it's not something
that would change at test runtime.
Instead of invoking get_algorithms.py script repeatedly (which may yield
different results), move the algorithm configuration to an isctest
module. This ensures the variables are consistent across the entire test
run.
Add two mechanisms to close open log files. The first is `rndc closelogs`. The second is `kill -USR1 <pid>`.
They are intended to be used with external log rotation tools.
Closes#4780
Merge branch '4780-add-support-for-external-log-rotation-tools' into 'main'
Closes#4780
See merge request isc-projects/bind9!9113
Some external log file rotation programs use signals to tell programs
to close log files. SIGHUP can be used to do this but it also does
a full reconfiguration. Configure named to accept SIGUSR1 as a
signal to close log files.
On 32 bit machines isc_meminfo_totalphys could return an incorrect value.
Closes#4799
Merge branch '4799-cid-498034-overflowed-return-value-integer_overflow' into 'main'
Closes#4799
See merge request isc-projects/bind9!9132
On a 32 bit machine casting to size_t can still lead to an overflow.
Cast to uint64_t. Also detect all possible negative values for
pages and pagesize to silence warning about possible negative value.
39#if defined(_SC_PHYS_PAGES) && defined(_SC_PAGESIZE)
1. tainted_data_return: Called function sysconf(_SC_PHYS_PAGES),
and a possible return value may be less than zero.
2. assign: Assigning: pages = sysconf(_SC_PHYS_PAGES).
40 long pages = sysconf(_SC_PHYS_PAGES);
41 long pagesize = sysconf(_SC_PAGESIZE);
42
3. Condition pages == -1, taking false branch.
4. Condition pagesize == -1, taking false branch.
43 if (pages == -1 || pagesize == -1) {
44 return (0);
45 }
46
5. overflow: The expression (size_t)pages * pagesize might be negative,
but is used in a context that treats it as unsigned.
CID 498034: (#1 of 1): Overflowed return value (INTEGER_OVERFLOW)
6. return_overflow: (size_t)pages * pagesize, which might have underflowed,
is returned from the function.
47 return ((size_t)pages * pagesize);
48#endif /* if defined(_SC_PHYS_PAGES) && defined(_SC_PAGESIZE) */
Stop updating `find.result_v4` and `find.result_v4` in `clean_finds_at_name`. The values are supposed to be static.
Closes#4118
Merge branch '4118-data-race-lib-dns-adb-c-1537-in-clean_finds_at_name' into 'main'
Closes#4118
See merge request isc-projects/bind9!9108
Don't verify the just signed zone as the RRSIGs could have expired before the signing process completes
Closes#4781#2476
Merge branch '4781-statschannel-setup-can-fail-due-to-short-validity-interval' into 'main'
Closes#4781 and #2476
See merge request isc-projects/bind9!9114
The updatecheck-kskonly.secure zone is being used to test dynamic
updates while the KSK is offline. It ensures that the DNSKEY RRset
will retain the RRSIG record, while the updated data is being signed
with the currently active ZSK.
When walking through ZSK rollovers, ensure that the newest ZSK (ZSK3)
is published before doing the dynamic update, preventing timing
related test failures.
Also fix the test log line ($ZSK_ID3 was not yet created at the time
of logging).
Adjust key state and timing metadata if dnssec-policy key lifetime configuration is updated, so that it also
affects existing keys.
Closes#4677
Merge branch '4677-dnssec-policy-key-lifetime-reconfigure' into 'main'
Closes#4677
See merge request isc-projects/bind9!9118
The key lifetime should no longer be adjusted if the key is being
retired earlier, for example because a manual rollover was started.
This would falsely be seen as a dnssec-policy lifetime reconfiguration,
and would adjust the retire/removed time again.
This also means we should update the status output, and the next
rollover scheduled is now calculated using (retire-active) instead of
key lifetime.
If dnssec-policy is reconfigured and the key lifetime has changed,
update existing keys with the new lifetime and adjust the retire
and removed timing metadata accordingly.
If the key has no lifetime yet, just initialize the lifetime. It
may be that the retire/removed timing metadata has already been set.
Skip keys which goal is not set to omnipresent. These keys are already
in the progress of retiring, or still unused.
Use a single source of truth, the git log, to generate the list of CHANGES. Use the .rst format and include it in the ARM for a quick reference with proper gitlab links to issues and merge requests.
Closes#75
Merge branch 'nicki/add-gitchangelog' into 'main'
Closes#75
See merge request isc-projects/bind9!9152
The files in contrib/ directory shouldn't be subject to our pylint
check. They can come from external sources and we don't subject these to
the same standards as the rest of the BIND9 code / scripts.
The configuration has been crafted to cater for BIND9 needs:
- Define actions that have an equivalent section in existing Release
Notes
- Assume the commits that have the necessary changelog makrup are merge
commits from GitLab and transform them into messages and proper links
- Put the resulting changelog into the proper place in
doc/arm/changelog.rst
- Have a separate configuration for changelog and release notes. Both of
these should be generated from the `git log`, with release notes being
subject to more scrutiny and further editing
The project hasn't seen any new development/changes since 2018 and it
seems unlikely we'd be able to get any changes into the upstream. Since
it's isolated into a single file and its task is fairly straighforward,
pull the code into our own repository and maintain it here as needed.
This also makes it easier to make any changes that are specific to our
project.
This commit ensures that we are not attempting to accept an expired
TCP connection as we are not interested in any data that could have
been accumulated in its internal buffers. Now we just drop them for
good.
When sending fails, the ns__client_request() would not reset the
connection and continue as nothing is happening. This comes from the
model that we don't care about failed UDP sends because datagrams are
unreliable anyway, but it greatly affects TCP connections with
keep-alive.
The worst case scenario is as follows:
1. the 3-way TCP handshake gets completed
2. the libuv calls the "uv_connection_cb" callback
3. the TCP connection gets queue because of the tcp-clients quota
4. the TCP client sends as many DNS messages as the buffers allow
5. the TCP connection gets dropped by the client due to the timeout
6. the TCP connection gets accepted by the server
7. the data already sent by the client gets read
8. all sending fails immediately because the TCP connection is dead
9. we consume all the data in the buffer in a very tight loop
As it doesn't make sense to trying to process more data on the TCP
connection when the sending is failing, drop the connection immediately
on the first sending error.
As ns_query_init() cannot fail now, remove the error paths, especially
in ns__client_setup() where we now don't have to care what to do with
the connection if setting up the client could fail. It couldn't fail
even before, but now it's formal.
Be more aggressive when throttling the reading - when we can't send the
outgoing TCP synchronously with uv_try_write(), we start throttling the
reading immediately instead of waiting for the send buffers to fill up.
This should not affect behaved clients that read the data from the TCP
on the other end.
Instead of outright refusing to add new RR types to the cache, be a bit
smarter:
1. If the new header type is in our priority list, we always add either
positive or negative entry at the beginning of the list.
2. If the new header type is negative entry, and we are over the limit,
we mark it as ancient immediately, so it gets evicted from the cache
as soon as possible.
3. Otherwise add the new header after the priority headers (or at the
head of the list).
4. If we are over the limit, evict the last entry on the normal header
list.
Add HTTPS, SVCB, SRV, PTR, NAPTR, DNSKEY and TXT records to the list of
the priority types that are put at the beginning of the slabheader list
for faster access and to avoid eviction when there are more types than
the max-types-per-name limit.
Due to omission it was possible to un-throttle a TCP connection
previously throttled due to the peer not reading back data we are
sending.
In particular, that affected DoH code, but it could also affect other
transports (the current or future ones) that pause/resume reading
according to its internal state.
Clear qctx->zversion when clearing qctx->zrdataset et al in
lib/ns/query.c:qctx_freedata. The uncleared pointer could lead to
an assertion failure if zone data needed to be re-saved which could
happen with stale data support enabled.
A different solution in the future might be adopted depending
on feedback and other new information, so it makes sense to mark
these options as EXPERIMENTAL until we have more data.
View matching on an incoming query checks the query's signature,
which can be a CPU-heavy task for a SIG(0)-signed message. Implement
an asynchronous mode of the view matching function which uses the
offloaded signature checking facilities, and use it for the incoming
queries.
Add support for using the offload threadpool to perform message
signature verifications. This should allow check SIG(0)-signed
messages without affecting the worker threads.
This is a tiny helper function which is used only once and can be
replaced with two function calls instead. Removing this makes
supporting asynchronous signature checking less complicated.
In order to protect from a malicious DNS client that sends many
queries with a SIG(0)-signed message, add a quota of simultaneously
running SIG(0) checks.
This protection can only help when named is using more than one worker
threads. For example, if named is running with the '-n 4' option, and
'sig0checks-quota 2;' is used, then named will make sure to not use
more than 2 workers for the SIG(0) signature checks in parallel, thus
leaving the other workers to serve the remaining clients which do not
use SIG(0)-signed messages.
That limitation is going to change when SIG(0) signature checks are
offloaded to "slow" threads in a future commit.
The 'sig0checks-quota-exempt' ACL option can be used to exempt certain
clients from the quota requirements using their IP or network addresses.
The 'sig0checks-quota-maxwait-ms' option is used to define a maximum
amount of time for named to wait for a quota to appear. If during that
time no new quota becomes available, named will answer to the client
with DNS_R_REFUSED.
By default we log a rekey failure on debug level. We should probably
change the log level to error. We make an exception for when the zone
is not loaded yet, it often happens at startup that a rekey is
run before the zone is fully loaded.
when signatures were not added because of too many types already
existing at a node, the diff was not being cleaned up; this led to
a memory leak being reported at shutdown.
kasp-max-types-per-name (named2.conf.in):
An unsigned zone with RR type count on a name right below the
configured limit. Then sign the zone using KASP. Adding a RRSIG would
push it over the RR type limit per name. Signing should fail, but
the server should not crash, nor end up in infinite resign-attempt loop.
kasp-max-records-per-type-dnskey (named1.conf.in):
Test with low max-record-per-rrset limit and a DNSSEC policy requiring
more than the limit. Signing should fail.
kasp-max-types-per-name (named1.conf.in):
Each RRSIG(covered type) is counted as an individual RR type. Test the
corner case where a signed zone, which is just below the limit-1,
adds a new type - doing so would trigger signing for the new type and
thus increase the number of "types" by 2, pushing it over the limit
again.
Add two new masterformat tests that use signing. In the case of
'under-limit-kasp', the signing will keep the number of records in the
RRset under the limit. In the case of 'on-limit-kasp', the signing
will push the number of records in the RRset over the limit, because
of the added RRSIG record.
Send a recursive query for a large number of RRsets, which should
fail when using the default max-types-per-name setting of 100, but
succeed when the cap is disabled.
Previously, the number of RR types for a single owner name was limited
only by the maximum number of the types (64k). As the data structure
that holds the RR types for the database node is just a linked list, and
there are places where we just walk through the whole list (again and
again), adding a large number of RR types for a single owner named with
would slow down processing of such name (database node).
Add a configurable limit to cap the number of the RR types for a single
owner. This is enforced at the database (rbtdb, qpzone, qpcache) level
and configured with new max-types-per-name configuration option that
can be configured globally, per-view and per-zone.
Send a recursive query for a large (2500 record) RRset, which should
fail when using the default max-records-per-type setting of 100, but
succeed when the cap is disabled.
Add two new masterformat tests - the 'huge' zone fits within the ns1
limit and loads on the primary ns1 server, but must not transfer to the
ns2 secondary, and the 'uber' zone should not even load on the primary
ns1 server.
Previously, the number of RRs in the RRSets were internally unlimited.
As the data structure that holds the RRs is just a linked list, and
there are places where we just walk through all of the RRs, adding an
RRSet with huge number of RRs inside would slow down processing of said
RRSets.
Add a configurable limit to cap the number of the RRs in a single RRSet.
This is enforced at the database (rbtdb, qpzone, qpcache) level and
configured with new max-records-per-type configuration option that can
be configured globally, per-view and per-zone.
The changes in this MR prevent the memory used for sending the outgoing
TCP requests to spike so much. That strictly remove the extra need for
own memory context, and thus since we generally prefer simplicity,
remove the extra memory context with own jemalloc arenas just for the
outgoing send buffers.
The single TCP read can create as much as 64k divided by the minimum
size of the DNS message. This can clog the processing thread and trash
the memory allocator because we need to do as much as ~20k allocations in
a single UV loop tick.
Limit the number of the DNS messages processed in a single UV loop tick
to just single DNS message and limit the number of the outstanding DNS
messages back to 23. This effectively limits the number of pipelined
DNS messages to that number (this is the limit we already had before).
As a single thread can process only one TCP send at the time, we don't
really need a memory pool for the TCP buffers, but it's enough to have
a single per-loop (client manager) static buffer that's being used to
assemble the DNS message and then it gets copied into own sending
buffer.
In the future, this should get optimized by exposing the uv_try API
from the network manager, and first try to send the message directly
and allocate the sending buffer only if we need to send the data
asynchronously.
Constantly allocating, reallocating and deallocating 64K TCP send
buffers by 'ns_client' instances takes too much CPU time.
There is an existing mechanism to reuse the ns_clent_t structure
associated with the handle using 'isc_nmhandle_getdata/_setdata'
(see ns_client_request()), but it doesn't work with TCP, because
every time ns_client_request() is called it gets a new handle even
for the same TCP connection, see the comments in
streamdns_on_complete_dnsmessage().
To solve the problem, we introduce an array of available (unused)
TCP buffers stored in ns_clientmgr_t structure so that a 'client'
working via TCP can have a chance to reuse one (if there is one)
instead of allocating a new one every time.
When TCP client would not read the DNS message sent to them, the TCP
sends inside named would accumulate and cause degradation of the
service. Throttle the reading from the TCP socket when we accumulate
enough DNS data to be sent. Currently this is limited in a way that a
single largest possible DNS message can fit into the buffer.
DoH: Avoid potential data races in our DoH implementation related to to HTTP/2 session object management and endpoints set object management
See merge request isc-private/bind9!614
This commit ensures that an HTTP endpoints set reference is stored in
a socket object associated with an HTTP/2 stream instead of
referencing the global set stored inside a listener.
This helps to prevent an issue like follows:
1. BIND is configured to serve DoH clients;
2. A client is connected and one or more HTTP/2 stream is
created. Internal pointers are now pointing to the data on the
associated HTTP endpoints set;
3. BIND is reconfigured - the new endpoints set object is created and
promoted to all listeners;
4. The old pointers to the HTTP endpoints set data are now invalid.
Instead referencing a global object that is updated on
re-configurations we now store a local reference which prevents the
endpoints set objects to go out of scope prematurely.
It was reported that HTTP/2 session might get closed or even deleted
before all async. processing has been completed.
This commit addresses that: now we are avoiding using the object when
we do not need it or specifically check if the pointers used are not
'NULL' and by ensuring that there is at least one reference to the
session object while we are doing incoming data processing.
This commit makes the code more resilient to such issues in the
future.
Changed the default value for 'allow-transfer' to 'none'; zone
transfers now require explicit authorization.
Updated all system tests to specify an allow-transfer ACL when needed.
Revised the ARM to specify that the default is 'none'.
Replace the ISC_LIST based deadnodes implementation with isc_queue which
is wait-free and we don't have to acquire neither the tree nor node lock
to append nodes to the queue and the cleaning process can also
copy (splice) the list into a local copy without acquiring the list.
Currently, there's little benefit to this as we need to hold those
locks anyway, but in the future as we move to RCU based implementation,
this will be ready.
To align the cleaning with our event loop based model, remove the
hardcoded count for the node locks and use the number of the event loops
instead. This way, each event loop can have its own cleaning as part of
the process. Use uniform random numbers to spread the nodes evenly
between the buckets (instead of hashing the domain name).
Add an isc_queue implementation that hides the gory details of cds_wfcq
into more neat API. The same caveats as with cds_wfcq.
TODO: Add documentation to the API.
Setting "Affects v9.x" labels on a merge request duplicates information
already present on the GitLab issue associated with that merge request.
For trivial merge requests that are not associated with any GitLab
issue, setting the "Affects v9.x" label(s) is considered unnecessary.
Trigger a failure for every merge request marked with at least one
"Affects v9.x" label.
Merge request titles auto-generated by GitLab are often a source of
confusion regarding the actual contents of a given merge request. Warn
for merge requests containing titles that look like auto-generated ones.
Using "-v9_x" and "-v9.x" version suffixes for branch names is now
deprecated since some automation logic does not handle these. Fail for
any merge request using such old-style version suffixes.
Backports are not expected to have any "Affects v9.x" labels set since
those are only meant to be set for merge requests that should have
backports created for them.
When in -4 mode check that "IPv6 disabled and no IPv4 primaries"
is logged and when in -6 mode check that "IPv4 disabled and no IPv6
primaries" is logged.
When the cache's memory context was in over memory state when the
cache was flushed it resulted in LRU cleaning removing newly entered
data in the new cache straight away until the old cache had been
destroyed enough to take it out of over memory state. When flushing
the cache create a new memory context for the new db to prevent this.
The `axfr_makedb()` didn't set the loop on the newly created database,
effectively killing delayed cleaning on such database. Move the
database creation into dns_zone API that knows all the gory details of
creating new database suitable for the zone.
The times it takes to run tests CI vary significantly enough
that it makes hypothesis test reach their deadlines and fail randomly
marking the tests as flaky.
This commit disables the deadlines when running in CI.
The most important being `dns_names` that generates dns.name.Name
objects based on given paramaters.
No guarantees are given when it comes the uniformity of generated
samples, however it plays nicely with the hypothesis' shrinking
algorithm.
Once we use hypothesis more widely (in at least one more test) this file
should be moved for it to be reused easily.
Check for more rcodes and various properties needed in the wildcard
test. Add a `name` module for various dns.name.Name operations (with
`prepend_label` function only now).
Expose `timeout` as a parameter of `query.tcp`/`query.udp`.
The system test from the BIND 9.19.24 release does not include the
isctest/vars/autoconf.py file from 9.19.25-dev, and therefore the job
will fail before the 9.19.25 release is published. In the meantime,
consider using the conf.sh file.
The rdataslab.c was full of code like this:
length = raw[0] * 256 + raw[1];
and
count2 = *current2++ * 256;
count2 += *current2++;
Refactor code like this into peek_uint16() and get_uint16 macros
to prevent code repetition and possible mistakes when copy and
pasting the same code over and over.
As a side note for an entertainment of a careful reader of the commit
messages: The byte manipulation was changed from multiplication and
addition to shift with or.
The difference in the assembly looks like this:
MUL and ADD:
movzx eax, BYTE PTR [rdi]
movzx edi, BYTE PTR [rdi+1]
sal eax, 8
or edi, eax
SHIFT and OR:
movzx edi, WORD PTR [rdi]
rol di, 8
movzx edi, di
If the result and/or buffer is then being used after the macro call,
there's more differences in favor of the SHIFT+OR solution.
The detach function declaration in `ISC__REFCOUNT_TRACE_DECL` had an
returned an accidental implicit int. While not allowed since C99, it
became an error by default in GCC 14.
`ISC_REFCOUNT_TRACE_IMPL` and `ISC_REFCOUNT_STATIC_TRACE_IMPL` expanded
into the wrong macros, trying to declare it again with the wrong number
of parameters.
- duplicated question
- duplicated answer
- qtype as an answer
- two question types
- question names
- nsec3 bad owner name
- short record
- short question
- mismatching question class
- bad record owner name
- mismatched class in record
- mismatched KEY class
- OPT wrong owner name
- invalid RRSIG "covers" type
- UPDATE malformed delete type
- TSIG wrong class
- TSIG not the last record
there were TSAN error reports because of conflicting uses of
node->dirty and node->nsec, which were in the same qword.
this could be resolved by separating them, but we could also
make them into atomic values and remove some node locking.
The fix_iterator() function had a lot of bugs in it and while fixing
them, the number of corner cases and the complexity of the function
got out of hand. Rewrite the function with the following modifications:
The function now requires that the iterator is pointing to a leaf node.
This removes the cases we have to deal when the iterator was left on a
dead branch.
From the leaf node, pop up the iterator stack until we encounter the
branch where the offset point is before the point where the search key
differs. This will bring us to the right branch, or at the first
unmatched node, in which case we pop up to the parent branch. From
there it is easier to retrieve the predecessor.
Once we are at the right branch, all we have to do is find the right
twig (which is either the twig for the character at the position where
the search key differs, or the previous twig) and walk down from there
to the greatest leaf or, in case there is no good twig, get the
previous twig from the successor and get the greatest leaf from there.
If there is no previous twig to select in this branch, because every
leaf from this branch node is greater than the one we wanted, we need
to pop up the stack again and resume at the parent branch. This is
achieved by calling prevleaf().
Move the fix_iterator out of the loop and only call it when we found
a leaf node. This leaf node may be the wrong leaf node, but fix_iterator
should correct that.
Also, when we don't need to set the iterator, just get any leaf. We
only need to have a leaf for the qpkey_compare and the end result does
not matter if compare was against an ancestor leaf or any leaf below
that point.
An obscured DNSKEY RRset at a delegation was incorrectly added to
the NSEC/NSEC3 type bit map leading to zone verification failures.
This adds such a RRset to the test zone.
DNSSEC only works when DNSKEYs are self signed. This only occurs
when the DNSKEY RRset is at the apex. Cause dnssec-signzone to
fail if it attempts to sign an non-apex DNSKEY RRset.
An assertion failure would be triggered when sending the TCP data ends
after the TCP reading gets closed. Implement proper reference counting
for the isc_httpd object.
When searching for a requested name in dns_qp_lookup(), we may add
a leaf node to the QP chain, then subsequently determine that the
branch we were on was a dead end. When that happens, the chain can be
left holding a pointer to a node that is *not* an ancestor of the
requested name.
We correct for this by unwinding any chain links with an offset
value greater or equal to that of the node we found.
The code below the if/else construction could only be run if the 'if'
code path was taken. Move the code into the 'if' code block so that
it is more easier to read.
The list isn't exactly maintained but it helped with some BIND history
tracking and is basically harmless so it might be worth holding onto it.
I have adapted the name to ASCII so IDN support won't be necessary.
The high-water allows administrators to better tune the recursive
clients limit without having to to poll the statistics channel in high
rates to get this number.
Returning the value allows for better high-water tracking without
running into edge cases like the following:
0. The counter is at value X
1. Increment the value (X+1)
2. The value is decreased multiple times in another threads (X+1-Y)
3. Get the value (X+1-Y)
4. Update-if-greater misses the X+1 value which should have been the
high-water
To avoid any escaping issues or messing with a language-specific format
when the variable has to be parsed, create a dedicated file for each
variable that is obtained from autoconf.
Make sure all initialization takes place in isctest.vars.__init__ and
export the initial env vars there. Remove the no longer needed env
fixture and use os.environ instead.
If OPENSSL_CONF is exported as an empty string, it will cause issues on
rhel9fips. Allow the environment variables to be set and exported, but
make sure to only export them if they have been set by the user.
The openssl config needs to be parsed for some tests that use SoftHSM2.
Rewrite the parsing to python and ensure the required variables are
properly set test-wide.
Provide a single point of access to all the variables used by tests. Use
a custom dict-like structure to access the underlying data without
making a copy. This allows the individual modules to update the contents
at runtime, which is used for some variables.
While this isn't required for pytest operation and execution of the
system test suite, it can be handy to allow test script development and
debugging. Especially setup scripts often source conf.sh and expect
environment variables to be loaded. If these scripts are executed
stand-alone, the environment variables need to be loaded from the python
package.
Remove conf.sh.in and move the environment variables into isctest/vars
python package. This enabled the removal of an ugly pytest hack which
loaded and parsed these variables from the environment.
qp and rbtdb produce stylistically different backup files. This
was causing the xferquota system test to fail. This has been
addressed by making the test independent of the stylistic differences.
The mechanism was published as RFC 8509. I've briefly looked at diff
between versions -08 and the RFC and did not find significant protocol
change. Quick manual check confirms what we seem to comply with the
published protocol.
Draft was eventually published as RFC 9276 but we did not update our
docs. Also add couple mentions in relevant places in the ARM and
dnssec-signzone man page, mainly around "do not touch" places.
Resolve "SVCB/HTTPS: Bind9 and Dig fail to parse packet if zero SvcPriority and existing SvcParams"
Closes#4704
See merge request isc-projects/bind9!9000
When parsing a zonefile named-checkzone (and others) could loop
infinitely if a directory was $INCLUDED. Record the error and treat
as EOF when looking for multiple errors.
This was found by Eric Sesterhenn from X41.
In nibble mode if the value to be converted was negative the parser
would loop forever. Process the value as an unsigned int instead
of as an int to prevent sign extension when shifting.
This was found by Eric Sesterhenn from X41.
This test is outdated because it tested the 'sig-validity-interval'
option that has been replaced by dnssec-policy's 'signatures-validity',
'signatures-refresh', and 'signatures-jitter' options.
Nevertheless, it tests if the jitter is spread correctly.
Update the test to make use of 'signatures-jitter', set the value
to 1 day (meaning resign in 499 days since 'signatures-validity' is
set to 500 days).
Note that this previously changed erroneously the refresh value to
449 days (should have been 499 days, but that is not allowed by
checkconf, since it is above 90% of 'signatures-validity').
After we have changed the maximum allowed iterations to 51 for signing,
the NSEC3 chain has changed and requires one more NSEC to be returned
in the answer (plus corresponding RRSIG). So the expected number or
records in the authority section is now 8.
If the key is offline and the keymgr runs, it will treat it as a missing key,
and generate a new key (according to the policy). Fix the test by putting
back the KSK temporarily when we run 'rndc loadkeys'.
1. When generating keys, don't set timing metadata. Otherwise keys
are considered to be in use and won't be selected when dnssec-policy
starts a new key rollover.
2. Add an extra check to make sure the new ZSK (zsk2) is prepublished.
Also add a check to make sure it has become active.
3. When using dnssec-settime, add -s to also write to key state files.
The config was recently modified to ensure ns4 won't leak any queries to
root servers. However, the test wasn't executed and it turns out the way
this was handled actually broke the test case. Add our custom root hint
to both of the views to ensure the test can still pass without leaking
any queries.
An RPZ response's SOA record TTL is set to 1 instead of the SOA TTL,
a boolean value is passed on to query_addsoa, which is supposed to be
a TTL value. I don't see what value is appropriate to be used for
overriding, so we will pass UINT32_MAX.
Now that this function also creates the DNSKEY record for the KSKs,
as well as other associated records such as CDS and CDNSKEY, rename
the function to something slightly better.
Creating the KSR happens on the "ZSK side". The KSK is offline and while
the public key and state file may be present, draft-icann-dnssec-keymgmt-01.txt
suggest that the KSR only contains ZSKs.
This is also what knot dns does, so it would also be in the spirit of
interoperability.
The final line in a KSR ";; KeySigningRequest generated at ..." was
missing the version number, that has now been fixed.
Thanks Libor Peltan for reporting.
The expireheader() call in the expire_ttl_headers() function
is erroneous as it passes the 'nlocktypep' and 'tlocktypep'
arguments in a wrong order, which then causes an assertion
failure.
Fix the order of the arguments so it corresponds to the function's
prototype.
in QP keys, characters that are not common in DNS names are
encoded as two-octet sequences. this caused a glitch in iterator
positioning when some lookups failed.
consider the case where we're searching for "\009" (represented
in a QP key as {0x03, 0x0c}) and a branch exists for "\000"
(represented as {0x03, 0x03}). we match on the 0x03, and continue
to search down. at the point where we find we have no match,
we need to pop back up to the branch before the 0x03 - which may
be multiple levels up the stack - before we position the iterator.
there were some structure names used in qpcache.c and qpzone.c that
were too similar to each other and could be confusing when debugging.
they have been changed as follows:
in qcache.c:
- changed_t was unused, and has been removed
- search_t -> qpc_search_t
- qpdb_rdatasetiter_t -> qpc_rditer_t
- qpdb_dbiterator_t -> qpc_dbiter_t
in qpzone.c:
- qpdb_changed_t -> qpz_changed_t
- qpdb_changedlist_t -> qpz_changedlist_t
- qpdb_version_t -> qpz_version_t
- qpdb_versionlist_t -> qpz_versionlist_t
- qpdb_search_t -> qpz_search_t
- qpdb_load_t -> qpz_search_t
when calling dns_qp_lookup() from qpcache, instead of passing
'foundname' so that a name would be constructed from the QP key,
we now just use the name field in the node data. this makes
dns_qp_lookup() run faster.
the same optimization has also been added to qpzone.
the documentation for dns_qp_lookup() has been updated to
discuss this performance consideration.
when the cache is over memory, we purge from the LRU list until
we've freed the approximate amount of memory to be added. this
approximation could fail because the memory allocated for nodenames
wasn't being counted.
add a dns_name_size() function so we can look up the size of nodenames,
then add that to the purgesize calculation.
in a cache database, unlike zones, NSEC3 records are stored in
the main tree. it is not necessary to maintain a separate 'nsec3'
tree, nor to have code in the dbiterator implementation to traverse
from one tree to another.
(if we ever implement synth-from-dnssec using NSEC3 records, we'll
need to revert this change. in the meantime, simpler code is better.)
the QP database doesn't support relative names as the RBTDB did, so
there's no need for a 'new_origin' flag or to handle `DNS_R_NEWORIGIN`
result codes.
- remove unneeded struct members and misleading comments.
- remove unused parameters for static functions.
- rename 'find_callback' to 'delegating' for consistency with qpzone;
the find callback mechanism is not used in QP databases.
- change dns_qpdata_t to qpcnode_t (QP cache node), and dns_qpdb_t to
qpcache_t, as these types are only accessed locally.
- also change qpdata_t in qpzone.c to qpznode_t (QP zone node), for
consistency.
- make the refcount declarations for qpcnode_t and qpznode_t static,
using the new ISC_REFCOUNT_STATIC macros.
In qpcache (and rbtdb), there are some functions that acquire
neither the tree lock nor the node lock when calling newref().
In theory, this could lead to a race in which a new reference
is added to a node that was about to be deleted.
We now detect this condition by passing the current tree and node
lock status to newref(). If the node was previously unreferenced
and we don't hold at least one read lock, we will assert.
The FreeBSD autoscaler has been configured to utilize the new "instance"
GitLab Runner executor to spawn "stress" test CI jobs on AWS EC2
dynamically. A shared GitLab Runner named "freebsd-instance-autoscaler"
has been set up in GitLab CI/CD to communicate with EC2, provisioning VM
instances on demand based on a FreeBSD 13 AMI image. This image is the
same as the one previously used for FreeBSD "stress" tests before the
implementation of autoscaling (specifically, the
"freebsd13-amd64-bind9stress.aws.lab.isc.org" GitLab Runner in CI/CD).
Since the BIND 9 QA repository has been made public, adjust the relevant
URLs and paths used in .gitlab-ci.yml so that they work with the public
version of that repository.
Because some tests don't have a legtimate handle, provide a temporary
return early that should be fixed and removed before squashing. This
short circuiting is still correct until DoQ/DoH3 support is introduced.
GCC might fail to compile because it expects a return after UNREACHABLE.
It should ideally just work anyway since UNREACHABLE is either a
noreturn or UB (__builtin_unreachable / C23 unreachable).
Either way, it should be optimized almost always so the fallback is
free or basically free anyway when it isn't optimized out.
the previous commit introduced a possible race in getsigningtime()
where the rdataset header could change between being found on the
heap and being bound.
getsigningtime() now looks at the first element of the heap, gathers the
locknum, locks the respective lock, and retrieves the header from the
heap again. If the locknum has changed, it will rinse and repeat.
Theoretically, this could spin forever, but practically, it almost never
will as the heap changes on the zone are very rare.
we simplify matters further by changing the dns_db_getsigningtime()
API call. instead of passing back a bound rdataset, we pass back the
information the caller actually needed: the resigning time, owner name
and type of the rdataset that was first on the heap.
in RBTDB, the heap was used by zone databases for resigning, and
by the cache for TTL-based cache cleaning. the cache use case required
very frequent updates, so there was a separate heap for each of the
node lock buckets.
qpzone is for zones only, so it doesn't need to support the cache
use case; the heap will only be touched when the zone is updated or
incrementally signed. we can simplify the code by using only a single
heap.
fix_iterator() and related functions are quite difficult to read.
perhaps it would be a little clearer if we didn't assign values
to variables that won't subsequently be used, or unnecessarily
pop the stack and then push the same value back onto it.
also, in dns_qp_lookup() we previously called fix_iterator(),
removed the leaf from the top of the iterator stack, and then
added it back on. this would be clearer if we just push the leaf
onto the stack when we need to, but leave the stack alone when
it's already complete.
under some circumstances it was possible for the iterator to
be set to the first leaf in a set of twigs, when it should have
been set to the last.
a unit test has been added to test this scenario. if there is a
a tree containing the following values: {".", "abb.", "abc."}, and
we query for "acb.", previously the iterator would be positioned at
"abb." instead of "abc.".
the tree structure is:
branch (offset 1, ".")
branch (offset 3, ".ab")
leaf (".abb")
leaf (".abc")
we find the branch with offset 3 (indicating that its twigs differ
from each other in the third position of the label, "abB" vs "abC").
but the search key differs from the found keys at position 2
("aC" vs "aB"). we look up the bit value in position 3 of the
search key ("B"), and incorrectly follow it onto the wrong twig
("abB").
to correct for this, we need to check for the case where the search
key is greater than the found key in a position earlier than the
branch offset. if it is, then we need to pop from the current leaf
to its parent, and get the greatest leaf from there.
a further change is needed to ensure that we don't do this twice;
when we've moved to a new leaf and the point of difference between
it and the search key even earlier than before, then we're definitely
at a predecessor node and there's no need to continue the loop.
The previous value of 30 minutes used to cache the ADB names and entries
was quite long. Change the value to 60 seconds for faster recovery
after cached intermittent failure of the remote nameservers.
The algorithm from the previous commit[1] is now used to calculate all
the expiration values through the code (ncache results, cname/dname
targets).
1. ISC_MIN(cur, ISC_MAX(now + ADB_ENTRY_WINDOW, now + rdataset->ttl))
Correct the logic to set the expiration period of expire_{v4,v6} as
follows:
1. If the trust is ultimate (local entry), immediately set the entry as
expired, so the changes to the local zones have immediate effect.
3. If the expiration is already set and smaller than the new value, then
leave the expiration value as it is.
2. Otherwise pick larger of `now + ADB_ENTRY_WINDOW` and `now + TTL` as
the new expiration value.
When ADB entry was created it was set to never expire. If we never
called any of the functions that adjust the expiration, it could linger
in the ADB forever.
Set the expiration (.expires) to now + ADB_ENTRY_WINDOW when creating
the new ADB entry to ensure the ADB entry will always expire.
In the past, our CI infrastructure was more sensitive to the number of
CI jobs running on it. We tried to limit long-running jobs in merge
request-triggered pipelines, as there are many of them, and spawned them
only in daily scheduled ones. Moving most of the CI infrastructure to
AWS has made it way better to run jobs in parallel, and the existence of
short respdiff jobs has lost its original merit. It can also be harmful
as some problems are detected only by the longer respdiff variant when a
faulty merge request has already been merged. We should run all long
respdiff tests in merge request-triggered pipelines.
Also, move the former respdiff-long job (now just "respdiff") to AWS as
old instance memory constraints (see
f09cf69594) are no longer an issue.
We now have ctx.kskflag, ctx.zskflag, and ctx.revflag, but zskflag is
not quite like the other two, as it doesn't have a special bit in the
DNS packet, and is used as a boolean.
This patch changes so that we use booleans for all three, and
construct the flags based on which ones are set.
patch by @aram
Add test cases for the 'request' command. Reuse the earlier
pregenerated ZSKs. We also need to set up some KSK files, that can
be done with 'dnssec-keygen -k <policy> -fK' now.
The 'check_keys()' function is adjusted such that the expected active
time of the successor key is set to the inactive time of the
predecessor. Some additional information is saved to make 'request'
testing easier.
Add code that can create a Key Signing Request (KSR) given a DNSSEC
policy, a set of keys and an interval.
Multiple keys that match the bundle and kasp parameters are sorted by
keytag, mainly for testing purposes.
Create some helper functions for code that is going to be reused by the
other commands (request, sign), such as setting and checking the context
parameters, and retrieving the dnssec-policy/kasp.
The 'dnssec-keygen' tool now allows the options '-k <dnssec-policy>'
and '-f <flags>' together to create keys from a DNSSEC policy that only
match the given role. Allow setting '-fZ' to only create ZSKs, while
'-fK' will only create KSKs.
Add a system test for testing dnssec-ksr, initally for the keygen
command. This should be able to create or select key files given a
DNSSEC policy and a time window.
Introduce a new DNSSEC tool, dnssec-ksr, for creating signed key
response (SKR) files, given one or more key signing requests (KSRs).
For now it is just a dummy tool, but the future purpose of this utility
is to pregenerate ZSKs and signed RRsets for DNSKEY, CDNSKEY, and CDS
for a given period that a KSK is to be offline.
Previously, only a single controlconf message would be processed from a
single TCP read even if the TCP read buffer contained multiple messages.
Refactor the isccc_ccmsg unit to store the extra buffer in the internal
buffer and use the already read data first before reading from the
network again.
Co-authored-by: Ondřej Surý <ondrej@isc.org>
Co-authored-by: Dominik Thalhammer <dominik@thalhammer.it>
The xmlGetGlobalState allocates per-thread memory that is not properly
cleaned up when the libxml2 is used from offloaded threads. Add the
function the the LeakSanitizer suppression list.
Prepare the statistics channel data in the offloaded worker thread, so
the networking thread is not blocked by the process gathering data from
various data structures. Only the netmgr send is then run on the
networkin thread when all the data is already there.
Protect the access to the trust byte in the ncache data with relaxed
atomic operation to mimick the current behaviour. This will teach
TSAN that the concurrent access is fine.
This prevents TSAN errors with the ncache code where the trust byte
access needs to be protected by a lock. The old code copied the
entire region before determining where the name ended. We now
determine where the name ends then copy just that data and in doing
so avoid reading the trust byte.
This commit makes the dispatch_test use the same timeouts that network
manager tests. We do that because the old values appear to be too
small for our heavy loaded CI machines, leading to spurious failures
on them. The network manager tests are much more stable in this
situation and they use somewhat larger timeout values.
We use a smaller connection timeouts for the tests which are expected
to timeout to not wait for too long.
The source address field of 'newnotify' was not updated from the
default (0.0.0.0) when the destination address was an IPv6 address.
This resulted in the messages failing to be sent. Set the source
address to :: when the destination address is an IPv6 address.
It's easier to maintain the template in a single place together with
the script used to in the template.
In future use script bind9/releng/create_checklist.py
from isc-private/bind-qa to generate release issue.
Since the dns_validator_destroy() function doesn't guarantee that
it destroys the validator, rename it to dns_validator_shutdown()
and require explicit dns_validator_detach() to follow.
Enforce the documented function requirement that the validator must
be completed when the function is called.
Make sure to set val->name to NULL when the function is called,
so that the owner of the validator may destroy the name, even if
the validator is not destroyed immediately. This should be safe,
because the name can be used further only for logging by the
offloaded work callbacks when they detect that the validator is
already canceled/complete, and the logging function has a condition
to use the name only when it is non-NULL.
If val->result is not ISC_R_SUCCESS, a similar message is logged
further down in the function. Remove the redundant log message.
Also remove an unnecessary code comment line.
isc_loop() can now take its place.
This also requires changes to the test harness - instead of running the
setup and teardown outside of th main loop, we now schedule the setup
and teardown to run on the loop (via isc_loop_setup() and
isc_loop_teardown()) - this is needed because the new the isc_loop()
call has to be run on the active event loop, but previously the
isc_loop_current() (and the variants like isc_loop_main()) would work
even outside of the loop because it needed just isc_tid() to work, but
not the full loop (which was mainly true for the main thread).
if we had a method to get the running loop, similar to how
isc_tid() gets the current thread ID, we can simplify loop
and loopmgr initialization.
remove most uses of isc_loop_current() in favor of isc_loop().
in some places where that was the only reason to pass loopmgr,
remove loopmgr from the function parameters.
an assertion could be triggered in the QPDB cache if a DNAME
was found above a queried NS, because the 'foundname' value was
not correctly updated to point to the zone cut.
the same mistake existed in qpzone and has been fixed there as well.
Commit a4e9ce500a added "pipelines" to CI
job trigger lists without updating the names of the YAML anchors
containing those lists accordingly. Update YAML anchor names so that
they are consistent with their own contents.
Since pre-release testing is usually carried out for branches in which
CHANGES entries are intentionally malformed to prevent entry numbering
conflicts down the road, do not run the "changes" GitLab CI job in
pipelines that are triggered by a parent pipeline (which can currently
only be a pre-release testing pipeline) to prevent triggering job
failures that would be meaningless anyway.
Checking the contents of the CHANGES file currently requires invoking
multiple shell scripts. These invocations are conflated with those for
other test scripts in the "misc" GitLab CI job. Extract the commands
checking the contents of the CHANGES file to a separate GitLab CI job,
"changes", to improve readability. Remove similar checks for the
CHANGES.SE file altogether as they are only relevant for BIND -S and
therefore should not be present in an open source branch.
Check that RFC 1918 leak detection does not trigger an assertion
when nxdomain redirection is enabled in the server but not for the
RFC 1918 reverse namespace.
If there are no more previous leaves, it means the queried name
precedes the entire range of names in the database, so we should just
move the iterator one step back and return, instead of continuing our
search for the predecessor.
This is similar to an earlier bug fixed in an earlier commit:
ea9a8cb392
When fixing the iterator, when every leaf on this branch is greater
than the one we wanted we go back to the parent branch and iterate back
to the predecessor from that point.
But if there are no more previous leafs, it means the queried name
precedes the entire range of names in the database, so we would just
move the iterator one step back and continue from there.
This could end in a loop because the queried name precedes the entire
range of names and so none of those names are the predecessor of the
queried name.
The condition in prereq.sh which attempts to match two string uses
integer equality operation. This results in an error, causing the
enginepkcs11 test to always be skipped. Use = operator for the string
comparison instead.
The autosign test uses sleep in many cases to wait for something to
happen. This inevitably leads to an instability that manifests in our
CI. Allow an automatic rerun of the test to improve its stability.
Variable assignment when calling subroutines might not be portable.
Notably, it doesn't work with FreeBSD shell, where the value of HOME
would be ignored in this case.
Since the commands are already executed in a subshell, export the HOME
variable to ensure it is properly handled in all shells.
To enable GitLab CI jobs in other projects to trigger pipelines in the
BIND 9 project using their CI_JOB_TOKEN, add "pipelines" to the relevant
GitLab CI job trigger lists.
The ThreadSanitizer version currently available from Fedora 39
repositories is unable to cope with very high ASLR entropy, which is the
default in some recent Linux distributions [1]. This causes all
TSAN-enabled builds to fail on the affected systems with an error like:
FATAL: ThreadSanitizer: unexpected memory mapping 0x7d00e0772000-0x7d00e0c00000
Work around the problem by reducing ASLR entropy for all TSAN-enabled
builds until the problem is resolved upstream.
[1] https://github.com/google/sanitizers/issues/1716
Initializing the conftest logging upon importing the isctest package
isn't practical when there are standalone pieces which can be used
outside of the testing framework, such as the asyncdnsserver module.
Implement a new Python class, AsyncDnsServer, which can be used by
ans.py scripts placed in ansX/ system test subdirectories. This enables
conveniently starting a feature-limited, non-standards-compliant, custom
DNS server instance. It can read and serve zone files, but it is also
able to evaluate any user-provided query-processing logic, allowing
query responses to be changed, delayed, or dropped altogether. These
are all actions commonly taken by custom DNS servers written in Python
that are used in BIND 9 system tests. Having a single "base"
implementation of such a custom DNS server reduces code duplication,
improving test maintainability.
Co-authored-by: Tom Krizek <tkrizek@isc.org>
Resolve "CID 488064: Passing null pointer "version" to "maybe_update_recordsandsize", which dereferences it"
Closes#4645 and #4646
See merge request isc-projects/bind9!8880
every node of a QP database contains a copy of the nodename,
which is used as the key for the QP-trie. previously, the name
was stored as a dns_fixedname object, which has room for up to
255 characters. we can reduce the space consumed by dynamically
allocating a dns_name object that's just long enough for the name
to be stored.
The dns_qpiter_next() was called without checking the return value. If
we cannot move the iterator forward, there is no use in calling the
step() function.
/lib/dns/qpzone.c: 2804 in activeempty()
2798 * of the name we were searching for. Step the iterator
2799 * forward, then step() will continue forward until it
2800 * finds a node with active data. If that node is a
2801 * subdomain of the one we were looking for, then we're
2802 * at an active empty nonterminal node.
2803 */
>>> CID 487882: Error handling issues (CHECKED_RETURN)
>>> Calling "dns_qpiter_next" without checking return value (as is done elsewhere 26 out of 27 times).
2804 dns_qpiter_next(it, NULL, NULL, NULL);
2805 return (step(search, it, FORWARD, next) &&
2806 dns_name_issubdomain(next, current));
2807 }
Be stricter in durations that are accepted. Basically we accept ISO 8601
formats, but fail to detect garbage after the integers in such strings.
For example, 'P7.5D' will be treated as 7 days. Pass 'endptr' to
'strtoll' and check if the endptr is at the correct suffix.
dns_db_addrdataset() enforces a requirement that version can only
be NULL for a cache database. code that checks for zone semantics
and version == NULL can never be reached.
qpzone does not support cache semantics, so dns_db_addrdataset(),
_deleterdataset() and _subtractrdataset() can't be run with
version == NULL; there's no need to check for it.
we can also clean up free_qpdb() a bit since current_version
is always non-NULL.
The Depends relation refers to types of rollovers in which a certain
record type is going to be swapped. Specifically, the Depends relation
says there should be no dependency on the predecessor key (the set
Dep(x, T) must be empty).
But if the key is phased out (all its states are in HIDDEN), there is
no longer a dependency. Since the relationship is still maintained
(Predecessor and Successor metadata), the keymgr_dep function still
returned true. In other words, the set Dep(x, T) is not considered
empty.
This slows down key rollovers, only retiring keys when the successor
key has been fully propagated.
When there is a secure chain of trust with a KSK that is not actively
signing the DNSKEY RRset, the code for validating the DNSKEY RRset
against the DS RRset could potentially skip DS records, thinking the
chain of trust is broken while there is a valid DS with corresponding
DNSKEY record present.
This is because we pass the result ISC_R_NOMORE on when we are done
checking for signatures, but then treat it as "no more DS records".
Chaning the return value to something else (DNS_R_NOVALIDSIG seems the
most appropriate here) fixes the issue.
Add a regression test case for the scenario where a secure chain of
trust includes an inactive KSK, that is a KSK that is not signing the
DNSKEY RRset.
the code in qpdb.c was previously shared by qp-cachedb.c and
qp-zonedb.c. since qp-zonedb.c no longer exists, it's not necessary
to keep these separate any longer. the two files have been merged,
and functions that were previously globally accessible have been
changed to static and renamed.
now that "qpzone" databases are available for use in zones, we no
longer need to retain the zone semantics in the "qp" database.
all zone-specific code has been removed from QPDB, and "configure
--with-zonedb" once again takes two values, rbt and qp.
some database API methods that are never used with a cache have
been removed from qpdb.c and qp-cachedb.c; these include newversion,
closeversion, subtractrdataset, and nodefullname.
because dns_qpmulti_commit() can be time consuming, it's inefficient
to open and commit a qpmulti transaction for each rdataset being loaded
into a database. we can improve load time by opening a qpmulti
transaction before adding a group of rdatasets and then committing it
afterward.
this commit adds 'setup' and 'commit' functions to dns_rdatacallbacks_t,
which can be called before and after the loops in which 'add' is
called in dns_master_load() and axfr_apply().
when copying the non-dnssec records in receive_secure_db(),
use DNS_DB_NONSEC3 so we don't accidentally create nodes in
the main tree for NSEC3 records. this was a long-standing error
in the code, but was harmless in the RBTDB.
QP database node data is not reference counted the same way RBT nodes
were: in the RBT, node->references could be zero if the node was in the
tree but was not in use by any caller, whereas in the QP trie, the
database itself uses reference counting of nodes internally.
this caused some subtle errors. in RBTDB, when the newref() function is
called and the node reference count was zero, the node lock reference
counter would also be incremented. in the QP trie, this can never
happen - because as long as the node is in the database its reference
count cannot be zero - and so the node lock reference counter was never
incremented.
this has been addressed by maintaining a separate "erefs" counter for
external references to the node. this is the same approach used in the
"qpdb-lite" database in commit e91fbd8dea.
while troubleshooting this issue, some compile errors were discovered
when building with DNS_DB_NODETRACE; those have also been fixed.
use the dns_qpmulti-based "qpzone" by default throughout BIND,
instead of the existing dns_qp-based "qp", when creating zone
databases. (cache databases still use "qp".)
the "--with-zonedb" option has been updated in configure.ac to permit
the use of both "qp" and "qpzone" databases.
in zone.c there was a test that prevented any database type other than
"qp" from hosting an RPZ. this was outdated, and has been removed.
previously, an RCU critical section was held open for the duration
of a snapshot. this should not be necessary, as the snapshot makes
local copies of QP trie metadata, and it causes problems when a
DB iterator is held open between two loop events. we now call
rcu_read_unlock() after setting up the snapshot.
finish importing the database API methods from RBTDB to qpzone:
issecure, nodecount, getnsec3parameters, findnsec3node, setsigningtime,
getsigningtime, getsize, setgluecachestats, locknode, unlocknode, and
addglue.
add database API methods needed to apply updates to an existing zone
database (newversion, addrdataset, subtractrdataset and deleterdataset).
it is now possible to apply journals to zone databases after loading, so
named-checkzone -J works correctly.
add database API method implementations needed to iterate and dump
a qpzone database to a file (createiterator, allrdatasets and
attachversion, plus dbiterator and rdatasetiter methods).
named-checkzone -D can now dump the contents of most zones,
but zone cuts are not correctly detected.
add database API methods needed for loading rdatasets into memory
(currentversion, beginload, endload), plus the methods used by
zone_postload() for zone consistency checks (getoriginnode, find,
findnode, findrdataset, attachnode, detachnode, deletedata).
the QP trie doesn't support the find callback mechanism available
in dns_rbt_findnode() which allows examination of intermediate nodes
while searching, so the detection of wildcard and delegation nodes
is now done by scanning QP chains after calling dns_qp_lookup().
Note that the lookup in previous_closest_nsec() cannot return
ISC_R_NOTFOUND. In RBTDB, we checked for this return value and
ovewrote the result with ISC_R_NOMORE if it occurred. In the
qpzone implementation, we insist that this return value cannot happen.
dns_qp_lookup() would only return ISC_R_NOTFOUND if we asked for a
name outside the zone's authoritative domain, and we never do that
when looking up a predecessor NSEC record.
named-checkzone is now able to load a zone and check it for errors,
but cannot dump it.
The dns_cache_flush() drops the old database and creates a new one, but
it forgets to pass the loop that runs the node pruning and cleaning
the rbtdb when flushing it next time. This causes the cleaning to skip
cleaning the parent nodes (with .down == NULL) leading to increased
memory usage over time until the database is unable to keep up and just
stays overmem all the time.
Reconstruct the variant of the prune_tree() parent cleaning to consider
all elibible parents in a single loop as we were doing before all the
changes that led to this commit.
Update code comments so that they more precisely describe what the
relevant bits of code actually do.
by default, QPDB is the database used by named and all tools and
unit tests. the old default of RBTDB can now be restored by using
"configure --with-zonedb=rbt --with-cachedb=rbt".
some tests have been fixed so they will work correctly with either
database.
CHANGES and release notes have been updated to reflect this change.
The change from RBT to QP has changed the contents of generated zone
files slightly: node names are now always absolute, so instead of using
$ORIGIN and relative names, generated zone files use full names for all
records.
This caused a failure in the xferquota system test, which was looking
for a relative name in secondary zone files. Replace the string
matching with a regular expression to fix the test.
When running resolver benchmark pipeline, a crash occurred:
https://gitlab.isc.org/isc-projects/bind9-shotgun-ci/-/pipelines/163946
In the code we are doing a lookup, it fails (meaning there is no node
with lookup name), we create the node and insert it and it fails.
But dns_qp_insert can only return ISC_R_SUCCESS or ISC_R_EXISTS.
So it must have been inserted in between. This is a race condition bug.
The first lookup only requires a write lock and if the lookup failed
the lock gets upgraded to a write lock and we insert the missing data.
To fix the race condition bug, we need to do a lookup again after we
have upgraded the lock to make sure it wasn't inserted in the mean
time.
the dyndb test requires a mechanism to retrieve the name associated
with a database node, and since the database no longer uses RBT for
its underlying storage, dns_rbt_fullnamefromnode() doesn't work.
addressed this by adding dns_db_nodefullname() to the database API.
If the iterator is paused, the tree is unlocked and may change.
In an RBT tree it's always possible to resume iteration as long
as a valid node pointer was still held, but now that the underlying
database structure is a QP trie, the iterator needs to be initialized
based on the existing structure of the trie or it will return
inconsistent results. We now call dns_qp_lookup() to reinitialize
the QP iterator whenever dbiterator_next() or dbiterator_prev() is
called on a paused iterator.
the change from RBT to QP has changed the contents of generated zone
files slightly: node names are now always absolute, so instead of using
$ORIGIN and relative names, generated zone files use full names for all
records.
this caused a failure in the stub system test, which was grepping for a
relative name in a dumped zone file. using "masterfile-style full" makes
the test pass regardless of the database being used.
QP database node data is not reference counted the same way RBT nodes
were: in the RBT, node->references could be zero if the node was in the
tree but was not in use by any caller, whereas in the QP trie, the
database itself uses reference counting of nodes internally.
this caused some subtle errors. in RBTDB, when the newref() function is
called and the node reference count was zero, the node lock reference
counter would also be incremented. in the QP trie, this can never
happen - because as long as the node is in the database its reference
count cannot be zero - and so the node lock reference counter was never
incremented.
reference counting will probably need to be refactored in more detail
later; the node lock reference count may not be needed at all. but
for now, as a temporary measure, we add a third reference counter,
'erefs' (external references), to the dns_qpdata structure. this is
counted separately from the main reference counter, and should match
the node reference count as it would have been in RBTDB.
this change revealed a number of places where the node reference counter
was being incremented on behalf of a caller without newref() being
called; those were cleaned up as well.
This is an adaptation of commit 3dd686261d2c4bcd15a96ebfea10baffa277732b
as a side effect of the switch from RBT to QBDB, NSEC3 records
are no longer created for empty non-terminal nodes when the
node only contains insecure delegations in an opt-out range.
such NSEC3 records are optional according to RFC 5155 (and,
for example, they are not created by dnssec-signzone), but they were
previously created by named, as a harmless side effect of the RBT
structure, which contains empty internal nodes that can be reached
by a DB iterator. these nodes are not present in the QPDB, so
NSEC3 records are not created unless they're actually required.
the autosign system test contained a test case (added in commit
ad91a70d as part of GL #4027) that checked whether ENT NSEC3
records were deleted when the delegations under the ENT removed.
this test no longer passes, because the NSEC3's are not created
in the first place, and therefore cannot be removed.
rather than "fix" the QPDB to add unnecessary NSEC3 records, this
commit instead revises the test to check for removal of ENT NSEC3
records when *not* using opt-out.
Fix reference counting: unreference nodes that are succesfully inserted
in the tree, detach created nodes, and cleanup the interior data in
dns_qpdata_destroy().
The name will be stored inside the node now so we can just copy it.
These are leftovers, most of the namefromnode code has been replaced
already in previous commits.
The qp approach pulled apart the chain and iterator into two separate
things. Replace the rbtnodechain with qpchain and qpiter. Most of the
times we are interested in the iterator only, the rbtnodechain was
mainly used as an an iterator to get the previous and next name in the
DNS canonical order.
Since dns_qpiter_prev() and dns_qpiter_next() store the name, origin,
and node in the provided parameters, often there is no need to call
a current() function anymore.
Getting the first or last item from the iterator is done by
re-initializing the iterator and then call dns_qpiter_next() or
dns_qpiter_prev() respectively.
The dbiterator no longer needs to maintain a chain, only an iterator.
All dns_qp_lookup() calls assume it is okay to find empty data, so
we don't need to do anything special for the DNS_RBTFIND_EMPTYDATA.
You can pass a callback function to dns_rbt_findnode(), something that
qp does not support. Instead, call the function afterwards. This has
the drawback that we do more lookup work if there was a zonecut.
With dns_qp_lookup() we also don't pass any options. In this case,
when DNS_RBTFIND_NOEXACT was set, we adapt the result after the lookup.
Replace dns_rbt_deletenode calls with dns_qp_deletename. For removing
the name from the nsec tree, we no longer first have to find it: we can
just remove the key (retrieved by name).
Replace dns_rbt_addnode calls with dns_qp_insert. With QP, it sometimes
makes more sense to first lookup the name and see if there is an
existing node (rather than create new data, insert, find out a node
already exists, and destroy the data again). This is done with
dns_qp_getname(), which is more lightweight than dns_qp_lookup(),
and we are only interested in if there is already a leaf node for this
name or not.
replace the string "rbt" throughout BIND with "qp" so that
qpdb databases will be used by default instead of rbtdb.
rbtdb databases can still be used by specifying "database rbt;"
in a zone statement.
- Copy rbtdb.c, rbt-zonedb.c and rbt-cachedb.c to qp-*.
- Added qpmethods.
- Added a new structure dns_qpdata that will replace dns_rbtnode.
- Replaced normal, nsec, and nsec3 dns_rbt trees with dns_qp tries.
- Replaced dns_rbt_create() calls with dns_qp_create().
- Replaced the dns_rbt_destroy() call with dns_qp_destroy().
- Create a dns_qpdata struct and create/destroy methods.
This commit will not build.
This is a regresssion test for GL #4621 where the NODATA responses
are SOA records that match the QNAME rather than the zone name. In
particular for NS queries.
[GL #3709] reordered the dns_rdataset_disassociate call to after
the dns_resolver_createfetch call resulting in qctx->nsrrset still
being associated when dns_resolver_createfetch is called in
resume_dslookup (7e4e125e). Revert that part of the change and add
comments as to why the multiple dns_rdataset_disassociate calls are
where they are.
The TCP dispatch connected callbacks could be called synchronously which
in turn could destroy xfrin before we return from dns_xfrin_create().
Delay the calling the callback called from tcp_dispatch_connect() by
calling it always asynchronously.
Instead of getting the loop from the zone every time, attach the xfrin
directly to the loop. This also allows to remove the extra safety tid
checks from the dns_xfrin unit.
the RRL test included a test case that tried to start named with
a broken configuration. the same error could be found with
named-checkconf, so it should have been tested in the checkconf
system test.
It was discovered that the TTL-based cleaning could build up
a significant backlog of the rdataset headers during the periods where
the top of the TTL heap isn't expired yet. Make the TTL-based cleaning
more aggressive by cleaning more headers from the heap when we are
adding new header into the RBTDB.
It was discovered that an expired header could sit on top of the heap
a little longer than desireable. Remove expired headers (headers with
rdh_ttl set to 0) from the heap completely, so they don't block the next
TTL-based cleaning.
Instead of juggling with node locks in a cycle, cleanup the node we are
just pruning and send any the parent that's also subject to the pruning
to the prune tree via normal way (e.g. enqueue pruning on the parent).
This simplifies the code and also spreads the pruning load across more
event loop ticks which is better for lock contention as less things run
in a tight loop.
The log message for commit 24381cc36d
explained:
In some older BIND 9 branches, the extra queuing overhead eliminated by
this change could be remotely exploited to cause excessive memory use.
Due to architectural shift, this branch is not vulnerable to that issue,
but applying the fix to the latter is nevertheless deemed prudent for
consistency and to make the code future-proof.
However, it turned out that having a single queue for the nodes to be
pruned increased lock contention to a level where cleaning up nodes from
the RBTDB took too long, causing the amount of memory used by the cache
to grow indefinitely over time.
This commit reverts the change to the pruning mechanism introduced by
commit 24381cc36d as BIND branches newer
than 9.16 were not affected by the excessive event queueing overhead
issue mentioned in the log message for the above commit.
This commit improves the documentation on the ephemeral TLS
configuration and describes in more detail what is happening with TLS
configurations on reconfiguration in general.
When the first parametrized test takes a bit longer than usual, the zone
transfer in ns3 may succeed before the second parametrized test is even
started, and then watch_log_from_here() won't find the "Transfer status:
success" message in the named log. Using watch_log_from_start() instead
makes sure the test is more stable.
dns__cacherbt_expireheader can unlink / free header_prev underneath
it. Use ISC_LIST_TAIL after calling dns__cacherbt_expireheader
instead to get the next pointer to be processed.
This commit ensures that worker threads are not sleeping (by using
select()) when '-T transferslowly/transferstuck' test options are
used. This commit converts synchronous implementation of the code into
an asynchronous one based on timers.
The resolver's code was not ready to failures when trying to establish
a connection via TCP-based transports (e.g. when creating TLS contexts
before establishing a TLS connection).
This commit fixes that.
This was mostly an artifact to tell which log lines belong to which test
from the time when the test output could be all mingled together. Now
this info is reduntant, because the pytest logger already includes both
the system test name, and the specific test.
Unify the different loggers (conftest, module, test) into a single
interface. Remove the need to select the proper logger by automatically
selecting the most-specific logger currently available.
This also removes the need to use the logger/mlogger fixtures manually
and pass these around. This was especially annoying and unwieldy when
splitting the test cases into functions, because logger had to always be
passed around. Instead, it is now possible to use the
isctest.log.(debug,info,warning,error) functions.
Preparation for further logging improvements - keep the watchlog
contents in a separate module inside isctest.log. Export the names in
the log package so the imports don't change for the users of these
classes.
This code has probably been accidentally added during some rebase. The
actual RNDCExecutor and related classes are in isctest/rndc.py. Remove
the duplicated and unused code from isctest/log.py, as it doesn't belong
there.
The warnings (see below) seem to be false-positives. Address them
by adding runtime checks.
resolver.c:1627:10: warning: Access to field 'tid' results in a dereference of a null pointer (loaded from variable 'fctx') [core.NullDereference]
1627 | REQUIRE(fctx->tid == isc_tid());
| ^~~~~~~~~
../../lib/isc/include/isc/util.h:332:34: note: expanded from macro 'REQUIRE'
332 | #define REQUIRE(e) ISC_REQUIRE(e)
| ^
../../lib/isc/include/isc/assertions.h:45:11: note: expanded from macro 'ISC_REQUIRE'
45 | ((void)((cond) || \
| ^~~~
resolver.c:10335:6: warning: Access to field 'depth' results in a dereference of a null pointer (loaded from variable 'fctx') [core.NullDereference]
10335 | if (fctx->depth > depth) {
| ^~~~~~~~~~~
2 warnings generated.
- the DNS_DB_NSEC3ONLY and DNS_DB_NONSEC3 flags are mutually
exclusive; it never made sense to set both at the same time.
to enforce this, it is now a fatal error to do so. the
dbiterator implementation has been cleaned up to remove
code that treated the two as independent: if nonsec3 is
true, we can be certain nsec3only is false, and vice versa.
- previously, iterating a database backwards omitted
NSEC3 records even if DNS_DB_NONSEC3 had not been set. this
has been corrected.
- when an iterator reaches the origin node of the NSEC3 tree, we
need to skip over it and go to the next node in the sequence.
the NSEC3 origin node is there for housekeeping purposes and
never contains data.
- the dbiterator_test unit test has been expanded, several
incorrect expectations have been fixed. (for example, the
expected number of iterations has been reduced by one; we were
previously counting the NSEC3 origin node and we should not
have been doing so.)
Fix a CHANGES entries numbering issue that was inadvertently introduced
when change 6344 was backported. This makes the affected CHANGES
numbers consistent across all branches and releases again.
A release note for CVE-2023-50868 was not included in BIND 9.19.21, even
though that vulnerability was already addressed in that release (by the
fix for CVE-2023-50387). Retroactively add a relevant release note for
BIND 9.19.21.
Since CVE-2023-50868 does not have a dedicated fix in BIND 9, mention
its CVE identifier in the CHANGES entry for CVE-2023-50387 (KeyTrap),
which accompanied the code change that addresses both of these
vulnerabilities.
- create_node() in rbt.c cannot fail
- the dns_rbt_*name() functions, which are wrappers around
dns_rbt_[add|find|delete]node(), were never used except in tests.
this change isn't really necessary since RBT is likely to go away
eventually anyway. but keeping the API as simple as possible while it
persists is a good thing, and may reduce confusion while QPDB is being
developed from RBTDB code.
these values pertain to whether a node is in the main, nsec, or nsec3
tree of an RBTDB. they need to be moved to a more generic location so
they can also be used by QPDB.
(this is in db.h rather than db_p.h because rbt.c needs access to it.
technically, that's a layer violation, but it's a long-existing one;
refactoring to get rid of it would be a large hassle, and eventually
we expect to remove rbt.c anyway.)
when the QPDB is implemented, we will need to have both qpdb_p.h and
rbtdb_p.h. in order to prevent name collisions or code duplication,
this commit adds a generic private header file, db_p.h, containing
structures and macros that will be used by both databases.
some functions and structs have been renamed to more specifically refer
to the RBT database, in order to avoid namespace collision with similar
things that will be needed by the QPDB later.
refactor the wildcard matching code to make it a bit easier to
understand, in hopes that it will reduce the difficulty of converting
from RBTDB to QPDB later.
there are also some minor optimizations: previously, after stepping
backward to find the predecessor, we stepped back foward *from* the
predecessor to find the successor. we now reset the rbtnode chain to
its original starting point before stepping forward; this eliminates
some unnecessary processing. and, if neither predecessor nor successor
is found, we return early rather than carrying on with an unnecessary
effort to match labels.
Coverity detected that address->type.sa was too small when copying
a struct sockaddr_sin6, use the alterative union element
address->type.sin6 instead.
We were missing a test where a single owner name would have multiple
types with a different case. The generated RRSIGs and NSEC records will
then have different case than the signed records and message parser have
to cope with that and treat everything as the same owner.
The case insensitive matching in isc_ht was basically completely broken
as only the hashvalue computation was case insensitive, but the key
comparison was always case sensitive.
The isc_log_t contains a isc_logconfig_t that is swapped, dereferenced
or accessed its fields through a mutex. Instead of protecting it with a
rwlock, use RCU.
When shutting down the whole server, the reading could stop and detach
from controlconnection before sending is done. If send callback then
detaches from the last controlconnection handle, the ccmsg would be
invalidated after the send callback and thus we must not access ccmsg
after calling the send_cb().
We need to stop reading when calling isc_ccmsg_disconnect() as the
reading handle doesn't have to be last because sending might be in
progress. After that, we can safely remove .reading member because the
reading would not be called after the disconnect has been called.
The ccmsg_senddone() should also not call the recv callback if the
sending failed, that's the job of the caller's send callback - in fact
it already does that, so the code in ccmsg_senddone() was superfluous.
To reduce memory pressure, we can add light per-loop (netmgr worker)
memory pools for isc_nmsocket_t structures. This will help in
situations where there's a lot of churn creating and destroying the
nmsockets.
Embedding isc_nmsocket_h2_t directly inside isc_nmsocket_t had increased
the size of isc_nmsocket_t to 1840 bytes. Making the isc_nmsocket_h2_t
to be a pointer to the structure and allocated on demand allows us to
reduce the size to 1208 bytes. While there are still some possible
reductions in the isc_nmsocket_t (embedded tlsstream, streamdns
structures), this was the far biggest drop in the memory usage.
The uv_req union member of struct isc__nm_uvreq contained libuv request
types that we don't use. Turns out that uv_getnameinfo_t is 1000 bytes
big and unnecessarily enlarged the whole structure. Remove all the
unused members from the uv_req union.
After removing sockaddr_unix from isc_sockaddr, we can also remove
sockaddr_storage and reduce the isc_sockaddr size from 152 bytes to just
48 bytes needed to hold IPv6 addresses.
When running `make check` on a platform which has older (but still
supported) pytest, e.g. 3.4.2 on EL8, the junit to trs conversion would
fail because the junit format has different structure. Make the junit
XML processing more lenient to support both the older and newer junit
XML formats.
The ditch.pl script is used to generate burst traffic without waiting
for the responses. When running other tests in parallel, this can result
in a ephemeral port clash, since the ditch.pl process closes the socket
immediately. In rare occasions when the message ID also clashes with
other tests' queries, it might result in an UnexpectedSource error from
dnspython.
Use a dedicated port EXTRAPORT8 which is reserved for each test as a
source port for the burst traffic.
As it was pointed out, the alignas() can't be used on objects larger
than `max_align_t` otherwise the compiler might miscompile the code to
use auto-vectorization on unaligned memory.
As we were only using alignas() as a way to prevent false memory
sharing, we can use manual padding in the affected structures.
In the benchmarks, DNS_QPGC_ALL was trying to hard to cleanup QP
and this was slowing down QP too much. Use DNS_QPGC_MAYBE instead
that we are going to use anyway for more realistic load - this also
shows the memory usage matching the real loads.
Stop the cname_and_other_data processing if we already know that the
result is true. Also, we know that CNAME will be placed in the priority
headers, so we can stop looking for CNAME if we haven't found CNAME and
we are past the priority headers.
Mark the infrastructure RRTypes as "priority" types and place them at
the beginning of the rdataslab header data graph. The non-priority
types either go right after the priority types (if any).
The cachedb was missing piece of code (already found in zonedb) that
would make lookups in the slabheaders to miss the RRSIGs for CNAME if
the order of CNAME and RRSIG(CNAME) was reversed in the node->data.
With _exit() instead of exit() in place, we don't need
isc__tls_setfatalmode() mechanism as the atexit() calls will not be
executed including OpenSSL atexit hooks.
Instead of crude 5x rcu_barrier() call in the isc__mem_destroy(), change
the mechanism to call rcu_barrier() until the memory use and references
stops decreasing. This should deal with any number of nested call_rcu()
levels.
Additionally, don't destroy the contextslock if the list of the contexts
isn't empty. Destroying the lock could make the late threads crash.
Since the fatal() isn't a correct but rather abrupt termination of the
program, we want to skip the various atexit() calls because not all
memory might be freed during fatal() call, etc. Using _exit() instead
of exit() has this effect - the program will end, but no destructors or
atexit routines will be called.
Currently we test the incoming zone transfers data in the statistics
channel by retransfering the zones in slow mode and capturing the XML
and JSON outputs in the meantime to check their validity. Add a new
transfer to the test, and check that the XML and JSON files correctly
indicate that we have 3 retransfers and 1 new (first time) transfer.
Expose the newly added 'first refresh' flag in the information
provided by the 'rndc staus' command, by showing the number of
zones, which are not yet fully ready, and their first refresh
is pending or is in-progress.
Add a new zone flag to indicate that a secondary type zone is
not yet fully ready, and a first time refresh is pending or is
in progress.
Expose this new flag in the statistics channel's "Incoming Zone
Transfers" section.
Explicitly use an empty 'trust-anchors' statement in the system
tests where it was used implicitly before.
In resolver/ns5/named.conf.in use the trust anchor in 'trusted.conf',
which was supposed to be used there.
Add checks into the 'checkconf' system test to make sure that the
'dnssec-validation yes' option fails without configured trusted
anchors, and succeeds with configured non-empty, as well as empty
trusted anchors.
Using the 'dnssec-validation yes' option now requires an explicitly
confgiured 'trust-anchors' statement (or 'managed-keys' or
'trusted-keys', both deprecated).
Using the 'dnssec-validation yes' option now requires an explicitly
confgiured 'trust-anchors' statement (or 'managed-keys' or
'trusted-keys', both deprecated).
"A parental agent is the entity that is allowed to change a zone's
delegation information" is untrue, because it is possible to use some
hidden server or a validating resolver.
Also the new text makes it more clear that named sends DS queries to
these servers.
Use the (1 << N) form for defining the flags, in order to avoid
errors like the one fixed in the previous commit.
Also convert the definitions to an enum, as done in some of our
recent refactoring work.
The DNS_GETDB_STALEFIRST flag is defined as 0x0C, which is the
combination of the DNS_GETDB_PARTIAL (0x04) and the
DNS_GETDB_IGNOREACL (0x08) flags (0x04 | 0x08 == 0x0C) , which is
an obvious error.
All the flags should be power of two, so they don't interfere with
each other. Fix the DNS_GETDB_STALEFIRST flag by setting it to 0x10.
Instead of running all the cryptographic validation in a tight loop,
spread it out into multiple event loop "ticks", but moving every single
validation into own isc_async_run() asynchronous event. Move the
cryptographic operations - both verification and DNSKEY selection - to
the offloaded threads (isc_work_enqueue), this further limits the time
we spend doing expensive operations on the event loops that should be
fast.
Limit the impact of invalid or malicious RRSets that contain crafted
records causing the dns_validator to do many validations per single
fetch by adding a cap on the maximum number of validations and maximum
number of validation failures that can happen before the resolving
fails.
Checking the DS at the parent only happens if dns_zone_getdnsseckeys()
returns success. However, if this function somehow fails, it can also
prevent the keymgr from running.
Before adding the check DS functionality, the keymgr should only run
if 'dns_dnssec_findmatchingkeys()' did not return an error (either
ISC_R_SUCCESS or ISC_R_NOTFOUND). After this change the correct
result code is used again.
Remove text in man page that says the program can also be used for
generate TSIG keys, this was removed in 9.13.0 but this text was still
not yet removed.
the 'low', 'high' and 'discount' parameters to 'fetch-quota-param'
are meant to be ratios with values between zero and one, but higher
values can be assigned. this could potentially lead to an assertion
in maybe_adjust_quota().
The fix for CVE-2023-4408 introduced a regression in the message
parser, which could cause a crash if an rdata type that can only
occur in the question was found in another section.
Use 'dns__message_putassociatedrdataset()' instead of
'dns__message_puttemprdataset()', because after calling the
'dns_rdatalist_tordataset()' function earlier the 'rdataset'
is associated.
the fix for CVE-2023-4408 introduced a regression in the message
parser, which could cause a crash if duplicate rdatasets were found
in the question section. this commit ensures that rdatasets are
correctly disassociated and freed when this occurs.
In the second test we are looking for key files and extract the key
id numbers. Because keys can be in different directories, we needed
to change the maxdepth when searching for keys.
For the second kasp system test, check that 'dnssec-keygen -k' (default
policy) creates valid files, the 'get_keyids' returned more than one
keytag, namely the ones that are inside the keys/ directory, that were
created for the predecessor test, check that 'dnssec-keygen -k'
(configuredd policy) creates valid files.
This caused the system test to spew out errors that key files were
missing (we were looking for key files in the current directory, but
when looking for key id numbers we included the keys/ directory). It
could also cause the next test to fail, check that 'dnssec-settime' by
default does not edit key state file, because the STATE_FILE environment
variable was overwritten with the key file path of one of the keys that
were created with the configured policy.
We fix this by adjusting the maxdepth for the test in question. Other
tests don't need adjusting because they use unique zone names.
Support for FreeBSD 12.4, the last FreeBSD 12.x release, ended on
December 31, 2023.
Link: https://www.freebsd.org/security/unsupported/
Move the --with-readline=editline ./configure option to FreeBSD 14.
This is now the default way to implement attaching to/detaching from
a pointer.
Also update cfg_keystore_fromconfig() to allow NULL value for the
keystore pointer. In most cases we detach it immediately after the
function call.
Update the minimum required version of pkcs11-provider that contains the
fixes needed in order to make it work with dnssec-policy.
Update documentation to not recommend using engine_pkcs11 in conjunction
with dnssec-policy.
Add a default key-directory parameter to the function that can
be returned if there is no keystore, or if the keystore directory
is NULL (the latter is also true for the built-in keystore).
The name "uri" was considered to be too generic and could potentially
clash with a future URI configuration option. Renamed to "pkcs11-uri".
Note that this option name was also preferred over "pkcs11uri", the
dash is considered to be the more clearer form.
When using the same PKCS#11 URI for a zone that uses different
DNSSEC policies, the PKCS#11 label could collide, i.e. the same
label could be used for different keys. Add the policy name to
the label to make it more unique.
Also, the zone name could contain characters that are interpreted
as special characters when parsing the PKCS#11 URI string. Mangle
the zone name through 'dns_name_tofilenametext()' to make it
PKCS#11 safe.
Move the creation to a separate function for clarity.
Furthermore, add a log message whenever a PKCS#11 object has been
successfully created.
Add test cases for zones in different views that are using PKCS#11
tokens to store its keys.
If it is using the same DNSSEC policy, only one PKCS#11 token should be
created and the same key should be used for the zone in both views.
If it is using a different DNSSEC policy, multiple PKCS#11 token should
be created and each view should use their respective key.
This log may still occur if there is a DNSKEY in the unsigned zone.
This may happen in a multi-signer setup for example.
Ideally this should not log a warning, but that requires looking up
keys a different way (by searching for key files only). However, that
requires adapting a bunch of system tests, and is out of scope for now.
- Shell function body should be in between curly braces.
- Some erroneous '|| return 1' are replaced with '|| ret=1'.
- Fix a variable name (was 'ret', should be '_ret').
- Clean up when setting up a new test.
The internal keymgr used 'isc_dir_open(&dir)' and 'isc_dir_close(&dir)',
but was not using the variable 'dir`, other than checking if the
directory can be opened. Errors like these will be be caught already
in the dst_api function calls.
The pkcs11-provider did not yet support getting X/Y coordinates
on newly generated EC PKEY keys, thus we attempted to get the
key from the label after it was generated in the keystore.
This has been fixed in:
https://github.com/latchset/pkcs11-provider/pull/293
Thus now we should be able to use the generated key structure
immediately.
The bullseye and bookworm images are not set up with pkcs11-provider,
so we need to add an additional prerequisite for running the
pkcs11engine test. Check the path of OPENSSL_CONF.
If not set, the created keys allows signing plus decrypt which is bad
practice. Setting the key usage explicitly will generate keys that
allow only signing.
The pkcs11-provider has changed to take a PKCS#11 URI instead of an
object identifier. Change the BIND 9 code accordingly to pass through
the label instead of just the object identifier.
See: https://github.com/latchset/pkcs11-provider/pull/284
Move dns_dnssec_findzonekeys from the dnssec.{c,h} source code to
zone.{c,h} (the header file already commented that this should be done
inside dns_zone_t).
Alter the function in such a way, that keys are searched for in the
key stores if a 'dnssec-policy' (kasp) is attached to the zone,
otherwise keep using the zone's key-directory.
When using dnssec-policy with dnssec-keygen in combination with setting
the key-directory on the command line, the commandline argument takes
priority over the key-directory from the default named.conf.
Add a test case where dnssec-policy uses key stores with a directory
other than the zone's key-directory.
This requires changing the kasp shell script to take into account that
keys can be in different directories. When looking for keys, the
'find' command now takes a maxdepth of 3 to also look for keys in
subdirectories. Note this maxdepth value is arbitrary, the added
'keystore.kasp' test only requires a maxdepth of 2.
Because of this change, the dnssec-keygen tests no longer work because
they are for the same zone (although different directories). Change
the test to use a different zone ('kasp2' instead of 'kasp').
When writing key files to disk, use the internally stored directory.
Add an access function 'dst_key_directory()'.
Most calls to keymgr functions no longer need to provide the
key-directory value. Only 'dns_keymgr_run' still needs access to
the zone's key-directory in case the key-store is set to the built-in
key-directory.
Refactor dns_dnssec_findmatchingkeys and dns_dnssec_keylistfromrdataset
to take into account the key store directories in case the zone is using
dnssec-policy (kasp). Add 'kasp' and 'keystores' parameters.
This requires the keystorelist to be stored inside the zone structure.
The calls to these functions in the DNSSEC tools can use NULL as the
kasp value, as dnssec-signzone does not (yet) support dnssec-policy,
and key collision is checked inside the directory where it is created.
Add cases for each algorithm to test the interaction between
dnssec-policy and engine_pkcs11. Ensure that named creates keys on
startup.
Also test dnssec-keygen when using a dnssec-policy with a PKCS#11
based key-store.
If there is a keystore configured with a PKCS#11 URI, zones that
are using a dnssec-policy that uses such a keystore should create keys
via the PKCS#11 interface. Those keys are generally stored inside an
HSM.
Some changes to the code are required, to store the engine reference
into the keystore.
When creating the kasp structure, instead of storing the name of the
key store on keys, store a reference to the key store object instead.
This requires to build the keystore list prior to creating the kasp
structures, in the dnssec tools, the check code and the server code.
We will create a builtin keystore called "key-directory" which means
use the zone's key-directory as the key store.
The check code changes, because now the keystore is looked up before
creating the kasp structure (and if the keystore is not found, this
is an error). Instead of looking up the keystore after all
'dnssec-policy' clauses have been read.
The check for printing zone list failed because of these additional
lines in the output:
good.conf:22: dnssec-policy: key algorithm 13 has predefined length; \
ignoring length value 256
I am not sure why this failure hasn't happened before already.
Similar to key-directory, check for zones in different views and
different key and signing policies. Zones must be using different key
directories to store key files on disk.
Now that a key directory can be linked with a dnssec-policy key, the
'keydirexist' checking needs to be reshuffled.
Add tests for bad configuration examples, named-checkconf should catch
those. Also add test cases for a mix of key-directory and key-store
directory.
Similar to key-directory, check if the key-store directory exists and
if it is an actual directory.
This commit fixes an accidental test bug in checkconf where if
the "warn key-dir" test failed, the result was ignored.
Add checkconf check to ensure that the used key-store in the keys
section exists. Error if that is not the case. We also don't allow
the special keyword 'key-directory' as that is internally used to
signal that the zone's key-directory should be used.
Add code for configuring keystore objects. Add this to the "kaspconf"
code, as it is related to 'dnssec-policy' and it is too small to create
a separate file for it.
Add new configuration for setting key stores. The new 'key-store'
statement allows users to configure key store backends. These can be
of type 'file' (that works the same as 'key-directory') or of type
'pkcs11'. In the latter case, keys should be stored in a HSM that is
accessible through a PKCS#11 interface.
Keys configured within 'dnssec-policy' can now also use the 'key-store'
option to set a specific key store.
Update the checkconf test to accomodate for the new configuration.
These tests have ns1 configured as a mock root server. Make sure it is
used in all config files of those tests, otherwise some queries could
leak to root nameservers.
Some tests don't have a mock root server configured, because they don't
need one. However, these tests might still leak queries to actual name
servers. Add a shared root hints file which can serve as a blackhole for
these queries.
As dnsrps and native test cases have been properly split up, the
ckdnsrps.sh script is no longer used anywhere, as the logic for
selecting these test cases is handled by pytest.
Previously, dnsrps test was executed as an optional part of the rpz and
rpzrecurse system tests. This was conceptually problematic, as the test
took the responsibility of running parts of the test framework -
cleaning files and setting up servers again.
Instead, allow these tests to execute either the native variant, or the
dnsrps one. To ensure the same test coverage, trigger both of these
variants as separate test cases from pytest.
cmocka.h and jemalloc.h/malloc_np.h has conflicting macro definitions.
While fixing them with push_macro for only malloc is done below, we only
need the non-standard mallocx interface which is easy to just define by
ourselves.
Because we don't use jemalloc functions directly, but only via the
libisc library, the dynamic linker might pull the jemalloc library
too late when memory has been already allocated via standard libc
allocator.
Add a workaround round isc_mem_create() that makes the dynamic linker
to pull jemalloc earlier than libc.
This commit improves TLS messages framing by avoiding an extra call to
SSL_write_ex(). Before that we would use an extra SSL_write_ex() call
to pass DNS message length to OpenSSL. That could create an extra TLS
frame, increasing number of bytes sent due to frame header and
padding.
This commit fixes that by making the code pass both DNS message length
and data at once, just like old TLS code did.
It should improve compatibility with some buggy clients that expect
both DNS message length and data to be in one TLS frame.
Older TLS DNS code worked like this, too.
The statistics channel does not expose the current number of TCP clients
connected, only the highwater. Therefore, users did not have an easy
means to collect statistics about TCP clients served over time. This
information could only be measured as a seperate mechanism via rndc by
looking at the TCP quota filled.
In order to expose the exact current count of connected TCP clients
(tracked by the "tcp-clients" quota) as a statistics counter, an
extra, dedicated Network Manager callback would need to be
implemented for that purpose (a counterpart of ns__client_tcpconn()
that would be run when a TCP connection is torn down), which is
inefficient. Instead, track the number of currently-connected TCP
clients separately for IPv4 and IPv6, as Network Manager statistics.
We need to skip some portions the system test in FIPS mode as some of
the algorithms used in the test are not available when using the FIPS
mode (e.g. TLS_CHACHA20_POLY1305_SHA256)
This commit removes wrong INSIST() condition as the assumption that if
'csock->recv_cb != NULL' iff 'csock->statichandle != NULL' is wrong.
There is no direct relation between 'csock->statichandle' and
'csock->recv_cb', as 'csock->statichandle' gets set when allocating a
handle regardless of 'csock->recv_cb' not being NULL, as it is
possible to attach to the handle without starting a read operation (at
the very least, it is correct to start writing before reading).
That condition made `cipher-suites` system test fail with crash on
some platforms in FIPS mode (namely, Oracle Linux 9) despite not being
related to FIPS at all.
The older version of the code was reporting that listeners are going
to be of the same type after reconfiguration when switching from DoT
to HTTPS listener, making BIND abort its executions.
That was happening due to the flaw in logic due to which the code
could consider a current listener and a configuration for the new one
to be of the same type (DoT) even when the new listener entry is
explicitly marked as HTTP.
The checks for PROXY in between the configuration were masking that
behaviour, but when porting it to 9.18 (when there is no PROXY
support), the behaviour was exposed.
Now the code mirrors the logic in 'interface_setup()' closely (as it
was meant to).
When processing UPDATE request DNSKEY, CDNSKEY and CDS record that
are managed by named are filtered out. The log message has been
updated to report the actual type rather that just DNSKEY.
Recreate listeners on DNS transport change when editing listen-on statements before reconfiguration
Closes#4528 and #4518
See merge request isc-projects/bind9!8644
This commit adds a system test that helps to verify that changing a
listener transport by editing "listen-on" statements before
reconfiguration works as expected.
This commit ensures that listeners are recreated on reconfiguration in
the case when their type changes (or when PROXY protocol type changes,
too).
Previously, if a "listen-on" statement was modified to represent a
different transport, BIND would not pick-up the change on
reconfiguration if listener type changes (e.g. DoH -> DoT) for a given
interface address and port combination. This commit fixes that by
recreating the listener.
Initially, that worked for most of the new transports as we would
recreate listeners on each reconfiguration for DoH and DoT. But at
some point we changed that in such a way that listeners were not
recreated to avoid rebinding a port as on some platforms only root can
do that for port numbers <1000, making some ports binding possible
only on start-up. We chose to asynchronously update listener socket
settings (like TLS contexts, HTTP settings) instead.
Now, we both avoid recreating the sockets if unnecessary and recreate
listeners when listener type changes.
This commit adds a new system test which verifies that using the
'cipher-suites' option actually works as expected (as well as adds
first TLSv1.3 specific tests).
This commits adds low-level wrappers on top of
'SSL_CTX_set_ciphersuites()'. These are going to be a foundation
behind the 'cipher-suites' option of the 'tls' statement.
Dig failed to print a comment about the reason of the unacceptable
query reply got from a server when there was no other query to
start in the lookup's chain.
Add an "else" block to print out the comment even when not starting
up the next query.
When getting a SERVFAIL reply from a query, 'host' tries to start
the next query in the lookup's list (also true for 'dig +nofail').
However, when running with the '-C' switch (or +nssearch for 'dig'),
all the queries in the lookup start from the beginning, so that logic
brings to a crash because of the attempted start of the query which
was already started.
Don't start the next query in the affected code path when in +nssearch
mode.
The conn_shutdown() function is called whenever a control channel
connection is supposed to be closed, e.g. after a response to the client
is sent or when named is being shut down. That function calls
isccc_ccmsg_invalidate(), which resets the magic number in the structure
holding the messages exchanged over a given control channel connection
(isccc_ccmsg_t). The expectation here is that all operations related to
the given control channel connection will have been completed by the
time the connection needs to be shut down.
However, if named shutdown is initiated while a control channel message
is still in flight, some netmgr callbacks might still be pending when
conn_shutdown() is called and isccc_ccmsg_t invalidated. This causes
the REQUIRE assertion checking the magic number in ccmsg_senddone() to
fail when the latter function is eventually called, resulting in a
crash.
Fix by splitting up isccc_ccmsg_invalidate() into two separate
functions:
- isccc_ccmsg_disconnect(), which initiates TCP connection shutdown,
- isccc_ccmsg_invalidate(), which cleans up magic number and buffer,
and then:
- replacing all existing uses of isccc_ccmsg_invalidate() with calls
to isccc_ccmsg_disconnect(),
- only calling isccc_ccmsg_invalidate() when all netmgr callbacks are
guaranteed to have been run.
Adjust function comments accordingly.
The "exceeded time limit waiting for literal 'too many DNS UPDATEs
queued' in ns1/named.run" is prone to fail due to a timing issue.
Despite out efforts to stabilize it, the check still often fails on
FreeBSD in our CI. Allow the test to be re-run on this platform.
This file was missing explicit dnssec-validation. Seems like it was
missed in our previous efforts, probably because of the different
filename / extension. Rename it to end with *.in to reflect that it is a
template file used by copy_setports.
The logic contained in dangerfile.py incorrectly warns about missing
release note changes for merge requests preparing release documentation
as such merge requests rename files in the doc/notes/ directory. This
(correctly) causes these files to be passed to dangerfile.py via
danger.git.created_files and danger.git.deleted_files rather than via
danger.git.modified_files, which in turn causes the logic checking the
use of the "Release Notes" label to assume that no release notes are
added, removed, or modified by a given merge request.
Fix by considering all types of file changes (modifications, additions,
and removals - which also covers file renaming) when checking whether a
given merge request modifies release notes. Update the warning messages
accordingly.
However, when trying to find release notes added by a given merge
request, deleted files must not be considered. Tweak the logic looking
for GitLab identifiers in the release notes added by a given merge
request so that it only scans modified and added (or renamed) files.
Instead of issuing a separate isc_async_run() call for every RBTDB node
that triggers tree pruning, maintain a list of nodes from which tree
pruning can be started from and only issue an isc_async_run() call if
pruning has not yet been triggered by another RBTDB node.
In some older BIND 9 branches, the extra queuing overhead eliminated by
this change could be remotely exploited to cause excessive memory use.
Due to architectural shift, this branch is not vulnerable to that issue,
but applying the fix to the latter is nevertheless deemed prudent for
consistency and to make the code future-proof.
If we are in the process of looking for the A records as part of
dns64 processing and the server-stale timeout triggers, redo the
dns64 changes that had been made to the orignal qctx.
The wrong result value was being saved for resumption with
nxdomain-redirect when performing the fetch. This lead to an assert
when checking that RFC 1918 reverse queries where not leaking to
the global internet.
When parsing messages use a hashmap instead of a linear search to reduce
the amount of work done in findname when there's more than one name in
the section.
There are two hashmaps:
1) hashmap for owner names - that's constructed for each section when we
hit the second name in the section and destroyed right after parsing
that section;
2) per-name hashmap - for each name in the section, we construct a new
hashmap for that name if there are more than one rdataset for that
particular name.
1) Restart the process with a write lock if we discover an expired key
while holding the read lock.
2) Move incrementing the key reference inside the lock block of code.
isc/atomic.h and its defined macros should be preferred over
stdatomic.h and explicit atomic operations.
Fix the redundant stdatomic.h header in histo.c found by the introduced
check.
If the DNSKEY, CDNSKEY or CDS RRset had different TTLs then the
filtering of these RRset resulted in dns_diff_apply failing with
"not exact". Identify tuple pairs that are just TTL changes and
allow them through the filter.
If the dnskey-ttl in the dnssec-policy doesn't match the DNSKEY's
ttl then the DNSKEY, CDNSKEY and CDS rrset should be updated by
named to reflect the expressed policy. Check that named does this
by creating a zone with a TTL that does not match the policy's TTL
and check that it is correctly updated.
In Net::DNS 1.42 $ns->main_loop no longer loops. Use current methods
for starting the server, wait for SIGTERM then cleanup child processes
using $ns->stop_server(), then remove the pid file.
Clang Static Analyzer is unable to grasp that when dns_rbt_addnode()
returns ISC_R_EXISTS, it always sets the pointer passed to it via its
'nodep' parameter to a non-NULL value. Add an extra safety check in the
conditional expression used in dns_rbt_addname() to silence that
warning.
Reconfiguring named using RNDC is a common action in BIND 9 system
tests. It involves sending the "reconfig" RNDC command to a named
instance and waiting until it is fully processed. Add a reconfigure()
method to the NamedInstance class in order to simplify and standardize
named reconfiguration using RNDC in Python-based system tests.
TODO:
- full reconfiguration support (w/templating *.in files)
- add an "rndc null" before every reconfiguration to show which file
is used (NamedInstance.add_mark_to_log() as it may be generically
useful?)
Ensure the type hints provided in helper code for Python-based system
tests are correct by continuously checking them using mypy in GitLab CI.
Check bin/tests/system/isctest.py exclusively for the time being because
it is the only Python file in the source tree which uses static typing
at the moment and working around the issues reported by mypy for other
(non-statically-typed) Python files present in the source tree would be
cumbersome.
The "checkds" system test contains a lot of duplicated code despite
carrying out the same set of actions for every tested scenario
(zone_check() → wait for logs to appear → keystate_check()). Extract
the parts of the code shared between all tests into a new function,
test_checkds(), and use pytest's test parametrization capabilities to
pass distinct sets of test parameters to this new function, in an
attempt to cleanly separate the fixed parts of this system test from the
variable ones. Replace format() calls with f-strings.
The "checkds" system test only uses dns.resolver.Resolver objects to
access their 'nameservers' and 'port' attributes. Instances of the
NamedInstance class also expose that information via their attributes,
so only pass NamedInstance objects around instead of needlessly
depending on dns.resolver.Resolver.
Make log file watching in Python-based system tests consistent by
employing the helper Python classes designed for that purpose. Drop the
custom code currently used.
Waiting for a specific log line to appear in a named.run file is a
common action in BIND 9 system tests. Implement a set of Python classes
which intend to simplify and standardize this task in Python-based
system tests.
Co-authored-by: Štěpán Balážik <stepan@isc.org>
The "addzone" and "shutdown" system tests currently invoke rndc using
test-specific helper code. Rework the relevant bits of those tests so
that they use the helper classes from bin/tests/system/isctest.py.
Controlling named instances using RNDC is a common action in BIND 9
system tests. However, there is currently no standardized way of doing
that from Python-based system tests, which leads to code duplication.
Add a set of Python classes and pytest fixtures which intend to simplify
and standardize use of RNDC in Python-based system tests.
For now, RNDC commands are sent to servers by invoking the rndc binary.
However, a switch to a native Python module able to send RNDC commands
without executing external binaries is expected to happen soon. Even
when that happens, though, having the capability to invoke the rndc
binary (in order to test it) will remain useful. Define a common Python
interface that such "RNDC executors" should implement (RNDCExecutor), in
order to make switching between them convenient.
Co-authored-by: Štěpán Balážik <stepan@isc.org>
it was possible for fix_iterator() to get stuck in a loop while
trying to find the predecessor of a missing node. this has been
fixed and a regression test has been added.
the fix_iterator() function moves an iterator so that it points
to the predecessor of the searched-for name when that name doesn't
exist in the database. the tests only checked the correctness of
the top of the stack, however, and missed some cases where interior
branches in the stack could be missing or duplicated. in these
cases, the iterator would produce inconsistent results when walked.
the predecessors test case in qp_test has been updated to walk
each iterator to the end and ensure that the expected number of
nodes are found.
When changing the NSEC3 chain, the new NSEC3 chain must be built before
the old NSEC3PARAM is removed. Check each delta in the conversion to
ensure this ordering is met.
When transitioning from NSEC3 to NSEC the NSEC3 must be built before
the NSEC3PARAM is removed. Check each delta in the conversion to
ensure this ordering is met.
When building NSEC3 chains update the NSEC3PARAM TTL to match
the SOA minimum. Delete all records using the old TTL then
re-add them using the new TTL.
Zone content integrity checks can significantly slow the conversion
of zones from raw to text. As this is more properly a job for
named-checkzone anyway, we now disable all zone checks by
default in named-compilezone.
Users relying on named-compilezone for integrity checks as
well as format conversion can run named-checkzone separately,
or re-enable the checks in named-compilezone by using:
"named-compilezone -n fail -k fail -r warn -T warn -W warn".
Just remove the key from consideration as it is being removed.
The old code could leak a key reference as dst_free_key was not
called every time we continued. This simplification will address
this as well.
Check the tid and cancel the request immediately or pass it to the
appropriate loop for processing. Call request->cb directly from
req_sendevent as it is now always called with the correct tid.
The xfrin_end() function is run when a zone transfer is finished or
canceled. One of the actions it takes for incremental transfers (IXFR)
is calling dns_journal_destroy() on the zone journal structure that is
stored in the relevant zone transfer context (xfr->ixfr.journal). That
immediately invalidates that structure as it is not reference-counted.
However, since the changes present in the IXFR stream are applied to the
journal asynchronously (via isc_work_enqueue()), it is possible that
some zone changes may still be in the process of being written to the
journal by the time xfrin_end() destroys the relevant structure. Such a
scenario leads to crashes.
Fix by not destroying the zone journal structure until the entire zone
transfer context is destroyed. xfrin_destroy() already conditionally
calls dns_journal_destroy() and when the former is called, all
asynchronous work for a given zone transfer process is guaranteed to be
complete.
Multiple zones should be able to read the same key and signing policy
at the same time. Since writing the kasp lock only happens during
reconfiguration, and the complete kasp list is being replaced, there
is actually no need for a lock. Reference counting ensures that a kasp
structure is not destroyed when still being attached to one or more
zones.
This significantly improves the load configuration time.
When transitioning from NSEC3 to NSEC the added records where not
being signed because the wrong time was being used to determine if
a key should be used or not. Check that these records are actually
signed.
When kasp support was added 'inception' was used as a proxy for
'now' and resulted in signatures not being generated or the wrong
signatures being generated. 'inception' is the time to be set
in the signatures being generated and is usually in the past to
allow for clock skew. 'now' determines what keys are to be used
for signing.
Resolve "dig in 9.19.13 crashes, when cancelling (with CTRL+C) a pending query to to a not reachable TCP port"
Closes#4138
See merge request isc-projects/bind9!8554
Remove the CFG_CLAUSEFLAG_EXPERIMENTAL flag from the
"trust-anchor-telemetry" statement as the behavior of the latter has not
been changed since its initial implementation and there are currently no
plans to do so. This silences a relevant log message that was emitted
even when the feature was explicitly disabled.
Each function queuing a do_nsfetch() call using isc_async_run() is
expected to increase the given zone's internal reference count
(zone->irefs), which is then correspondingly decreased in either
do_nsfetch() itself (when the dns_resolver_createfetch() fails) or in
nsfetch_done() (when recursion is finished).
However, do_nsfetch() can also return early if either the zone itself or
the relevant view's resolver object is being shut down. In that case,
do_nsfetch() simply returns without decreasing the internal reference
count for the zone. This leaves a dangling zone reference around, which
leads to hangs during named shutdown.
Fix by executing the same cleanup code for early returns from
do_nsfetch() as for a failed dns_resolver_createfetch() call in that
function as the reference count will not be decreased in nsfetch_done()
in any of these cases.
The loop in shutdown_listener() assumes that the reference count for
every controlconnection_t object on the listener->connections linked
list will drop down to zero after the conn_shutdown() call in the loop's
body. However, when the timing is just right, some netmgr callbacks for
a given control connection may still be awaiting processing by the same
event loop that executes shutdown_listener() when the latter is run.
Since these netmgr callbacks must be run in order for the reference
count for the relevant controlconnection_t objects to drop to zero, when
the scenario described above happens, shutdown_listener() runs into an
infinite loop due to one of the controlconnection_t objects on the
listener->connections linked list never going away from the head of that
list.
Fix by safely iterating through the listener->connections list and
initiating shutdown for all controlconnection_t objects found. This
allows any pending netmgr callbacks to be run by the same event loop in
due course, i.e. after shutdown_listener() returns.
The check_loaded() function compares the zone's loadtime value and
an expected loadtime value, which is based on the zone file's mtime
extracted from the filesystem.
For the secondary zones there may be cases, when the zone file isn't
ready yet before the zone transfer is complete and the zone file is
dumped to the disk, so a so zero value mtime is retrieved.
In such cases wait one second and retry until timeout. Also modify
the affected check to allow a possible difference of the same amount
of seconds as the chosen timeout value.
The atomic_init() function makes sense to use with structure's
members when creating a new instance of a strucutre. In other
places, use atomic store operations instead, in order to avoid
data races.
Move the code to find the predecessor into one function, as it is shares
quite some similarities: In both cases we first need to find the
immediate predecessor/successor, then we need to find the immediate
predecessor if the iterator is not already pointing at it.
This one is similar to the bug when searching for a key, reaching a
dead-end branch that doesn't match, because the branch offset point
is after the point where the search key differs.
This fixes the case where we are multiple levels deep. In other
words, we had a more-than-one matches *after* the point where the
search key differs.
For example, consider the following qp-trie:
branch: "[e]", "[m]":
- leaf: "a.b.c.d.e"
- branch: "moo[g]", "moo[k]", "moo[n]":
- leaf: "moog"
- branch: "mook[e]", "mook[o]"
- leaf: "mooker"
- leaf: "mooko"
- leaf: "moon"
If searching for a key "monky", we would reach the branch with
twigs "moo[k]" and "moo[n]". The key matches on the 'k' on offset=4,
and reaches the branch with twigs "mook[e]" and "mook[o]". This time
we cannot find a twig that matches our key at offset=5, there is no
twig for 'y'. The closest name we found was "mooker".
Note that on a branch it can't detect it is on a dead branch because the
key is not encapsulated in a branch node.
In the previous code we considered "mooker" to be the successor of
"monky" and so we needed to the predecessor of "mooker" to find the
predecessor for "monky". However, since the search key alread differed
before entering this branch, this is not enough. We would be left with
"moog" as the predecessor of "monky", while in this example "a.b.c.d.e"
is the actual predecessor.
Instead, we need to go up a level, find the predecessor and check
again if we are on the right branch, and repeat the process until we
are.
Unit tests to cover the scenario are now added.
There was yet another edge case in which an iterator could be
positioned at the wrong node after dns_qp_lookup(). When searching for
a key, it's possible to reach a leaf that matches at the given offset,
but because the offset point is *after* the point where the search key
differs from the leaf's contents, we are now at the wrong leaf.
In other words, the bug fixed the previous commit for dead-end branches
must also be applied on matched leaves.
For example, if searching for the key "monpop", we could reach a branch
containing "moop" and "moor". the branch offset point - i.e., the point
after which the branch's leaves differ from each other - is the
fourth character ("p" or "r"). The search key matches the fourth
character "p", and takes that twig to the next node (which can be
a branch for names starting with "moop", or could be a leaf node for
"moop").
The old code failed to detect this condition, and would have
incorrectly left the iterator pointing at some successor, and not
at the predecessor of the "moop".
To find the right predecessor in this case, we need to get to the
previous branch and get the previous from there.
This has been fixed and the unit test now includes several new
scenarios for testing search names that match and unmatch on the
offset but have a different character before the offset.
A merge request might have no description at all (i.e. None, rather than
an empty string). This might happen when the MR is created via an API.
Check a description is present before trying to find a backport string
in it.
Since the list of lines added to Git-tracked text files in a given
branch is not part of the Danger DSL [1], it is determined using custom
code in dangerfile.py. The current implementation of that logic is less
than perfect as it examines the diff between the current tip of the
target branch and the source branch rather than the diff between the
merge base of the two branches and the source branch. Consider a Git
history like this:
* F (target)
...
* E
* D
* C
| * B (source)
|/
* A (merge base)
If danger-python or Hazard are run for commit B, the current logic for
determining the list of added lines in dangerfile.py examines the diff
between commits F and B rather than between commits A and B. Therefore,
the added_lines() function returns not just the lines added by commit B
on top of commit A, but also the list of lines that were removed between
commits A and F, which leads to confusing results.
Fix by using the triple-dot diff operator in the Git invocation whose
output is used as the source of information for determining the list of
lines added by a given branch.
Since Hazard fetches the target branch itself when it is run, remove the
explicit "git fetch" invocation that fetches the target branch from
GitLab (shortening its local history to a single commit in the process)
before "git diff" is invoked.
[1] https://danger.systems/js/reference.html#GitDSL
There was a change in the branch that uncovered preexisting data races
when testing with respdiff under TSAN. This isn't a new issue that
should stop the releases. Allow the check to temporarily fail until the
underlying issue GL #4475 is addressed.
As we are in overmem state we want to free more memory than we are
adding so we need to add in an allowance for the rbtnodes that may
have been added and the names stored with them. There is the node
for the owner name and a possible ENT node if there was a node split.
Only cleanup headers that are less than equal to the rbt's last_used
time. Adjust the rbt's last_used time when the target cleaning was
not achieved to the oldest value of the remaining set of headers.
When updating delegating NS and glue records last_used was not being
updated when it should have been.
When adding zero TTL records to the tail of the LRU lists set
last_used to rbtdb->last_used + 1 rather than now. This appoximately
preserves the lists LRU order.
certain dig options which were deprecated and became nonoperational
several releases ago still had documentation in the dig man page and
warnings printed when they were used: these included +mapped,
+sigchase, +topdown, +unexpected, +trusted-key, and the -i and -n
options. these are now all fatal errors.
another option was described as deprecated in the man page, but
the code to print a warning was never added. it has been added now.
these options control default timing of retries in the resolver
for experimental purposes; they are not known to useful in production
environments. they will be removed in the future; for now, we
only log a warning if they are used.
there was another edge case in which an iterator could be positioned at
the wrong node after dns_qp_lookup(). when searching for a key, it's
possible to reach a dead-end branch that doesn't match, because the
branch offset point is *after* the point where the search key differs
from the branch's contents.
for example, if searching for the key "mop", we could reach a branch
containing "moon" and "moor". the branch offset point - i.e., the
point after which the branch's leaves differ from each other - is the
fourth character ("n" or "r"). however, both leaves differ from the
search key at position *three* ("o" or "p"). the old code failed to
detect this condition, and would have incorrectly left the iterator
pointing at some lower value and not at "moor".
this has been fixed and the unit test now includes this scenario.
in some cases it was possible for the iterator to be positioned in the
wrong place by dns_qp_lookup(). previously, when a leaf node was found
which matched the search key at its parent branch's offset point, but
did not match after that point, the code incorrectly assumed the leaf
it had found was a successor to the searched-for name, and stepped the
iterator back to find a predecessor. however, it was possible for the
non-matching leaf to be the predecessor, in which case stepping the
iterator back was wrong.
(for example: a branch contains "aba" and "abcd", and we are searching
for "abcde". we step down to the twig matching the letter "c" in
position 3. "abcd" is the predecessor of "abcde", so the iterator is
already correctly positioned, but because the twig was an exact match,
we would have moved it back one step to "aba".)
this previously went unnoticed due to a mistake in the qp_test unit
test, which had the wrong expected result for the test case that should
have detected the error. both the code and the test have been fixed.
the 'predecessor' argument to dns_qp_lookup() turns out not to
be sufficient for our needs: the predecessor node in a QP database
could have become "empty" (for the current version) because of an
update or because cache data expired, and in that case the caller
would have to iterate more than one step back to find the predecessor
node that it needs.
it may also be necessary for a caller to iterate forward, in
order to determine whether a node has any children.
for both of these reasons, we now replace the 'predecessor'
argument with an 'iter' argument. if set, this points to memory
with enough space for a dns_qpiter object.
when an exact match is found by the lookup, the iterator will be
pointing to the matching node. if not, it will be pointing to the
lexical predecessor of the nae that was searched for.
a dns_qpiter_current() method has been added for examining
the current value of the iterator without moving it in either
direction.
This commit extends the 'doth' system tests with additional secondary
NS instance that reuses the same 'tls' entry for connecting the the
primary to download zones. This configurations were known to crash
secondaries in some cases.
During initialisation or reconfiguration, it is possible that multiple
threads are trying to create a TLS context and associated data (like
TLS certs store) concurrently. In some cases, a thread might be too
late to add newly created data to the TLS contexts cache, in which
case it needs to be discarded. In the code that handles that case, it
was not taken into account that, in some cases, the TLS certs store
could not have been created or should not be deleted, as it is being
managed by the TLS contexts cache already. Deleting the store in such
cases might lead to crashes.
This commit fixes the issue.
This commit adds a system test suite for PROXYv2. The idea on which it
is based is simple:
1. Firstly we check that 'allow-proxy' and 'allow-proxy-on' (whatever
is using the new 'isc_nmhandle_real_localaddr/peeraddr()') do what
they intended to do.
2. Anything else that needs an interface or peer address (ACL
functionality, for example) is using the old
'isc_nmhandle_localaddr/peeraddr()' - which are now returning
addresses received via PROXY (if any) instead of the real connection
addresses. The beauty of it that we DO NOT need to verify every bit of
the code relying on these functions: whatever works in one place will
work everywhere else, as these were the only functions that allowed
any higher level code to get peer and interface addresses.
This way it is relatively easy to see if PROXYv2 works as intended.
The main intention of PROXY protocol is to pass endpoints information
to a back-end server (in our case - BIND). That means that it is a
valid way to spoof endpoints information, as the addresses and ports
extracted from PROXYv2 headers, from the point of view of BIND, are
used instead of the real connection addresses.
Of course, an ability to easily spoof endpoints information can be
considered a security issue when used uncontrollably. To resolve that,
we introduce 'allow-proxy' and 'allow-proxy-on' ACL options. These are
the only ACL options in BIND that work with real PROXY connections
addresses, allowing a DNS server operator to specify from what clients
and on which interfaces he or she is willing to accept PROXY
headers. By default, for security reasons we do not allow to accept
them.
This commit modifies TLS Stream and DNS-over-HTTPS transports so that
they do not use the "sock->iface" and "sock->peer" of the lower level
transport directly.
That did not cause any problems before, as things worked as expected,
but with the introduction of PROXYv2 support we use handles to store
the information in both PROXY Stream and UDP Proxy
transports. Therefore, in order to propagate the information (like
addresses), extracted from PROXYv2 headers, from the lower level
transports to the higher-level ones, we need to get that information
from the lower-level handles rather than sockets. That means that we
should get the peer and interface addresses using the intended
APIs ("isc_nmhandle_peeraddr()" and "isc_nmhandle_localaddr()").
This commit extends "listen-on" statement with "proxy" options that
allows one to enable PROXYv2 support on a dedicated listener. It can
have the following values:
- "plain" to send PROXYv2 headers without encryption, even in the case
of encrypted transports.
- "encrypted" to send PROXYv2 headers encrypted right after the TLS
handshake.
Due to increased number of the NM unit tests and, thus, increased load
on the system timeout recovery tests can sometimes fail, in particular
on FreeBSD. This commit fixes that. Besides, it seems that use of
T_SOFT here was unintentional to begin with.
This commit adds PROXYv2 support into dig by the means of adding
+[no]proxy and +[no]proxy-plain options. Since this commit dig
supports sending PROXYv2 headers on all supported DNS-transports.
The support for PROXYv2 is modelled after that one in kdig.
This commit adds a unit test suite for the new PROXY over UDP
transport. Most of the code is reused from the UDP unit test suite, as
the new transport aims to be fully compatible with UDP on the API
level.
This commit mostly moves the code around to make the parts of the UDP
unit test suite reusable. That changes the unit test suite structure
to resemble that of stream based unit tests.
The motivation behind this is to reuse most of the code for the new
PROXY over UDP uni tests suite.
Add the new isc__nm_dump_active_manager() function that can be used
for debugging purposes: it dumps all active sockets withing the
network manager instance.
This commit adds a new transport that supports PROXYv2 over UDP. It is
built on top of PROXYv2 handling code (just like PROXY Stream). It
works by processing and stripping the PROXYv2 headers at the beginning
of a datagram (when accepting a datagram) or by placing a PROXYv2
header to the beginning of an outgoing datagram.
The transport is built in such a way that incoming datagrams are being
handled with minimal memory allocations and copying.
In the previous versions of the NM, detecting the case when worker is
shutting down was not that important and actual status code did not
matter much. However, that might be not the case all the time.
This commit makes necessary modifications to the code.
This commit makes it possible to use PROXY Stream not only over TCP,
but also over TLS. That is, now PROXY Stream can work in two modes as
far as TLS is involved:
1. PROXY over (plain) TCP - PROXYv2 headers are sent unencrypted before
TLS handshake messages. That is the main mode as described in the
PROXY protocol specification (as it is clearly stated there), and most
of the software expects PROXYv2 support to be implemented that
way (e.g. HAProxy);
2. PROXY over (encrypted) TLS - PROXYv2 headers are sent after the TLS
handshake has happened. For example, this mode is being used (only ?)
by "dnsdist". As far as I can see, that is, in fact, a deviation from
the spec, but I can certainly see how PROXYv2 could end up being
implemented this way elsewhere.
This commit modifies TLS Stream to make it possible to use over PROXY
Stream. That is required to add PROVYv2 support into TLS-based
transports (DNS over HTTP, DNS over TLS).
This commit adds a specialised test suite for the PROXY Stream
transport by reusing most of the testing code from other unit tests
for other stream-based transports.
This commit adds a new stream-based transport with an interface
compatible with TCP. The transport is built on top of TCP transport
and the new PROXYv2 handling code. Despite being built on top of TCP,
it can be easily extended to work on top of any TCP-like stream-based
transport. The intention of having this transport is to add PROXYv2
support into all existing stream-based DNS transport (DNS over TCP,
DNS over TLS, DNS over HTTP) by making the work on top of this new
transport.
The idea behind the transport is simple after accepting the connection
or connecting to a remote server it enters PROXYv2 handling mode: that
is, it either attempts to read (when accepting the connection) or send
(when establishing a connection) a PROXYv2 header. After that it works
like a mere wrapper on top of the underlying stream-based
transport (TCP).
The commit adds a fairly comprehensive unit test suite for our new
PROXYv2 handling code. The unit tests suite ensures both the
correctness of the code and ensures that the part responsible for
handling incoming headers is very strict regarding what to accept as
valid.
This commit adds a set of utilities for dealing with PROXYv2 headers,
both parsing and generating them. The code has no dependencies from
the networking code and is (for the most part) a "separate library".
The part responsible for handling incoming PROXYv2 headers is
structured as a state machine which accepts data as input and calls a
callback to notify the upper-level code about the data processing
status.
Such a design, among other things, makes it easy to write a thorough
unit test suite for that, as there are fewer dependencies as well as
will not stand in the way of any changes in the networking code.
Following the discontinuation of the OpenBSD system test in CI, transfer
the execution of the system test using the "make check" method to the
Debian "sid" system test CI job.
The system tests on OpenBSD consistently exhibit lower stability
compared to our other CI platforms. Some of these challenges are
intrinsic to the system test itself and require attention. However,
there are OpenBSD issues, which seem to be more widespread on this
platform than others. In our daily CI pipelines, OpenBSD system tests
often bear the brunt of all failed CI jobs.
It's possible that our OpenBSD CI image could be optimized, but we
currently lack the domain-specific knowledge needed to make
improvements.
BIND 9 will now treat the response as insecure when processing NSEC3
records with iterations larger than 50.
Earlier, we limited the number of iterations to 150 (in #2445).
RFC 9276 says: Because there has been a large growth of open (public)
DNSSEC validating resolvers that are subject to compute resource
constraints when handling requests from anonymous clients, this
document recommends that validating resolvers reduce their iteration
count limits over time. Specifically, validating resolver operators and
validating resolver software implementers are encouraged to continue
evaluating NSEC3 iteration count deployment trends and lower their
acceptable iteration limits over time.
After evaluation, we decided that the next major BIND release should
lower the maximum allowed NSEC3 iterations to 50, which should be
fine for 99,87% of the domain names.
The system tests need to be updated because non-zero iterations are no
longer accepted.
The autosign system test changes its iterations from 1 to 0 in one
test case. This requires the hash to be updated.
The checkconf system test needs to change the iterations in the good
configuration files to 0, and in the bad ones to 1 (any non-zero value
would suffice, but we test the corner case here). Also, the expected
failure message is change, so needs to be adjusted.
The nsec3 system test also needs iteration configuration adjustments.
In addition, the test script no longer needs the ITERATIONS environment
variable.
In the process of updating the system tests, I noticed an error
in the dnssec-policy "nsec3-other", where the salt length in one
configuration file is different than in the other (they need to be
the same). Furthermore, the 'rndc signing -nsec3param' test case
is operated on the zone 'nsec-change.kasp', so is moved so that the
tests on the same zone are grouped together.
Create a utility package for code shared by the python tests. The
utility functions should use reasonable defaults and be split up into
modules according to their functionality.
Ensure assert rewriting is enabled for the modules to get the most
useful output from pytest.
By default, the useful assertion message rewrite is used by pytest for
test modules only. Since another module is imported with shared
functionality, ensure it has pytest's assertion message rewriting
enabled to obtain more debug information in case it fails.
This file is executed outside of pytest with pure python, which doesn't
do any AssertionError message rewriting like pytest. Ensure the assert
messages in this file provide a useful debug message.
Resolve "Crash on shutdown when DNSSEC validation is running: ENSURE(isc_mempool_getallocated(*namepoolp) == 0) failed"
Closes#4462 and #4384
See merge request isc-projects/bind9!8526
With shared name memory pools (f5af981831)
the message needs to be destroyed before the view is detached which
in turn detaches the resolver which checks that all resources have
been returned.
When all the servers are exhausted for UDP setup emit "no servers
could be reached" in udp_ready(). This message can also be emitted
for a recv_done() error and for TCP connection failure similarly.
Rewrite and reorganize the test documentation to focus on the pytest
runner, omit any mentions of the legacy runner which are no longer
relevant, and mention a few pytest tricks.
The new unit isc_mem_overmem unit test sets hi and lo water marks and
then does allocations to go over:
0. x < lo_water
1. lo_water < x < hi_water
2. x > hi_water
3. lo_water < x < hi_water
4. < lo_water
Previously, there were two methods of working with the overmem
condition:
1. hi/lo water callback - when the overmem condition was reached
for the first time, the water callback was called with HIWATER
mark and .is_overmem boolean was set internally. Similarly,
when the used memory went below the lo water mark, the water
callback would be called with LOWATER mark and .is_overmem
was reset. This check would be called **every** time memory
was allocated or freed.
2. isc_mem_isovermem() - a simple getter for the internal
.is_overmem flag
This commit refactors removes the first method and move the hi/lo water
checks to the isc_mem_isovermem() function, thus we now have only a
single method of checking overmem condition and the check for hi/lo
water is removed from the hot path for memory contexts that doesn't use
overmem checks.
The ns2/named-alt3.conf.in config file was removed in
f8e264ba6d. From then on, system test
reports:
sed: can't read ns2/named-alt3.conf.in: No such file or directory"
Drop the last remnant of ns2/named-alt3.conf.in.
The definitions of the "ci-variables" and "cross-version-config-tests"
GitLab CI jobs were accidentally added in the .gitlab-ci.yml section
that claims to only contain job templates. Move the definitions of
these two jobs to a more appropriate location in .gitlab-ci.yml, without
changing the job definitions themselves.
All currently supported BIND 9 branches use xz-packed tarballs for
source code distribution. Having a variable with a lengthy name that
only holds two characters does not improve readability - it was only
useful for maintaining .gitlab-ci.yml consistency between BIND 9.11 and
all the newer branches, but that era has come to an end a while ago.
Replace all occurrences of the TARBALL_EXTENSION variable in
.gitlab-ci.yml with a fixed string ("xz") to simplify the contents of
that file.
It was possible that controlconf connections could be shutdown twice
when shutting down the server, because they would receive the
signal (ISC_R_SHUTTINGDOWN result) from netmgr and then the shutdown
procedure would be called second time via controls_shutdown().
Split the shutdown procedure from control_recvmessage(), so we can call
it independently from netmgr callbacks and make sure it will be called
only once. Do the similar thing for the listeners.
The post-mortem meeting is now considered an on-demand event. The past
few security release cycles proved that there is rarely a need to
discuss things in this form, so there is little point in carrying out
the relevant steps for every single vulnerability - which does not
prevent us from doing so if the actual need arises.
Applying version bumps in open source branches breaks automatic rebasing
of the bind-9.x-sub branches. Ensure the latter are manually rebased
after each version bump to prevent the "rebase" job in GitLab CI from
failing.
The patches/ subdirectory needs to be present in each prerelease
directory before the ASN releases get pre-published or else the latter
will not contain standalone patches.
when transferring in a non-inline-signing secondary for the first time,
we previously never set the value of zone->loadtime, so it remained
zero. this caused a test failure in the statschannel system test,
and that test case was temporarily disabled. the value is now set
correctly and the test case has been reinstated.
The maximum DNS message size is 65535 octets. Check that the buffer
being passed to dns_message_renderbegin does not exceed this as the
compression code assumes that all offsets are no bigger than this.
The AES algorithm for DNS cookies was being kept for legacy reasons, and
it can be safely removed in the next major release. Remove both the AES
usage for DNS cookies and the AES implementation itself.
Please see the 998765fea5 commit for
the description of the original issue. The commit had fixed the
logic error, but it was reintroduced again later with the
a1afa31a5a commit, where the check of
the 'db_registered' flag was removed in dns__catz_update_cb(). The
check was removed, because the registration function was made
idempotent, so double registration is not an issue, but the check
also prevented from unneeded registration, on which the original
fix relied.
This commit just removes the update callback registration code from
the dns__catz_update_cb() function instead of bringing back the check,
because after code flow analysis, it is now clear that it's not required
at all. The "call onupdate() artificially" comment (which was mentioned
by the removed code) is speaking about the dns_catz_dbupdate_callback()
function, which is called by server.c on (re)configuration, and that
function already takes care of update callback's registration since the
998765fea5 commit was applied, so there
is no need to do that here again.
Concurrent threads can access a hashmap for reading by creating and
then destroying an iterator, in which case the integer number of the
active iterators is increased or decreased from different threads,
introducing a data race. Use atomic operations to protect the variable.
The AES algorithm for DNS cookies was being kept for legacy reasons,
and it can be safely removed in the next major release. Mark is as
deprecated, so the `named-checkconf` prints a warning when in use.
The 'dns_tsigkeyring_t' structure has a read/write lock to protect
its 'keys' member, which is a 'isc_hashmap_t' pointer and needs to
be protected.
The dns_tsigkeyring_dump() function, however, doesn't use the lock,
which can introduce a race with another thread, if the other thread
tries to modify the hashmap.
Add a read lock around the code, which iterates over the hashmap.
Not every element tagged `skipped` in the JUnitXML tree has to contain
the `type` attribute. An example of that is a test that results in
xpass.
This has been verified with pytest version 7.4.2 and prior.
Add a GitLab CI job that is only run for tags and makes signing BIND 9
releases more convenient by utilizing a signing VM that is registered as
a GitLab CI runner. This pulls the signing process into the release
pipelines in GitLab CI, resulting in job artifacts containing the
signatures for BIND 9 releases, which in turns simplifies the subsequent
release publication steps.
EXPERIMENT: when DNS_DB_GLUEOK is set, dns_view_find() will now return
glue if it is found it a local zone database, without checking to see
if a better answer has been cached previously.
If a child zone is served by the same servers as a parent zone and
a NS query is made for the zone name then the addresses of the
nameservers are returned in the additional section are tagged as
trust additional.
Commit 4db150437e incorrectly removed the
call to isc_mem_setwater() from dns_cache_setcachesize(). The water()
function is a no-op, but we still need to set high- and low-water marks
in the memory context, otherwise overmem conditions will not be
detected.
Run comparative performance tests against the latest released version of
the same branch. This is done for different protocols with an
appropriate load the server is expected to be able to handle.
Currently, the results need to be inspected manually, since a success of
the job doesn't indicate there is no issue. Instead, the job provides an
URL to an overview with latency, memory and CPU charts which display the
test results with the current code against the reference version. There
should be no major unexplained and reproducible differences in the
charts.
The shotgun performance tests are executed in a different repository, in
a couple of different pipelines. To hide away the complexity, this
script takes the pipeline ID of the triggered pipeline and then takes
care of the rest - waits for the pipeline to finish, locates the child
pipeline and the relevant results. The output from this script is a
convenient link to the charts with the results once they're available.
GitLab also has a mechanism which can wait for another pipeline.
However, it can't be utilized here, since there are variables which
need to be passed in when the pipeline is triggered (like protocol to be
tested, load, runtime etc.). This isn't currently supported by the
GitLab feature.
Multiple CI jobs may utilize a baseline version, i.e. the version that
the current code should be tested against when doing comparative
testing. To avoid repeating the non-trivial detection of the baseline
version, move it into a separate job which creates an environment file
that subsequent jobs may require via `needs` option. It is then possible
to use the variable(s) defined in the script section of the new job.
When serve-stale is enabled and recursive resolution fails, the fallback
to lookup stale data always happens in the cache database. Any
authoritative data is ignored, and only information learned through
recursive resolution is examined.
If there is data in the cache that could lead to an answer, and this can
be just the root delegation, the resolver will iterate further, getting
closer to the answer that can be found by recursing down the root, and
eventually puts the final response in the cache.
Change the fallback to serve-stale to use 'query_getdb()', that finds
out the best matching database for the given query.
Add a test case where serve-stale is enabled on a server that also
servers a local authoritative zone.
The particular case tests a lame delegation and checks if falling
back to serving stale data does not attempt to retrieve the query
by recursing from the root down.
Increasing the initial and freemax sizes for dns_message memory pools
restores the root zone performance. The former sizes were suited for
per-dns_message memory pools and we need to bump the sizes up for
per-thread memory pools.
Commit 42d43aa075 made --with-liburcu
depend on --enable-developer. This broke pairwise testing as this new
dependency was not codified in configure.ac. Since the --with-liburcu
option is currently just a convenience for developers, there is no need
to test building against all possible RCU variants in GitLab CI until
they actually work with BIND 9. Update the pairwise testing
"configuration" in configure.ac so that builds with non-standard RCU
variants are not tested.
The client connection timeout was set to just one second, which might
not be enough on busy systems (and the CI machines are oh-boy-busy).
Bump the server timeouts to 10 seconds and client timeouts to 5 seconds,
this will make the unit test run a little bit longer, but it should be
more reliable.
Previously, the isccc_ccmsg_invalidate() was called from conn_free() and
this could lead to netmgr calling control_recvmessage() after we
detached the reading controlconnection_t reference, but it wouldn't be
the last reference because controlconnection_t is also attached/detached
when sending response or running command asynchronously.
Instead, move the isccc_ccmsg_invalidate() call to control_recvmessage()
error handling path to make sure that control_recvmessage() won't be
ever called again from the netmgr.
The controlconf channel runs single-threaded on the main thread.
Replace the listener->connections locking with check that we are still
running on the thread with TID 0.
The lock-file configuration (both from configuration file and -X
argument to named) has better alternatives nowadays. Modern process
supervisor should be used to ensure that a single named process is
running on a given configuration.
Alternatively, it's possible to wrap the named with flock(1).
This is first step in removing the lock-file configuration option, it
marks both the `lock-file` configuration directive and -X option to
named as deprecated.
When -X is used the 'lock-file' option change detection condition
is invalid, because it compares the 'lock-file' option's value to
the '-X' argument's value instead of the older 'lock-file' option
value (which was ignored because of '-X').
Don't warn about changing 'lock-file' option if '-X' is used.
It is obvious that the '!cfg_obj_asstring(obj)' check should be
'cfg_obj_asstring(obj)' instead, because it is an AND logic chain
which further uses 'obj' as a string.
Fix the error.
When 'lock-file <lockfile>' is used in configuration at the same time
as using '-X none' in 'named' invocation, there is an invalid
logic that would lead to a isc_mem_strdup() call on a NULL value.
Also, contradicting to ARM, 'lock-file none' is overriding the '-X'
argument.
Fix the overall logic, and make sure that the '-X' takes precedence to
'lock-file'.
When 'lock-file <lockfile1>' was used in configuration at the same time
as using `-X <lockfile2>` in `named` invocation, there was an invalid
logic that would lead to a double isc_mem_strdup() call on the
<lockfile2> value.
Skip the second allocation if `lock-file` is being used in
configuration, so the <lockfile2> is used only single time.
All changes in this commit were automated using the command:
shfmt -w -i 2 -ci -bn . $(find . -name "*.sh.in")
By default, only *.sh and files without extension are checked, so
*.sh.in files have to be added additionally. (See mvdan/sh#944)
The Userspace-RCU variants other than membarrier is untested and at
least in QSBR case it's broken. Allow changing the Userspace-RCU
variant only in the developer's mode.
The `.reader` member of dns_qpmulti_t was accessed without RCU
protection; reader_open() calls rcu_dereference() on it, and this
call needs to be inside an RCU critical section.
A similar problem was identified in the dns_qpmulti_snapshot() - the
RCU critical section was completely missing.
These are relicts of the isc_qsbr - in the QSBR mode the rcu_read_lock()
and rcu_read_unlock() are no-ops and whole event loop is a critical section.
When named fails to starts due to not being able to obtain
a lock on the lock file that lock file should remain. Check
that the lock file exists before and after the attempt to
start a second instance of named.
The lock file was being removed when we hadn't successfully locked
it which defeated the purpose of the lockfile. Adjust cleanup_lockfile
such that it only unlinks the lockfile if we have successfully locked
the lockfile and it is still active (lockfile != NULL).
Do a light refactoring and cleanups that replaces common list walking
patterns with ISC_LIST_FOREACH macros and split some nested loops into
separate static functions to reduce the nesting depth.
Add complementary macros to ISC_LIST_FOREACH(_SAFE) that walk the lists
in reverse.
* ISC_LIST_FOREACH_REV(list, elt, link) - walk the static list from
tail to head
* ISC_LIST_FOREACH_REV_SAFE(list, elt, link, next) - walk the list
from tail to head in a manner that's safe against list member
deletions
There was a lot of internal code looking like this:
INSIST(dns_rdataset_isassociated(rdataset));
dns_rdataset_disassociated(rdataset)
isc_mempool_put(msg->rdspool, rdataset);
Deduplicate the code into local dns__message_puttemprdataset() routine,
and drop the INSIST() which is checked in dns_rdataset_disassociate().
Refactor the dispatch unit test to use more local variables (previously
dispatchmgr, dispatch and dispentry were all global), and add two new
tests:
* dispatch_getcp - test whether the TCP connection will get reused
* dispatch_newtcp - test that the TCP connection will not get reused
when DNS_DISPATCHOPT_UNSHARED is in effect
The current dispatch code could reuse the TCP connection when
dns_dispatch_gettcp() would be used first. This is problematic as the
dns_resolver doesn't use TCP connection sharing, but dns_request could
get the TCP stream that was created outside of the dns_request.
Add new DNS_DISPATCHOPT_UNSHARED option to dns_dispatch_createtcp() that
would prevent the TCP stream to be reused. Use that option in the
dns_resolver call to dns_dispatch_createtcp() to prevent dns_request
from reusing the TCP connections created by dns_resolver.
Additionally, the dns_xfrin unit added TCP connection sharing for
incoming transfers. While interleaving *xfr streams on a TCP connection
should work this should be a deliberate change and be property of the
server that can be controlled. Additionally some level of parallel TCP
streams is desirable. Revert to the old behaviour by removing the
dns_dispatch_gettcp() calls from dns_xfrin and use the new option to
prevent from sharing the transfer streams with dns_request.
The xfrin_recv_done() was accessing xfr->result where we stored the
result of the offloaded work from a thread that could receive data while
processing the transfer on the offloaded thread.
Completely remove the offloaded result from the dns_xfrin_t structure
and keep it local for *xfr_apply() and *xfr_apply_done() as the failure
is already recorded in .shutdown_result and we now that the processing
has failed because .shuttingdown has been already set.
When using sd_notify(3) to send a message to the service manager
about named being reloaded, systemd also requires the MONOTONIC_USEC
field to be set to the current monotonic time in microseconds,
otherwise the 'systemctl reload' command fails.
Add the MONOTONIC_USEC field to the message.
See 'man 5 systemd.service' for more information.
The dns__catz_update_cb() does not expect that 'catzs->zones'
can become NULL during shutdown.
Add similar checks in the dns__catz_update_cb() and dns_catz_zone_get()
functions to protect from such a case. Also add an INSIST in the
dns_catz_zone_add() function to explicitly state that such a case
is not expected there, because that function is called only during a
reconfiguration.
We test EDNS requests returning FORMERR where named is expected
to retry without EDNS.
We test EDNS requests returning NOTIMP where named is expected
to fail the transfer as the remote end is not protocol compliant.
The server is expected to retry the transfer using SOA and if
the returned serial is greater than the current serial AXFR.
Check the log that IXFR is request.
To reduce the amount of log spam when root servers change their
addresses keep a table of upcoming changes by expected date and time
and suppress reporting differences for them until then.
Add initial entry for B.ROOT-SERVERS.NET, Nov 27, 2023.
Instead of processing received data synchronously, store the incoming
differences in the list and process them asynchronously when we need to
commit the data into the database and/or journal.
Instead of locking the struct dns_xfrin members that get accessed from
the statistics, convert those into atomic types and use atomic accesses
to prevent ThreadSanitizer from blowing up.
In fact, even the atomic operations are not really needed here, because
all writes are done from a single thread and we don't really require
consistency from the statistics. It's easier to use atomics here, but
it is slightly confusing as it suggests there might be multithreaded
accesses to those variables while in fact, the only off-thread access
happens when collecting the statistics.
The ixfr_putdata() and axfr_putdata() had a logic to apply dns_diff when
the number of pending tuples went over 100. Since we are going to
offload the XFR data processing, we don't need to do that anymore.
OpenSSL 1.1 has already reached end-of-life and since we are
experiencing a weird memory leak in the mirror system test on just
Ubuntu 20.04 (Focal) with OpenSSL 1.1, we disable the legacy code for
enabling memory contexts for OpenSSL < 3.0.0 in this commit.
In order to check whether there are enough inserted values the
code uses the 'tests' variable (loop counter), which is unreliable,
because the loop sometimes removes an item instead of inserting
one (when the randomly generated item already exists).
Instead of the loop counter, use the existing variable 'inserted',
which should indicate the correct number of the inserted items.
Drop timeout before resending a UDP request from 15 seconds to 5
seconds and add 1 second to the total time to allow for the reply
to the third request to arrive. This will speed up the time it
takes for named to recover from a lost packet when refreshing a
zone and for it to determine that a primary is down.
Now that inline-signing is ignored when there is no dnssec-policy,
add 'dnssec-policy default;' to the zones when attempting to add them
via 'rndc addzone'.
Add the missing documentation for 'dnssec-policy/inline-signing'.
Update the zone-only option 'inline-signing' to indicate that the
use of inline signing should be set in 'dnssec-policy' and that this
is merely a way to override the value for the given zone.
(cherry picked from commit 2b7381950d17fe4d289959e5f76f020cc462200a)
Ignore the option 'inline-signing' unless there is a 'dnssec-policy'
configured for the zone. Having inline signing enabled while the zone
is not DNSSEC signed does not make sense.
If there is a 'dnssec-policy' the 'inline-signing' zone-only option
can be used to override the value for the given zone.
The 'dynamic-signed-inline-signing.kasp' zone was set up with
the environment variable 'ksktimes', but that should be 'csktimes'
which is set one line above. Since the values are currently the same
the behavior is identical, but of course it should use the correct
variable.
The 'step4.enable-dnssec.autosign' zone was set up twice. This is
unnecessary.
Update the function 'set_resigntime()' so that raw versions of
inline-signing zones are not scheduled to be resigned.
Also update the check in the same function for zone is dynamic, there
exists a function 'dns_zone_isdynamic()' that does a similar thing
and is more complete.
Also in 'zone_postload()' check whether the zone is not the raw
version of an inline-signing zone, preventing calculating the next
resign time.
Add a test scenario for a dynamic zone that uses inline-signing which
accidentally has signed the raw version of the zone.
This should not trigger resign scheduling on the raw version of the
zone.
The dns_aclenv_t contains two dns_acl_t - localhost and localnets that
can be swapped with a different ACLs as we configure BIND 9. Instead of
protecting those two pointers with heavyweight read-write lock, use RCU
mechanism to dereference and swap the pointers.
In units that support detailed reference tracing via ISC_REFCOUNT
macros, we were doing:
/* Define to 1 for detailed reference tracing */
#undef <unit>_TRACE
This would prevent using -D<unit>_TRACE=1 in the CFLAGS.
Convert the above mentioned snippet with just a comment how to enable
the detailed reference tracing:
/* Add -D<unit>_TRACE=1 to CFLAGS for detailed reference tracing */
Basically all local data is considered trusted, and proper ACLs and
limits need to be explicitly configured. We are also free to let
protocol non-compliant servers burn in flames.
It was possible to reach add_link() without visiting an
intermediate node first, and the check for a duplicate entry
could then cause a crash.
Credit to OSS-Fuzz for discovering this error.
Read the Docs is capable of building the PDF version of the BIND 9 ARM
using just the contents of the doc/arm/ directory - it does not need the
build system to facilitate that. Since the BIND 9 ARM is also built in
other formats when "make doc" is run, drop the parts of the build system
that enable building the PDF version as they pull in complexity without
bringing much added value in return. Update related files accordingly.
There was a microoptimization for smoothing srtt with bitshifts. Revert
the code to use * 98 / 100, it doesn't really make that difference on
modern CPUs, for comparison here:
muldiv:
imul eax, edi, 98
imul rax, rax, 1374389535
shr rax, 37
ret
shift:
mov eax, edi
sal eax, 9
sub eax, edi
shr eax, 9
ret
If factor == DNS_ADB_RTTADJAGE and addr->entry->lastage == now we would
load value into new_srtt and then immediatelly store it back which
triggers the synchronization between threads using .srtt values.
Use atomics on couple of ADB entry members (.srtt, .flags, .expires, and
.lastage) to remove ADB entry locking from couple of hot spots. The
most prominent place is copy_namehook_lists() that gets called under ADB
name lock and if the namehook list is long it acquires-releases quite a
few ADB entry locks. Changing those ADB entry members to atomics
allowed us to new_adbaddrinfo() not require locked ADB entry and since
adbentry_overquota() already used atomics and handling lame information
was dropped in the previous commit, we could not make the
copy_namehook_lists() lockless.
The other hotspot is dns_adb_adjustsrtt() and dns_adb_agesrtt() that can
now use atomics because .srtt is already atomic_uint.
And the last place that could now use atomics is dns_adb_changeflags().
Keeping the information about lame server in the ADB was done in !322 to
fix following security issue:
[CVE-2021-25219] Disable "lame-ttl" cache
The handling of the lame servers needs to be redesigned and it is not
going to be enabled any time soon, and the current code is just dead
code that takes up space, code and stands in the way of making ADB work
faster.
Remove all the internals needed for handling the lame servers in the ADB
for now. It might get reintroduced later if and when we redesign ADB.
The usage of the newline in the replacement part of the 'sed' call
works in GNU systems, but not in OpenBSD. Use 'awk' instead.
Also use the extended syntax of regular expressions for 'grep', which
is similarly more portable across the supported systems.
now that we're using qpmulti for the summary database, we
no longer need to hold search_lock for it. we do still need
it for the radix tree and the trigger counts.
now that we have the QP chain mechanism, we can convert the
RPZ summary database to use a QP trie instead of an RBT.
also revised comments throughout the file accordingly, and
incidentally cleaned up calls to new_node(), which can no
longer fail.
depending on how the QP trie is traversed during a lookup, it is
possible for a search to terminate on a leaf which is a partial
match, without that leaf being added to the chain. to ensure the
chain is correct in this case, when a partial match condition is
detected via qpkey_compare(), we will call add_link() again, just
in case. (add_link() will check for a duplicated node, so it will
be harmless if it was already done.)
This was generated from dnsperf queryfile with following script:
#!/usr/bin/env python3
names = {}
import sys
i = 0
for line in iter(sys.stdin.readline, ''):
name = line.rstrip('\n')
if not name in names:
names[name] = line
print(f"{i},{name}")
i += 1
if i >= 1024*1024:
break
The name_match() was errorneously converting struct item into dns_name
pointer. Correctly retype void *node to struct item * first and then
use item.fixed.name to pass the name to dns_name_equal() function.
The load_names benchmark expected the input CSV with domains would fill
the whole item array and it would crash when the number of lines would
be less than that.
Fix the expectations by using the real number or lines read to calculate
the array start and end position for each benchmark thread.
The autoscaling GitLab CI runners currently used for most GitLab CI jobs
spin up AWS EC2 instances that are at least as powerful as the dedicated
instances used for running "stress" tests. Move all Linux-based
"stress" tests to autoscaling GitLab CI runners to enable deprovisioning
Linux AWS instances reserved for running "stress" tests. Leave FreeBSD
"stress" tests intact as there is currently no support for autoscaling
BSD instances.
At times, a problem might occur where a test is not responding,
especially in the CI, determining the specific test responsible can be
difficult. Fortunately, when running tests with the pytest runner,
pytest sets the PYTEST_CURRENT_TEST environment variable to the current
test nodeid and stage. Afterward, the variable can be examined to
identify the test that has stopped responding.
The monitoring script needs to be started in the background. Still, the
shell executor used for BSD and FIPS testing can't handle the background
process cleanly, and the script step will wait for the background
process for the entire duration of the background process (currently
3000 seconds). Therefore, run the monitoring script only when the Docker
executor is used where this is not a problem.
All these pointers are guaranteed to be non-NULL.
Additionally, update a comment to remove obviously outdated
information about the function's requirements.
Move the block on the error path, where the link is checked, to a place
where it makes sense, to avoid accessing an unitialized link when
jumping to the 'cleanup_query' label from 4 different places. The link
is initialized only after those jumps happen.
In addition, initilize the link when creating the object, to avoid
similar errors.
dns_qp_findname_ancestor() now takes an optional 'predecessor'
parameter, which if non-NULL is updated to contain the DNSSEC
predecessor of the name searched for. this is done by constructing
an iterator stack while carrying out the search, so it can be used
to step backward if needed.
since dns_qp_findname_ancestor() can now return a chain object, it is no
longer necessary to provide a _NOEXACT search option. if we want to look
up the closest ancestor of a name, we can just do a normal search, and
if successful, retrieve the second-to-last node from the QP chain.
this makes ancestor lookups slightly more complicated for the caller,
but allows us to simplify the code in dns_qp_findname_ancestor(), making
it easier to ensure correctness. this was a fairly rare use case:
outside of unit tests, DNS_QPFIND_NOEXACT was only used in the zone
table, which has now been updated to use the QP chain. the equivalent
RBT feature is only used by the resolver for cache lookups of 'atparent'
types (i.e, DS records).
- make iterators reversible: refactor dns_qpiter_next() and add a new
dns_qpiter_prev() function to support iterating both forwards and
backwards through a QP trie.
- added a 'name' parameter to dns_qpiter_next() (as well as _prev())
to make it easier to retrieve the nodename while iterating, without
having to construct it from pointer value data.
- the helper functions for accessing twigs beneath a branch
(branch_twig_pos(), branch_twig_ptr(), etc) were somewhat confusing
to read, since several of them were implemented by calling other
helper functions. they now all show what they're really doing.
- branch_twigs_vector() has been renamed to simply branch_twigs().
- revised some unrelated comments in qp_p.h for clarity.
dns_qp_findname_ancestor() now takes an optional 'chain' parameter;
if set, the dns_qpchain object it points to will be updated with an
array of pointers to the populated nodes between the tree root and the
requested name. the number of nodes in the chain can then be accessed
using dns_qpchain_length() and the individual nodes using
dns_qpchain_node().
add a 'foundname' parameter to dns_qp_findname_ancestor(),
and use it to set the found name in dns_nametree.
this required adding a dns_qpkey_toname() function; that was
done by moving qp_test_keytoname() from the test library to qp.c.
added some more test cases and fixed bugs with the handling of
relative and empty names.
this loads a file containing DNS names and measures the time it takes to:
1) iterate it,
2) look up each name with dns_qp_getname()
3) look up each name with dns_qp_findname_ancestor()
4) look up a modified name based on the name, to check performance
when the name is not found.
the refactoring of isc_job_run() and isc_async_run() in 9.19.12
intefered with the way the qpmulti benchmark uses uv_idle.
it has now been modified to use isc_job/isc_async instead.
When the given zone is not associated with a zone manager, the function
currently returns ISC_R_NOTFOUND, which is documented as the return
value for the case in which no incoming zone transfer is found. Make
the function return ISC_R_FAILURE in such a case instead.
Also update the description of the function as the value it returns is
not meant to indicate whether an ongoing incoming transfer for the given
zone exists. The boolean variables that the function sets via the
pointers provided as its parameters, combined with either keeping
'*xfrp' set to NULL or updating it to a valid pointer, can be used by
the caller to infer all the necessary information.
The TRY0 macro doesn't set the 'result' variable, so the error
log message is never printed. Remove the 'result' variable and
modify the function's control flow to be similar to the the
zone_xmlrender() function, with a separate error returning path.
The .free_pools bitfield would not be set on some levels of
optimizations - workaround the compiler bug by reordering the setting
the .freepools in the initializer.
The structure member is populated only moments before its
destruction, and is not used anywhere, except for the
destructor. Use a local variable instead.
The 'end_serial' and some other members of the 'dns_xfrin_t'
structure can be accessed by the statistics channel, causing
a data race with the zone transfer process.
Use the existing 'statslock' mutex for protecting those members.
'rndc thaw' initiates asynchrous loading of all the zones
similar to 'rndc load'. Wait for the test zone's load to
complete before testing that it is updatable again.
Instead of creating new memory pools for each new dns_message, change
dns_message_create() method to optionally accept externally created
dns_fixedname_t and dns_rdataset_t memory pools. This allows us to
preallocate the memory pools in ns_client and dns_resolver units for the
lifetime of dns_resolver_t and ns_clientmgr_t.
The "Needs Refresh" flag is exposed in two places in the statistics
channel: first - there is a state called "Needs Refresh", when the
process hasn't started yet, but the zone needs a refresh, and second
- there there is a field called "Additional Refresh Queued", when the
process is ongoing, but another refresh is queued for the same zone.
The DNS_ZONEFLG_NEEDREFRESH flag, however, is set only when there is
an ongoing zone transfer and a new notify is received. That is, the
flag is not set for the first case above.
In order to fix the issue, use the DNS_ZONEFLG_NEEDREFRESH flag only
when the zone transfer is running, otherwise, decide whether a zone
needs a refresh using its refresh and expire times.
The XFRST_INITIALSOA state in the xfrin module is named like that,
because the first RR in a zone transfer must be SOA. However, the
name of the state is a bit confusing (especially when exposed to
the users with statistics channel), because it can be mistaken with
the refresh SOA request step, which takes place before the zone
transfer starts.
Rename the state to XFRST_ZONEXFRREQUEST (i.e. Zone Transfer Request).
During that step the state machine performs several operations -
establishing a connection, sending a request, and receiving/parsing
the first RR in the answer.
Currently in the statsistics channel's incoming zone transfers list
the local and remote addresses are shown only when the zone transfer
is already running. Since we have now introduced the "Refresh SOA"
state, which shows the state of the SOA query before the zone transfer
is started, this commit implements a feature to show the local and
remote addresses for the SOA query, when the state is "Refresh SOA".
Improve the "Duration (s)" field, so that it can show the duration of
all the major states of an incoming zone transfer process, while they
are taking place. In particular, it will now show the duration of the
"Pending", "Refresh SOA" and "Deferred" states too, before the actual
zone transfer starts.
With adding this state to the statistics channel, it can now show
the zone transfer in this state instead of as "Pending" when the
zone.c module is performing a refresh SOA request, before actually
starting the transfer process. This will help to understand
whether the process is waiting because of the rate limiter (i.e.
"Pending"), or the rate limiter is passed and it is now waiting for
the refresh SOA query to complete or time out.
Add two more secondary zones to ns3 to be transferred from ns1,
using its IPv6 address for which the 'tcp-only' is set to 'yes'.
Check the statistics channel's incoming zone transfers information
to confirm that the expected transports were used for each of the
SOA query cases (UDP, TCP, TLS), and also for zone transfers (TCP,
TLS).
Add a new field in the incoming zone transfers section of the
statistics channel to show the transport used for the SOA request.
When the transfer is started beginning from the XFRST_SOAQUERY state,
it means that the SOA query will be performed by xfrin itself, using
the same transport. Otherwise, it means that the SOA query was already
performed by other means (e.g. by zone.c:soa_query()), and, in that
case, we use the SOA query transport type information passed by the
'soa_transport_type' argument, when the xfrin object was created.
This allows the statistics channel to be viewed in a browser while
the transfer is in progress. Also set the transfer format to
one-answer to extend the amount of time the re-transfer takes.
When running the statschannel test on its own, use
<http://10.53.0.3:5304/xml/v3/xfrins> to see the output.
Note: the port is subject to future change.
dns_xfrin_t state may be accessed from different threads when
when reporting transfer state. Ensure access is thread safe by
using atomics and locks where appropriate.
Use the named -T transferslowly test options to slow down a zone
transfer from the primary server, and test that it's correctly
exposed in the statistics channel of the secondary server, while
it's in-progress.
In some dns_transport getter functions it's possible to use a
const dns_transport_t as the first argument instead of just
dns_transport_t. Convert the function prototypes to use const.
Apply the semantic patch to catch all the places where we pass 'char' to
the <ctype.h> family of functions (isalpha() and friends, toupper(),
tolower()).
Add a semantic patch to catch all the places where we pass 'char' to the
<ctype.h> family of functions (isalpha() and friends, toupper(),
tolower()). While it generally works because the way how these
functions are constructed in the libc, it's safer to do the explicit
cast.
The new :cve: Sphinx role takes a CVE number as an argument and creates
a hyperlink to the relevant ISC Knowledgebase document that might have
more up-to-date or verbose information than the relevant release note.
This makes reaching ISC Knowledgebase pages directly from the release
notes easier.
Make all CVE references in the release notes use the new Sphinx role.
The QID table hashing used a custom merging of the sockaddr, port and id
into a single hashvalue. Normalize the QID table hashing function to
use isc_hash32 API for all the values.
Instead of copying address back and forth when hashing addr+port, we can
use incremental hashing. Additionally, switch from 64-bit
isc_hash_function to 32-bit isc_hash32() as the resulting value is
32-bit.
The Unix Domain Sockets support in BIND 9 has been completely disabled
since BIND 9.18 and it has been a fatal error since then. Cleanup the
code and the documentation that suggest that Unix Domain Sockets are
supported.
The previous symlink name convention was prone to name collisions If a
system test contained both a shell test and a pytest module of the same
name (e.g. dnstap test has both tests.sh and tests_dnstap.py), then
these would have the same convenience symlink, which could cause test
setup issues as well as confusion when examining test artifacts.
Update the naming convention to include the full pytest module name.
This results in a slightly more verbose names for shell tests (e.g.
dnstap_sh_dnstap instead of the previous dnstap_dnstap), but it removes
the chance of a collision.
Reorganize individual port fixtures and re-use the ports fixture to
obtain their number. Store it as integer and only cast it to string when
setting it as environment variable.
Remove code fork for legacy runner, reorganize imports and move a
pylint-silencing snippet to the top of the file. The rest of the code
was just unindented.
In order to python system tests, pytest (runner) has to be used
directly. This makes it possible to simplify the pytest runner and make
its behavior simpler and easier to extend.
The legacy runner can still be used to run shell system tests.
The legacy runner no longer uses make check. Ensure the legacy runner
script doesn't interact with that automake target in any way. The legacy
runner script remains available to execute the legacy runner, but there
is no out-of-the box support for running tests in parallel. Other tools
such as xargs can be utilized for that.
Invoking pytest directly provides a better formatted output and more
flexibility. However, it's prudent to verify that `make check` keeps
working as expected. Use it in the OpenBSD job which isn't executed as
frequently and its output is of least concern.
Pytest provides JUnit output and uses different exit codes from
Automake. Use the conversion script to interpret the JUnit test results
from python rather than relying on the status code.
It's important to parse the JUnit result file rather than relying on the
exit code from pytest, which has a different meaning. Include a .trs test
result for each test case and set an exit code which is most appropriate
as the aggregate result (e.g. it will be set to 77 (SKIP) if there's at
least one test case that was skipped).
Dependencies for these tests are already checked in prereq.sh - if the
dependencies are missing, these tests will be skipped. The extra
dependency check in Makefile.am is extraneous and only applied for the
legacy test runner.
To allow concurrent invocations of pytest, it is necessary to assign
ports properly to avoid conflicts. In order to do that, pytest needs to
know a complete list of all test modules.
When pytest is invoked from run.sh, the current working directory is the
system test directory. To properly detect other tests, the conftest.py
has to look in the bin/tests/system directory, rather than the current
working directory.
When adding a new system test, it might easy to forget to add the
required files for the pytest runner or break a naming convention. Add
danger checks to cover these cases.
To conform with the expected naming convention, the pytest glue file for
the `allow-query` test should use underscore as the word separator in
the python file name: allow-query/tests_sh_allow_query.py
The _common directory is a special case directory which contains shared
files for other system test directories. Make sure it's tracked in git
and not deleted during temporary directory cleanup.
The old name "common" clashes with the convention of system test
directory naming. It appears as a system test directory, but it only
contains helper files.
To reduce confusion and to allow automatic detection of issues with
possibly missing test files, rename the helper directory to "_common".
The leading underscore indicates the directory is different and the its
name can no longer be confused with regular system test directories.
When iterating the table, we can't add new nodes to the hashmap because
we can't assure that we are not adding the new node before the iterator.
This also applies to rehashing - which might be triggered by both
isc_hashmap_add() and isc_hashmap_delete(), but not
isc_hashmap_iter_delcurrent_next().
When isc_hashmap_iter_delcurrent_next calls hashmap_delete_node
nodes from the front of the table could be added to the end of
the table resulting in them being returned twice. Detect when
this is happening and prevent those nodes being returned twice
buy reducing the effective size of the table by one each time
it happens.
Reusing TCP connections with dns_dispatch_gettcp() used linear linked
list to lookup existing outgoing TCP connections that could be reused.
Replace the linked list with per-loop cds_lfht hashtable to speedup the
lookups. We use cds_lfht because it allows non-unique node insertion
that we need to check for dispatches in different connection states.
Instead of high number of dispatches (4 * named_g_udpdisp)[1], make the
dispatches bound to threads and make dns_dispatchset_t create a dispatch
for each thread (event loop).
This required couple of other changes:
1. The dns_dispatch_createudp() must be called on loop, so the isc_tid()
is already initialized - changes to nsupdate and mdig were required.
2. The dns_requestmgr had only a single dispatch per v4 and v6. Instead
of using single dispatch, use dns_dispatchset_t for each protocol -
this is same as dns_resolver.
Looking up unique message ID in the dns_dispatch has been using custom
hash tables. Rewrite the custom hashtable to use cds_lfht API, removing
one extra lock in the cold-cache resolver hot path.
Refactor isc_hashmap to allow custom matching functions. This allows us
to have better tailored keys that don't require fixed uint8_t arrays,
but can be composed of more fields from the stored data structure.
Hi and thanks for filing an issue! It will be read with care by human beings.
It would be a tremendous help if you could follow these steps first:
- [ ] Search the existing issues in GitLab (both open and closed) to see if your report might be a duplicate. We have a large database here and many issues have already been fixed in the latest versions!
- [ ] Make sure this is **not** a support question. If you have specific trouble configuring or debugging your setup, please use the bind-users mailing list: https://lists.isc.org/mailman/listinfo/bind-users
- [ ] You have read and understood the "out in the open" support policy: https://blog.powerdns.com/2016/01/18/open-source-support-out-in-the-open/ . Even though it was written by the PowerDNS folks, we follow it as well!
Before continuing, **please select the appropriate issue template in the drop-down menu above, under the heading _Description_**.
:bulb: **Click [here][checklist_explanations] (internal resource) for general information about the security incident handling process.**
@@ -30,14 +30,14 @@ confidential!
- [ ] [:link:][step_deputy] **(IM)** Pick a Deputy Incident Manager
- [ ] [:link:][step_respond] **(IM)** Respond to the bug reporter
- [ ] [:link:][step_etherpad] **(IM)** Create an Etherpad for post-mortem
- [ ] [:link:][step_public_mrs] **(SwEng)** Ensure there are no public merge requests which inadvertently disclose the issue
- [ ] [:link:][step_assign_cve_id] **(IM)** Assign a CVE identifier
- [ ] [:link:][step_note_cve_info] **(SwEng)** Update this issue with the assigned CVE identifier and the CVSS score
- [ ] [:link:][step_note_cve_info] **(SwEng)** Update this issue with the assigned CVE identifier, the CVSS score, and CWE category
- [ ] [:link:][step_versions_affected] **(SwEng)** Determine the range of product versions affected (including the Subscription Edition)
- [ ] [:link:][step_workarounds] **(SwEng)** Determine whether workarounds for the problem exist
- [ ] [:link:][step_coordinate] **(SwEng)** If necessary, coordinate with other parties
- [ ] [:link:][step_earliest] **(Support)** Prepare and send out "earliest" notifications
- [ ] [:link:][step_earliest_prepare]**(Support)** Prepare "earliest" notification text
- [ ] [:link:][step_earliest_send] **(Support)** Update "earliest" notification ticket in support portal Earliest queue which will notify earliest customers
- [ ] [:link:][step_advisory_mr] **(Support)** Create a merge request for the Security Advisory and include all readily available information in it
- [ ] [:link:][step_reproducer_mr] **(SwEng)** Prepare a private merge request containing a system test reproducing the problem
- [ ] [:link:][step_notify_support] **(SwEng)** Notify Support when a reproducer is ready
@@ -48,53 +48,50 @@ confidential!
- [ ] [:link:][step_backports] **(SwEng)** Prepare backports of the merge request addressing the problem for all affected (and still maintained) branches of a given product
- [ ] [:link:][step_finish_advisory] **(Support)** Finish preparing the Security Advisory
- [ ] [:link:][step_meta_issue] **(QA)** Create (or update) the private issue containing links to fixes & reproducers for all CVEs fixed in a given release cycle
- [ ] [:link:][step_changes] **(QA)** (BIND 9 only) Reserve a block of `CHANGES` placeholders once the complete set of vulnerabilities fixed in a given release cycle is determined
- [ ] [:link:][step_merge_fixes] **(QA)** Merge the CVE fixes in CVE identifier order
- [ ] [:link:][step_patches] **(QA)** Prepare a standalone patch for the last stable release of each affected (and still maintained) product branch
- [ ] [:link:][step_asn_releases] **(QA)** Prepare ASN releases (as outlined in the Release Checklist)
### At T-5
- [ ] [:link:][step_send_asn] **(Support)** Send ASN to eligible customers
- [ ] [:link:][step_preannouncement]**(Support)** (BIND 9 only) Send a pre-announcement email to the *bind-announce* mailing list to alert users that the upcoming release will include security fixes
- [ ] [:link:][step_asn_links]**(Marketing)** (BIND 9 only) Update the BIND -S information document in the support portal with download links to the new versions
- [ ] [:link:][step_asn_send] **(Support)** Notify eligible customers by adding a ticket to the 5 Day queue in RT with the text of the advisory (earliest, and T-5)
- [ ] [:link:][step_preannouncement] **(Marketing)** (BIND 9 only) Send a pre-announcement email to the *bind-announce* mailing list to alert users that the upcoming release will include security fixes
### At T-4
- [ ] [:link:][step_verify_asn] **(Support)** Verify that all ASN-eligible customers have received the notification email
### At T-3
- [ ] [:link:][step_asn_send] **(Support)** Notify eligible customers by adding a ticket to the 3 Day queue in RT with the text of the advisory (T-3)
### At T-1
- [ ] [:link:][step_check_customers] **(Support)** Verify that any new or reinstated customers have received the notification email
- [ ] [:link:][step_packager_emails] **(First IM)** Send notifications to OS packagers
### On the Day of Public Disclosure
- [ ] [:link:][step_clearance] **(IM)** Grant Support clearance to proceed with public release
- [ ] [:link:][step_publish] **(Support)** Publish the releases (as outlined in the release checklist)
- [ ] [:link:][step_clearance] **(IM)** Grant QA & Marketing clearance to proceed with public release
- [ ] [:link:][step_publish] **(QA/Marketing)** Publish the releases (as outlined in the release checklist)
- [ ] [:link:][step_matrix] **(Support)** (BIND 9 only) Add the new CVEs to the vulnerability matrix in the Knowledge Base
- [ ] [:link:][step_publish_advisory] **(Support)** Bump Document Version for the Security Advisory and publish it in the Knowledge Base
- [ ] [:link:][step_notifications] **(First IM)** Send notification emails to third parties
- [ ] [:link:][step_mitre] **(First IM)** Advise MITRE about the disclosed CVEs
**Closed issues assigned to the milestone without a release note:**
- []()
- []()
- []()
**Merge requests merged into the milestone without a release note:**
- []()
- []()
- []()
**Merge requests merged into the milestone without a `CHANGES` entry:**
- []()
- []()
- []()
## Release Checklist
### Before the Code Freeze
- [ ]***(QA)*** Rebase -S editions on top of current open-source versions: `git checkout bind-9.18-sub && git rebase origin/bind-9.18`
- [ ]***(QA)*** [Inform](https://gitlab.isc.org/isc-private/bind-qa/-/blob/master/bind9/releng/inform_supp_marketing.py) Support and Marketing of impending release (and give estimated release dates).
- [ ]***(QA)*** Ensure there are no permanent test failures on any platform. Check [public](https://gitlab.isc.org/isc-projects/bind9/-/pipelines?scope=all&source=schedule) and [private](https://gitlab.isc.org/isc-private/bind9/-/pipelines?scope=all&source=schedule) scheduled pipelines.
- [ ]***(QA)*** Check [Perflab](https://perflab.isc.org/) to ensure there has been no unexplained drop in performance for the versions being released.
- [ ]***(QA)*** Check whether all issues assigned to the release milestone are resolved[^1].
- [ ]***(QA)*** Ensure that there are no outstanding [merge requests in the private repository](https://gitlab.isc.org/isc-private/bind9/-/merge_requests/)[^1] (Subscription Edition only).
- [ ]***(QA)*** [Ensure](https://gitlab.isc.org/isc-private/bind-qa/-/blob/master/bind9/releng/check_backports.py) all merge requests marked for backporting have been indeed backported.
- [ ]***(QA)*** [Announce](https://gitlab.isc.org/isc-private/bind-qa/-/blob/master/bind9/releng/inform_code_freeze.py) (on Mattermost) that the code freeze is in effect.
### Before the Tagging Deadline
- [ ]***(QA)*** Inspect the current output of the `cross-version-config-tests` job to verify that no unexpected backward-incompatible change was introduced in the current release cycle.
- [ ]***(QA)*** Ensure release notes are correct, ask Support and Marketing to check them as well. [Example](https://gitlab.isc.org/isc-private/bind9/-/merge_requests/510)
- [ ]***(QA)*** Add a release marker to `CHANGES`. Examples: [9.18](https://gitlab.isc.org/isc-projects/bind9/-/commit/f14d8ad78c0506fd4247187f2177f8eceeb6b3b9), [9.16](https://gitlab.isc.org/isc-projects/bind9/-/commit/1bcdf21874f99a00da389d723e0ad07dfd70f9f1)
- [ ]***(QA)*** Add a release marker to `CHANGES.SE` (Subscription Edition only). [Example](https://gitlab.isc.org/isc-private/bind9/-/commit/0f03d5737bcbdaa1bf713c6db1887b14938c3421)
- [ ]***(QA)*** Update BIND 9 version in `configure.ac` ([9.18+](https://gitlab.isc.org/isc-projects/bind9/-/commit/3c85ab7f4c35e6d8acef1393606002a0a8730100)) or `version` ([9.16](https://gitlab.isc.org/isc-projects/bind9/-/merge_requests/7692/diffs?commit_id=1bcdf21874f99a00da389d723e0ad07dfd70f9f1)).
- [ ]***(QA)*** Rebuild `configure` using Autoconf on `docs.isc.org` (9.16).
- [ ]***(QA)*** Update GitLab settings for all maintained branches to disallow merging to them: [public](https://gitlab.isc.org/isc-projects/bind9/-/settings/repository), [private](https://gitlab.isc.org/isc-private/bind9/-/settings/repository)
- [ ]***(QA)*** Tag the releases in the private repository (`git tag -s -m "BIND 9.x.y" v9.x.y`).
### Before the ASN Deadline (for ASN Releases) or the Public Release Date (for Regular Releases)
- [ ]***(QA)*** Check that the formatting is correct for HTML and PDF versions of release notes.
- [ ]***(QA)*** Check that the formatting of the generated man pages is correct.
- [ ]***(QA)*** Verify GitLab CI results [for the tags](https://gitlab.isc.org/isc-private/bind9/-/pipelines?scope=tags) created and sign off on the releases to be published.
- [ ]***(QA)*** Update GitLab settings for all maintained branches to allow merging to them again: [public](https://gitlab.isc.org/isc-projects/bind9/-/settings/repository), [private](https://gitlab.isc.org/isc-private/bind9/-/settings/repository)
- [ ]***(QA)*** Prepare (using [`version_bump.py`](https://gitlab.isc.org/isc-private/bind-qa/-/blob/master/bind9/releng/version_bump.py)) and merge MRs resetting the release notes and updating the version string for each maintained branch.
- [ ]***(QA)*** Announce (on Mattermost) that the code freeze is over.
- [ ]***(QA)*** Request signatures for the tarballs, providing their location and checksums. Ask [signers on Mattermost](https://mattermost.isc.org/isc/channels/bind-9-qa).
- [ ]***(Signers)*** Ensure that the contents of tarballs and tags are identical.
- [ ]***(QA)*** Verify tarball signatures and check tarball checksums again: Run `publish_bind.sh` on repo.isc.org to pre-publish.
- [ ]***(Support)*** Pre-publish ASN and/or Subscription Edition tarballs so that packages can be built.
- [ ]***(QA)*** Build and test ASN and/or Subscription Edition packages (in [cloudsmith branch in private repo](https://gitlab.isc.org/isc-private/rpms/bind/-/tree/cloudsmith)). [Example](https://gitlab.isc.org/isc-private/rpms/bind/-/commit/e2512f4cfaf991827a635e374e7e93b27a5f38ba)
- [ ]***(QA)*** Prepare the `patches/` subdirectory for each security release (if applicable).
- [ ]***(QA)*** Notify Support that the releases have been prepared.
- [ ]***(Support)*** Send out ASNs (if applicable).
### On the Day of Public Release
- [ ]***(Support)*** Wait for clearance from Security Officer to proceed with the public release (if applicable).
- [ ]***(Support)*** Place tarballs in public location on FTP site.
- [ ]***(Support)*** Publish links to downloads on ISC website. [Example](https://gitlab.isc.org/website/theme-staging-site/-/commit/1ac7b30b73cb03228df4cd5651fa4e774ac35625)
- [ ]***(Support)*** Add the new releases to the [vulnerability matrix in the Knowledge Base](https://kb.isc.org/docs/aa-00913).
- [ ]***(Support)*** Use the [Printing Press project](https://gitlab.isc.org/isc-private/printing-press/-/wikis/home#adding-new-documents) to prepare a release announcement email and send it to the *bind-announce* mailing list.
- [ ]***(Support)*** Write email to *bind-users* (if a major release). [Example](https://lists.isc.org/pipermail/bind-users/2022-January/105624.html)
- [ ]***(Support)*** Send eligible customers updated links to the Subscription Edition (update the -S edition delivery tickets, even if those links were provided earlier via an ASN ticket).
- [ ]***(Support)*** Update tickets in case of waiting support customers.
- [ ]***(QA)*** Build and test any outstanding private packages in [private repo](https://gitlab.isc.org/isc-private/rpms/bind/-/tree/cloudsmith). [Example](https://gitlab.isc.org/isc-private/rpms/bind/-/commit/2007d566db81dd9dfd79e571e2f600a3bc284da4)
- [ ]***(SwEng)*** Update Docker files [here](https://gitlab.isc.org/isc-projects/bind9-docker/-/branches) and make sure push is synchronized to [GitHub](https://github.com/isc-projects/bind9-docker). [Docker Hub](https://hub.docker.com/r/internetsystemsconsortium/bind9) should pick it up automatically. [Example](https://gitlab.isc.org/isc-projects/bind9-docker/-/commit/cada7e10e9af951595c98bfffc4bd42512faac05)
- [ ]***(QA)*** Inform Marketing of the release.
- [ ]***(Marketing)*** Post a short note to Mastodon.
- [ ]***(Marketing)*** Update [Wikipedia entry for BIND](https://en.wikipedia.org/wiki/BIND).
- [ ]***(Marketing)*** Write blog article (if a major release).
- [ ]***(QA)*** Ensure all new tags are annotated and signed. `git show --show-signature v9.19.12`
- [ ]***(QA)*** Push tags for the published releases to the public repository.
- [ ]***(QA)*** Using [`merge_tag.py`](https://gitlab.isc.org/isc-private/bind-qa/-/blob/master/bind9/releng/merge_tag.py), merge published release tags back into the their relevant development/maintenance branches.
- [ ]***(QA)*** Ensure `allow_failure: true` is removed from the `cross-version-config-tests` job if it was set during the current release cycle.
- [ ]***(QA)*** Sanitize confidential issues which are assigned to the current release milestone and do not describe a security vulnerability, then make them public.
- [ ]***(QA)*** Sanitize [confidential issues](https://gitlab.isc.org/isc-projects/bind9/-/issues/?sort=milestone_due_desc&state=opened&confidential=yes) which are assigned to older release milestones and describe security vulnerabilities, then make them public if appropriate[^2].
- [ ]***(QA)*** Update QA tools used in GitLab CI (e.g. Black, PyLint, Sphinx) by modifying the relevant [`Dockerfile`](https://gitlab.isc.org/isc-projects/images/-/merge_requests/228/diffs).
- [ ]***(QA)*** Run a pipeline to rebuild all [images](https://gitlab.isc.org/isc-projects/images) used in GitLab CI.
- [ ]***(QA)*** Update [`metadata.json`](https://gitlab.isc.org/isc-private/bind-qa/-/blob/master/bind9/releng/metadata.json) with the upcoming release information.
[^1]: If not, use the time remaining until the tagging deadline to ensure all outstanding issues are either resolved or moved to a different milestone.
[^2]: As a rule of thumb, security vulnerabilities which have reproducers merged to the public repository are considered okay for full disclosure.
| `-DOPENSSL_API_COMPAT=10100` | Build using the deprecated OpenSSL APIs so that the `engine` API is available when building with OpenSSL 3.0.0 for PKCS#11 support |
| [security] | Fix for a significant security flaw |
| [experimental] | Used for new features when the syntax or other aspects of the design are still in flux and may change |
| [port] | Portability enhancement |
| [maint] | Updates to built-in data such as root server addresses and keys |
| [tuning] | Changes to built-in configuration defaults and constants to improve performance |
| [performance] | Other changes to improve server performance |
| [protocol] | Updates to the DNS protocol such as new RR types |
| [test] | Changes to the automatic tests, not affecting server functionality |
| [cleanup] | Minor corrections and refactoring |
| [doc] | Documentation |
| [contrib] | Changes to the contributed tools and libraries in the 'contrib' subdirectory |
| [placeholder] | Used in the main development branch to reserve change numbers for use in other branches, e.g., when fixing a bug that only exists in older releases |
In general, [func] and [experimental] tags only appear in new-feature
releases (i.e., those with version numbers ending in zero). Some new
functionality may be backported to older releases on a case-by-case basis.
All other change types may be applied to all currently supported releases.
#### Bug report identifiers
Most notes in the CHANGES file include a reference to a bug report or
``$INCLUDE``. This option cannot be combined with :option:`-O raw <-O>`
or serial-number updating.
..option:: -E engine
This option specifies the hardware to use for cryptographic
operations, such as a secure key store used for signing, when applicable.
When BIND 9 is built with OpenSSL, this needs to be set to the OpenSSL
engine identifier that drives the cryptographic accelerator or
hardware service module (usually ``pkcs11``).
..option:: -F
This options turns on FIPS (US Federal Information Processing Standards)
@@ -89,6 +80,11 @@ Options
where ``digest-type`` is an allowed algorithm such as SHA-256 (2), or SHA-384 (4).
Only works in combination with smart signing (``-S``).
..option:: -J filename
This option tells :program:`dnssec-signzone` to read the journal from the given file
when loading the zone file.
..option:: -K directory
This option specifies the directory to search for DNSSEC keys. If not
@@ -273,7 +269,7 @@ Options
with cached copies of the old DNSKEY RRset. The :option:`-Q` option forces
:program:`dnssec-signzone` to remove signatures from keys that are no longer
active. This enables ZSK rollover using the procedure described in
:rfc:`4641#4.2.1.1` ("Pre-Publish Key Rollover").
:rfc:`6781#4.1.1.1` ("Pre-Publish Key Rollover").
..option:: -q
@@ -290,7 +286,7 @@ Options
This option is similar to :option:`-Q`, except it forces
:program:`dnssec-signzone` to remove signatures from keys that are no longer
published. This enables ZSK rollover using the procedure described in
:rfc:`4641#4.2.1.2` ("Double Signature Zone Signing Key
:rfc:`6781#4.1.1.2` ("Double Signature Zone Signing Key
Rollover").
..option:: -S
@@ -374,6 +370,7 @@ Options
..note::
``-3 -`` is the recommended configuration. Adding salt provides no practical benefits.
See :rfc:`9276`.
..option:: -H iterations
@@ -382,6 +379,7 @@ Options
..warning::
Values greater than 0 cause interoperability issues and also increase the risk of CPU-exhausting DoS attacks.
See :rfc:`9276`.
..option:: -A
@@ -390,6 +388,7 @@ Options
..warning::
Do not use this option unless all its implications are fully understood. This option is intended only for extremely large zones (comparable to ``com.``) with sparse secure delegations.
@@ -152,6 +152,15 @@ Currently supported commands are:
See also :option:`rndc delzone` and :option:`rndc modzone`.
..option:: closelogs
This command closes currently open log files. It is intended to be used
by external log rotation tools following this proceedure.
1) rename the log files
2) run ``rndc closelogs``
3) optionally compress the log files
..option:: delzone [-clean] zone [class [view]]
This command deletes a zone while the server is running.
@@ -257,6 +266,11 @@ Currently supported commands are:
See also :option:`rndc stop`.
..option:: skr -import file zone [class [view]]
This command allows you to import a SKR file for the specified zone, to
support offline KSK signing.
..option:: loadkeys [zone [class [view]]]
This command fetches all DNSSEC keys for the given zone from the key directory. If
@@ -440,17 +454,20 @@ Currently supported commands are:
..option:: zone [class [view]]
If a zone is specified, this command reloads only the given zone.
If no zone is specified, the reloading happens asynchronously.
..program:: rndc
..option:: retransfer zone [class [view]]
..option:: retransfer [-force] zone [class [view]]
This command retransfers the given secondary zone from the primary server.
If the zone is configured to use ``inline-signing``, the signed
version of the zone is discarded; after the retransfer of the
unsigned version is complete, the signed version is regenerated
with new signatures.
with new signatures. With the optional ``-force`` argument provided
if there is an ongoing zone transfer it will be aborted before a new zone
transfer is scheduled.
..option:: scan
@@ -604,7 +621,8 @@ Currently supported commands are:
refused. If the zone has changed and the ``ixfr-from-differences``
option is in use, the journal file is updated to reflect
changes in the zone. Otherwise, if the zone has changed, any existing
journal file is removed.
journal file is removed. If no zone is specified, the reloading happens
asynchronously.
See also :option:`rndc freeze`.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Ondřej Surý
Alessio Podda
Arаm Sаrgsyаn
Mark Andrews
Michal Nowak
Nicki Křížek
Vicky RiskandNicki Křížek
Suzanne GoldlustandNicki Křížek
Aydın Mercan
Michał KępieńandAydın Mercan
JINMEI TatuyaandNicki Křížek
Matthijs MekkingandNicki Křížek
Evan Hunt
Petr Špaček
James AddisonandPetr Špaček
Yedaya KatsmanandMark Andrews
Tom KrizekandNicki Křížek
Emanuel PetrandNicki Křížek
Artem Boldariev
Štěpán BalážikandPetr Špaček
Petr MenšíkandPetr Špaček
Dominik ThalhammerandOndřej Surý
Tony FinchandMark Andrews