Commit Graph

38057 Commits

Author SHA1 Message Date
Ondřej Surý
f3753d591f Use thread_local EVP_MD_CTX in isc_iterated_hash()
As this code is on hot path (NSEC3) this introduces an additional
optimization of the EVP_MD API - instead of calling EVP_MD_CTX_new() on
every call to isc_iterated_hash(), we create two thread_local objects
for each thread - a basectx and mdctx, initialize basectx once and then
use EVP_MD_CTX_copy_ex() to flip the initialized state into mdctx.  This
saves us couple more valuable microseconds from the isc_iterated_hash()
call.
2023-01-18 19:36:21 +01:00
Ondřej Surý
25db8d0103 Use OpenSSL 1.x SHA_CTX API in isc_iterated_hash()
If the OpenSSL SHA1_{Init,Update,Final} API is still available, use it.
The API has been deprecated in OpenSSL 3.0, but it is significantly
faster than EVP_MD API, so make an exception here and keep using it
until we can't.
2023-01-18 19:36:17 +01:00
Ondřej Surý
36654df732 Use OpenSSL EVP_MD API directly in isc_iterated_hash()
Instead of going through another layer, use OpenSSL EVP_MD API directly
in the isc_iterated_hash() implementation.  This shaves off couple of
microseconds in the microbenchmark.
2023-01-18 18:32:57 +01:00
Ondřej Surý
e6bfb8e456 Avoid implicit algorithm fetch for OpenSSL EVP_MD family
The implicit algorithm fetch causes a lock contention and significant
slowdown for small input buffers.  For more details, see:

https://github.com/openssl/openssl/issues/19612

Instead of using EVP_DigestInit_ex() initialize empty MD_CTX objects for
each algorithm and use EVP_MD_CTX_copy_ex() to initialize MD_CTX from a
static copy.  Additionally avoid implicit algorithm fetching by using
EVP_MD_fetch() for OpenSSL 3.0.
2023-01-18 18:32:57 +01:00
Ondřej Surý
cfbe01c62f Add microbenchmark for isc_iterated_hash()
Add microbenchmark for isc_iterated_hash() to measure the speed of NSEC3
per second.
2023-01-18 18:32:57 +01:00
Ondřej Surý
05b60f9cd8 Merge branch 'ondrej/openssl3-no-deprecated' into 'main'
Enable OpenSSL 3.x build with -DOPENSSL_NO_DEPRECATED=1

See merge request isc-projects/bind9!7394
2023-01-18 17:18:09 +00:00
Ondřej Surý
61ad7a59e0 Enable OpenSSL 3.x build with -DOPENSSL_NO_DEPRECATED=1
Add a CI job where we explicitly disable any use of OpenSSL functions
that have been deprecated in OpenSSL 3.x.
2023-01-18 11:13:19 +01:00
Evan Hunt
680fdce937 Merge branch 'each-remove-reserved-dispatches' into 'main'
remove dead code for reserved dispatches

See merge request isc-projects/bind9!7391
2023-01-18 02:53:40 +00:00
Evan Hunt
d79a3823ad remove dead code for reserved dispatches
named formerly reserved a set of dispatch objects for use when
sending requests from user-specified source ports. this objects
are no longer used and have been removed.
2023-01-17 18:23:52 -08:00
Evan Hunt
1070f1f56a Merge branch '3781-deprecate-source-port' into 'main'
mark "port" as deprecated for source address options

Closes #3781

See merge request isc-projects/bind9!7307
2023-01-18 01:51:43 +00:00
Evan Hunt
22d1951aa6 CHANGES and release note for [GL #3781] 2023-01-17 17:29:43 -08:00
Matthijs Mekking
531914e660 Test deprecate source port configuration
Add 'port' token to deprecated.conf. Also add options
'use-v4-udp-ports', 'use-v6-udp-ports', 'avoid-v4-udp-ports',
and 'avoid-v6-udp-ports'.

All of these should trigger warnings (except when deprecation warnings
are being ignored).
2023-01-17 17:29:31 -08:00
Evan Hunt
470ccbc8ed mark "port" as deprecated for source address options
Deprecate the use of "port" when configuring query-source(-v6),
transfer-source(-v6), notify-source(-v6), parental-source(-v6),
etc. Also deprecate use-{v4,v6}-udp-ports and avoid-{v4,v6}udp-ports.
2023-01-17 17:29:21 -08:00
Evan Hunt
175ee6b22e Merge branch '3789-remove-dscp' into 'main'
fully remove DSCP

Closes #3789

See merge request isc-projects/bind9!7338
2023-01-18 00:47:24 +00:00
Evan Hunt
d6768d6000 CHANGES and release note revision for [GL #3789] 2023-01-17 16:18:21 -08:00
Evan Hunt
287722ac12 fully remove DSCP
The "dscp" option is now marked as "ancient" and it is a configuration
error to use it or to configure DSCP values for any source-address
option.
2023-01-17 16:18:21 -08:00
Ondřej Surý
71f13b9ad9 Merge branch '3801-reduce-memory-bloat-caused-by-delayed-view-detach' into 'main'
Detach the views in zone_shutdown(), not in zone_free()

Closes #3801

See merge request isc-projects/bind9!7382
2023-01-17 21:47:14 +00:00
Ondřej Surý
b049e329ef Add CHANGES and release note for [GL #3801] 2023-01-17 21:48:30 +01:00
Ondřej Surý
7e8b53720d Commit the change of view for view->managed_keys
When we change the view in the view->managed_keys, we never commit the
change, keeping the previous view possibly attached forever.

Call the dns_zone_setviewcommit() immediately after changing the view as
we are detaching the previous view anyway and there's no way to recover
from that.
2023-01-17 21:48:30 +01:00
Ondřej Surý
13bb821280 Detach the views in zone_shutdown(), not in zone_free()
The .view (and possibly .prev_view) would be kept attached to the
removed zone until the zone is fully removed from the memory in
zone_free().  If this process is delayed because server is busy
something else like doing constant `rndc reconfig`, it could take
seconds to detach the view, possibly keeping multiple dead views in the
memory.  This could quickly lead to a massive memory bloat.

Release the views early in the zone_shutdown() call, and don't wait
until the zone is freed.
2023-01-17 21:48:30 +01:00
Artem Boldariev
42ab66b971 Merge branch '3767-dot-no-alpn-transfer-shutdown-crash' into 'main'
XoT: properly handle the case when checking for ALPN failed

Closes #3767

See merge request isc-projects/bind9!7360
2023-01-17 18:50:10 +00:00
Artem Boldariev
16fa692b56 Modify CHANGES [GL #3767]
Mention that downloading a zone via TLS- from a server which does not
negotiate "dot" ALPN tag could have crashed BIND on shutdown.
2023-01-17 19:00:19 +02:00
Artem Boldariev
34a1aab1cb XoT: properly handle the case when checking for ALPN failed
During XoT it is important to check for "dot" ALPN tag to be
negotiated (according to the RFC 9103). We were doing that, however, the
situation was not handled properly, leading to non-cancelled zone
transfers that would crash (abort()) BIND on shutdown.

In this particular case 'result' might equal 'ISC_R_SUCCESS'. When
this is the case, the part of the code supposed to handle failures
will not cancel the zone transfer.

This situation cannot happen when BIND is a secondary of other BIND
instance. Only primaries following the RFC not closely enough could
trigger such a behaviour.
2023-01-17 19:00:19 +02:00
Tom Krizek
9c32089fa9 Merge branch '3804-fix-feature-detection-in-pytests' into 'main'
Fix feature detection for pytest markers in tests

Closes #3804

See merge request isc-projects/bind9!7383
2023-01-17 13:48:57 +00:00
Tom Krizek
473cb530f4 Fix feature detection for pytest markers in tests
The condition was accidentally reversed during refactoring in
9730ac4c56 . It would result in skipped
tests on builds with proper support and false negatives on builds
without proper feature support.

Credit for reporting the issue and the fix goes to Stanislav Levin.
2023-01-17 14:18:22 +01:00
Tony Finch
2b89ef56a8 Merge branch 'fanf-time-backwards' into 'main'
Fix a typo in the NS_PER_ macros

See merge request isc-projects/bind9!7380
2023-01-17 09:01:32 +00:00
Tony Finch
290899661d Fix a typo in the NS_PER_ macros
Milliseconds and microseconds were swapped.
2023-01-16 20:33:57 +00:00
Ondřej Surý
d7bcdf8bd6 Merge branch 'feature/main/zt-rwlock.h' into 'main'
Include isc_rwlocktype_t type definition in zt.h

See merge request isc-projects/bind9!7376
2023-01-16 11:06:48 +00:00
Petr Menšík
395d6fca26 Include isc_rwlocktype_t type definition in zt.h
After changes zt.h uses rwlock type, which is not enforced to be always
defined. Ensure full type definition is ready by including appropriate
header.
2023-01-16 11:45:31 +01:00
Tom Krizek
0bc0cabaa1 Merge branch 'tkrizek/system-tests-pytest-prep' into 'main'
Various tweaks of system test framework

See merge request isc-projects/bind9!7347
2023-01-16 09:13:01 +00:00
Tom Krizek
03d7b45d81 Update the TEST_PARALLEL_JOBS value in CI
The authoritative source for this value is in the project's CI/CD
Variables Setting. The reason to keep it in .gitlab-ci.yaml as well is
to have functional testing in forks without the need to manually specify
this variable in Settings.

The tests have been executed with 4 jobs for some time now. This
"change" only brings .gitlab-ci.yaml file up to date, it doesn't
actually change the number of jobs we currently use to test.
2023-01-13 17:01:30 +01:00
Tom Krizek
9593ff9347 Tweak dupsigs test output
Use a different visual separator, since "====", "----" and "____" is
used by pytest to separate the log output.
2023-01-13 17:01:30 +01:00
Tom Krizek
e24d3b21d0 Look for ifconfig.sh.in in testsock.pl parent dir
Instead of using the current working directory to find the ifconfig.sh
script, look for the ifconfig.sh.in template in the directory where the
testsock.pl script is located. This enables the testsock.pl script to be
called from any working directory.

Using the ifconfig.sh.in template is sufficient, since it contains
the necessary information to be extracted: the max= value (which is
hard-coded in the template).
2023-01-13 17:01:30 +01:00
Tom Krizek
d9a97200d5 Factor out script to handle system test core dumps
Move the core dump detection functionality for system test runs into a
separate script. This enables reuse by the pytest runner. The
functionality remains the same.
2023-01-13 17:01:30 +01:00
Tom Krizek
61330a7863 testcrypto.sh: run in TMPDIR if possible
Avoid creating any temporary files in the current workdir.

Additional/changing files in the bin/tests/system directory are
problematic for pytest/xdist collection phase, which assumes the list of
files doesn't change between the collection phase of the main pytest
thread and the subsequent collection phase of the xdist worker threads.

Since the testcrypto.sh is also called during pytest initialization
through conf.sh.common (to detect feature support), this could
occasionally cause a race condition when the list of files would be
different for the main pytest thread and the xdist worker.
2023-01-13 17:01:28 +01:00
Michał Kępień
6f0c821c48 Merge branch 'michal/set-up-version-and-release-notes-for-bind-9.19.10' into 'main'
Set up version and release notes for BIND 9.19.10

See merge request isc-projects/bind9!7362
2023-01-13 14:47:05 +00:00
Michał Kępień
bf14b1a74c Set up release notes for BIND 9.19.10 2023-01-13 15:35:32 +01:00
Michał Kępień
2f39be23b1 Update BIND version to 9.19.10-dev 2023-01-13 15:35:32 +01:00
Michał Kępień
6ecc0a2693 Merge branch 'michal/add-placeholder-entries-to-CHANGES' into 'main'
Add placeholder entries to CHANGES

See merge request isc-projects/bind9!7356
2023-01-11 16:48:05 +00:00
Michał Kępień
a2cc802060 Add placeholder entries to CHANGES
Add placeholders for the following issues:

  - [GL #3523]
  - [GL #3599]
  - [GL #3619]
  - [GL #3622]
2023-01-11 17:44:12 +01:00
Arаm Sаrgsyаn
a5e23c3158 Merge branch 'aram/dns_fwdtable_addfwd-cleanup-bugfix' into 'main'
Fix dns_fwdtable_addfwd() error path cleanup bug

See merge request isc-projects/bind9!7205
2023-01-11 13:19:31 +00:00
Aram Sargsyan
cf4003fa58 Use sizeof(*ptr) for allocating/freeing memory in forward.c
As shown in the previous commit, using sizeof(type_t) is a little
bit more error-prone when copy-pasting code, so extracting the
size information from the pointer which is being dealt with seems
like a better alternative.
2023-01-11 12:41:06 +00:00
Aram Sargsyan
0cc1b06d98 Fix dns_fwdtable_addfwd() error path cleanup bug
Free 'sizeof(dns_forwarder_t)' bytes of memory instead of
'sizeof(dns_sockaddr_t)' bytes, because `fwd` is a pointer
to a 'dns_forwarder_t' type structure.
2023-01-11 12:41:06 +00:00
Mark Andrews
40077f6f75 Merge branch '3764-adjust-descriptors-for-some-unit-tests' into 'main'
Resolve "Adjust descriptor limit for some unit tests"

Closes #3764

See merge request isc-projects/bind9!7294
2023-01-11 11:04:07 +00:00
Mark Andrews
cdcd605ede Increase the number of available file descriptors
notify_test and query_test run out of descriptors on some platforms.

    % ./notify_test
    [==========] Running 1 test(s).
    [ RUN      ] notify_start
    netmgr/tcp.c:369: REQUIRE(csock->fd >= 0) failed, back trace
    0   libisc-9.19.9-dev.dylib             0x00000001045c5080 default_callback + 72
    1   libisc-9.19.9-dev.dylib             0x00000001045c4ffc isc_assertion_failed + 56
    2   libisc-9.19.9-dev.dylib             0x00000001045b8bcc start_tcp_child + 304
    3   libisc-9.19.9-dev.dylib             0x00000001045b8710 isc_nm_listentcp + 636
    4   libisc-9.19.9-dev.dylib             0x00000001045b3e0c isc_nm_listenstreamdns + 344
    5   libns-9.19.9-dev.dylib              0x0000000104b90630 ns_interface_listentcp + 152
    6   libns-9.19.9-dev.dylib              0x0000000104b8f65c interface_setup + 488
    7   libns-9.19.9-dev.dylib              0x0000000104b8de98 do_scan + 2028
    8   libns-9.19.9-dev.dylib              0x0000000104b8d640 ns_interfacemgr_scan + 212
    9   notify_test                         0x000000010418ddd0 scan_interfaces + 44
    10  libisc-9.19.9-dev.dylib             0x00000001045d5fd8 isc__job_cb + 116
    11  libuv.1.dylib                       0x000000010545afe4 uv__run_idle + 152
    12  libuv.1.dylib                       0x0000000105455cd0 uv_run + 204
    13  libisc-9.19.9-dev.dylib             0x00000001045e1120 loop_run + 460
    14  libisc-9.19.9-dev.dylib             0x00000001045df824 loop_thread + 44
    15  libisc-9.19.9-dev.dylib             0x00000001045df6dc isc_loopmgr_run + 456
    16  notify_test                         0x000000010418b900 run_test_notify_start + 88
    17  libcmocka.0.dylib                   0x00000001054968c0 cmocka_run_one_test_or_fixture + 448
    18  libcmocka.0.dylib                   0x0000000105494ca4 _cmocka_run_group_tests + 848
    19  notify_test                         0x000000010418be60 main + 120
    20  libdyld.dylib                       0x0000000181509430 start + 4
    Abort
    % ./query_test
    [==========] Running 4 test(s).
    [ RUN      ] ns__query_sfcache
    netmgr/tcp.c:369: REQUIRE(csock->fd >= 0) failed, back trace
    0   libisc-9.19.9-dev.dylib             0x000000010294d080 default_callback + 72
    1   libisc-9.19.9-dev.dylib             0x000000010294cffc isc_assertion_failed + 56
    2   libisc-9.19.9-dev.dylib             0x0000000102940bcc start_tcp_child + 304
    3   libisc-9.19.9-dev.dylib             0x0000000102940710 isc_nm_listentcp + 636
    4   libisc-9.19.9-dev.dylib             0x000000010293be0c isc_nm_listenstreamdns + 344
    5   libns-9.19.9-dev.dylib              0x00000001026cc630 ns_interface_listentcp + 152
    6   libns-9.19.9-dev.dylib              0x00000001026cb65c interface_setup + 488
    7   libns-9.19.9-dev.dylib              0x00000001026c9e98 do_scan + 2028
    8   libns-9.19.9-dev.dylib              0x00000001026c9640 ns_interfacemgr_scan + 212
    9   query_test                          0x00000001026a1018 scan_interfaces + 44
    10  libisc-9.19.9-dev.dylib             0x000000010295dfd8 isc__job_cb + 116
    11  libuv.1.dylib                       0x0000000103996fe4 uv__run_idle + 152
    12  libuv.1.dylib                       0x0000000103991cd0 uv_run + 204
    13  libisc-9.19.9-dev.dylib             0x0000000102969120 loop_run + 460
    14  libisc-9.19.9-dev.dylib             0x0000000102967824 loop_thread + 44
    15  libisc-9.19.9-dev.dylib             0x00000001029676dc isc_loopmgr_run + 456
    16  query_test                          0x000000010269cf34 run_test_ns__query_sfcache + 88
    17  libcmocka.0.dylib                   0x00000001028068c0 cmocka_run_one_test_or_fixture + 448
    18  libcmocka.0.dylib                   0x0000000102804ca4 _cmocka_run_group_tests + 848
    19  query_test                          0x000000010269eab8 main + 116
    20  libdyld.dylib                       0x0000000181509430 start + 4
    Abort
    %

expected_creads can exceed the number of file descriptors on some
platforms.

    % ./udp_test
    [==========] Running 18 test(s).
    [ RUN      ] mock_listenudp_uv_udp_open
    [       OK ] mock_listenudp_uv_udp_open
    [ RUN      ] mock_listenudp_uv_udp_bind
    [       OK ] mock_listenudp_uv_udp_bind
    [ RUN      ] mock_listenudp_uv_udp_recv_start
    [       OK ] mock_listenudp_uv_udp_recv_start
    [ RUN      ] mock_udpconnect_uv_udp_open
    [       OK ] mock_udpconnect_uv_udp_open
    [ RUN      ] mock_udpconnect_uv_udp_bind
    [       OK ] mock_udpconnect_uv_udp_bind
    [ RUN      ] mock_udpconnect_uv_udp_connect
    [       OK ] mock_udpconnect_uv_udp_connect
    [ RUN      ] mock_udpconnect_uv_recv_buffer_size
    [       OK ] mock_udpconnect_uv_recv_buffer_size
    [ RUN      ] mock_udpconnect_uv_send_buffer_size
    [       OK ] mock_udpconnect_uv_send_buffer_size
    [ RUN      ] udp_noop
    [       OK ] udp_noop
    [ RUN      ] udp_noresponse
    [       OK ] udp_noresponse
    [ RUN      ] udp_shutdown_connect
    [       OK ] udp_shutdown_connect
    [ RUN      ] udp_shutdown_read
    [       OK ] udp_shutdown_read
    [ RUN      ] udp_cancel_read
    [       OK ] udp_cancel_read
    [ RUN      ] udp_timeout_recovery
    [       OK ] udp_timeout_recovery
    [ RUN      ] udp_double_read
    [       OK ] udp_double_read
    [ RUN      ] udp_recv_one
    [       OK ] udp_recv_one
    [ RUN      ] udp_recv_two
    [       OK ] udp_recv_two
    [ RUN      ] udp_recv_send
    udp__connect_cb(0x0, too many open files, 0x0)
    udp__connect_cb(0x0, too many open files, 0x0)
    udp__connect_cb(0x0, too many open files, 0x0)
    udp__connect_cb(0x0, too many open files, 0x0)
    udp__connect_cb(0x0, too many open files, 0x0)
    udp__connect_cb(0x0, too many open files, 0x0)
    %
2023-01-11 10:44:07 +00:00
Ondřej Surý
5bcfd46523 Merge branch 'ondrej/use-pthread_barrier-if-available' into 'main'
Prefer the pthread_barrier implementation over uv_barrier

See merge request isc-projects/bind9!7348
2023-01-11 09:54:13 +00:00
Ondřej Surý
d07c4a98da Prefer the pthread_barrier implementation over uv_barrier
Prefer the pthread_barrier implementation on platforms where it is
available over uv_barrier implementation.  This also solves the problem
with thread sanitizer builds on macOS that doesn't have pthread barrier.
2023-01-11 09:51:02 +01:00
Ondřej Surý
5139b04e27 Merge branch 'fanf-git-replay-merge-tags' into 'main'
Auto-tag merge requests created by git-replay-merge

See merge request isc-projects/bind9!7254
2023-01-11 08:02:46 +00:00
Tony Finch
434e25c098 Auto-tag merge requests created by git-replay-merge
The target branch is added to the title of the MR, and a "Backport"
label is added. If the target branch starts with "v" a version label
is added too.

While I am here, clean up remnants of the old gitlab API (which has
been replaced by push options for our purposes) and improve the
command-line parsing.
2023-01-11 08:02:29 +00:00
Ondřej Surý
2037b1d944 Merge branch 'ondrej/remove-udp-and-tcp-listen-lock' into 'main'
Get rid of locking during UDP and TCP listen

See merge request isc-projects/bind9!7343
2023-01-11 06:51:14 +00:00