Commit Graph

33325 Commits

Author SHA1 Message Date
Michał Kępień
5ca495ad4f Add release note
(cherry picked from commit 16708682ad)
2021-06-10 10:26:51 +02:00
Michał Kępień
46001e4de3 Add CHANGES entry
(cherry picked from commit c223d81691)
2021-06-10 10:26:51 +02:00
Michał Kępień
f88c90f47f Fix "no DS" proofs for wildcard+CNAME delegations
When answering a query requires wildcard expansion, the AUTHORITY
section of the response needs to include NSEC(3) record(s) proving that
the QNAME does not exist.

When a response to a query is an insecure delegation, the AUTHORITY
section needs to include an NSEC(3) proof that no DS record exists at
the parent side of the zone cut.

These two conditions combined trip up the NSEC part of the logic
contained in query_addds(), which expects the NS RRset to be owned by
the first name found in the AUTHORITY section of a delegation response.
This may not always be true, for example if wildcard expansion causes an
NSEC record proving QNAME nonexistence to be added to the AUTHORITY
section before the delegation is added to the response.  In such a case,
named incorrectly omits the NSEC record proving nonexistence of QNAME
from the AUTHORITY section.

The same block of code is affected by another flaw: if the same NSEC
record proves nonexistence of both the QNAME and the DS record at the
parent side of the zone cut, this NSEC record will be added to the
AUTHORITY section twice.

Fix by looking for the NS RRset in the entire AUTHORITY section and
adding the NSEC record to the delegation using query_addrrset() (which
handles duplicate RRset detection).

(cherry picked from commit 7a87bf468b)
2021-06-10 10:26:51 +02:00
Michał Kępień
82b7e6ccef Add AUTHORITY tests for CNAME-sourced delegations
Add a set of system tests which check the contents of the AUTHORITY
section for signed, insecure delegation responses constructed from CNAME
records and wildcards, both for zones using NSEC and NSEC3.

(cherry picked from commit 26ec4b9a89)
2021-06-10 10:26:51 +02:00
Michał Kępień
5b9c25bef8 Merge branch 'michal/fix-the-variable-checked-by-a-post-load-assertion-v9_16' into 'v9_16'
[v9_16] Fix the variable checked by a post-load assertion

See merge request isc-projects/bind9!5168
2021-06-10 08:06:31 +00:00
Mark Andrews
3593651559 Fix the variable checked by a post-load assertion
Instead of checking the value of the variable modified two lines earlier
(the number of SOA records present at the apex of the old version of the
zone), one of the RUNTIME_CHECK() assertions in zone_postload() checks
the number of SOA records present at the apex of the new version of the
zone, which is already checked before.  Fix the assertion by making it
check the correct variable.

(cherry picked from commit 098639dc59)
2021-06-10 10:04:21 +02:00
Mark Andrews
63e1a742ee Merge branch '2760-db-unit-test-failure-v9_16' into 'v9_16'
Adjust acceptable count values

See merge request isc-projects/bind9!5159
2021-06-09 23:20:54 +00:00
Mark Andrews
c7216ae382 Adjust acceptable count values
usleep(100000) can be slightly less than 10ms so allow the count
to reach 11.

(cherry picked from commit 2bc454dc2d)
2021-06-10 08:33:46 +10:00
Mark Andrews
dab8e3fa0e Merge branch '2720-threadsanitizer-data-race-lib-isc-unix-time-c-110-in-isc_time_isepoch-v9_16' into 'v9_16'
Address race between zone_settimer and set_key_expiry_warning by

See merge request isc-projects/bind9!5157
2021-06-09 22:05:28 +00:00
Mark Andrews
edd0fe1dca Address race between zone_settimer and set_key_expiry_warning by
adding missing lock.

    WARNING: ThreadSanitizer: data race
    Read of size 4 at 0x000000000001 by thread T1 (mutexes: read M1, write M2):
    #0 isc_time_isepoch lib/isc/unix/time.c:110
    #1 zone_settimer lib/dns/zone.c:14649
    #2 dns_zone_maintenance lib/dns/zone.c:6281
    #3 dns_zonemgr_forcemaint lib/dns/zone.c:18190
    #4 view_loaded server.c:9654
    #5 call_loaddone lib/dns/zt.c:301
    #6 doneloading lib/dns/zt.c:575
    #7 zone_asyncload lib/dns/zone.c:2259
    #8 task_run lib/isc/task.c:845
    #9 isc_task_run lib/isc/task.c:938
    #10 isc__nm_async_task lib/isc/netmgr/netmgr.c:855
    #11 process_netievent lib/isc/netmgr/netmgr.c:934
    #12 process_queue lib/isc/netmgr/netmgr.c:1003
    #13 process_all_queues lib/isc/netmgr/netmgr.c:775
    #14 async_cb lib/isc/netmgr/netmgr.c:804
    #15 <null> <null>
    #16 isc__trampoline_run lib/isc/trampoline.c:191
    #17 <null> <null>

    Previous write of size 4 at 0x000000000001 by thread T2:
    #0 isc_time_set lib/isc/unix/time.c:93
    #1 set_key_expiry_warning lib/dns/zone.c:6430
    #2 del_sigs lib/dns/zone.c:6711
    #3 zone_resigninc lib/dns/zone.c:7113
    #4 zone_maintenance lib/dns/zone.c:11111
    #5 zone_timer lib/dns/zone.c:14588
    #6 task_run lib/isc/task.c:845
    #7 isc_task_run lib/isc/task.c:938
    #8 isc__nm_async_task lib/isc/netmgr/netmgr.c:855
    #9 process_netievent lib/isc/netmgr/netmgr.c:934
    #10 process_queue lib/isc/netmgr/netmgr.c:1003
    #11 process_all_queues lib/isc/netmgr/netmgr.c:775
    #12 async_cb lib/isc/netmgr/netmgr.c:804
    #13 <null> <null>
    #14 isc__trampoline_run lib/isc/trampoline.c:191
    #15 <null> <null>

    SUMMARY: ThreadSanitizer: data race lib/isc/unix/time.c:110 in isc_time_isepoch

(cherry picked from commit 3d66e97a28)
2021-06-09 23:56:47 +10:00
Matthijs Mekking
35a1a6e5c1 Merge branch '2725-nsec3param-changes-on-restart-v9_16' into 'v9_16'
Fix NSEC3 resalt on restart (9.16)

See merge request isc-projects/bind9!5154
2021-06-09 08:01:25 +00:00
Matthijs Mekking
203652baef Add release note and change entry for [#2725]
(cherry picked from commit d51aed7112)
2021-06-09 09:18:51 +02:00
Matthijs Mekking
7893064f2e Fix NSEC3 resalting upon restart
When named restarts, it will examine signed zones and checks if the
current denial of existence strategy matches the dnssec-policy. If not,
it will schedule to create a new NSEC(3) chain.

However, on startup the zone database may not be read yet, fooling
BIND that the denial of existence chain needs to be created. This
results in a replacement of the previous NSEC(3) chain.

Change the code such that if the NSEC3PARAM lookup failed (the result
did not return in ISC_R_SUCCESS or ISC_R_NOTFOUND), we will try
again later. The nsec3param structure has additional variables to
signal if the lookup is postponed. We also need to save the signal
if an explicit resalt was requested.

In addition to the two added boolean variables, we add a variable to
store the NSEC3PARAM rdata. This may have a yet to be determined salt
value. We can't create the private data yet because there may be a
mismatch in salt length and the NULL salt value.

(cherry picked from commit 0ae3ffdc1c)
2021-06-09 09:18:44 +02:00
Matthijs Mekking
22e5666925 Add test for NSEC3PARAM not changed after restart
Add a test case where 'named' is restarted and ensure that an already
signed zone does not change its NSEC3 parameters.

The test case first tests the current zone and saves the used salt
value. Then after restart it checks if the salt (and other parameters)
are the same as before the restart.

This test case changes 'set_nsec3param'. This will now reset the salt
value, and when checking for NSEC3PARAM we will store the salt and
use it when testing the NXDOMAIN response. This does mean that for
every test case we now have to call 'set_nsec3param' explicitly (and
can not omit it because it is the same as the previous zone).

Finally, slightly changed some echo output to make debugging friendlier.

(cherry picked from commit 08a9e7add1)
2021-06-09 09:18:37 +02:00
Ondřej Surý
4cf637cd9a Merge branch '2732-pause-the-dbiterator-in-dumptostream-v9_16' into 'v9_16'
Pause the dbiterator when dumping the zone to the disk

See merge request isc-projects/bind9!5151
2021-06-04 10:05:36 +00:00
Ondřej Surý
6a43b1e711 Pause the dbiterator when dumping the zone to the disk
When we rewrote the zone dumping to use the separate threadpool, the
dumping would acquire the read lock for the whole time the zone dumping
process is dumping the zone.

When combined with incoming IXFR that tries to acquire the write lock on
the same rwlock, we would end up blocking all the other readers.

In this commit, we pause the dbiterator every time we get next record
and before start dumping it to the disk.

(cherry picked from commit 7e59b8a4a1)
2021-06-04 11:32:31 +02:00
Mark Andrews
884346628d Merge branch '2751-serve-stale-tests-false-negative-v9_16' into 'v9_16'
Address test race condition in serve-stale

See merge request isc-projects/bind9!5145
2021-06-03 09:03:18 +00:00
Mark Andrews
6d84bff565 Address test race condition in serve-stale
the dig.out.test# files could still be being written when the
content greps where being made.

(cherry picked from commit af95cb8ccc)
2021-06-03 18:43:24 +10:00
Mark Andrews
554578e8cb Merge branch '2750-provide-more-insight-into-why-the-timer_test-is-failing-v9_16' into 'v9_16'
Report which assertion failed when calling set_global_error

See merge request isc-projects/bind9!5144
2021-06-03 08:10:08 +00:00
Mark Andrews
a74b2a4448 Report which assertion failed when calling set_global_error
(cherry picked from commit 66d1df57cb)
2021-06-03 17:36:51 +10:00
Mark Andrews
e2e5e1601a Merge branch '2724-statschannel-system-test-sometimes-hangs-v9_16' into 'v9_16'
Add timeout to url get requests

See merge request isc-projects/bind9!5140
2021-06-02 23:35:16 +00:00
Mark Andrews
3f20e71430 Add timeout to url get requests
to prevent the system test taking forever on failures.

(cherry picked from commit 02726cb66e)
2021-06-03 08:39:12 +10:00
Ondřej Surý
906cd59d88 Merge branch '2746-fix-the-typo-in-setsockopt_off-v9_16' into 'v9_16'
Fix copy&paste error in setsockopt_off

See merge request isc-projects/bind9!5139
2021-06-02 17:23:02 +00:00
Ondřej Surý
bff37dc79b Add CHANGES and release note for [GL #2746]
(cherry picked from commit 22aa929aac)
2021-06-02 18:10:44 +02:00
Ondřej Surý
e528b3241d Fix copy&paste error in setsockopt_off
Because of copy&paste error the setsockopt_off macro would enable
the socket option instead of disabling it.

(cherry picked from commit f14d870d15)
2021-06-02 18:10:44 +02:00
Michał Kępień
e3de88945c Merge branch '2540-check-dname-resolution-via-itself-v9_16' into 'v9_16'
[v9_16] Check DNAME resolution via itself

See merge request isc-projects/bind9!5136
2021-06-02 13:10:43 +00:00
Mark Andrews
c4410f0f4f Check DNAME resolution via itself
(cherry picked from commit cbdea694e8)
2021-06-02 14:49:30 +02:00
Michał Kępień
74afaa6f2b Merge branch '2467-add-a-system-test-checking-a-malformed-ixfr-v9_16' into 'v9_16'
[v9_16] Add a system test checking a malformed IXFR

See merge request isc-projects/bind9!5133
2021-06-02 11:56:32 +00:00
Mark Andrews
2b73101db6 Add a system test checking a malformed IXFR
Make sure an incoming IXFR containing an SOA record which is not placed
at the apex of the transferred zone does not result in a broken version
of the zone being served by named and/or a subsequent crash.

(cherry picked from commit 5547003a3d)
2021-06-02 13:29:05 +02:00
Ondřej Surý
af07340444 Merge branch 'ondrej/fix-uv_udp_connect-detection-v9_16' into 'v9_16'
Cleanup the remaining of HAVE_UV_<func> macros

See merge request isc-projects/bind9!5129
2021-06-02 10:25:58 +00:00
Ondřej Surý
ce0083474e Cleanup the remaining of HAVE_UV_<func> macros
While cleaning up the usage of HAVE_UV_<func> macros, we forgot to
cleanup the HAVE_UV_UDP_CONNECT in the actual code and
HAVE_UV_TRANSLATE_SYS_ERROR and this was causing Windows build to fail
on uv_udp_send() because the socket was already connected and we were
falsely assuming that it was not.

The platforms with autoconf support were not affected, because we were
still checking for the functions from the configure.

(cherry picked from commit 67afea6cfc)
2021-06-02 12:01:29 +02:00
Ondřej Surý
e773ec0b6a Merge branch '2732-zone-dumping-is-blocking-the-networking-io-v9_16' into 'v9_16'
Improve the zone dumping impact on the networking (v9.16)

See merge request isc-projects/bind9!5108
2021-05-31 15:39:44 +00:00
Ondřej Surý
6ca678aca3 Add CHANGES and release note for [GL #2732]
(cherry picked from commit 3e433b87fb)
2021-05-31 16:57:20 +02:00
Ondřej Surý
e95dadb40d Indicate to the kernel that we won't be needing the zone dumps
Add a call to posix_fadvise() to indicate to the kernel, that `named`
won't be needing the dumped zone files any time soon with:

 * POSIX_FADV_DONTNEED - The specified data will not be accessed in the
   near future.

Notes:

 POSIX_FADV_DONTNEED attempts to free cached pages associated with the
 specified region. This is useful, for example, while streaming large
 files. A program may periodically request the kernel to free cached
 data that has already been used, so that more useful cached pages are
 not discarded instead.

(cherry picked from commit e83b6569da)
2021-05-31 16:57:20 +02:00
Ondřej Surý
c8eddf4f33 Refactor zone dumping code to use netmgr async threadpools
Previously, dumping the zones to the files were quantized, so it doesn't
slow down network IO processing.  With the introduction of network
manager asynchronous threadpools, we can move the IO intensive work to
use that API and we don't have to quantize the work anymore as it the
file IO won't block anything except other zone dumping processes.

(cherry picked from commit 8a5c62de83)
2021-05-31 16:57:19 +02:00
Ondřej Surý
2e849353b3 Add isc_task_getnetmgr() function
Add a function to pull the attached netmgr from inside the executed
task.  This is needed for any task that needs to call the netmgr API.

(cherry picked from commit 7670f98377)
2021-05-31 16:57:19 +02:00
Ondřej Surý
1417e39055 Add asynchronous work API to the network manager
The libuv has a support for running long running tasks in the dedicated
threadpools, so it doesn't affect networking IO.

This commit adds isc_nm_work_enqueue() wrapper that would wraps around
the libuv API and runs it on top of associated worker loop.

The only limitation is that the function must be called from inside
network manager thread, so the call to the function should be wrapped
inside a (bound) task.

(cherry picked from commit 87fe97ed91)
2021-05-31 16:57:19 +02:00
Ondřej Surý
c1703f5ce6 Use UV_VERSION_HEX to decide whether we need libuv shim functions
Instead of having a configure check for every missing function that has
been added in later version of libuv, we now use UV_VERSION_HEX to
decide whether we need the shim or not.

(cherry picked from commit 211bfefbaa)
2021-05-31 16:57:19 +02:00
Ondřej Surý
4db28d79b1 Add uv_os_getenv() and uv_os_setenv() compatibility shims
The uv_os_getenv() and uv_os_setenv() functions were introduced in the
libuv >= 1.12.0.  Add simple compatibility shims for older versions.

(cherry picked from commit 7477d1b2ed)
2021-05-31 16:57:19 +02:00
Ondřej Surý
0ce462ab8e Add uv_req_get_data() and uv_req_set_data() compatibility shims
The uv_req_get_data() and uv_req_set_data() functions were introduced in
libuv >= 1.19.0, so we need to add compatibility shims with older libuv
versions.

(cherry picked from commit f752840db3)
2021-05-31 16:57:19 +02:00
Ondřej Surý
68cb38fc60 Cleanup the uv_import check
The uv_import() is not needed anymore, so we can remove the autoconf
check for it.

(cherry picked from commit 7b02848865)
2021-05-31 16:57:19 +02:00
Michał Kępień
2ed9e86580 Merge branch 'michal/regenerate-man-pages-with-docutils-0.16-v9_16' into 'v9_16'
[v9_16] Regenerate man pages with docutils 0.16

See merge request isc-projects/bind9!5120
2021-05-31 12:55:11 +00:00
Michał Kępień
1d239012a9 Regenerate man pages with docutils 0.16
Commit 070c5fff49 updated the man pages
to contents produced using:

  - Sphinx 4.0.2
  - sphinx-rtd-theme 0.5.2
  - docutils 0.17.1

However, sphinx-rtd-theme 0.5.2 is incompatible with versions 0.17+ of
the docutils package.  This problem was addressed in the Docker image
used for building man pages by downgrading the docutils package to
version 0.16.

Regenerate the man pages again, this time using:

  - Sphinx 4.0.2
  - sphinx-rtd-theme 0.5.2
  - docutils 0.16

This is necessary to prevent the "docs" GitLab CI job from failing.

(cherry picked from commit 6a2daddf5b)
2021-05-31 14:36:11 +02:00
Evan Hunt
4a93c54b56 Merge branch 'kchen-servestale-fix-v9_16' into 'v9_16'
Several serve-stale fixes

See merge request isc-projects/bind9!5118
2021-05-30 19:53:54 +00:00
Matthijs Mekking
89b0a0aa52 Reuse rdatset->ttl when dumping ancient RRsets
Rather than having an expensive 'expired' (fka 'stale_ttl') in the
rdataset structure, that is only used to be printed in a comment on
ancient RRsets, reuse the TTL field of the RRset.

(cherry picked from commit f7f543d99b)
2021-05-30 12:30:36 -07:00
Kevin Chen
3924f78748 Several serve-stale improvements
Commit a83c8cb0af updated masterdump so
that stale records in "rndc dumpdb" output no longer shows 0 TTLs.  In
this commit we change the name of the `rdataset->stale_ttl` field to
`rdataset->expired` to make its purpose clearer, and set it to zero in
cases where it's unused.

Add 'rbtdb->serve_stale_ttl' to various checks so that stale records
are not purged from the cache when they've been stale for RBTDB_VIRTUAL
(300) seconds.

Increment 'ns_statscounter_usedstale' when a stale answer is used.

Note: There was a question of whether 'overmem_purge' should be
purging ancient records, instead of stale ones.  It is left as purging
stale records, since stale records could take up the majority of the
cache.

This submission is copyrighted Akamai Technologies, Inc. and provided
under an MPL 2.0 license.

This commit was originally authored by Kevin Chen, and was updated by
Matthijs Mekking to match recent serve-stale developments.

(cherry picked from commit 0cdf85d204)
2021-05-30 12:30:36 -07:00
Evan Hunt
5daa9066d0 Merge branch '2733-serve-stale-prefetch-crash-v9_16' into 'v9_16'
Fix crash with serve-stale in combination with prefetch

See merge request isc-projects/bind9!5117
2021-05-30 07:57:27 +00:00
Matthijs Mekking
8433a39c54 Add CHANGES and notes for [#2733]
(cherry picked from commit 7ca253818e)
2021-05-30 00:33:42 -07:00
Evan Hunt
de480dcbb6 add a system test for the prefetch bug
Ensure that if prefetch is triggered as a result of a query
restart, it won't have the TRYSTALE_ONTIMEOUT flag set.

(cherry picked from commit 8c047feb3a)
2021-05-30 00:33:42 -07:00
Matthijs Mekking
a2ec3a1e4c Reset DNS_FETCHOPT_TRYSTALE_ONTIMEOUT on resume
Once we resume a query, we should clear DNS_FETCHOPT_TRYSTALE_ONTIMEOUT
from the options to prevent triggering the stale-answer-client-timeout
on subsequent fetches.

If we don't this may cause a crash when for example when prefetch is
triggered after a query restart.

(cherry picked from commit c0dc5937c7)
2021-05-30 00:33:42 -07:00