Commit Graph
34027 Commits
Author SHA1 Message Date
Ondřej Surý 3e433b87fb Add CHANGES and release note for [GL #2732] 2021-05-31 14:52:05 +02:00
Ondřej Surý e83b6569da Indicate to the kernel that we won't be needing the zone dumps
Add a call to posix_fadvise() to indicate to the kernel, that `named`
won't be needing the dumped zone files any time soon with:

 * POSIX_FADV_DONTNEED - The specified data will not be accessed in the
   near future.

Notes:

 POSIX_FADV_DONTNEED attempts to free cached pages associated with the
 specified region. This is useful, for example, while streaming large
 files. A program may periodically request the kernel to free cached
 data that has already been used, so that more useful cached pages are
 not discarded instead.
2021-05-31 14:52:05 +02:00
Ondřej SurýandOndřej Surý 8a5c62de83 Refactor zone dumping code to use netmgr async threadpools
Previously, dumping the zones to the files were quantized, so it doesn't
slow down network IO processing.  With the introduction of network
manager asynchronous threadpools, we can move the IO intensive work to
use that API and we don't have to quantize the work anymore as it the
file IO won't block anything except other zone dumping processes.
2021-05-31 14:52:05 +02:00
Ondřej SurýandOndřej Surý 7670f98377 Add isc_task_getnetmgr() function
Add a function to pull the attached netmgr from inside the executed
task.  This is needed for any task that needs to call the netmgr API.
2021-05-31 14:52:05 +02:00
Ondřej SurýandOndřej Surý 87fe97ed91 Add asynchronous work API to the network manager
The libuv has a support for running long running tasks in the dedicated
threadpools, so it doesn't affect networking IO.

This commit adds isc_nm_work_enqueue() wrapper that would wraps around
the libuv API and runs it on top of associated worker loop.

The only limitation is that the function must be called from inside
network manager thread, so the call to the function should be wrapped
inside a (bound) task.
2021-05-31 14:52:05 +02:00
Ondřej Surý 211bfefbaa Use UV_VERSION_HEX to decide whether we need libuv shim functions
Instead of having a configure check for every missing function that has
been added in later version of libuv, we now use UV_VERSION_HEX to
decide whether we need the shim or not.
2021-05-31 14:52:05 +02:00
Ondřej Surý 7477d1b2ed Add uv_os_getenv() and uv_os_setenv() compatibility shims
The uv_os_getenv() and uv_os_setenv() functions were introduced in the
libuv >= 1.12.0.  Add simple compatibility shims for older versions.
2021-05-31 14:52:05 +02:00
Ondřej Surý f752840db3 Add uv_req_get_data() and uv_req_set_data() compatibility shims
The uv_req_get_data() and uv_req_set_data() functions were introduced in
libuv >= 1.19.0, so we need to add compatibility shims with older libuv
versions.
2021-05-31 14:52:05 +02:00
Ondřej Surý 7b02848865 Cleanup the uv_import check
The uv_import() is not needed anymore, so we can remove the autoconf
check for it.
2021-05-31 14:52:05 +02:00
Michał Kępień db8ff45f0b Merge branch 'michal/regenerate-man-pages-with-docutils-0.16' into 'main'
Regenerate man pages with docutils 0.16

See merge request isc-projects/bind9!5119
2021-05-31 12:34:45 +00:00
Michał Kępień 6a2daddf5b Regenerate man pages with docutils 0.16
Commit bdb777b2a2 updated the man pages
to contents produced using:

  - Sphinx 4.0.2
  - sphinx-rtd-theme 0.5.2
  - docutils 0.17.1

However, sphinx-rtd-theme 0.5.2 is incompatible with versions 0.17+ of
the docutils package.  This problem was addressed in the Docker image
used for building man pages by downgrading the docutils package to
version 0.16.

Regenerate the man pages again, this time using:

  - Sphinx 4.0.2
  - sphinx-rtd-theme 0.5.2
  - docutils 0.16

This is necessary to prevent the "docs" GitLab CI job from failing.
2021-05-31 11:22:52 +02:00
Evan Hunt 6702f23dcd Merge branch 'kchen-servestale-fix' into 'main'
Several serve-stale fixes

See merge request isc-projects/bind9!199
2021-05-30 19:28:59 +00:00
Matthijs MekkingandEvan Hunt f7f543d99b Reuse rdatset->ttl when dumping ancient RRsets
Rather than having an expensive 'expired' (fka 'stale_ttl') in the
rdataset structure, that is only used to be printed in a comment on
ancient RRsets, reuse the TTL field of the RRset.
2021-05-30 11:48:36 -07:00
Kevin ChenandEvan Hunt 0cdf85d204 Several serve-stale improvements
Commit a83c8cb0af updated masterdump so
that stale records in "rndc dumpdb" output no longer shows 0 TTLs.  In
this commit we change the name of the `rdataset->stale_ttl` field to
`rdataset->expired` to make its purpose clearer, and set it to zero in
cases where it's unused.

Add 'rbtdb->serve_stale_ttl' to various checks so that stale records
are not purged from the cache when they've been stale for RBTDB_VIRTUAL
(300) seconds.

Increment 'ns_statscounter_usedstale' when a stale answer is used.

Note: There was a question of whether 'overmem_purge' should be
purging ancient records, instead of stale ones.  It is left as purging
stale records, since stale records could take up the majority of the
cache.

This submission is copyrighted Akamai Technologies, Inc. and provided
under an MPL 2.0 license.

This commit was originally authored by Kevin Chen, and was updated by
Matthijs Mekking to match recent serve-stale developments.
2021-05-30 11:45:35 -07:00
Evan Hunt fc1e3efc41 Merge branch '2733-serve-stale-prefetch-crash' into 'main'
Fix crash with serve-stale in combination with prefetch

Closes #2733

See merge request isc-projects/bind9!5111
2021-05-30 07:31:38 +00:00
Matthijs MekkingandEvan Hunt 7ca253818e Add CHANGES and notes for [#2733] 2021-05-30 00:04:01 -07:00
Evan Hunt 8c047feb3a add a system test for the prefetch bug
Ensure that if prefetch is triggered as a result of a query
restart, it won't have the TRYSTALE_ONTIMEOUT flag set.
2021-05-30 00:04:01 -07:00
Matthijs MekkingandEvan Hunt c0dc5937c7 Reset DNS_FETCHOPT_TRYSTALE_ONTIMEOUT on resume
Once we resume a query, we should clear DNS_FETCHOPT_TRYSTALE_ONTIMEOUT
from the options to prevent triggering the stale-answer-client-timeout
on subsequent fetches.

If we don't this may cause a crash when for example when prefetch is
triggered after a query restart.
2021-05-30 00:03:51 -07:00
Michal Nowak 9fbe80d646 Merge branch 'mnowak/add-fedora-34' into 'main'
Add Fedora 34

See merge request isc-projects/bind9!5075
2021-05-28 10:50:51 +00:00
Michal Nowak 5f27aaa0ff Add Fedora 34 2021-05-28 11:57:31 +02:00
Evan Hunt 5ca27cc33d Merge branch '2731-servestale-dns64' into 'main'
fix a crash when using stale data with dns64

Closes #2731

See merge request isc-projects/bind9!5102
2021-05-27 18:52:31 +00:00
Evan Hunt d8b793760c CHANGES, release note 2021-05-27 10:37:16 -07:00
Matthijs MekkingandEvan Hunt c64589bf46 Test with stale timeout cache miss, then fetch completes
Add a test case where a client request is received and the stale
timeout occurs, but it is not served stale data because there is no entry
in the cache, then is served an authoritative answer once the background
fetch completes. This ensures that a stale timeout only affects a
subsequent response if the client was answered.
2021-05-27 10:35:48 -07:00
Evan Hunt 8bd8e995f1 clean up query correctly if already answered by serve-stale
when a serve-stale answer has been sent, the client continues waiting
for a proper answer. if a final completion event for the client does
arrive, it can just be cleaned up without sending a response, similar
to a canceled fetch.
2021-05-27 10:35:48 -07:00
Evan Hunt 453e905d7e add a test of DNS64 processing with a stale negative response
- send a query for an AAAA which will be resolved as a mapped A
- disable authoritative responses
- wait for the negative AAAA response to become stale
- send another query, wait for the stale answer
- re-enable authorative responses so that a real answer arrives
- currently, this triggers an assertion in query.c
2021-05-27 10:33:31 -07:00
Ondřej Surý e7f5c9582a Merge branch '2708-named-doesn-t-compile-with-gcc-10' into 'main'
Resolve "named doesn't compile with GCC 10."

Closes #2708

See merge request isc-projects/bind9!5057
2021-05-27 06:22:14 +00:00
Mark AndrewsandOndřej Surý d68b009cfe Remove priority from attribute constructor/destructor
On some platforms, the __attribute__ constructor and destructor won't
take priorities and the compilation failed.  On such platform would be
macOS.  For this reason, the constructor/destructor in the libisc was
reworked to not use priorities, but have a single constructor and
destructor that calls the appropriate routines in correct order.

This commit removes the extra priority because it's now not needed and
it also breaks a compilation on macOS with GCC 10.
2021-05-27 08:02:21 +02:00
Mark Andrews 728422d939 Merge branch '2282-shutdown-system-test-needs-to-be-tweaked-to-account-for-recent-netmgr-changes' into 'main'
Resolve ""shutdown" system test needs to be tweaked to account for recent netmgr changes"

Closes #2282

See merge request isc-projects/bind9!4923
2021-05-27 02:56:57 +00:00
Diego FronzaandMark Andrews b19cd2d83b Handling NoNameservers exception
In the shutdown system test multiple queries are sent to a resolver
instance, in the meantime we terminate the same resolver process for
which the queries were sent to, either via rndc stop or a SIGTERM
signal, that means the resolver may not be able to answer all those
queries, since it has initiated the shutdown process.

The dnspython library raises a dns.resolver.NoNameservers exception when
a resolver object fails to receive an answer from the specified list
of nameservers (resolver.nameservers list), we need to handle this
exception as this is something that may happen since we asked the
resolver to terminate, as a result it may not answer clients even if
an answer is available, as the operation will be canceled.
2021-05-27 12:37:49 +10:00
Ondřej Surý 91cd1ef9ff Merge branch 'ondrej/revert-OpenBSD-system-wide-connection-timeout' into 'main'
Revert "Configure the system-wide TCP connection timeout on OpenBSD"

See merge request isc-projects/bind9!5100
2021-05-26 14:24:58 +00:00
Ondřej Surý a0f6275ef8 Revert "Configure the system-wide TCP connection timeout on OpenBSD"
This reverts commit 8ff15fc028.
2021-05-26 16:16:00 +02:00
Mark Andrews d3a8aa4daa Merge branch 'marka-missing-initialisations' into 'main'
Add missing initialisations

See merge request isc-projects/bind9!5097
2021-05-26 08:52:46 +00:00
Mark Andrews 715a2c7fc1 Add missing initialisations
configuring with --enable-mutex-atomics flagged these incorrectly
initialised variables on systems where pthread_mutex_init doesn't
just zero out the structure.
2021-05-26 08:15:08 +00:00
Ondřej Surý 5a10cde89a Merge branch '2722-bad-sizeof-declaration-in-main' into 'main'
Fix the sizeof() for array holding the pointers to clientmgr

Closes #2722

See merge request isc-projects/bind9!5098
2021-05-26 08:10:45 +00:00
Ondřej Surý 2db5290579 Fix the sizeof() for array holding the pointers to clientmgr
The size of the array holding the pointers to clientmgr was created so
big it could hold the actual clientmgr objects, not just the pointer.
This commit fixes the size to be just the ncpus * sizeof(pointer).
2021-05-26 10:03:52 +02:00
Ondřej Surý da282bad6a Merge branch '2721-shutdown-race-in-interfacemgr-cleanup' into 'main'
Cleanup the struct isc_nmiface

Closes #2721

See merge request isc-projects/bind9!5096
2021-05-26 08:00:25 +00:00
Ondřej Surý a227562f13 Cleanup the struct isc_nmiface
In previous MR, I forgot to remove the `struct isc_nmiface`, this commit
rectifies that.
2021-05-26 09:55:10 +02:00
Ondřej Surý 5d325e952d Merge branch '2721-shutdown-race-in-interfacemgr' into 'main'
Make a local copy of interface address when creating new socket

Closes #2721

See merge request isc-projects/bind9!5094
2021-05-26 07:48:00 +00:00
Ondřej Surý 50270de8a0 Refactor the interface handling in the netmgr
The isc_nmiface_t type was holding just a single isc_sockaddr_t,
so we got rid of the datatype and use plain isc_sockaddr_t in place
where isc_nmiface_t was used before.  This means less type-casting and
shorter path to access isc_sockaddr_t members.

At the same time, instead of keeping the reference to the isc_sockaddr_t
that was passed to us when we start listening, we will keep a local
copy. This prevents the data race on destruction of the ns_interface_t
objects where pending nmsockets could reference the sockaddr of already
destroyed ns_interface_t object.
2021-05-26 09:43:12 +02:00
Mark Andrews bef3a9b01f Merge branch '2685-max-ixfr-ratio-appears-to-be-forcing-axfr-very-prematurely-on-bind-9-16-15' into 'main'
Resolve "max-ixfr-ratio appears to be forcing AXFR very prematurely on BIND 9.16.15"

Closes #2685

See merge request isc-projects/bind9!5011
2021-05-25 22:46:52 +00:00
Mark Andrews 80ca95a95c Add release note for [GL #2685] 2021-05-26 08:16:35 +10:00
Mark Andrews d99c312938 Add CHANGES note for [GL #2685] 2021-05-26 08:16:35 +10:00
Mark Andrews 0a45af2e2f Consolidate xhdr fixups 2021-05-26 08:16:35 +10:00
Mark Andrews 68d203ff1c Check that IXFR delta size is correct 2021-05-25 22:27:54 +10:00
Mark Andrews 00609f5094 Correct size calculation in dns_journal_iter_init()
* dns_journal_next() leaves the read point in the journal after the
transaction header so journal_seek() should be inside the loop.
* we need to recover from transaction header inconsistencies

Additionally when correcting for <size, serial0, serial1, 0> the
correct consistency check is isc_serial_gt() rather than
isc_serial_ge().  All instances updated.
2021-05-25 22:27:54 +10:00
Michal Nowak b6bdeb06be Merge branch '2629-man-pages-aren-t-installed-when-building-9-17-x' into 'main'
Install man pages when sphinx-build tool is missing

Closes #2629

See merge request isc-projects/bind9!4945
2021-05-25 09:34:11 +00:00
Michal Nowak 823bf3e79b Install BIND with "make DESTDIR=<PATH> install"
BIND installation should be done by setting DESTDIR during "make
install" not by setting prefix via ./configure.

Make sure that installation with DESTDIR=<PATH> works by checking that
named binary and it's respective man page were installed and that
well-known BIND9 directories - and only them - are present in DESTDIR.

Also rename install path variable from BIND_INSTALL_PATH to
INSTALL_PATH to avoid namespace clash in stress tests which use
BIND_INSTALL_PATH variable to configure path to BIND9 binaries.
2021-05-25 11:21:33 +02:00
Michal Nowak c0566dd8d7 Install man pages when sphinx-build tool is missing
The exclusion of doc/man/ when the sphinx-build tool is missing was
excessive as some targets in doc/man/ are necessary to install man
pages.
2021-05-25 11:21:32 +02:00
Ondřej Surý 9359e1a8c0 Merge branch 'ondrej/dont-set-mctx-name-in-resolver' into 'main'
Don't set memory context name in resolver.c

See merge request isc-projects/bind9!5090
2021-05-25 05:27:59 +00:00
Ondřej Surý d0d37aa6d1 Don't set memory context name in resolver.c
We now attach to existing memory context instead of creating a new
memory context, so we should not set its name.
2021-05-25 07:25:44 +02:00