In order to implement a generic inheritance mechanism of values,
isc_cfgmgr_getval attempts to lookup for the given name in the parent
clauses if the value is not found in the current clause.
This also introduce the notion of "builtin" mode, which enable to prefix
the builtin configuration (bin/named/config.c) with "default.", and
handle it as a specific case when the parent is a top-level clause,
effectively enabling to also support inheritance of options values to
the builtin-ones.
Environment variable NAMED_CFGMGR_PERSIST being set enable to keep LMDB
DB created by cfgmgr. For now usecase is only for debugging (i.e.
inspecting what has been actually written in the DB).
- pack the cfgmgr type enum, so it likely takes 1 byte rather than 4
bytes
- when putting a string in LMDB, let's allocate only the size of type +
strlen(string) + 1, instead of the size of the whole cfgmgr_val_t +
strlen(string) + 1. It needs a special threatment when reading, but
probably worth it considering cfgmgr_val_t is not tiny (and will
likely grows up)
- revert previous changes from configure.ac script in order to re-use
the existing logic, as well as the gitlab ci flag which check build
with two combinations.
- Conditionally build LMDB source code based on HAVE_LMDB. So if
--without-lmdb is used, HAVE_LMDB won't be set and LMDB source code
will be compiled and included in libisc. This enable code using LMDB
to link no matter LMDB is OS-provided or not.
- Regarding the LMDB headers, and in order to make future LMDB update
easier, all original OpenLDAP code is wrapped under
lib/isc/openldap-lmdb including the headers. This avoid extra
changes (putting aside liscence header) to mdb.c to find out its
headers. The counter-part is the relative inclusion of this header
from isc/lmdb.h header, but it's probably a good idea to not make
this header public anyway.
- update LMDB license to add BIND9 SPDX headers but mentioning LMDB
lisence
As several places uses custom LMDB macros, this centralize those
definitions inside a dedicated `lib/isc/include/isc/lmdb.h` header
file, so future uses cases not DNS-centrics could fit into there as
well.
MDB_CREATE is removed as well because:
- it was useless (it's needed only for named DB, which is not the case
for all use case so far)
- it was used in `mdb_env_open` which is not expecting this flag: it
should be passed (if needed) to `mdb_dbi_open`. It was likely
ignored so far.
Few points to note (and possibly discuss):
- cfgmgr is build on top of LMDB, as it brings transaction (and so
thread safe) support out of the box
- LMDB keys are build in a way that we can support repeatable (see
https://pad.isc.org/p/cfgmgr-proposal-v2 even if most of it is
outdated, the section 2.0.2 about the way it's build is still
relevant, otherwise I hope the code changes here are clear enough)
- Each thread own its own cfgmgr context (which basically means LMDB
transaction, and where the API points to inside the configuration)
- In order to avoid allocations everytime we get/set a value (as well
as few other operations) a single buffer is pre-allocated per-thread
and per-transaction. Now, few internal helper functions directly use
it (instead or, let's say, work on a parameter) and this might be
confusing and error prone, I'm happy to change that if it is a
worry.
- The data type which can be read/wrote from cfgmgr is not exhaustive,
more data type will be added (i.e. duration type, uint64 if needed,
etc.)
- This current implementation does not support inheritance (i.e. a
non-specified view option won't use the option one). It's something
we need to discuss, I see some options that could be put on top of
that's here.
- The "default" values, however, should be fine out-of-the-box: I
think the default configuration in bin/named/config.c can be
parse/"added" in cfgmgr first then user one on top of that (because
writting an existing value override it). Obviously there would be
no-way to "go back" to the default config only, but I don't see such
use case in existing code. Even if we'd need such thing, that would
be quite invasive and a full config reload would be advisable. (so
we could add an API to entirely drop the cfgmgr data and start from
scratch. But I don't see such use case right now anyway, so it's not
there)
- I hope the things a user of cfgmgr must know should be clearly
explained in the cfgmgr.h file (if it's not the case, then I need
to fix it -- That said I'll likely re-work the doc anyway).
- I initially implemented a mechanism which would dynamically re-size
key buffers in case keys are very long, but I remove this as LMDB
doesn't supports key more than 511 bytes anyway. Instead I made
assertions every time we build a key to make sure we don't exceed
this value.
Instead of relying on unreliable order of execution of the library
constructors and destructors, move them to individual binaries. The
advantage is that the execution time and order will remain constant and
will not depend on the dynamic load dependency solver.
Merge branch 'ondrej/move-the-constructors-destructors-to-binaries' into 'main'
See merge request isc-projects/bind9!10069
As the default_call_rcu_thread can't be forced to flush all the work
during the executable shutdown, create one call_rcu_thread explicitly
and assign it to the all created threads.
This allows this explicit call_rcu_thread to be unassociated from the
main thread and freed before the executable destructor exits.
Instead of relying on unreliable order of execution of the library
constructors and destructors, move them to individual binaries. The
advantage is that the execution time and order will remain constant and
will not depend on the dynamic load dependency solver.
This requires more work, but that was mitigated by a simple requirement,
any executable using libisc and libdns, must include <isc/lib.h> and
<dns/lib.h> respectively (in this particular order). In turn, these two
headers must not be included from within any library as they contain
inlined functions marked with constructor/destructor attributes.
Previously, active resolver fetches were only dumped when the `fetches-per-zone` configuration option was enabled. Now, active resolver fetches are dumped along with the number of `clients-per-server` counters per resolver fetch.
Merge branch 'ondrej/make-dns_resolver_dumpfetches-dump-fetches' into 'main'
See merge request isc-projects/bind9!10107
Previously, the dns_resolver_dumpfetches() would go over the fetch
counters. Alas, because of the earlier optimization, the fetch counters
would be increased only when fetches-per-zone was not 0, otherwise the
whole counting was skipped for performance reasons.
Instead of using the auxiliary fetch counters hash table, use the real
hash table that stores the fetch contexts to dump the ongoing fetches to
the recursing file.
Additionally print more information about the fetch context like start
and expiry times, number of fetch responses, number of queries and count
of allowed and dropped fetches.
Previously, a data race could cause a newly created fetch context for a new client to be used
before it had been fully initialized, which would cause the query to become stuck; queries for the same
data would be either paused indefinitely or dropped because of
the `clients-per-query` limit. This has been fixed.
Closes#5053
Merge branch '5053-fetch-context-create-data-race' into 'main'
See merge request isc-projects/bind9!10146
The order of the fetch context hash table rwlock and the individual
fetch context was reversed when calling the release_fctx() function.
This was causing a problem when iterating the hash table, and thus the
ordering has been corrected in a way that the hash table rwlock is now
always locked on the outside and the fctx lock is the interior lock.
In the next commit, we need to know whether the timer has been started
or stopped. Add isc_timer_running() function that returns true if the
timer has been started.
Update Sphinx-related Python packages to their current versions pulled
in by "pip install sphinx-rtd-theme" run in a fresh virtual environment.
Merge branch 'michal/update-sphinx-and-sphinx_rtd_theme' into 'main'
See merge request isc-projects/bind9!10138
With RPZ in use, `named` could terminate unexpectedly because of a race condition when a reconfiguration command was received using `rndc`. This has been fixed.
Closes#5146
Merge branch '5146-rpz-reconfig-bug-fix' into 'main'
See merge request isc-projects/bind9!10079
After a reconfiguration the old view can be left without a valid
'rpzs' member, because when the RPZ is not changed during the named
reconfiguration 'rpzs' "migrate" from the old view into the new
view, so when a query resumes it can find that 'qctx->view->rpzs'
is NULL which query_resume() currently doesn't expect to happen if
it's recursing and 'qctx->rpz_st' is not NULL.
Fix the issue by adding a NULL-check. In order to not split the log
message to two different log messages depending on whether
'qctx->view->rpzs' is NULL or not, change the message to not log
the RPZ policy's "version" which is just a runtime counter and is
most likely not very useful for the users.
The isc_counter_create() doesn't need the return value (it was always
ISC_R_SUCCESS), use the macros to implement the reference counting,
little style cleanup, and expand the unit test.
Merge branch 'ondrej/cleanup-isc_counter-unit' into 'main'
See merge request isc-projects/bind9!10126
The isc_counter_create() doesn't need the return value (it was always
ISC_R_SUCCESS), use the macros to implement the reference counting,
little style cleanup, and expand the unit test.
Previously, when parsing responses, named incorrectly rejected responses without matching RRSIG records for NSEC/DS/NSEC3 records in the authority section. This rejection, if appropriate, should have been left for the validator to determine and has been fixed.
Closes#5185
Merge branch '5185-remove-rrsig-check-from-dns_message_parse' into 'main'
See merge request isc-projects/bind9!10125
Checking whether the authority section is properly signed should
be left to the validator. Checking in getsection (dns_message_parse)
was way too early and resulted in resolution failures of lookups
that should have otherwise succeeded.
Previously a hard-coded limitation of maximum two key or message
verification checks were introduced when checking the message's
SIG(0) signature. It was done in order to protect against possible
DoS attacks. The logic behind choosing the number 2 was that more
than a single key should only be required during key rotations, and
in that case two keys are enough. But later it became apparent that
there are other use cases too where even more keys are required, see
issue number #5050 in GitLab.
This change introduces two new configuration options for the views,
`sig0key-checks-limit` and `sig0message-checks-limit`, which define how
many keys are allowed to be checked to find a matching key, and how
many message verifications are allowed to take place once a matching
key has been found. The latter protects against expensive cryptographic
operations when there are keys with colliding tags and algorithm
numbers, with default being 2, and the former protects against a bit
less expensive key parsing operations and defaults to 16.
Closes#5050
Merge branch '5050-sig0-let-considering-more-than-two-keys' into 'main'
See merge request isc-projects/bind9!9967
Previously a hard-coded limitation of maximum two key or message
verification checks were introduced when checking the message's
SIG(0) signature. It was done in order to protect against possible
DoS attacks. The logic behind choosing the number two was that more
than one key should only be required only during key rotations, and
in that case two keys are enough. But later it became apparent that
there are other use cases too where even more keys are required, see
issue number #5050 in GitLab.
This change introduces two new configuration options for the views,
sig0key-checks-limit and sig0message-checks-limit, which define how
many keys are allowed to be checked to find a matching key, and how
many message verifications are allowed to take place once a matching
key has been found. The latter protects against expensive cryptographic
operations when there are keys with colliding tags and algorithm
numbers, with default being 2, and the former protects against a bit
less expensive key parsing operations and defaults to 16.
Running jobs which were entered into the isc_quota queue is the
responsibility of the isc_quota_release() function, which, when
releasing a previously acquired quota, checks whether the queue
is empty, and if it's not, it runs a job from the queue without touching
the 'quota->used' counter. This mechanism is susceptible to a possible
hangup of a newly queued job in case when between the time a decision
has been made to queue it (because used >= max) and the time it was
actually queued, the last quota was released. Since there is no more
quotas to be released (unless arriving in the future), the newly
entered job will be stuck in the queue.
Fix the issue by adding checks in both isc_quota_release() and
isc_quota_acquire_cb() to make sure that the described hangup does
not happen. Also see code comments.
Closes#4965
Merge branch '4965-isc_quota-bug-fix' into 'main'
See merge request isc-projects/bind9!10082
Running jobs which were entered into the isc_quota queue is the
responsibility of the isc_quota_release() function, which, when
releasing a previously acquired quota, checks whether the queue
is empty, and if it's not, it runs a job from the queue without touching
the 'quota->used' counter. This mechanism is susceptible to a possible
hangup of a newly queued job in case when between the time a decision
has been made to queue it (because used >= max) and the time it was
actually queued, the last quota was released. Since there is no more
quotas to be released (unless arriving in the future), the newly
entered job will be stuck in the queue.
Fix the wrong memory ordering for 'quota->used', as the relaxed
ordering doesn't ensure that data modifications made by one thread
are visible in other threads.
Add checks in both isc_quota_release() and isc_quota_acquire_cb()
to make sure that the described hangup does not happen. Also see
code comments.
A new option 'min-transfer-rate-in <bytes> <minutes>' has been added
to the view and zone configurations. It can abort incoming zone
transfers which run very slowly due to network related issues, for
example. The default value is set to 10240 bytes in 5 minutes.
Closes#3914
Merge branch '3914-detect-and-restart-stalled-zone-transfers' into 'main'
See merge request isc-projects/bind9!9098
Expose the average transfer rate (in bytes-per-second) during the
last full 'min-transfer-rate-in <bytes> <minutes>' minutes interval.
If no such interval has passed yet, then the overall average rate is
reported instead.
Add a new big zone, run a zone transfer in slow mode, and check
whether the zone transfer gets canceled because 100000 bytes are
not transferred in 5 seconds (as it's running in slow mode).
Colin Vidal
Evan HuntandColin Vidal
Ondřej Surý
Michał Kępień
Arаm Sаrgsyаn
Mark Andrews