when signatures were not added because of too many types already
existing at a node, the diff was not being cleaned up; this led to
a memory leak being reported at shutdown.
(cherry picked from commit 2825bdb1ae5be801e7ed603ba2455ed9a308f1f7)
Previously, the number of RR types for a single owner name was limited
only by the maximum number of the types (64k). As the data structure
that holds the RR types for the database node is just a linked list, and
there are places where we just walk through the whole list (again and
again), adding a large number of RR types for a single owner named with
would slow down processing of such name (database node).
Add a configurable limit to cap the number of the RR types for a single
owner. This is enforced at the database (rbtdb, qpzone, qpcache) level
and configured with new max-types-per-name configuration option that
can be configured globally, per-view and per-zone.
(cherry picked from commit 00d16211d6368b99f070c1182d8c76b3798ca1db)
Previously, the number of RRs in the RRSets were internally unlimited.
As the data structure that holds the RRs is just a linked list, and
there are places where we just walk through all of the RRs, adding an
RRSet with huge number of RRs inside would slow down processing of said
RRSets.
Add a configurable limit to cap the number of the RRs in a single RRSet.
This is enforced at the database (rbtdb, qpzone, qpcache) level and
configured with new max-records-per-type configuration option that can
be configured globally, per-view and per-zone.
(cherry picked from commit 3fbd21f69a1bcbd26c4c00920e7b0a419e8762fc)
When calling dns_resolver_createfetch in resolver.c with a callback
of resume_dslookup, clear DNS_FETCHOPT_TRYSTALE_ONTIMEOUT from
options as DNS_EVENT_TRYSTALE is not an expected event type and
triggers a REQUIRE.
When the cache's memory context was in over memory state when the
cache was flushed it resulted in LRU cleaning removing newly entered
data in the new cache straight away until the old cache had been
destroyed enough to take it out of over memory state. When flushing
the cache create a new memory context for the new db to prevent this.
(cherry picked from commit 5e77edd074)
When named is started with -4 or -6 and the primaries for a zone
do not have an IPv4 or IPv6 address respectively issue a log message.
(cherry picked from commit 2cd4303249)
The `axfr_makedb()` didn't set the loop on the newly created database,
effectively killing delayed cleaning on such database. Move the
database creation into dns_zone API that knows all the gory details of
creating new database suitable for the zone.
(cherry picked from commit 3310cac2b0)
- duplicated question
- duplicated answer
- qtype as an answer
- two question types
- question names
- nsec3 bad owner name
- short record
- short question
- mismatching question class
- bad record owner name
- mismatched class in record
- mismatched KEY class
- OPT wrong owner name
- invalid RRSIG "covers" type
- UPDATE malformed delete type
- TSIG wrong class
- TSIG not the last record
(cherry picked from commit 6e9ed4983e)
DNSKEY was incorrectly being added to the NESC/NSEC3 type bit map
when it was obscured by the delegation. This lead to zone verification
failures.
(cherry picked from commit ec3c624814)
If a query sent using the dns_request API times out when the view it was
associated with gets torn down, the dns_dispatch_resume() call in
req_response() may be issued with the 'resp' argument set to NULL,
triggering an assertion failure. Consider the following scenario ([A]
and [B] are thread identifiers):
1. [A] Read timeout for a dispatch query fires.
2. [A] udp_recv() is called. It locks the dispatch, determines it
timed out, prepares for calling the higher-level callback with
ISC_R_TIMEDOUT, and unlocks the dispatch (lib/dns/dispatch.c:633).
3. [B] The last reference to a view is released.
dns_requestmgr_shutdown() is called, canceling all in-flight
requests for that view. (Note that udp_recv() in thread [A] already
unlocked the dispatch, so its state can be modified.) As a part of
this process, request_cancel() calls dns_dispatch_done() on
request->dispentry, setting it to NULL.
4. [A] udp_recv() calls the higher-level callback (req_response()) with
ISC_R_TIMEDOUT.
5. [A] Since the request timed out, req_response() retries sending it.
In the process, it calls dns_dispatch_resume(), passing
request->dispentry as the 'resp' argument.
6. [A] Since 'resp' is NULL, the REQUIRE(VALID_RESPONSE(resp));
assertion in dns_dispatch_resume() fails.
Fix by checking whether the request has been canceled before calling
dns_dispatch_resume(), similarly to how it is done in req_connected()
and req_senddone().
With weak zone attachments being used for catzs, catzs->view->zonetable
may be NULL so we need to account for this which dns_view_findzone
does. This is already done in main.
Use dns_view_weakattach and dns_view_weakdetach to maintain a
reference to the view referenced through catzs->view.
(cherry picked from commit 307e3ed9a6)
Allow SVBC (HTTPS) alias form with parameters to be accepted from
the wire and when transfered. This is for possible future extensions.
(cherry picked from commit 799046929c)
When parsing a zonefile named-checkzone (and others) could loop
infinitely if a directory was $INCLUDED. Record the error and treat
as EOF when looking for multiple errors.
This was found by Eric Sesterhenn from X41.
(cherry picked from commit efd27bb82d)
In nibble mode if the value to be converted was negative the parser
would loop forever. Process the value as an unsigned int instead
of as an int to prevent sign extension when shifting.
This was found by Eric Sesterhenn from X41.
(cherry picked from commit 371824f078)
If the zone is signed with a different way than 'dnssec-policy', use
the legacy way of jittering signatures, that is calculate jitter by
taking the two values of 'sig-validity-interval' and subtracting the
second value from the first value.
Having a value higher than signatures-validity does not make sense
and should be treated as a configuration error.
(cherry picked from commit c3d8932f79)
When calculating the RRSIG validity, jitter is now derived from the
config option rather than from the refresh value.
(cherry picked from commit 67f403a423)
dns_db_addrdataset() enforces a requirement that version can only be
NULL for a cache database. code that checks for zone semantics and
version == NULL can never be reached.
(cherry picked from commit b3c8b5cfb2)
The Depends relation refers to types of rollovers in which a certain
record type is going to be swapped. Specifically, the Depends relation
says there should be no dependency on the predecessor key (the set
Dep(x, T) must be empty).
But if the key is phased out (all its states are in HIDDEN), there is
no longer a dependency. Since the relationship is still maintained
(Predecessor and Successor metadata), the keymgr_dep function still
returned true. In other words, the set Dep(x, T) is not considered
empty.
This slows down key rollovers, only retiring keys when the successor
key has been fully propagated.
(cherry picked from commit 0aac81cf80)
The dns_cache_flush() drops the old database and creates a new one, but
it forgets to create the task(s) that runs the node pruning and cleaning
the rbtdb when flushing it next time. This causes the cleaning to skip
cleaning the parent nodes (with .down == NULL) leading to increased
memory usage over time until the database is unable to keep up and just
stays overmem all the time.
(cherry picked from commit 79040a669c)
Previously, rbtdb->task had quantum of 1 because it was originally used
just for freeing RBTDB contents, which can happen on a "best effort"
basis (does not need to be prioritized). However, when tree pruning was
implemented, it also started sending events to that task, enabling the
latter to become clogged up with a significant event backlog because it
only pruned a single RBTDB node per event.
To prioritize tree pruning (as it is necessary for enforcing the
configured memory use limit for the cache memory context), create a
second task with a virtually unlimited quantum (UINT_MAX) and send the
tree-pruning events to this new task, to ensure that all nodes scheduled
for pruning will be processed before further nodes are queued in a
similar fashion.
This change enables dropping the prunenodes list and restoring the
originally-used logic that allocates and sends a separate event for each
node to prune.
(cherry picked from commit 231b2375e5)
Reconstruct the variant of the prune_tree() parent cleaning to consider
all elibible parents in a single loop as we were doing before all the
changes that led to this commit.
Update code comments so that they more precisely describe what the
relevant bits of code actually do.
(cherry picked from commit 3a01c749f9)
The dns_cache_flush() drops the old database and creates a new one, but
it forgets to create the task(s) that runs the node pruning and cleaning
the rbtdb when flushing it next time. This causes the cleaning to skip
cleaning the parent nodes (with .down == NULL) leading to increased
memory usage over time until the database is unable to keep up and just
stays overmem all the time.
Previously, rbtdb->task had quantum of 1 because it was originally used
just for freeing RBTDB contents, which can happen on a "best effort"
basis (does not need to be prioritized). However, when tree pruning was
implemented, it also started sending events to that task, enabling the
latter to become clogged up with a significant event backlog because it
only pruned a single RBTDB node per event.
To prioritize tree pruning (as it is necessary for enforcing the
configured memory use limit for the cache memory context), create a
second task with a virtually unlimited quantum (UINT_MAX) and send the
tree-pruning events to this new task, to ensure that all nodes scheduled
for pruning will be processed before further nodes are queued in a
similar fashion.
This change enables dropping the prunenodes list and restoring the
originally-used logic that allocates and sends a separate event for each
node to prune.
Reconstruct the variant of the prune_tree() parent cleaning to consider
all elibible parents in a single loop as we were doing before all the
changes that led to this commit.
Update code comments so that they more precisely describe what the
relevant bits of code actually do.
(cherry picked from commit 454c75a33a)
It was discovered that the TTL-based cleaning could build up
a significant backlog of the rdataset headers during the periods where
the top of the TTL heap isn't expired yet. Make the TTL-based cleaning
more aggressive by cleaning more headers from the heap when we are
adding new header into the RBTDB.
(cherry picked from commit d8220ca4ca)
(cherry picked from commit b4d9f1cbab)
It was discovered that an expired header could sit on top of the heap
a little longer than desireable. Remove expired headers (headers with
rdh_ttl set to 0) from the heap completely, so they don't block the next
TTL-based cleaning.
(cherry picked from commit a9383e4b95)
(cherry picked from commit 756555dbcf)
It was discovered that the TTL-based cleaning could build up
a significant backlog of the rdataset headers during the periods where
the top of the TTL heap isn't expired yet. Make the TTL-based cleaning
more aggressive by cleaning more headers from the heap when we are
adding new header into the RBTDB.
(cherry picked from commit d8220ca4ca)
It was discovered that an expired header could sit on top of the heap
a little longer than desireable. Remove expired headers (headers with
rdh_ttl set to 0) from the heap completely, so they don't block the next
TTL-based cleaning.
(cherry picked from commit a9383e4b95)
Instead of juggling with node locks in a cycle, cleanup the node we are
just pruning and send any the parent that's also subject to the pruning
to the prune tree via normal way (e.g. enqueue pruning on the parent).
This simplifies the code and also spreads the pruning load across more
event loop ticks which is better for lock contention as less things run
in a tight loop.
(cherry picked from commit 0b32d323e0)
(cherry picked from commit a4c225cb6d)
The log message for commit a9af1ac5ae
explained:
In some older BIND 9 branches, the extra queuing overhead eliminated by
this change could be remotely exploited to cause excessive memory use.
Due to architectural shift, this branch is not vulnerable to that issue,
but applying the fix to the latter is nevertheless deemed prudent for
consistency and to make the code future-proof.
However, it turned out that having a single queue for the nodes to be
pruned increased lock contention to a level where cleaning up nodes from
the RBTDB took too long, causing the amount of memory used by the cache
to grow indefinitely over time.
This commit reverts the change to the pruning mechanism introduced by
commit a9af1ac5ae as BIND branches newer
than 9.16 were not affected by the excessive event queueing overhead
issue mentioned in the log message for the above commit.
(cherry picked from commit eed17611d8)
(cherry picked from commit 4b32456705)
Instead of juggling with node locks in a cycle, cleanup the node we are
just pruning and send any the parent that's also subject to the pruning
to the prune tree via normal way (e.g. enqueue pruning on the parent).
This simplifies the code and also spreads the pruning load across more
event loop ticks which is better for lock contention as less things run
in a tight loop.
(cherry picked from commit 0b32d323e0)
The log message for commit a9af1ac5ae
explained:
In some older BIND 9 branches, the extra queuing overhead eliminated by
this change could be remotely exploited to cause excessive memory use.
Due to architectural shift, this branch is not vulnerable to that issue,
but applying the fix to the latter is nevertheless deemed prudent for
consistency and to make the code future-proof.
However, it turned out that having a single queue for the nodes to be
pruned increased lock contention to a level where cleaning up nodes from
the RBTDB took too long, causing the amount of memory used by the cache
to grow indefinitely over time.
This commit reverts the change to the pruning mechanism introduced by
commit a9af1ac5ae as BIND branches newer
than 9.16 were not affected by the excessive event queueing overhead
issue mentioned in the log message for the above commit.
(cherry picked from commit eed17611d8)
