Commit Graph
13536 Commits
Author SHA1 Message Date
Aram Sargsyan aef3f26aaf Fix a bug in isc_rwlock_trylock()
When isc_rwlock_trylock() fails to get a read lock because another
writer was faster, it should wake up other waiting writers in case
there are no other readers, but the current code forgets about
the currently active writer when evaluating 'cntflag'.

Unset the WRITER_ACTIVE bit in 'cntflag' before checking to see if
there are other readers, otherwise the waiting writers, if they exist,
might not wake up.

(cherry picked from commit 73b6d9e9e5)
2025-01-08 10:30:05 +00:00
Michal Nowak 2ac61b21bb Fix formatting in lib/ns/query.c broken by !9284 2024-08-26 14:25:55 +02:00
Evan HuntandOndřej Surý 3e0643a6dc implement 'max-query-restarts'
implement, document, and test the 'max-query-restarts' option
which specifies the query restart limit - the number of times
we can follow CNAMEs before terminating resolution.

(cherry picked from commit 104f3b82fb)
(cherry picked from commit 2e04f0380c)
2024-08-20 17:35:07 +00:00
Evan HuntandOndřej Surý a7a479c258 reduce the max-recursion-queries default to 32
the number of iterative queries that can be sent to resolve a
name now defaults to 32 rather than 100.

(cherry picked from commit 7e3b425dc2)
(cherry picked from commit a11367ade3)
2024-08-20 17:35:07 +00:00
Evan HuntandOndřej Surý 95376b45f5 make "max_restarts" a configurable value
MAX_RESTARTS is no longer hard-coded; ns_server_setmaxrestarts()
and dns_client_setmaxrestarts() can now be used to modify the
max-restarts value at runtime. in both cases, the default is 11.

(cherry picked from commit c5588babaf)
(cherry picked from commit bfbc6a6c84)
2024-08-20 17:35:07 +00:00
Evan HuntandOndřej Surý d1797c9151 reduce MAX_RESTARTS to 11
the number of steps that can be followed in a CNAME chain
before terminating the lookup has been reduced from 16 to 11.
(this is a hard-coded value, but will be made configurable later.)

(cherry picked from commit 05d78671bb)
(cherry picked from commit dd88a4cdfc)
2024-08-20 17:35:07 +00:00
Evan HuntandOndřej Surý 2b756caa4c add debug logging when creating or attaching to a query counter
fctx_create() now logs at debug level 9 when the fctx attaches
to an existing counter or creates a new one.

(cherry picked from commit 825f3d68c5)
(cherry picked from commit 14bce7e275)
2024-08-20 17:35:07 +00:00
Evan HuntandOndřej Surý f89f8af34a apply max-recursion-queries quota to validator queries
previously, validator queries for DNSKEY and DS records were
not counted toward the quota for max-recursion-queries; they
are now.

(cherry picked from commit af7db89513)
(cherry picked from commit 18e39d989f)
2024-08-20 17:35:07 +00:00
Evan HuntandOndřej Surý 1e54e9bb92 attach query counter to NS fetches
there were cases in resolver.c when queries for NS records were
started without passing a pointer to the parent fetch's query counter;
as a result, the max-recursion-queries quota for those queries started
counting from zero, instead of sharing the limit for the parent fetch,
making the quota ineffective in some cases.

(cherry picked from commit d3b7e92783)
(cherry picked from commit 5ab4cae4ed)
2024-08-20 17:35:07 +00:00
JINMEI TatuyaandOndřej Surý aca98dfaa5 add a trivial wrapper for uv_stream_get_write_queue_size
(cherry picked from commit b9bef2cc89)
2024-08-05 10:31:15 +00:00
Ondřej SurýandPetr Špaček e5a62e9407 Add the ability specify the signing / verification time
When fuzzing it is useful for all signing operations to happen
at a specific time for reproducability.  Add two variables to
the message structure (fuzzing and fuzztime) to specify if a
fixed time should be used and the value of that time.

(cherry picked from commit 3e85d8c3d6)
2024-08-02 11:18:25 +02:00
Matthijs MekkingandOndřej Surý d72adf4b92 Log error when update fails
The new "too many records" error can make an update fail without the
error being logged. This commit fixes that.

(cherry picked from commit 558923e5405894cf976d102f0d246a28bdbb400c)
2024-07-29 12:27:25 +02:00
Ondřej Surý 92a680a3ef Be smarter about refusing to add many RR types to the database
Instead of outright refusing to add new RR types to the cache, be a bit
smarter:

1. If the new header type is in our priority list, we always add either
   positive or negative entry at the beginning of the list.

2. If the new header type is negative entry, and we are over the limit,
   we mark it as ancient immediately, so it gets evicted from the cache
   as soon as possible.

3. Otherwise add the new header after the priority headers (or at the
   head of the list).

4. If we are over the limit, evict the last entry on the normal header
   list.

(cherry picked from commit 57cd34441a)
2024-07-29 12:27:25 +02:00
Evan HuntandOndřej Surý a080317de0 fix a memory leak that could occur when signing
when signatures were not added because of too many types already
existing at a node, the diff was not being cleaned up; this led to
a memory leak being reported at shutdown.

(cherry picked from commit 2825bdb1ae5be801e7ed603ba2455ed9a308f1f7)
2024-07-29 12:27:25 +02:00
Ondřej Surý 89f1779bc2 Add a limit to the number of RR types for single name
Previously, the number of RR types for a single owner name was limited
only by the maximum number of the types (64k).  As the data structure
that holds the RR types for the database node is just a linked list, and
there are places where we just walk through the whole list (again and
again), adding a large number of RR types for a single owner named with
would slow down processing of such name (database node).

Add a configurable limit to cap the number of the RR types for a single
owner.  This is enforced at the database (rbtdb, qpzone, qpcache) level
and configured with new max-types-per-name configuration option that
can be configured globally, per-view and per-zone.

(cherry picked from commit 00d16211d6368b99f070c1182d8c76b3798ca1db)
2024-07-29 12:27:25 +02:00
Ondřej Surý f63d72fb7e Add a limit to the number of RRs in RRSets
Previously, the number of RRs in the RRSets were internally unlimited.
As the data structure that holds the RRs is just a linked list, and
there are places where we just walk through all of the RRs, adding an
RRSet with huge number of RRs inside would slow down processing of said
RRSets.

Add a configurable limit to cap the number of the RRs in a single RRSet.
This is enforced at the database (rbtdb, qpzone, qpcache) level and
configured with new max-records-per-type configuration option that can
be configured globally, per-view and per-zone.

(cherry picked from commit 3fbd21f69a1bcbd26c4c00920e7b0a419e8762fc)
2024-07-28 16:55:28 +02:00
Mark AndrewsandNicki Křížek a61be8eef0 Clear qctx->zversion
Clear qctx->zversion when clearing qctx->zrdataset et al in
lib/ns/query.c:qctx_freedata.  The uncleared pointer could lead to
an assertion failure if zone data needed to be re-saved which could
happen with stale data support enabled.

(cherry picked from commit 179fb3532ab8d4898ab070b2db54c0ce872ef709)
2024-07-24 13:51:41 +02:00
Petr ŠpačekandNicki Křížek 0820a90b41 Remove support for SIG(0) message verification
(cherry picked from commit 857fd5c346e3309ee8e280c29174b46579af5a13)
2024-07-24 13:15:27 +02:00
Ondřej SurýandNicki Křížek 26c9da5f28 Be smarter about refusing to add many RR types to the database
Instead of outright refusing to add new RR types to the cache, be a bit
smarter:

1. If the new header type is in our priority list, we always add either
   positive or negative entry at the beginning of the list.

2. If the new header type is negative entry, and we are over the limit,
   we mark it as ancient immediately, so it gets evicted from the cache
   as soon as possible.

3. Otherwise add the new header after the priority headers (or at the
   head of the list).

4. If we are over the limit, evict the last entry on the normal header
   list.

(cherry picked from commit 57cd34441a)
2024-07-24 12:12:04 +02:00
Ondřej SurýandNicki Křížek d56d2a32b8 Expand the list of the priority types
Add HTTPS, SVCB, SRV, PTR, NAPTR, DNSKEY and TXT records to the list of
the priority types that are put at the beginning of the slabheader list
for faster access and to avoid eviction when there are more types than
the max-types-per-name limit.

(cherry picked from commit b27c6bcce8)
2024-07-24 12:11:59 +02:00
Ondřej SurýandNicki Křížek dfcadc2085 Add a limit to the number of RR types for single name
Previously, the number of RR types for a single owner name was limited
only by the maximum number of the types (64k).  As the data structure
that holds the RR types for the database node is just a linked list, and
there are places where we just walk through the whole list (again and
again), adding a large number of RR types for a single owner named with
would slow down processing of such name (database node).

Add a hard-coded limit (100) to cap the number of the RR types for a single
owner.  The limit can be changed at the compile time by adding following
define to CFLAGS:

    -DDNS_RBTDB_MAX_RTYPES=<limit>
2024-07-24 10:56:59 +02:00
Ondřej SurýandNicki Křížek fdabf4b957 Add a limit to the number of RRs in RRSets
Previously, the number of RRs in the RRSets were internally unlimited.
As the data structure that holds the RRs is just a linked list, and
there are places where we just walk through all of the RRs, adding an
RRSet with huge number of RRs inside would slow down processing of said
RRSets.

The fix for end-of-life branches make the limit compile-time only for
simplicity and the limit can be changed at the compile time by adding
following define to CFLAGS:

    -DDNS_RDATASET_MAX_RECORDS=<limit>

(cherry picked from commit c5c4d00c38530390c9e1ae4c98b65fbbadfe9e5e)
2024-07-24 10:56:57 +02:00
Mark AndrewsandNicki Křížek b36e1cfbcd Clear DNS_FETCHOPT_TRYSTALE_ONTIMEOUT
When calling dns_resolver_createfetch in resolver.c with a callback
of resume_dslookup, clear DNS_FETCHOPT_TRYSTALE_ONTIMEOUT from
options as DNS_EVENT_TRYSTALE is not an expected event type and
triggers a REQUIRE.

(cherry picked from commit 6faea6da3d646557d234d63ddd5d524d222e8082)
2024-06-06 18:28:35 +02:00
Mark AndrewsandEvan Hunt 9a0769d817 Clang-format header file changes 2024-05-17 16:45:43 -07:00
Michal Nowak 99eb2ebaa9 Update sources to Clang 18 formatting
(cherry picked from commit f454fa6dea)
2024-04-23 16:59:58 +02:00
Mark AndrewsandPetr Špaček 6ec335798a Check that sig0 name is the root.
(cherry picked from commit 5b5f1ba0b2)
2024-04-07 05:22:59 +00:00
Mark AndrewsandPetr Špaček f8a464a54a Always clean sig0name in msgresetsigs() and dns_message_renderreset()
The fuzzing harness operates on dns_message_t in non-standard ways
and if 'sig0name' is non-NULL when msgresetsigs() and
dns_message_renderreset() are called it should be cleaned up.

(cherry picked from commit 450fab92b1)
2024-04-07 05:22:59 +00:00
Michał Kępień c340f9d6e4 Merge tag 'v9.16.49' into bind-9.16 2024-03-20 14:37:45 +01:00
Mark Andrews 3bcd6385d4 Only call memmove if the rdata length is non zero
This avoids undefined behaviour on zero length rdata where the
data pointer is NULL.

(cherry picked from commit 228cc557fe)
2024-03-14 11:06:25 +11:00
Michał Kępień b138931fc0 Account for changes to struct dns_rbtnode
Commit eba7fb5f9f modified the definition
of struct dns_rbtnode.  Doing that changes the layout of map-format zone
files.  Bump MAPAPI and update the offsets used in map-format zone file
checks in the "masterformat" system test, as these changes were
inadvertently omitted from the aforementioned change.

(cherry picked from commit 52fe0b6be7)
2024-03-07 09:57:48 +01:00
Michał Kępień 52fe0b6be7 Account for changes to struct dns_rbtnode
Commit 540a5b5a2c modified the definition
of struct dns_rbtnode.  Doing that changes the layout of map-format zone
files.  Bump MAPAPI and update the offsets used in map-format zone file
checks in the "masterformat" system test, as these changes were
inadvertently omitted from the aforementioned change.
2024-03-07 09:42:38 +01:00
Ondřej SurýandMichał Kępień 5f98eba608 Move the task creation into cache_create_db()
The dns_cache_flush() drops the old database and creates a new one, but
it forgets to create the task(s) that runs the node pruning and cleaning
the rbtdb when flushing it next time.  This causes the cleaning to skip
cleaning the parent nodes (with .down == NULL) leading to increased
memory usage over time until the database is unable to keep up and just
stays overmem all the time.

(cherry picked from commit d4bc4e5cc6)
2024-03-06 19:17:32 +01:00
Ondřej SurýandMichał Kępień eba7fb5f9f Create a second pruning task for rbtdb with unlimited quantum
Previously, rbtdb->task had quantum of 1 because it was originally used
just for freeing RBTDB contents, which can happen on a "best effort"
basis (does not need to be prioritized).  However, when tree pruning was
implemented, it also started sending events to that task, enabling the
latter to become clogged up with a significant event backlog because it
only pruned a single RBTDB node per event.

To prioritize tree pruning (as it is necessary for enforcing the
configured memory use limit for the cache memory context), create a
second task with a virtually unlimited quantum (UINT_MAX) and send the
tree-pruning events to this new task, to ensure that all nodes scheduled
for pruning will be processed before further nodes are queued in a
similar fashion.

This change enables dropping the prunenodes list and restoring the
originally-used logic that allocates and sends a separate event for each
node to prune.

(cherry picked from commit 540a5b5a2c)
2024-03-06 19:17:32 +01:00
Ondřej SurýandMichał Kępień a548312191 Restore the parent cleaning logic in prune_tree()
Reconstruct the variant of the prune_tree() parent cleaning to consider
all elibible parents in a single loop as we were doing before all the
changes that led to this commit.

Update code comments so that they more precisely describe what the
relevant bits of code actually do.

(cherry picked from commit 12c42a6c07)
2024-03-06 19:17:32 +01:00
Ondřej SurýandMichał Kępień d4bc4e5cc6 Move the task creation into cache_create_db()
The dns_cache_flush() drops the old database and creates a new one, but
it forgets to create the task(s) that runs the node pruning and cleaning
the rbtdb when flushing it next time.  This causes the cleaning to skip
cleaning the parent nodes (with .down == NULL) leading to increased
memory usage over time until the database is unable to keep up and just
stays overmem all the time.

(cherry picked from commit 79040a669c)
2024-03-06 18:43:49 +01:00
Ondřej SurýandMichał Kępień 540a5b5a2c Create a second pruning task for rbtdb with unlimited quantum
Previously, rbtdb->task had quantum of 1 because it was originally used
just for freeing RBTDB contents, which can happen on a "best effort"
basis (does not need to be prioritized).  However, when tree pruning was
implemented, it also started sending events to that task, enabling the
latter to become clogged up with a significant event backlog because it
only pruned a single RBTDB node per event.

To prioritize tree pruning (as it is necessary for enforcing the
configured memory use limit for the cache memory context), create a
second task with a virtually unlimited quantum (UINT_MAX) and send the
tree-pruning events to this new task, to ensure that all nodes scheduled
for pruning will be processed before further nodes are queued in a
similar fashion.

This change enables dropping the prunenodes list and restoring the
originally-used logic that allocates and sends a separate event for each
node to prune.

(cherry picked from commit 231b2375e5)
2024-03-06 18:43:49 +01:00
Ondřej SurýandMichał Kępień 12c42a6c07 Restore the parent cleaning logic in prune_tree()
Reconstruct the variant of the prune_tree() parent cleaning to consider
all elibible parents in a single loop as we were doing before all the
changes that led to this commit.

Update code comments so that they more precisely describe what the
relevant bits of code actually do.

(cherry picked from commit 454c75a33a)
2024-03-06 18:43:49 +01:00
Michał Kępień 0b59306166 Check the prunelink member of the correct node
Commit 37101c7c8a checks the prunelink
member of the node that was just pruned, not its parent node that was
intended to be examined.  Fix by checking the prunelink member of the
parent node, so that adding the latter to its relevant prunenodes list
twice is properly guarded against.

(cherry picked from commit 7d9be24bb1)
2024-03-02 06:37:53 +01:00
Michał Kępień 7d9be24bb1 Check the prunelink member of the correct node
Commit 4b6fc97af6 checks the prunelink
member of the node that was just pruned, not its parent node that was
intended to be examined.  Fix by checking the prunelink member of the
parent node, so that adding the latter to its relevant prunenodes list
twice is properly guarded against.
2024-03-02 06:36:37 +01:00
Michał Kępień 37101c7c8a Do not re-add a node to the same prunenodes list
If a node cleaned up by prune_tree() happens to belong to the same node
bucket as its parent, the latter is directly appended to the prunenodes
list currently processed by prune_tree().  However, the relevant code
branch does not account for the fact that the parent might already be on
the list it is trying to append it to.  Fix by only calling
ISC_LIST_APPEND() for parent nodes not yet added to their relevant
prunenodes list.

(cherry picked from commit 4b6fc97af6)
2024-03-01 18:19:39 +01:00
Michał Kępień 4b6fc97af6 Do not re-add a node to the same prunenodes list
If a node cleaned up by prune_tree() happens to belong to the same node
bucket as its parent, the latter is directly appended to the prunenodes
list currently processed by prune_tree().  However, the relevant code
branch does not account for the fact that the parent might already be on
the list it is trying to append it to.  Fix by only calling
ISC_LIST_APPEND() for parent nodes not yet added to their relevant
prunenodes list.
2024-03-01 18:12:37 +01:00
Michał Kępień cb9928aaeb Gracefully handle resending a node to prune_tree()
Commit 801e888d03 made the prune_tree()
function use send_to_prune_tree() for triggering pruning of deleted leaf
nodes' parents.  This enabled the following sequence of events to
happen:

 1. Node A, which is a leaf node, is passed to send_to_prune_tree() and
    its pruning is queued.

 2. Node B is added to the RBTDB as a child of node A before the latter
    gets pruned.

 3. Node B, which is now a leaf node itself (and is likely to belong to
    a different node bucket than node A), is passed to
    send_to_prune_tree() and its pruning gets queued.

 4. Node B gets pruned.  Its parent, node A, now becomes a leaf again
    and therefore the prune_tree() call that handled node B calls
    send_to_prune_tree() for node A.

 5. Since node A was already queued for pruning in step 1 (but not yet
    pruned), the INSIST(!ISC_LINK_LINKED(node, prunelink)); assertion
    fails for node A in send_to_prune_tree().

The above sequence of events is not a sign of pathological behavior.
Replace the assertion check with a conditional early return from
send_to_prune_tree().

(cherry picked from commit f6289ad931)
2024-02-29 18:06:12 +01:00
Michał Kępień f6289ad931 Gracefully handle resending a node to prune_tree()
Commit 2df147cb12 made the prune_tree()
function use send_to_prune_tree() for triggering pruning of deleted leaf
nodes' parents.  This enabled the following sequence of events to
happen:

 1. Node A, which is a leaf node, is passed to send_to_prune_tree() and
    its pruning is queued.

 2. Node B is added to the RBTDB as a child of node A before the latter
    gets pruned.

 3. Node B, which is now a leaf node itself (and is likely to belong to
    a different node bucket than node A), is passed to
    send_to_prune_tree() and its pruning gets queued.

 4. Node B gets pruned.  Its parent, node A, now becomes a leaf again
    and therefore the prune_tree() call that handled node B calls
    send_to_prune_tree() for node A.

 5. Since node A was already queued for pruning in step 1 (but not yet
    pruned), the INSIST(!ISC_LINK_LINKED(node, prunelink)); assertion
    fails for node A in send_to_prune_tree().

The above sequence of events is not a sign of pathological behavior.
Replace the assertion check with a conditional early return from
send_to_prune_tree().
2024-02-29 17:38:52 +01:00
Ondřej Surý d7e3c782fd Make the TTL-based cleaning more aggressive
It was discovered that the TTL-based cleaning could build up
a significant backlog of the rdataset headers during the periods where
the top of the TTL heap isn't expired yet.  Make the TTL-based cleaning
more aggressive by cleaning more headers from the heap when we are
adding new header into the RBTDB.

(cherry picked from commit d8220ca4ca)
(cherry picked from commit 496fe6bc60)
2024-02-29 16:14:05 +01:00
Ondřej Surý 1082495439 Remove expired rdataset headers from the heap
It was discovered that an expired header could sit on top of the heap
a little longer than desireable.  Remove expired headers (headers with
rdh_ttl set to 0) from the heap completely, so they don't block the next
TTL-based cleaning.

(cherry picked from commit a9383e4b95)
(cherry picked from commit abe080d16e)
2024-02-29 16:14:05 +01:00
Ondřej Surý 496fe6bc60 Make the TTL-based cleaning more aggressive
It was discovered that the TTL-based cleaning could build up
a significant backlog of the rdataset headers during the periods where
the top of the TTL heap isn't expired yet.  Make the TTL-based cleaning
more aggressive by cleaning more headers from the heap when we are
adding new header into the RBTDB.

(cherry picked from commit d8220ca4ca)
2024-02-29 16:09:36 +01:00
Ondřej Surý abe080d16e Remove expired rdataset headers from the heap
It was discovered that an expired header could sit on top of the heap
a little longer than desireable.  Remove expired headers (headers with
rdh_ttl set to 0) from the heap completely, so they don't block the next
TTL-based cleaning.

(cherry picked from commit a9383e4b95)
2024-02-29 16:09:34 +01:00
Ondřej Surý 801e888d03 Reduce lock contention during RBTDB tree pruning
The log message for commit c3377cbfaa
explained:

    Instead of issuing a separate isc_task_send() call for every RBTDB node
    that triggers tree pruning, maintain a list of nodes from which tree
    pruning can be started from and only issue an isc_task_send() call if
    pruning has not yet been triggered by another RBTDB node.

    The extra queuing overhead eliminated by this change could be remotely
    exploited to cause excessive memory use.

However, it turned out that having a single queue for the nodes to be
pruned increased lock contention to a level where cleaning up nodes from
the RBTDB took too long, causing the amount of memory used by the cache
to grow indefinitely over time.

This commit makes the prunenodes list bucketed, adds a quantum of 10
items per prune_tree() run, and simplifies parent node cleaning in the
prune_tree() logic.

Instead of juggling node locks in a cycle, only clean up the node
currently being pruned and queue its parent (if it is also eligible) for
pruning in the same way (by sending an event).

This simplifies the code and also spreads the pruning load across more
task loop ticks, which is better for lock contention as less things run
in a tight loop.

(cherry picked from commit 2df147cb12)
2024-02-29 12:47:25 +01:00
Ondřej SurýandMichał Kępień 2df147cb12 Reduce lock contention during RBTDB tree pruning
The log message for commit c3377cbfaa
explained:

    Instead of issuing a separate isc_task_send() call for every RBTDB node
    that triggers tree pruning, maintain a list of nodes from which tree
    pruning can be started from and only issue an isc_task_send() call if
    pruning has not yet been triggered by another RBTDB node.

    The extra queuing overhead eliminated by this change could be remotely
    exploited to cause excessive memory use.

However, it turned out that having a single queue for the nodes to be
pruned increased lock contention to a level where cleaning up nodes from
the RBTDB took too long, causing the amount of memory used by the cache
to grow indefinitely over time.

This commit makes the prunenodes list bucketed, adds a quantum of 10
items per prune_tree() run, and simplifies parent node cleaning in the
prune_tree() logic.

Instead of juggling node locks in a cycle, only clean up the node
currently being pruned and queue its parent (if it is also eligible) for
pruning in the same way (by sending an event).

This simplifies the code and also spreads the pruning load across more
task loop ticks, which is better for lock contention as less things run
in a tight loop.
2024-02-29 12:18:00 +01:00
Mark AndrewsandOndřej Surý 742dc803a8 Do not use header_prev in expire_lru_headers
dns__cacherbt_expireheader can unlink / free header_prev underneath
it.  Use ISC_LIST_TAIL after calling dns__cacherbt_expireheader
instead to get the next pointer to be processed.

(cherry picked from commit 7ce2e86024)
2024-02-23 12:38:31 +01:00