Commit Graph
12659 Commits
Author SHA1 Message Date
Mark AndrewsandOndřej Surý c2ee9eea3a Refactor totext_loc
(cherry picked from commit 2ca4d35037)
2020-08-26 16:44:01 +02:00
Mark AndrewsandOndřej Surý baf93342d0 Correctly encode LOC records with non integer negative altitudes.
(cherry picked from commit 337cc878fa)
2020-08-26 16:44:01 +02:00
Mark AndrewsandOndřej Surý 06b76b2b16 Check LOC's altitude field is properly parsed and encoded.
(cherry picked from commit 888dfd78c7)
2020-08-26 16:44:00 +02:00
Mark AndrewsandOndřej Surý 7eb5d61703 Tighten LOC parsing to reject period and/or m as a value.
(cherry picked from commit 9225c67835)
2020-08-26 16:44:00 +02:00
Ondřej SurýandMark Andrews 5674f76590 Use the Fibonacci Hashing for the RBTDB glue table
The rbtdb version glue_table has been refactored similarly to rbt.c hash
table, so it does use 32-bit hash function return values and apply
Fibonacci Hashing to lookup the index to the hash table instead of
modulo.  For more details, see the lib/dns/rbt.c commit log.

(cherry picked from commit 01684cc219)
2020-08-26 21:49:59 +10:00
Mark Andrews 511747307f rbtversion->glue_table_size must be read when holding a lock
(cherry picked from commit 33d0e8d168)
2020-08-26 21:49:59 +10:00
Mark AndrewsandOndřej Surý 3fce53b0e3 Cast the original rcode to (dns_ttl_t) when setting extended rcode
Shifting (signed) integer left could trigger undefined behaviour when
the shifted value would overflow into the sign bit (e.g. 2048).

The issue was found when using AFL++ and UBSAN:

    message.c:2274:33: runtime error: left shift of 2048 by 20 places cannot be represented in type 'int'
    SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior message.c:2274:33 in

(cherry picked from commit a347641782)
2020-08-25 16:40:42 +02:00
Michal Nowak 79e8f1076a Fix warnings in when build with --enable-buffer-useinline
sockaddr.c:147:49: error: pointer targets in passing argument 2 of ‘isc__buffer_putmem’ differ in signedness
    rdata.c:1780:30: error: pointer targets in passing argument 2 of ‘isc__buffer_putmem’ differ in signedness

(cherry picked from commit dd425254a7)
2020-08-25 16:08:44 +02:00
Mark Andrews da4f189ea8 Add missing isc_mutex_init to manytasks subtest.
(cherry picked from commit 2eb5c29c83)
2020-08-25 09:58:30 +10:00
Evan Hunt 1c7e3c8515 Merge tag 'v9_16_6' into v9_16
BIND 9.16.6
2020-08-20 12:08:57 -07:00
Mark AndrewsandOndřej Surý d8a57d32b1 A6: return FORMERR in fromwire if bits are non zero.
oss_fuzz: Issue 24864: bind9:dns_rdata_fromwire_text_fuzzer: Overwrites-const-input in dns_rdata_fromwire_text_fuzzer

(cherry picked from commit 8452404bd7)
2020-08-18 11:11:40 +02:00
Mark AndrewsandOndřej Surý 6b1675a62c RRSIG: reject records with empty SIG section
(cherry picked from commit f6d7b8c20d)
2020-08-18 11:11:40 +02:00
Mark Andrews 9675d83b96 X25: Check that record is all ASCII digits
(cherry picked from commit 7e49689746)
2020-08-14 00:38:45 +10:00
Mark Andrews ee10a93cbc WKS: reject records with zero octets at the end of the bitmap
(cherry picked from commit 9d446142d8)
2020-08-14 00:38:45 +10:00
Mark Andrews e44da35092 TLSA: fix fromwire length checks
(cherry picked from commit 3429c35f52)
2020-08-14 00:38:45 +10:00
Mark Andrews b4a66cffa8 SIG: reject records with a zero length signature
(cherry picked from commit 9b93e5d684)
2020-08-14 00:38:45 +10:00
Mark Andrews 10e8ad22c5 NXT: fix fromwire bitmap checks
(cherry picked from commit 73dd849655)
2020-08-14 00:38:45 +10:00
Mark Andrews c712394b34 NSEC3PARAM: check that saltlen is consistent with the rdata length
(cherry picked from commit 7dc8e720ff)
2020-08-14 00:38:45 +10:00
Mark Andrews 26be6c817b NSEC3: reject records with a zero length hash field
(cherry picked from commit 031ee9e279)
2020-08-14 00:38:45 +10:00
Mark Andrews ebd8033a96 IPSECKEY: require non-zero length public keys
(cherry picked from commit d7f7014803)
2020-08-14 00:38:45 +10:00
Mark Andrews baf7d114af CERT: reject records with a empty certificate field
(cherry picked from commit a238f37239)
2020-08-14 00:38:45 +10:00
Mark Andrews f09691feff Get rid of type 'RESERVED0'.
(cherry picked from commit 3c492b3ef1)
2020-08-14 00:38:45 +10:00
Mark Andrews 5806d856cf base32_decode*() could incorrectly decode a input.
base32_decode_char() added a extra zero octet to the output
if the fifth character was a pad character.  The length
of octets to copy to the output was set to 3 instead of 2.

(cherry picked from commit 6c7e50c267)
2020-08-14 00:11:06 +10:00
Mark Andrews f3b25f1ffb Address use after free between view, resolver and nta.
Hold a weak reference to the view so that it can't go away while
nta is performing its lookups.  Cancel nta timers once all external
references to the view have gone to prevent them triggering new work.

(cherry picked from commit 0b2555e8cf)
2020-08-11 11:55:44 +10:00
Ondřej SurýandMichał Kępień 25846cfec4 Reduce the default RBT hash table size to 16 entries (4 bits)
The hash table rework MRs (!3865, !3871) increased the default RBT hash
table size from 64 to 65,536 entries (for 64-bit architectures, that is
512 bytes before vs. 524,288 bytes after).  This works fine for RBTs
used for cache databases, but since three separate RBT databases are
created for every zone loaded (RRs, NSEC, NSEC3), memory usage would
skyrocket when BIND 9 is used as an authoritative DNS server with many
zones.

The default RBT hash table size before the rework was 64 entries, this
commit reduces it to 16 entries because our educated guess is that most
zones are just couple of entries (SOA, NS, A, AAAA, MX) and rehashing
small hash tables is actually cheap.  The rework we did in the previous
MRs tries to avoid growing the hash tables for big-to-huge caches where
growing the hash table comes at a price because the whole cache needs to
be locked.

(cherry picked from commit 1e043a011b)
(cherry picked from commit f0ccc17f30)
2020-08-10 11:31:13 +02:00
Ondřej Surý f0ccc17f30 Reduce the default RBT hash table size to 16 entries (4 bits)
The hash table rework MRs (!3865, !3871) increased the default RBT hash
table size from 64 to 65,536 entries (for 64-bit architectures, that is
512 bytes before vs. 524,288 bytes after).  This works fine for RBTs
used for cache databases, but since three separate RBT databases are
created for every zone loaded (RRs, NSEC, NSEC3), memory usage would
skyrocket when BIND 9 is used as an authoritative DNS server with many
zones.

The default RBT hash table size before the rework was 64 entries, this
commit reduces it to 16 entries because our educated guess is that most
zones are just couple of entries (SOA, NS, A, AAAA, MX) and rehashing
small hash tables is actually cheap.  The rework we did in the previous
MRs tries to avoid growing the hash tables for big-to-huge caches where
growing the hash table comes at a price because the whole cache needs to
be locked.

(cherry picked from commit 1e043a011b)
2020-08-10 10:32:26 +02:00
Mark Andrews 8a4dd25562 Silence 'may be used uninitialized' 2020-08-08 16:12:12 +10:00
Matthijs Mekking 624f1b9531 rndc dnssec -checkds set algorithm
In the rare case that you have multiple keys acting as KSK and that
have the same keytag, you can now set the algorithm when calling
'-checkds'.

(cherry picked from commit 46fcd927e7)
2020-08-07 13:34:10 +02:00
Matthijs Mekking 4892006a92 Make 'parent-registration-delay' obsolete
With the introduction of 'checkds', the 'parent-registration-delay'
option becomes obsolete.

(cherry picked from commit a25f49f153)
2020-08-07 13:30:50 +02:00
Matthijs Mekking 5dcf56f216 Fix time printing in key files
Don't strip off the final character when printing times in key files.

With the introduction of 'rndc dnssec -status' we introduced
'isc_stdtime_tostring()'. This changed in behavior such that it was no
longer needed to strip of the final '\n' of the string format
datetime. However, in 'printtime()' it still stripped the final
character.

(cherry picked from commit e3eb55fd1c)
2020-08-07 13:30:30 +02:00
Matthijs Mekking 81d0c63ecb Implement 'rndc dnssec -checkds'
Add a new 'rndc' command 'dnssec -checkds' that allows the user to
signal named that a new DS record has been seen published in the
parent, or that an existing DS record has been withdrawn from the
parent.

Upon the 'checkds' request, 'named' will write out the new state for
the key, updating the 'DSPublish' or 'DSRemoved' timing metadata.

This replaces the "parent-registration-delay" configuration option,
this was unreliable because it was purely time based (if the user
did not actually submit the new DS to the parent for example, this
could result in an invalid DNSSEC state).

Because we cannot rely on the parent registration delay for state
transition, we need to replace it with a different guard. Instead,
if a key wants its DS state to be moved to RUMOURED, the "DSPublish"
time must be set and must not be in the future. If a key wants its
DS state to be moved to UNRETENTIVE, the "DSRemoved" time must be set
and must not be in the future.

By default, with '-checkds' you set the time that the DS has been
published or withdrawn to now, but you can set a different time with
'-when'. If there is only one KSK for the zone, that key has its
DS state moved to RUMOURED. If there are multiple keys for the zone,
specify the right key with '-key'.

(cherry picked from commit 04d8fc0143)
2020-08-07 13:30:19 +02:00
Tinderbox User a195123ad0 prep 9.16.6 2020-08-06 08:14:40 +00:00
Ondřej SurýandMichał Kępień ac3862a5da Fix crash in pk11_numbits() when native-pkcs11 is used
When pk11_numbits() is passed a user provided input that contains all
zeroes (via crafted DNS message), it would crash with assertion
failure.  Fix that by properly handling such input.
2020-08-05 15:51:40 +02:00
Mark AndrewsandMichał Kępień 0eec632d6a Always keep a copy of the message
this allows it to be available even when dns_message_parse()
returns a error.
2020-08-05 15:47:25 +02:00
Evan HuntandMichał Kępień 81514ff925 permanently disable QNAME minimization in a fetch when forwarding
QNAME minimization is normally disabled when forwarding. if, in the
course of processing a fetch, we switch back to normal recursion at
some point, we can't safely start minimizing because we may have
been left in an inconsistent state.
2020-08-05 15:44:18 +02:00
Evan HuntandMichał Kępień 9a372f2bce Use different allocators for UDP and TCP
Each worker has a receive buffer with space for 20 DNS messages of up
to 2^16 bytes each, and the allocator function passed to uv_read_start()
or uv_udp_recv_start() will reserve a portion of it for use by sockets.
UDP can use recvmmsg() and so it needs that entire space, but TCP reads
one message at a time.

This commit introduces separate allocator functions for TCP and UDP
setting different buffer size limits, so that libuv will provide the
correct buffer sizes to each of them.
2020-08-05 12:57:58 +02:00
Ondřej SurýandMatthijs Mekking f9711481ad Expire the 0 TTL RRSet quickly rather using them for serve-stale
When a received RRSet has TTL 0, they would be preserved for
serve-stale (default `max-stale-cache` is 12 hours) rather than expiring
them quickly from the cache database.

This commit makes sure the RRSet didn't have TTL 0 before marking the
entry in the database as "stale".

(cherry picked from commit 6ffa2ddae0)
2020-08-05 09:09:16 +02:00
Ondřej SurýandMatthijs Mekking b48e9ab201 Add stale-cache-enable option and disable serve-stable by default
The current serve-stale implementation in BIND 9 stores all received
records in the cache for a max-stale-ttl interval (default 12 hours).

This allows DNS operators to turn the serve-stale answers in an event of
large authoritative DNS outage.  The caching of the stale answers needs
to be enabled before the outage happens or the feature would be
otherwise useless.

The negative consequence of the default setting is the inevitable
cache-bloat that happens for every and each DNS operator running named.

In this MR, a new configuration option `stale-cache-enable` is
introduced that allows the operators to selectively enable or disable
the serve-stale feature of BIND 9 based on their decision.

The newly introduced option has been disabled by default,
e.g. serve-stale is disabled in the default configuration and has to be
enabled if required.

(cherry picked from commit ce53db34d6)
2020-08-05 09:09:16 +02:00
Mark Andrews 2dc26ebdb6 Map DNS_R_BADTSIG to FORMERR
Now that the log message has been printed set the result code to
DNS_R_FORMERR.  We don't do this via dns_result_torcode() as we
don't want upstream errors to produce FORMERR if that processing
end with DNS_R_BADTSIG.

(cherry picked from commit 20488d6ad3)
2020-08-04 23:04:34 +10:00
Diego Fronza fca1000ee9 Fix ns_statscounter_recursclients underflow
The basic scenario for the problem was that in the process of
resolving a query, if any rrset was eligible for prefetching, then it
would trigger a call to query_prefetch(), this call would run in
parallel to the normal query processing.

The problem arises due to the fact that both query_prefetch(), and,
in the original thread, a call to ns_query_recurse(), try to attach
to the recursionquota, but recursing client stats counter is only
incremented if ns_query_recurse() attachs to it first.

Conversely, if fetch_callback() is called before prefetch_done(),
it would not only detach from recursionquota, but also decrement
the stats counter, if query_prefetch() attached to te quota first
that would result in a decrement not matched by an increment, as
expected.

To solve this issue an atomic bool was added, it is set once in
ns_query_recurse(), allowing fetch_callback() to check for it
and decrement stats accordingly.

For a more compreensive explanation check the thread comment below:
https://gitlab.isc.org/isc-projects/bind9/-/issues/1719#note_145857
2020-08-03 19:18:04 -03:00
Witold KręcickiandOndřej Surý a12076cc52 netmgr: retry binding with IP_FREEBIND when EADDRNOTAVAIL is returned.
When a new IPv6 interface/address appears it's first in a tentative
state - in which we cannot bind to it, yet it's already being reported
by the route socket. Because of that BIND9 is unable to listen on any
newly detected IPv6 addresses. Fix it by setting IP_FREEBIND option (or
equivalent option on other OSes) and then retrying bind() call.

(cherry picked from commit a0f7d28967)
2020-07-31 13:33:06 +02:00
Mark Andrews 14fe6e77a7 Always check the return from isc_refcount_decrement.
Created isc_refcount_decrement_expect macro to test conditionally
the return value to ensure it is in expected range.  Converted
unchecked isc_refcount_decrement to use isc_refcount_decrement_expect.
Converted INSIST(isc_refcount_decrement()...) to isc_refcount_decrement_expect.

(cherry picked from commit bde5c7632a)
2020-07-31 12:54:47 +10:00
Mark Andrews 1981fb1327 Refactor the code that counts the last log version to keep
When silencing the Coverity warning in remove_old_tsversions(), the code
was refactored to reduce the indentation levels and break down the long
code into individual functions.  This improve fix for [GL #1989].

(cherry picked from commit aca18b8b5b)
2020-07-31 10:01:36 +10:00
Ondřej Surý 0fff3008ac Change the dns_name hashing to use 32-bit values
Change the dns_hash_name() and dns_hash_fullname() functions to use
isc_hash32() as the maximum hashtable size in rbt is 0..UINT32_MAX
large.

(cherry picked from commit a9182c89a6)
2020-07-30 11:57:24 +02:00
Ondřej Surý ebb2b055cc Add isc_hash32() and rename isc_hash_function() to isc_hash64()
As the names suggest the original isc_hash64 function returns 64-bit
long hash values and the isc_hash32() returns 32-bit values.

(cherry picked from commit f59fd49fd8)
2020-07-30 11:57:24 +02:00
Ondřej Surý 1e5df7f3bf Add HalfSipHash 2-4 reference implementation
The HalfSipHash implementation has 32-bit keys and returns 32-bit
value.

(cherry picked from commit 344d66aaff)
2020-07-30 11:57:24 +02:00
Ondřej Surý d89eb403f3 Remove OpenSSL based SipHash 2-4 implementation
Creation of EVP_MD_CTX and EVP_PKEY is quite expensive, so until we fix the code
to reuse the OpenSSL contexts and keys we'll use our own implementation of
siphash instead of trying to integrate with OpenSSL.

(cherry picked from commit 21d751dfc7)
2020-07-30 11:57:24 +02:00
Ondřej Surý aa72c31422 Fix the rbt hashtable and grow it when setting max-cache-size
There were several problems with rbt hashtable implementation:

1. Our internal hashing function returns uint64_t value, but it was
   silently truncated to unsigned int in dns_name_hash() and
   dns_name_fullhash() functions.  As the SipHash 2-4 higher bits are
   more random, we need to use the upper half of the return value.

2. The hashtable implementation in rbt.c was using modulo to pick the
   slot number for the hash table.  This has several problems because
   modulo is: a) slow, b) oblivious to patterns in the input data.  This
   could lead to very uneven distribution of the hashed data in the
   hashtable.  Combined with the single-linked lists we use, it could
   really hog-down the lookup and removal of the nodes from the rbt
   tree[a].  The Fibonacci Hashing is much better fit for the hashtable
   function here.  For longer description, read "Fibonacci Hashing: The
   Optimization that the World Forgot"[b] or just look at the Linux
   kernel.  Also this will make Diego very happy :).

3. The hashtable would rehash every time the number of nodes in the rbt
   tree would exceed 3 * (hashtable size).  The overcommit will make the
   uneven distribution in the hashtable even worse, but the main problem
   lies in the rehashing - every time the database grows beyond the
   limit, each subsequent rehashing will be much slower.  The mitigation
   here is letting the rbt know how big the cache can grown and
   pre-allocate the hashtable to be big enough to actually never need to
   rehash.  This will consume more memory at the start, but since the
   size of the hashtable is capped to `1 << 32` (e.g. 4 mio entries), it
   will only consume maximum of 32GB of memory for hashtable in the
   worst case (and max-cache-size would need to be set to more than
   4TB).  Calling the dns_db_adjusthashsize() will also cap the maximum
   size of the hashtable to the pre-computed number of bits, so it won't
   try to consume more gigabytes of memory than available for the
   database.

   FIXME: What is the average size of the rbt node that gets hashed?  I
   chose the pagesize (4k) as initial value to precompute the size of
   the hashtable, but the value is based on feeling and not any real
   data.

For future work, there are more places where we use result of the hash
value modulo some small number and that would benefit from Fibonacci
Hashing to get better distribution.

Notes:
a. A doubly linked list should be used here to speedup the removal of
   the entries from the hashtable.
b. https://probablydance.com/2018/06/16/fibonacci-hashing-the-optimization-that-the-world-forgot-or-a-better-alternative-to-integer-modulo/

(cherry picked from commit e24bc324b4)
2020-07-30 11:57:24 +02:00
Michał Kępień b6c33087b0 Fix idle timeout for connected TCP sockets
When named acting as a resolver connects to an authoritative server over
TCP, it sets the idle timeout for that connection to 20 seconds.  This
fixed timeout was picked back when the default processing timeout for
each client query was hardcoded to 30 seconds.  Commit
000a8970f8 made this processing timeout
configurable through "resolver-query-timeout" and decreased its default
value to 10 seconds, but the idle TCP timeout was not adjusted to
reflect that change.  As a result, with the current defaults in effect,
a single hung TCP connection will consistently cause the resolution
process for a given query to time out.

Set the idle timeout for connected TCP sockets to half of the client
query processing timeout configured for a resolver.  This allows named
to handle hung TCP connections more robustly and prevents the timeout
mismatch issue from resurfacing in the future if the default is ever
changed again.

(cherry picked from commit 953d704bd2)
2020-07-30 11:16:09 +02:00
Diego Fronza a8ce7b461c Fix rpz wildcard name matching
Whenever an exact match is found by dns_rbt_findnode(),
the highest level node in the chain will not be put into
chain->levels[] array, but instead the chain->end
pointer will be adjusted to point to that node.

Suppose we have the following entries in a rpz zone:
example.com     CNAME rpz-passthru.
*.example.com   CNAME rpz-passthru.

A query for www.example.com would result in the
following chain object returned by dns_rbt_findnode():

chain->level_count = 2
chain->level_matches = 2
chain->levels[0] = .
chain->levels[1] = example.com
chain->levels[2] = NULL
chain->end = www

Since exact matches only care for testing rpz set bits,
we need to test for rpz wild bits through iterating the nodechain, and
that includes testing the rpz wild bits in the highest level node found.

In the case of an exact match, chain->levels[chain->level_matches]
will be NULL, to address that we must use chain->end as the start point,
then iterate over the remaining levels in the chain.
2020-07-27 17:02:16 -03:00