Commit Graph

33437 Commits

Author SHA1 Message Date
Ondřej Surý
a55bdb28f9 Assigning uint64_t from buffer might be misaligned in netmgr tests
Resolve possible 8-byte unaligned access when assigning the magic
value from the received buffer.
2021-03-04 15:02:24 +01:00
Ondřej Surý
cb9df06f7d Merge branch '2533-mdig-move-cleanup' into 'main'
Move cleanup of queries to later in the shutdown sequence

Closes #2533

See merge request isc-projects/bind9!4768
2021-03-04 14:01:15 +00:00
Mark Andrews
4015af02d8 Move cleanup of queries to later in the shutdown sequence
to avoid TSAN report

    WARNING: ThreadSanitizer: data race
      Write of size 8 at 0x000000000001 by main thread:
        #0 free <null>
        #1 default_memfree lib/isc/mem.c:440
        #2 mem_put lib/isc/mem.c:363
        #3 isc__mem_free lib/isc/mem.c:1012
        #4 main bin/tools/mdig.c:2231

      Previous read of size 1 at 0x000000000005 by thread T1:
        #0 dns_name_fromtext lib/dns/name.c:1121
        #1 sendquery bin/tools/mdig.c:596
        #2 sendqueries bin/tools/mdig.c:779
        #3 dispatch lib/isc/task.c:1153
        #4 run lib/isc/task.c:1345
        #5 isc__trampoline_run lib/isc/trampoline.c:184
        #6 <null> <null>

      Thread T1 (running) created by main thread at:
        #0 pthread_create <null>
        #1 isc_thread_create pthreads/thread.c:79
        #2 isc_taskmgr_create lib/isc/task.c:1435
        #3 main bin/tools/mdig.c:2148

    SUMMARY: ThreadSanitizer: data race in __interceptor_free
2021-03-04 13:21:56 +01:00
Michal Nowak
1f8463a6e9 Merge branch '2357-add-CHANGES-entry' into 'main'
Add CHANGES entry for GL #2357

Closes #2357

See merge request isc-projects/bind9!4770
2021-03-04 11:45:06 +00:00
Michal Nowak
d98f72de25 Add CHANGES entry for GL #2357 2021-03-04 12:26:58 +01:00
Ondřej Surý
b532eb3029 Merge branch '2552-comparison-between-signed-and-unsigned-on-arm7' into 'main'
Fix comparison between signed and unsigned integer expressions

Closes #2552

See merge request isc-projects/bind9!4766
2021-03-04 10:24:00 +00:00
Ondřej Surý
d3bb3ae64f Fix comparison between signed and unsigned integer expressions
Simple typecast to size_t should be enough to silence the warning on
ARMv7, even though the code is in fact correct, because the readlen is
checked for being < 0 in the block before the warning.
2021-03-04 11:21:43 +01:00
Ondřej Surý
2c9c16e334 Merge branch '2551-char-is-unsigned-on-arm64' into 'main'
Use int type to store result from isc_commandline_parse()

Closes #2551

See merge request isc-projects/bind9!4765
2021-03-04 10:20:18 +00:00
Ondřej Surý
8153729d3a Use int type to store result from isc_commandline_parse()
The C standard actually doesn't define char as signed or unsigned, and
it could be either according to underlying architecture.  It turns out
that while it's usually signed type, it isn't on arm64 where it's
unsigned.

isc_commandline_parse() return int, just use that instead of the char.
2021-03-04 10:43:00 +01:00
Mark Andrews
fc57dd4c41 Merge branch '2298-multiple-definition-of-librpz_dnsrpzd_path' into 'main'
Resolve "multiple definition of `librpz_dnsrpzd_path'"

Closes #2298

See merge request isc-projects/bind9!4703
2021-03-04 08:22:16 +00:00
Mark Andrews
e12cf5eb57 Add CHANGES for [GL #2298] 2021-03-04 19:21:44 +11:00
Mark Andrews
8f016dd57a Fixed librpz_dnsrpzd_path being a duplicate symbol
librpz_dnsrpzd_path should have been declared extern in dns/librpz.h
2021-03-04 19:20:59 +11:00
Evan Hunt
add81d6480 Merge branch '2505-journal-compatibility' into 'main'
allow dns_journal_rollforward() to read old journal files

Closes #2505

See merge request isc-projects/bind9!4720
2021-03-04 03:18:42 +00:00
Evan Hunt
82b82bb821 CHANGES, release note 2021-03-03 17:54:47 -08:00
Evan Hunt
a0aefa1de6 create 'journal' system test
tests that version 1 journal files containing version 1 transaction
headers are rolled forward correctly on server startup, then updated
into version 2 journals. also checks journal file consistency and
'max-journal-size' behavior.
2021-03-03 17:54:47 -08:00
Evan Hunt
a4972324a6 print journal index data and test for consistency
'named-journalprint -x' now prints the journal's index table and
the offset of each transaction in the journal, so that index consistency
can be confirmed.
2021-03-03 17:54:47 -08:00
Mark Andrews
fb2d0e2897 extend named-journalprint to be able to force the journal version
named-journalprint can now upgrade or downgrade a journal file
in place; the '-u' option upgrades and the '-d' option downgrades.
2021-03-03 17:54:47 -08:00
Evan Hunt
ee19966326 allow dns_journal_rollforward() to read old journal files
when the 'max-ixfr-ratio' option was added, journal transaction
headers were revised to include a count of RR's in each transaction.
this made it impossible to read old journal files after an upgrade.

this branch restores the ability to read version 1 transaction
headers. when rolling forward, printing journal contents, if
the wrong transaction header format is found, we can switch.

when dns_journal_rollforward() detects a version 1 transaction
header, it returns DNS_R_RECOVERABLE.  this triggers zone_postload()
to force a rewrite of the journal file in the new format, and
also to schedule a dump of the zone database with minimal delay.
journal repair is done by dns_journal_compact(), which rewrites
the entire journal, ignoring 'max-journal-size'. journal size is
corrected later.

newly created journal files now have "BIND LOG V9.2" in their headers
instead of "BIND LOG V9". files with the new version string cannot be
read using the old transaction header format. note that this means
newly created journal files will be rejected by older versions of named.

named-journalprint now takes a "-x" option, causing it to print
transaction header information before each delta, including its
format version.
2021-03-03 17:54:47 -08:00
Ondřej Surý
695caaf8d6 Merge branch 'ondrej/call-isc__initialize-shutdown-from-DllMain' into 'main'
Call isc__initialize()/isc__shutdown() from win32 DllMain

See merge request isc-projects/bind9!4759
2021-03-02 07:48:19 +00:00
Ondřej Surý
a50f5d0cf5 Call isc__initialize()/isc__shutdown() from win32 DllMain
Call the libisc isc__initialize() constructor and isc__shutdown()
destructor from DllMain instead of having duplicate code between
those and DllMain() code.
2021-03-01 14:24:57 +01:00
Evan Hunt
8795b12c49 Merge branch 'ondrej/improve-mempool-AddressSanitizer-integration' into 'main'
Improve allocation-deallocation tracking with AddressSanitizer

See merge request isc-projects/bind9!4748
2021-02-26 18:26:28 +00:00
Ondřej Surý
888bdfc1ff Add mempool get/put tracking with AddressSanitizer
When AddressSanitizer is in use, disable the internal mempool
implementation and redirect the isc_mempool_get to isc_mem_get
(and similarly for isc_mempool_put).  This is the method recommended
by the AddressSanitizer authors for tracking allocations and
deallocations instead of custom poison/unpoison code (see
https://github.com/google/sanitizers/wiki/AddressSanitizerManualPoisoning).
2021-02-26 10:05:42 -08:00
Ondřej Surý
aa8f730a22 Merge branch '2396-add-thread-trampoline-for-thread-accounting' into 'main'
Resolve "BIND 9.16 unit tests failing reliably on x86_64 NUMA machines"

Closes #2396

See merge request isc-projects/bind9!4687
2021-02-26 12:44:15 +00:00
Ondřej Surý
4f2ca15201 Add CHANGES note for GL #2396 2021-02-25 16:21:43 +01:00
Ondřej Surý
a0181056a8 Change the isc_thread_self() return type to uintptr_t
The pthread_self(), thrd_current() or GetCurrentThreadId() could
actually be a pointer, so we should rather convert the value into
uintptr_t instead of unsigned long.
2021-02-25 16:21:10 +01:00
Ondřej Surý
bea333f7c9 Use globally assigned thread_id in the isc_hp API
Convert the isc_hp API to use the globally available isc_tid_v instead
of locally defined tid_v.  This should solve most of the problems on
machines with many number of cores / CPUs.
2021-02-25 16:21:10 +01:00
Ondřej Surý
cbbecfcc82 Add isc_trampoline API to have simple accounting around threads
The current isc_hp API uses internal tid_v variable that gets
incremented for each new thread using hazard pointers.  This tid_v
variable is then used as a index to global shared table with hazard
pointers state.  Since the tid_v is only incremented and never
decremented the table could overflow very quickly if we create set of
threads for short period of time, they finish the work and cease to
exist.  Then we create identical set of threads and so on and so on.
This is not a problem for a normal `named` operation as the set of
threads is stable, but the problematic place are the unit tests where we
test network manager or other APIs (task, timer) that create threads.

This commits adds a thin wrapper around any function called from
isc_thread_create() that adds unique-but-reusable small digit thread id
that can be used as index to f.e. hazard pointer tables.  The trampoline
wrapper ensures that the thread ids will be reused, so the highest
thread_id number doesn't grow indefinitely when threads are created and
destroyed and then created again.  This fixes the hazard pointer table
overflow on machines with many cores. [GL #2396]
2021-02-25 16:21:10 +01:00
Matthijs Mekking
6dbdffd7b8 Merge branch '2503-stale-answer-client-timeout-crash' into 'main'
Resolve "New stale-answer-client-timeout crashes BIND 9.16 and 9.17"

Closes #2503

See merge request isc-projects/bind9!4714
2021-02-25 11:03:13 +00:00
Matthijs Mekking
a404eaaffd Add CHANGES and release notes for GL #2503 2021-02-25 11:32:53 +01:00
Matthijs Mekking
f8b7b597e9 Don't servfail on staleonly lookups
When a staleonly lookup doesn't find a satisfying answer, it should
not try to respond to the client.

This is not true when the initial lookup is staleonly (that is when
'stale-answer-client-timeout' is set to 0), because no resolver fetch
has been created at this point. In this case continue with the lookup
normally.
2021-02-25 11:32:17 +01:00
Matthijs Mekking
9e061faaae Don't allow recursion on staleonly lookups
Fix a crash that can happen in the following scenario:

A client request is received. There is no data for it in the cache,
(not even stale data). A resolver fetch is created as part of
recursion.

Some time later, the fetch still hasn't completed, and
stale-answer-client-timeout is triggered. A staleonly lookup is
started. It will also find no data in the cache.

So 'query_lookup()' will call 'query_gotanswer()' with ISC_R_NOTFOUND,
so this will call 'query_notfound()' and this will start recursion.

We will eventually end up in 'ns_query_recurse()' and that requires
the client query fetch to be NULL:

    REQUIRE(client->query.fetch == NULL);

If the previously started fetch is still running this assertion
fails.

The crash is easily prevented by not requiring recursion for
staleonly lookups.

Also remove a redundant setting of the staleonly flag at the end of
'query_lookup_staleonly()' before destroying the query context.

Add a system test to catch this case.
2021-02-25 11:32:17 +01:00
Matthijs Mekking
e53af87f2c Merge branch '2498-nsec3-dynamic-update-dnssec-policy' into 'main'
Resolve "Regression in BIND 9.16.10, DNSSEC fails due to improper NSEC3 creation witihin named"

Closes #2498

See merge request isc-projects/bind9!4739
2021-02-25 09:49:29 +00:00
Matthijs Mekking
89c47b3b42 Add changes and notes for [#2498] 2021-02-25 17:21:17 +11:00
Matthijs Mekking
4b176c850b Fix dnssec-policy NSEC3 on dynamic zones
When applying dnssec-policy on a dynamic zone (e.g. that allows Dynamic
Updates), the NSEC3 parameters were put on the queue, but they were
not being processed (until a reload of the zone or reconfiguration).

Process the NSEC3PARAM queue on zone postload when handling a
dynamic zone.
2021-02-25 17:21:17 +11:00
Matthijs Mekking
0c0f10b53f Add tests for NSEC3 on dynamic zones
GitLab issue #2498 is a bug report on NSEC3 with dynamic zones. Tests
for it in the nsec3 system test directory were missing.
2021-02-25 17:21:17 +11:00
Mark Andrews
95cf93081c Merge branch '2507-cid-320483-api-usage-errors-lock' into 'main'
Resolve "CID 320483:  API usage errors  (LOCK)"

Closes #2507

See merge request isc-projects/bind9!4721
2021-02-25 05:54:20 +00:00
Mark Andrews
3ac53daa06 Address unbalanced lock/unlock
Also address race between reading and testing mpctx->allocated
and incrementing mpctx->allocated.
2021-02-25 13:08:07 +11:00
Ondřej Surý
965848a11a Merge branch '2519-disable-assertion-in-DLL_THREAD_ATTACH-DLL_THREAD_DETACH' into 'main'
Disable safe-guard assertion in DLL_THREAD_ATTACH/DLL_THREAD_DETACH

Closes #2519

See merge request isc-projects/bind9!4738
2021-02-24 08:03:16 +00:00
Ondřej Surý
c5887c4312 Disable safe-guard assertion in DLL_THREAD_ATTACH/DLL_THREAD_DETACH
The BIND 9 libraries on Windows define DllMain() optional entry point
into a dynamic-link library (DLL).  When the system starts or terminates
a process or thread, it calls the entry-point function for each loaded
DLL using the first thread of the process.

When the DLL is being loaded into the virtual address space of the
current process as a result of the process starting up, we make a call
to DisableThreadLibraryCalls() which should disable the
DLL_THREAD_ATTACH and DLL_THREAD_DETACH notifications for the specified
dynamic-link library (DLL).

This seems not be the case because we never check the return value of
the DisableThreadLibraryCalls() call, and it could in fact fail.  The
DisableThreadLibraryCalls() function fails if the DLL specified by
hModule has active static thread local storage, or if hModule is an
invalid module handle.

In this commit, we remove the safe-guard assertion put in place for the
DLL_THREAD_ATTACH and DLL_THREAD_DETACH events and we just ignore them.
BIND 9 doesn't create/destroy enough threads for it actually to make any
difference, and in fact we do use static thread local storage in the
code.
2021-02-24 08:31:42 +01:00
Mark Andrews
de00c105bb Merge branch 'marka-placeholder' into 'main'
add placeholder

See merge request isc-projects/bind9!4737
2021-02-23 22:53:34 +00:00
Mark Andrews
57c70624d9 add placeholder 2021-02-24 09:49:27 +11:00
Michal Nowak
6a21522579 Merge branch '2446-query-c-5430-16-runtime-error' into 'main'
Initialize checknames field in dns_view_create()

Closes #2446

See merge request isc-projects/bind9!4706
2021-02-23 15:43:50 +00:00
Michal Nowak
0c6fa16477 Initialize checknames field in dns_view_create()
The 'checknames' field wasn't initialized in dns_view_create(), but it
should otherwise AddressSanitizer identifies the following runtime error
in query_test.c.

    runtime error: load of value 190, which is not a valid value for type '_Bool'
2021-02-23 16:08:13 +01:00
Michal Nowak
40b6db58a1 Revert "Initialize checknames field in query_test.c"
This reverts commit c75484c4dff04698c183b456a6cc85f951264e75.
2021-02-23 16:08:13 +01:00
Michal Nowak
efe11d4383 Initialize checknames field in query_test.c
'checknames' field of struct dns_view is not initialized by
dns_view_create(). ASAN identified this as runtime error:

    runtime error: load of value 190, which is not a valid value for type '_Bool'
2021-02-23 16:08:13 +01:00
Michal Nowak
63c7300f97 Merge branch 'mnowak/alpine-3.13' into 'main'
Add Alpine Linux 3.13

See merge request isc-projects/bind9!4724
2021-02-23 14:45:19 +00:00
Michal Nowak
909c85f7a9 Add Alpine Linux 3.13 2021-02-23 15:33:59 +01:00
Michal Nowak
ed13fb7ad2 Merge branch 'mnowak/pairwise-pict-keep-stderr' into 'main'
Do not remove stderr from pict output

See merge request isc-projects/bind9!4727
2021-02-23 14:25:38 +00:00
Michal Nowak
079debaa10 Do not remove stderr from pict output
Removing stderr from the pict tool serves no purpose and drops valuable
information, we might use when debugging failed pairwise CI job, such
as:

    Input Error: A parameter names must be unique
2021-02-23 15:23:58 +01:00
Mark Andrews
076bb4f989 Merge branch '2508-cid-320481-null-pointer-dereferences-reverse_inull' into 'main'
Resolve "CID 320481:  Null pointer dereferences  (REVERSE_INULL)"

Closes #2508

See merge request isc-projects/bind9!4722
2021-02-23 13:04:38 +00:00