Commit Graph

32611 Commits

Author SHA1 Message Date
Ondřej Surý
a00ca65ae6 Remove the .key from the beginning of the line in rst file
The handling of . (dot) characted at the beginning of the line has
changed between the sphinx-doc versions, and it was constantly giving us
trouble when generating man pages when using different sphinx-doc.  This
commit just changes the source rst file, so there's no more . (dot) the
beginning of the line.
2020-09-30 21:12:15 +02:00
Mark Andrews
cf64834630 Merge branch '2180-threadsanitizer-data-race-bin-named-server-c-9678-25-in-view_loaded' into 'main'
Resolve "ThreadSanitizer: data race bin/named/server.c:9678:25 in view_loaded"

Closes #2180

See merge request isc-projects/bind9!4167
2020-09-30 14:37:37 +00:00
Mark Andrews
b00ba7ac94 make (named_server_t).reload_status atomic
WARNING: ThreadSanitizer: data race
    Write of size 4 at 0x000000000001 by thread T1:
    #0 view_loaded bin/named/server.c:9678:25
    #1 call_loaddone lib/dns/zt.c:308:3
    #2 doneloading lib/dns/zt.c:582:3
    #3 zone_asyncload lib/dns/zone.c:2322:3
    #4 dispatch lib/isc/task.c:1152:7
    #5 run lib/isc/task.c:1344:2

    Previous read of size 4 at 0x000000000001 by thread T2:
    #0 named_server_status bin/named/server.c:11903:14
    #1 named_control_docommand bin/named/control.c:272:12
    #2 control_command bin/named/controlconf.c:390:17
    #3 dispatch lib/isc/task.c:1152:7
    #4 run lib/isc/task.c:1344:2

    Location is heap block of size 409 at 0x000000000011 allocated by main thread:
    #0 malloc <null>
    #1 default_memalloc lib/isc/mem.c:713:8
    #2 mem_get lib/isc/mem.c:622:8
    #3 mem_allocateunlocked lib/isc/mem.c:1268:8
    #4 isc___mem_allocate lib/isc/mem.c:1288:7
    #5 isc__mem_allocate lib/isc/mem.c:2453:10
    #6 isc___mem_get lib/isc/mem.c:1037:11
    #7 isc__mem_get lib/isc/mem.c:2432:10
    #8 named_server_create bin/named/server.c:9978:27
    #9 setup bin/named/main.c:1256:2
    #10 main bin/named/main.c:1523:2

    Thread T1 (running) created by main thread at:
    #0 pthread_create <null>
    #1 isc_thread_create lib/isc/pthreads/thread.c:73:8
    #2 isc_taskmgr_create lib/isc/task.c:1434:3
    #3 create_managers bin/named/main.c:915:11
    #4 setup bin/named/main.c:1223:11
    #5 main bin/named/main.c:1523:2

    Thread T2 (running) created by main thread at:
    #0 pthread_create <null>
    #1 isc_thread_create lib/isc/pthreads/thread.c:73:8
    #2 isc_taskmgr_create lib/isc/task.c:1434:3
    #3 create_managers bin/named/main.c:915:11
    #4 setup bin/named/main.c:1223:11
    #5 main bin/named/main.c:1523:2

    SUMMARY: ThreadSanitizer: data race bin/named/server.c:9678:25 in view_loaded
2020-09-30 14:19:09 +00:00
Mark Andrews
6e6ad6d303 Merge branch '2190-in-new-yaml-output-when-truncation-is-query-time-usec' into 'main'
dig: print timestamps with microsecond precision if "-u" is used in YAML output mode

Closes #2190

See merge request isc-projects/bind9!4193
2020-09-30 14:11:49 +00:00
Mark Andrews
b7512e87b1 add CHANGES 2020-09-30 23:56:49 +10:00
Mark Andrews
3fe7fa3db2 Output timestamps in yaml to the microsecond if -u is specified. 2020-09-30 23:56:18 +10:00
Mark Andrews
519b070618 Add ISO time stamps to the microsecond 2020-09-30 23:56:18 +10:00
Mark Andrews
f2762acd13 Merge branch '2188-bug-in-message-c-673-ensure-isc_mempool_getallocated-msg-namepool-0-failed' into 'main'
Resolve "Bug in message.c:673: ENSURE(isc_mempool_getallocated(msg->namepool) == 0) failed"

Closes #2188

See merge request isc-projects/bind9!4189
2020-09-30 13:43:20 +00:00
Mark Andrews
5b5f1ba0b2 Check that sig0 name is the root. 2020-09-30 13:24:29 +00:00
Mark Andrews
b736d86697 Have -d print out the message 2020-09-30 13:24:29 +00:00
Mark Andrews
450fab92b1 Always clean sig0name in msgresetsigs() and dns_message_renderreset()
The fuzzing harness operates on dns_message_t in non-standard ways
and if 'sig0name' is non-NULL when msgresetsigs() and
dns_message_renderreset() are called it should be cleaned up.
2020-09-30 13:24:29 +00:00
Mark Andrews
fc5c9e6bce Add packet from issue #4189 2020-09-30 13:24:29 +00:00
Ondřej Surý
0665f01c97 Merge branch 'marka-regen-ddns-confgen.8in' into 'main'
regenerate ddns-confgen.8in

See merge request isc-projects/bind9!4198
2020-09-30 12:19:26 +00:00
Mark Andrews
eeafbddccc regenerate ddns-confgen.8in 2020-09-30 15:52:54 +10:00
Ondřej Surý
6e156834e2 Merge branch '2124-fix-assertion-failure-in-dns-message' into 'main'
Resolve "Bind 9.16.6 Assertion failure message.c:4733: REQUIRE(msg->state == (-1)) failed"

Closes #2124

See merge request isc-projects/bind9!4194
2020-09-29 08:14:23 +00:00
Ondřej Surý
6179a38810 Add CHANGES and release note for GL #2124 2020-09-29 08:22:09 +02:00
Ondřej Surý
33eefe9f85 The dns_message_create() cannot fail, change the return to void
The dns_message_create() function cannot soft fail (as all memory
allocations either succeed or cause abort), so we change the function to
return void and cleanup the calls.
2020-09-29 08:22:08 +02:00
Diego Fronza
7deaf9a93c cocci: Add semantic patch to refactor dns_message_destroy()
dns_message_t objects are now being handled using reference counting
semantics, so now dns_message_destroy() is not called directly anymore,
dns_message_detach must be called instead.
2020-09-29 08:22:08 +02:00
Diego Fronza
cde6227a68 Properly handling dns_message_t shared references
This commit fix the problems that arose when moving the dns_message_t
object from fetchctx_t to the query structure.

Since the lifetime of query objects are different than that of a fetchctx
and the dns_message_t object held by the query may be being used by some
external module, e.g. validator, even after the query may have been destroyed,
propery handling of the references to the message were added in this commit to
avoid accessing an already destroyed object.

Specifically, in rctx_done(), a reference to the message is attached at
the beginning of the function and detached at the end, since a possible call
to fctx_cancelquery() would release the dns_message_t object, and in the next
lines of code a call to rctx_nextserver() or rctx_chaseds() would require
a valid pointer to the same object.

In valcreate() a new reference is attached to the message object, this
ensures that if the corresponding query object is destroyed before the
validator attempts to access it, no invalid pointer access occurs.

In validated() we have to attach a new reference to the message, since
we destroy the validator object at the beginning of the function,
and we need access to the message in the next lines of the same function.

rctx_nextserver() and rctx_chaseds() functions were adapted to receive
a new parameter of dns_message_t* type, this was so they could receive a
valid reference to a dns_message_t since using the response context respctx_t
to access the message through rctx->query->rmessage could lead to an already
released reference due to the query being canceled.
2020-09-29 08:22:08 +02:00
Diego Fronza
02f9e125c1 Fix invalid dns message state in resolver's logic
The assertion failure REQUIRE(msg->state == DNS_SECTION_ANY),
caused by calling dns_message_setclass within function resquery_response()
in resolver.c, was happening due to wrong management of dns message_t
objects used to process responses to the queries issued by the resolver.

Before the fix, a resolver's fetch context (fetchctx_t) would hold
a pointer to the message, this same reference would then be used over all
the attempts to resolve the query, trying next server, etc... for this to work
the message object would have it's state reset between each iteration, marking
it as ready for a new processing.

The problem arose in a scenario with many different forwarders configured,
managing the state of the dns_message_t object was lacking better
synchronization, which have led it to a invalid dns_message_t state in
resquery_response().

Instead of adding unnecessarily complex code to synchronize the object,
the dns_message_t object was moved from fetchctx_t structure to the
query structure, where it better belongs to, since each query will produce
a response, this way whenever a new query is created an associated
dns_messate_t is also created.

This commit deals mainly with moving the dns_message_t object from fetchctx_t
to the query structure.
2020-09-29 08:22:08 +02:00
Diego Fronza
12d6d13100 Refactored dns_message_t for using attach/detach semantics
This commit will be used as a base for the next code updates in order
to have a better control of dns_message_t objects' lifetime.
2020-09-29 08:22:08 +02:00
Mark Andrews
e6f2f79ff2 Merge branch '2189-some-comments-in-lib-dns-stats-c-use-incorrect-notation-for-bit-values' into 'main'
Resolve "some comments in lib/dns/stats.c use incorrect notation for bit values"

Closes #2189

See merge request isc-projects/bind9!4191
2020-09-29 00:38:14 +00:00
Mark Andrews
6727e23a47 Update comments to have binary notation 2020-09-29 10:36:07 +10:00
Michał Kępień
90e1acfd92 Merge branch '114-out-of-tree-system-tests' into 'main'
Add out-of-tree system test job

Closes #114

See merge request isc-projects/bind9!3895
2020-09-28 19:39:22 +00:00
Michal Nowak
47075f64c3 Do not remove $systest for out-of-tree builds
Previously, the $systest directory was being removed for out-of-tree
builds at the end of each system test.  Because of that, running tests
which depend on compiled objects was breaking subsequent "make check"
invocations:

    make: Target 'check' not remade because of errors.
    Making all in dyndb/driver
    /bin/bash: line 20: cd: dyndb/driver: No such file or directory
    Making all in dlzexternal/driver
    /bin/bash: line 20: cd: dlzexternal/driver: No such file or directory

Address by first removing build/test artifacts for a given test and then
removing empty directories inside (and potentially including) $systest.
2020-09-28 21:38:15 +02:00
Michal Nowak
483e5af534 Add an out-of-tree system test job to GitLab CI
Make sure the new job does not get run for every pipeline as it is not
expected to break often and it is similar enough to other system test
jobs.  Change the name of the variable holding the path to the
out-of-tree build directory to a more generic one.
2020-09-28 21:38:15 +02:00
Ondřej Surý
36195cf797 Merge branch 'ondrej/clear-the-uv-event-loop-before-exiting' into 'main'
Clear the libuv event loop before exiting

See merge request isc-projects/bind9!4181
2020-09-28 10:56:33 +00:00
Ondřej Surý
e5ab137ba3 Refactor the pausing/unpausing and finishing the nm_thread
The isc_nm_pause(), isc_nm_resume() and finishing the nm_thread() from
nm_destroy() has been refactored, so all use the netievents instead of
directly touching the worker structure members.  This allows us to
remove most of the locking as the .paused and .finished members are
always accessed from the matching nm_thread.

When shutting down the nm_thread(), instead of issuing uv_stop(), we
just shutdown the .async handler, so all uv_loop_t events are properly
finished first and uv_run() ends gracefully with no outstanding active
handles in the loop.
2020-09-28 11:17:11 +02:00
Michał Kępień
b8b3ddb02f Merge branch '1725-drop-function-wrapping' into 'main'
Drop function wrapping as it is redundant for now

Closes #1725

See merge request isc-projects/bind9!4174
2020-09-28 07:11:48 +00:00
Michał Kępień
b60d7345ed Fix function overrides in unit tests on macOS
Since Mac OS X 10.1, Mach-O object files are by default built with a
so-called two-level namespace which prevents symbol lookups in BIND unit
tests that attempt to override the implementations of certain library
functions from working as intended.  This feature can be disabled by
passing the "-flat_namespace" flag to the linker.  Fix unit tests
affected by this issue on macOS by adding "-flat_namespace" to LDFLAGS
used for building all object files on that operating system (it is not
enough to only set that flag for the unit test executables).
2020-09-28 09:09:21 +02:00
Michał Kępień
8bdba2edeb Drop function wrapping as it is redundant for now
As currently used in the BIND source tree, the --wrap linker option is
redundant because:

  - static builds are no longer supported,

  - there is no need to wrap around existing functions - what is
    actually required (at least for now) is to replace them altogether
    in unit tests,

  - only functions exposed by shared libraries linked into unit test
    binaries are currently being replaced.

Given the above, providing the alternative implementations of functions
to be overridden in lib/ns/tests/nstest.c is a much simpler alternative
to using the --wrap linker option.  Drop the code detecting support for
the latter from configure.ac, simplify the relevant Makefile.am, and
remove lib/ns/tests/wrap.c, updating lib/ns/tests/nstest.c accordingly
(it is harmless for unit tests which are not calling the overridden
functions).
2020-09-28 09:09:21 +02:00
Mark Andrews
aee29706f2 Merge branch '2185-nsdname-wait-recurse-speed-test-fails-under-tsan' into 'main'
Resolve "nsdname-wait-recurse speed test fails under tsan"

Closes #2185

See merge request isc-projects/bind9!4184
2020-09-28 00:05:56 +00:00
Mark Andrews
bdf8194af8 Wait for 'rpz: policy: reload done' to signalled before proceeding.
RPZ rules cannot be fully relied upon until the summary RPZ database is
updated after an "rndc reload".  Wait until the relevant message is
logged after an "rndc reload" to prevent false positives in the
"rpzrecurse" system test caused by the RPZ rules not yet being in effect
by the time ns3 is queried.
2020-09-28 09:14:21 +10:00
Mark Andrews
22d10df0f0 add test markers to ns3 named.log 2020-09-28 09:14:21 +10:00
Evan Hunt
bfe9d3a70d Merge branch '1041-filter-aaaa-purge-memory-pool-upon-plugin-destruction' into 'main'
filter-aaaa: Purge memory pool upon plugin destruction

Closes #1041

See merge request isc-projects/bind9!1957
2020-09-25 21:02:20 +00:00
Michał Kępień
555e1f446c Add CHANGES entry
5238.	[bug]		filter-aaaa: named crashed upon shutdown if it was in
			the process of recursing for A RRsets. [GL #1040]
2020-09-25 13:32:34 -07:00
Evan Hunt
86eddebc83 Purge memory pool upon plugin destruction
The typical sequence of events for AAAA queries which trigger recursion
for an A RRset at the same name is as follows:

 1. Original query context is created.
 2. An AAAA RRset is found in cache.
 3. Client-specific data is allocated from the filter-aaaa memory pool.
 4. Recursion is triggered for an A RRset.
 5. Original query context is torn down.

 6. Recursion for an A RRset completes.
 7. A second query context is created.
 8. Client-specific data is retrieved from the filter-aaaa memory pool.
 9. The response to be sent is processed according to configuration.
10. The response is sent.
11. Client-specific data is returned to the filter-aaaa memory pool.
12. The second query context is torn down.

However, steps 6-12 are not executed if recursion for an A RRset is
canceled.  Thus, if named is in the process of recursing for A RRsets
when a shutdown is requested, the filter-aaaa memory pool will have
outstanding allocations which will never get released.  This in turn
leads to a crash since every memory pool must not have any outstanding
allocations by the time isc_mempool_destroy() is called.

Fix by creating a stub query context whenever fetch_callback() is called,
including cancellation events. When the qctx is destroyed, it will ensure
the client is detached and the plugin memory is freed.
2020-09-25 13:32:34 -07:00
Mark Andrews
b2408003d4 Merge branch '2184-add-rfc8096-to-list-of-rfcs-in-doc-general-rst' into 'main'
Resolve "Add RFC8906 to list of RFCs in doc/arm/general.rst"

Closes #2184

See merge request isc-projects/bind9!4180
2020-09-24 00:28:59 +00:00
Suzanne Goldlust
bc604793d3 Accidentally wrote RFC8096 instead of 8906 2020-09-23 19:41:58 +00:00
Suzanne Goldlust
4c36b6bdf0 Add RFC8096 to list in doc/arm/general.rst 2020-09-23 19:34:42 +00:00
Matthijs Mekking
bfa8c6a822 Merge branch '1870-rndc-dumpdb-expired-v2' into 'main'
Resolve "[Support#12071] [RT#46548] Output stale/expired data with 'rndc dumpdb'"

Closes #1870

See merge request isc-projects/bind9!4088
2020-09-23 14:38:54 +00:00
Matthijs Mekking
d14c2d0d73 rndc dumpdb -expired: print when RRsets expired
When calling 'rndc dumpdb -expired', also print when the RRset expired.
2020-09-23 16:09:26 +02:00
Matthijs Mekking
1728599684 Add notes and CHANGES for #1870
This is a new features so it requires a CHANGE and release notes entry.
2020-09-23 16:09:24 +02:00
Matthijs Mekking
388cc666e5 Handle ancient rrsets in bind_rdataset
An ancient RRset is one still in the cache but expired, and awaiting
cleanup.
2020-09-23 16:08:29 +02:00
Matthijs Mekking
17d5bd4493 Include expired rdatasets in iteration functions
By changing the check in 'rdatasetiter_first' and 'rdatasetiter_next'
from "now > header->rdh_ttl" to "now - RBDTB_VIRTUAL > header->rdh_ttl"
we include expired rdataset entries so that they can be used for
"rndc dumpdb -expired".
2020-09-23 16:08:29 +02:00
Matthijs Mekking
1c3e6f4045 Add test for 'rdnc dumpdb -expired'
This test makes sure that expired records are dumped with rndc's
'dumpdb' command if the '-expired' flag is used.
2020-09-23 16:08:29 +02:00
Matthijs Mekking
86a1bbfe28 Minor changes to serve-stale tests
Minor changes are:
- Replace the "$RNDCCMD dumpdb" logic with "rndc_dumpdb" from
  conf.sh.common (it does the same thing).
- Update a comment to match the grep calls below it (comment said the
  rest should be expired, while the grep calls indicate that they
  are still in the cache, the comment now explains why).
2020-09-23 16:08:29 +02:00
Matthijs Mekking
8beda7d2ea Add -expired flag to rndc dumpdb command
This flag is the same as -cache, but will use a different style format
that will also print expired entries (awaiting cleanup) from the cache.
2020-09-23 16:08:29 +02:00
Mark Andrews
c7156d2116 Merge branch '2162-threadsanitizer-data-race-in-epoll_ctl' into 'main'
Resolve "ThreadSanitizer: data race in epoll_ctl"

Closes #2162

See merge request isc-projects/bind9!4171
2020-09-23 04:19:58 +00:00
Mark Andrews
c37b251eb9 It appears that you can't change what you are polling for while connecting.
WARNING: ThreadSanitizer: data race
    Read of size 8 at 0x000000000001 by thread T1 (mutexes: write M1):
    #0 epoll_ctl <null>
    #1 watch_fd lib/isc/unix/socket.c:704:8
    #2 wakeup_socket lib/isc/unix/socket.c:897:11
    #3 process_ctlfd lib/isc/unix/socket.c:3362:3
    #4 process_fds lib/isc/unix/socket.c:3275:10
    #5 netthread lib/isc/unix/socket.c:3516:10

    Previous write of size 8 at 0x000000000001 by thread T2 (mutexes: write M2):
    #0 connect <null>
    #1 isc_socket_connect lib/isc/unix/socket.c:4737:7
    #2 resquery_send lib/dns/resolver.c:2892:13
    #3 fctx_query lib/dns/resolver.c:2202:12
    #4 fctx_try lib/dns/resolver.c:4300:11
    #5 resquery_connected lib/dns/resolver.c:3130:4
    #6 dispatch lib/isc/task.c:1152:7
    #7 run lib/isc/task.c:1344:2

    Location is file descriptor 513 created by thread T2 at:
    #0 connect <null>
    #1 isc_socket_connect lib/isc/unix/socket.c:4737:7
    #2 resquery_send lib/dns/resolver.c:2892:13
    #3 fctx_query lib/dns/resolver.c:2202:12
    #4 fctx_try lib/dns/resolver.c:4300:11
    #5 resquery_connected lib/dns/resolver.c:3130:4
    #6 dispatch lib/isc/task.c:1152:7
    #7 run lib/isc/task.c:1344:2

    Mutex M1 (0x000000000016) created at:
    #0 pthread_mutex_init <null>
    #1 isc__mutex_init lib/isc/pthreads/mutex.c:288:8
    #2 setup_thread lib/isc/unix/socket.c:3584:3
    #3 isc_socketmgr_create2 lib/isc/unix/socket.c:3825:3
    #4 create_managers bin/named/main.c:932:11
    #5 setup bin/named/main.c:1223:11
    #6 main bin/named/main.c:1523:2

    Mutex M2 is already destroyed.

    Thread T1 'isc-socket-1' (running) created by main thread at:
    #0 pthread_create <null>
    #1 isc_thread_create lib/isc/pthreads/thread.c:73:8
    #2 isc_socketmgr_create2 lib/isc/unix/socket.c:3826:3
    #3 create_managers bin/named/main.c:932:11
    #4 setup bin/named/main.c:1223:11
    #5 main bin/named/main.c:1523:2

    Thread T2 (running) created by main thread at:
    #0 pthread_create <null>
    #1 isc_thread_create lib/isc/pthreads/thread.c:73:8
    #2 isc_taskmgr_create lib/isc/task.c:1434:3
    #3 create_managers bin/named/main.c:915:11
    #4 setup bin/named/main.c:1223:11
    #5 main bin/named/main.c:1523:2

    SUMMARY: ThreadSanitizer: data race in epoll_ctl
2020-09-23 13:54:06 +10:00