Mark Andrews
28f1cbb007
Add comment about cookie sizes
...
(cherry picked from commit 304df53991
2020-11-27 10:20:40 +11:00
Mark Andrews
cdf73095ab
Tighten DNS COOKIE response handling
...
Fallback to TCP when we have already seen a DNS COOKIE response
from the given address and don't have one in this UDP response. This
could be a server that has turned off DNS COOKIE support, a
misconfigured anycast server with partial DNS COOKIE support, or a
spoofed response. Falling back to TCP is the correct behaviour in
all 3 cases.
(cherry picked from commit 0e3b1f5a25
2020-11-27 09:28:09 +11:00
Michał Kępień
fa989e4236
Use proper cmocka macros for pointer checks
...
Make sure pointer checks in unit tests use cmocka assertion macros
dedicated for use with pointers instead of those dedicated for use with
integers or booleans.
(cherry picked from commit f440600126
2020-11-26 13:15:28 +01:00
Tinderbox User and Michał Kępień
66faf3215c
prep 9.11.25
2020-11-26 12:33:58 +01:00
Mark Andrews and Michał Kępień
c5fff690ad
Remove now redundant check for state != NULL
...
(cherry picked from commit ee135d8946
2020-11-25 13:42:12 +01:00
Michał Kępień
47c206f386
Silence cppcheck 2.2 false positive in udp_recv()
...
cppcheck 2.2 reports the following false positive:
lib/dns/dispatch.c:1220:14: warning: Either the condition 'resp==NULL' is redundant or there is possible null pointer dereference: resp. [nullPointerRedundantCheck]
if (disp != resp->disp) {
^
lib/dns/dispatch.c:1191:11: note: Assuming that condition 'resp==NULL' is not redundant
if (resp == NULL) {
^
lib/dns/dispatch.c:1220:14: note: Null pointer dereference
if (disp != resp->disp) {
^
Apparently this version of cppcheck gets confused about conditional
"goto" statements because line 1220 can never be reached if 'resp' is
NULL.
Move a code block to prevent the above false positive from being
reported without affecting the processing logic.
(cherry picked from commit 0b6216d1c7
2020-11-25 13:42:12 +01:00
Mark Andrews
62158e1882
Address TSAN error between dns_rbt_findnode() and subtractrdataset().
...
Having dns_rbt_findnode() in previous_closest_nsec() check of
node->data is a optimisation that triggers a TSAN error with
subtractrdataset(). find_closest_nsec() still needs to check if
the NSEC record are active or not and look for a earlier NSEC records
if it isn't. Set DNS_RBTFIND_EMPTYDATA so node->data isn't referenced
without the node lock being held.
WARNING: ThreadSanitizer: data race
Read of size 8 at 0x000000000001 by thread T1 (mutexes: read M1, read M2):
#0 dns_rbt_findnode lib/dns/rbt.c:1708
#1 previous_closest_nsec lib/dns/rbtdb.c:3760
#2 find_closest_nsec lib/dns/rbtdb.c:3942
#3 zone_find lib/dns/rbtdb.c:4091
#4 dns_db_findext lib/dns/db.c:536
#5 query_lookup lib/ns/query.c:5582
#6 ns__query_start lib/ns/query.c:5505
#7 query_setup lib/ns/query.c:5229
#8 ns_query_start lib/ns/query.c:11380
#9 ns__client_request lib/ns/client.c:2166
#10 processbuffer netmgr/tcpdns.c:230
#11 dnslisten_readcb netmgr/tcpdns.c:309
#12 read_cb netmgr/tcp.c:832
#13 <null> <null>
#14 <null> <null>
Previous write of size 8 at 0x000000000001 by thread T2 (mutexes: write M3):
#0 subtractrdataset lib/dns/rbtdb.c:7133
#1 dns_db_subtractrdataset lib/dns/db.c:742
#2 diff_apply lib/dns/diff.c:368
#3 dns_diff_apply lib/dns/diff.c:459
#4 do_one_tuple lib/dns/update.c:247
#5 update_one_rr lib/dns/update.c:275
#6 delete_if_action lib/dns/update.c:689
#7 foreach_rr lib/dns/update.c:471
#8 delete_if lib/dns/update.c:716
#9 dns_update_signaturesinc lib/dns/update.c:1948
#10 receive_secure_serial lib/dns/zone.c:15637
#11 dispatch lib/isc/task.c:1152
#12 run lib/isc/task.c:1344
#13 <null> <null>
Location is heap block of size 130 at 0x000000000028 allocated by thread T3:
#0 malloc <null>
#1 default_memalloc lib/isc/mem.c:713
#2 mem_get lib/isc/mem.c:622
#3 mem_allocateunlocked lib/isc/mem.c:1268
#4 isc___mem_allocate lib/isc/mem.c:1288
#5 isc__mem_allocate lib/isc/mem.c:2453
#6 isc___mem_get lib/isc/mem.c:1037
#7 isc__mem_get lib/isc/mem.c:2432
#8 create_node lib/dns/rbt.c:2239
#9 dns_rbt_addnode lib/dns/rbt.c:1202
#10 dns_rbtdb_create lib/dns/rbtdb.c:8668
#11 dns_db_create lib/dns/db.c:118
#12 receive_secure_db lib/dns/zone.c:16154
#13 dispatch lib/isc/task.c:1152
#14 run lib/isc/task.c:1344
#15 <null> <null>
Mutex M1 (0x000000000040) created at:
#0 pthread_rwlock_init <null>
#1 isc_rwlock_init lib/isc/rwlock.c:39
#2 dns_rbtdb_create lib/dns/rbtdb.c:8527
#3 dns_db_create lib/dns/db.c:118
#4 receive_secure_db lib/dns/zone.c:16154
#5 dispatch lib/isc/task.c:1152
#6 run lib/isc/task.c:1344
#7 <null> <null>
Mutex M2 (0x000000000044) created at:
#0 pthread_rwlock_init <null>
#1 isc_rwlock_init lib/isc/rwlock.c:39
#2 dns_rbtdb_create lib/dns/rbtdb.c:8600
#3 dns_db_create lib/dns/db.c:118
#4 receive_secure_db lib/dns/zone.c:16154
#5 dispatch lib/isc/task.c:1152
#6 run lib/isc/task.c:1344
#7 <null> <null>
Mutex M3 (0x000000000046) created at:
#0 pthread_rwlock_init <null>
#1 isc_rwlock_init lib/isc/rwlock.c:39
#2 dns_rbtdb_create lib/dns/rbtdb.c:8600
#3 dns_db_create lib/dns/db.c:118
#4 receive_secure_db lib/dns/zone.c:16154
#5 dispatch lib/isc/task.c:1152
#6 run lib/isc/task.c:1344
#7 <null> <null>
Thread T1 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create pthreads/thread.c:73
#2 isc_nm_start netmgr/netmgr.c:232
#3 create_managers bin/named/main.c:909
#4 setup bin/named/main.c:1223
#5 main bin/named/main.c:1523
Thread T2 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create pthreads/thread.c:73
#2 isc_taskmgr_create lib/isc/task.c:1434
#3 create_managers bin/named/main.c:915
#4 setup bin/named/main.c:1223
#5 main bin/named/main.c:1523
Thread T3 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create pthreads/thread.c:73
#2 isc_taskmgr_create lib/isc/task.c:1434
#3 create_managers bin/named/main.c:915
#4 setup bin/named/main.c:1223
#5 main bin/named/main.c:1523
SUMMARY: ThreadSanitizer: data race lib/dns/rbt.c:1708 in dns_rbt_findnode
(cherry picked from commit 244f84a84b
2020-11-11 08:25:55 +11:00
Mark Andrews
b2bccc68a5
Lock zone before calling zone_namerd_tostr()
...
WARNING: ThreadSanitizer: data race
Read of size 8 at 0x000000000001 by thread T1:
#0 inline_raw lib/dns/zone.c:1375
#1 zone_namerd_tostr lib/dns/zone.c:15316
#2 dns_zone_name lib/dns/zone.c:15391
#3 xfrin_log lib/dns/xfrin.c:1605
#4 xfrin_destroy lib/dns/xfrin.c:1477
#5 dns_xfrin_detach lib/dns/xfrin.c:739
#6 xfrin_connect_done lib/dns/xfrin.c:970
#7 tcpdnsconnect_cb netmgr/tcpdns.c:786
#8 tcp_connect_cb netmgr/tcp.c:292
#9 <null> <null>
#10 <null> <null>
Previous write of size 8 at 0x000000000001 by thread T2 (mutexes: write M1):
#0 zone_shutdown lib/dns/zone.c:14462
#1 dispatch lib/isc/task.c:1152
#2 run lib/isc/task.c:1344
#3 <null> <null>
Location is heap block of size 2769 at 0x000000000013 allocated by thread T3:
#0 malloc <null>
#1 default_memalloc lib/isc/mem.c:713
#2 mem_get lib/isc/mem.c:622
#3 mem_allocateunlocked lib/isc/mem.c:1268
#4 isc___mem_allocate lib/isc/mem.c:1288
#5 isc__mem_allocate lib/isc/mem.c:2453
#6 isc___mem_get lib/isc/mem.c:1037
#7 isc__mem_get lib/isc/mem.c:2432
#8 dns_zone_create lib/dns/zone.c:984
#9 configure_zone bin/named/server.c:6502
#10 do_addzone bin/named/server.c:13391
#11 named_server_changezone bin/named/server.c:13788
#12 named_control_docommand bin/named/control.c:207
#13 control_command bin/named/controlconf.c:392
#14 dispatch lib/isc/task.c:1152
#15 run lib/isc/task.c:1344
#16 <null> <null>
(cherry picked from commit 84f43903da
2020-11-10 17:16:43 +11:00
Mark Andrews
ee53b9558f
Call nta_detach() before dns_view_weakdetach() so view is available.
...
(cherry picked from commit ea956976d1
2020-11-03 23:50:19 +11:00
Mark Andrews
4a4605fbef
Handle DNS_R_NCACHENXRRSET in fetch_callback_{dnskey,validator}()
...
DNS_R_NCACHENXRRSET can be return when zones are in transition state
from being unsigned to signed and signed to unsigned. The validation
should be resumed and should result in a insecure answer.
(cherry picked from commit 718e597def
2020-10-30 08:25:03 +11:00
Mark Andrews and Michał Kępień
adc090a478
address 'make depend' issues
...
(cherry picked from commit 06fa0d7b4e
2020-10-28 11:00:34 +01:00
Diego Fronza
6b97de3813
Fix transfer of glue records in stub zones if master has minimal-responses set
...
Stub zones don't make use of AXFR/IXFR for the transfering of zone
data, instead, a single query is issued to the master asking for
their nameserver records (NS).
That works fine unless master is configured with 'minimal-responses'
set to yes, in which case glue records are not provided by master
in the answer with nameservers authoritative for the zone, leaving
stub zones with incomplete databases.
This commit fix this problem in a simple way, when the answer with
the authoritative nameservers is received from master (stub_callback),
for each nameserver listed (save_nsrrset), a A and AAAA records for
the name is verified in the additional section, and if not present
a query is created to resolve the corresponsing missing glue.
A struct 'stub_cb_args' was added to keep relevant information for
performing a query, like TSIG key, udp size, dscp value, etc, this
information is borrowed from, and created within function 'ns_query',
where the resolving of nameserver from master starts.
A new field was added to the struct 'dns_stub', an atomic integer,
namely pending_requests, which is used to keep how many queries are
created when resolving nameserver addresses that were missing in
the glue.
When the value of pending_requests is zero we know we can release
resources, adjust zone timers, dump to zone file, etc.
2020-10-26 10:45:27 -03:00
Mark Andrews
86d9d04fd8
Hold qid->lock when calling deref_portentry() as
...
socket_search() need portentry to be unchanging.
WARNING: ThreadSanitizer: data race
Write of size 8 at 0x000000000001 by thread T1 (mutexes: write M1):
#0 deref_portentry lib/dns/dispatch.c:630
#1 deactivate_dispsocket lib/dns/dispatch.c:861
#2 udp_recv lib/dns/dispatch.c:1105
#3 udp_exrecv lib/dns/dispatch.c:1028
#4 dispatch lib/isc/task.c:1152
#5 run lib/isc/task.c:1344
#6 <null> <null>
Previous read of size 8 at 0x000000000001 by thread T2 (mutexes: write M1, write M2):
#0 socket_search lib/dns/dispatch.c:661
#1 get_dispsocket lib/dns/dispatch.c:744
#2 dns_dispatch_addresponse lib/dns/dispatch.c:3120
#3 resquery_send lib/dns/resolver.c:2467
#4 fctx_query lib/dns/resolver.c:2217
#5 fctx_try lib/dns/resolver.c:4245
#6 fctx_timeout lib/dns/resolver.c:4570
#7 dispatch lib/isc/task.c:1152
#8 run lib/isc/task.c:1344
#9 <null> <null>
(cherry picked from commit 5c253c416d
2020-10-24 08:23:31 +11:00
Mark Andrews
a2d2161be4
DNS_ZONEFLAG_NOIXFR should be DNS_ZONEFLG_NOIXFR
...
(cherry picked from commit 3a044444bd
2020-10-24 00:29:07 +11:00
Tinderbox User and Michał Kępień
6995173371
prep 9.11.24
2020-10-22 09:22:48 +02:00
Mark Andrews
56e8628e92
Restore the dns_message_reset() call before the dns_dispatch_getnext()
...
This was accidentally lost in the process of moving rmessage from fctx
to query. Without this dns_message_setclass() will fail.
2020-10-08 13:10:31 +11:00
Mark Andrews
24e204d013
Add the ability select individual tests to rdata_test
...
(cherry picked from commit 6293682020
2020-10-01 23:05:10 +10:00
Diego Fronza and Ondřej Surý
79c62017eb
Properly handling dns_message_t shared references
...
This commit fix the problems that arose when moving the dns_message_t
object from fetchctx_t to the query structure.
Since the lifetime of query objects are different than that of a
fetchctx and the dns_message_t object held by the query may be being
used by some external module, e.g. validator, even after the query may
have been destroyed, propery handling of the references to the message
were added in this commit to avoid accessing an already destroyed
object.
Specifically, in resquery_response(), a reference to the message is
attached at the beginning of the function and detached at the end, since
a possible call to fctx_cancelquery() would release the dns_message_t
object, and in the next lines of code a call to add_bad() would require
a valid pointer to the same object.
In valcreate() a new reference is attached to the message object, this
ensures that if the corresponding query object is destroyed before the
validator attempts to access it, no invalid pointer access occurs.
In validated() we have to attach a new reference to the message, since
we destroy the validator object at the beginning of the function, and we
need access to the message in the next lines of the same function.
2020-09-30 14:04:04 +02:00
Diego Fronza and Ondřej Surý
d391a0b4c5
Fix invalid dns message state in resolver's logic
...
The assertion failure REQUIRE(msg->state == DNS_SECTION_ANY),
caused by calling dns_message_setclass within function resquery_response()
in resolver.c, was happening due to wrong management of dns message_t
objects used to process responses to the queries issued by the resolver.
Before the fix, a resolver's fetch context (fetchctx_t) would hold
a pointer to the message, this same reference would then be used over all
the attempts to resolve the query, trying next server, etc... for this to work
the message object would have it's state reset between each iteration, marking
it as ready for a new processing.
The problem arose in a scenario with many different forwarders configured,
managing the state of the dns_message_t object was lacking better
synchronization, which have led it to a invalid dns_message_t state in
resquery_response().
Instead of adding unnecessarily complex code to synchronize the object,
the dns_message_t object was moved from fetchctx_t structure to the
query structure, where it better belongs to, since each query will produce
a response, this way whenever a new query is created an associated
dns_messate_t is also created.
This commit deals mainly with moving the dns_message_t object from fetchctx_t
to the query structure.
2020-09-30 14:04:04 +02:00
Diego Fronza and Ondřej Surý
8781aef52e
Refactored dns_message_t for using attach/detach semantics
...
This commit will be used as a base for the next code updates in order
to have a better control of dns_message_t objects' lifetime.
2020-09-30 13:33:43 +02:00
Mark Andrews
a0b123a06e
Address lock order reversals when shutting down a view.
...
Release view->lock before calling dns_resolver_shutdown,
dns_adb_shutdown, and dns_requestmgr_shutdown.
2020-09-23 00:42:26 +00:00
Mark Andrews
cd54ac9abe
Address lock-order-inversion
...
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock)
Cycle in lock order graph: M1 (0x000000000001) => M2 (0x000000000002) => M1
Mutex M2 acquired here while holding mutex M1 in thread T1:
#0 pthread_rwlock_wrlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:52:4
#2 zone_postload lib/dns/zone.c:5101:2
#3 receive_secure_db lib/dns/zone.c:16206:11
#4 dispatch lib/isc/task.c:1152:7
#5 run lib/isc/task.c:1344:2
Mutex M1 previously acquired by the same thread here:
#0 pthread_mutex_lock <null>
#1 receive_secure_db lib/dns/zone.c:16204:2
#2 dispatch lib/isc/task.c:1152:7
#3 run lib/isc/task.c:1344:2
Mutex M1 acquired here while holding mutex M2 in thread T1:
#0 pthread_mutex_lock <null>
#1 get_raw_serial lib/dns/zone.c:2518:2
#2 zone_gotwritehandle lib/dns/zone.c:2559:4
#3 dispatch lib/isc/task.c:1152:7
#4 run lib/isc/task.c:1344:2
Mutex M2 previously acquired by the same thread here:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 zone_gotwritehandle lib/dns/zone.c:2552:2
#3 dispatch lib/isc/task.c:1152:7
#4 run lib/isc/task.c:1344:2
Thread T1 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:73:8
#2 isc_taskmgr_create lib/isc/task.c:1434:3
#3 create_managers bin/named/main.c:915:11
#4 setup bin/named/main.c:1223:11
#5 main bin/named/main.c:1523:2
SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) in pthread_rwlock_wrlock
(cherry picked from commit 1090876693
2020-09-22 22:29:09 +10:00
Mark Andrews
267fe9a6b7
Pause dbiterator ealier to prevent lock-order-inversion
...
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock)
Cycle in lock order graph: M1 (0x000000000000) => M2 (0x000000000000) => M1
Mutex M2 acquired here while holding mutex M1 in thread T1:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 findnodeintree lib/dns/rbtdb.c:2877:2
#3 findnode lib/dns/rbtdb.c:2941:10
#4 dns_db_findnode lib/dns/db.c:439:11
#5 resume_addnsec3chain lib/dns/zone.c:3776:11
#6 rss_post lib/dns/zone.c:20659:3
#7 setnsec3param lib/dns/zone.c:20471:3
#8 dispatch lib/isc/task.c:1152:7
#9 run lib/isc/task.c:1344:2
Mutex M1 previously acquired by the same thread here:
#0 pthread_mutex_lock <null>
#1 rss_post lib/dns/zone.c:20658:3
#2 setnsec3param lib/dns/zone.c:20471:3
#3 dispatch lib/isc/task.c:1152:7
#4 run lib/isc/task.c:1344:2
Mutex M1 acquired here while holding mutex M2 in thread T2:
#0 pthread_mutex_lock <null>
#1 zone_nsec3chain lib/dns/zone.c:8666:5
#2 zone_maintenance lib/dns/zone.c:11063:4
#3 zone_timer lib/dns/zone.c:14098:2
#4 dispatch lib/isc/task.c:1152:7
#5 run lib/isc/task.c:1344:2
Mutex M2 previously acquired by the same thread here:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 resume_iteration lib/dns/rbtdb.c:9357:2
#3 dbiterator_next lib/dns/rbtdb.c:9647:3
#4 dns_dbiterator_next lib/dns/dbiterator.c:87:10
#5 zone_nsec3chain lib/dns/zone.c:8656:13
#6 zone_maintenance lib/dns/zone.c:11063:4
#7 zone_timer lib/dns/zone.c:14098:2
#8 dispatch lib/isc/task.c:1152:7
#9 run lib/isc/task.c:1344:2
(cherry picked from commit 9e584a4511
2020-09-17 18:27:14 +10:00
Mark Andrews
9781ce414a
Pause the database iterator to release rwlock
...
(cherry picked from commit 2e63de94aa
2020-09-17 18:26:28 +10:00
Mark Andrews
215526caf4
Pause dbiterator to release rwlock to prevent lock-order-inversion.
...
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock)
Cycle in lock order graph: M1 (0x000000000000) => M2 (0x000000000001) => M1
Mutex M2 acquired here while holding mutex M1 in thread T1:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 getsigningtime lib/dns/rbtdb.c:8198:2
#3 dns_db_getsigningtime lib/dns/db.c:979:11
#4 set_resigntime lib/dns/zone.c:3887:11
#5 dns_zone_markdirty lib/dns/zone.c:11119:4
#6 update_action lib/ns/update.c:3376:3
#7 dispatch lib/isc/task.c:1152:7
#8 run lib/isc/task.c:1344:2
Mutex M1 previously acquired by the same thread here:
#0 pthread_mutex_lock <null>
#1 dns_zone_markdirty lib/dns/zone.c:11089:2
#2 update_action lib/ns/update.c:3376:3
#3 dispatch lib/isc/task.c:1152:7
#4 run lib/isc/task.c:1344:2
Mutex M1 acquired here while holding mutex M2 in thread T1:
#0 pthread_mutex_lock <null>
#1 zone_nsec3chain lib/dns/zone.c:8502:3
#2 zone_maintenance lib/dns/zone.c:11056:4
#3 zone_timer lib/dns/zone.c:14091:2
#4 dispatch lib/isc/task.c:1152:7
#5 run lib/isc/task.c:1344:2
Mutex M2 previously acquired by the same thread here:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 resume_iteration lib/dns/rbtdb.c:9357:2
#3 dbiterator_current lib/dns/rbtdb.c:9695:3
#4 dns_dbiterator_current lib/dns/dbiterator.c:101:10
#5 zone_nsec3chain lib/dns/zone.c:8539:3
#6 zone_maintenance lib/dns/zone.c:11056:4
#7 zone_timer lib/dns/zone.c:14091:2
#8 dispatch lib/isc/task.c:1152:7
#9 run lib/isc/task.c:1344:2
Thread T1 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:73:8
#2 isc_taskmgr_create lib/isc/task.c:1434:3
#3 create_managers bin/named/main.c:915:11
#4 setup bin/named/main.c:1223:11
#5 main bin/named/main.c:1523:2
SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) in pthread_rwlock_rdlock
(cherry picked from commit fbed962204
2020-09-17 18:26:27 +10:00
Mark Andrews
38734d80b5
Pause dbiterator to release rwlock to prevent lock-order-inversion.
...
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock)
Cycle in lock order graph: M1 (0x000000000001) => M2 (0x000000000000) => M1
Mutex M2 acquired here while holding mutex M1 in thread T1:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 zone_sign lib/dns/zone.c:9247:3
#3 zone_maintenance lib/dns/zone.c:11047:4
#4 zone_timer lib/dns/zone.c:14090:2
#5 dispatch lib/isc/task.c:1152:7
#6 run lib/isc/task.c:1344:2
Mutex M1 previously acquired by the same thread here:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 resume_iteration lib/dns/rbtdb.c:9357:2
#3 dbiterator_next lib/dns/rbtdb.c:9647:3
#4 dns_dbiterator_next lib/dns/dbiterator.c:87:10
#5 zone_sign lib/dns/zone.c:9488:13
#6 zone_maintenance lib/dns/zone.c:11047:4
#7 zone_timer lib/dns/zone.c:14090:2
#8 dispatch lib/isc/task.c:1152:7
#9 run lib/isc/task.c:1344:2
Mutex M1 acquired here while holding mutex M2 in thread T2:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 findnodeintree lib/dns/rbtdb.c:2877:2
#3 findnode lib/dns/rbtdb.c:2941:10
#4 dns_db_findnode lib/dns/db.c:439:11
#5 dns_db_getsoaserial lib/dns/db.c:780:11
#6 dump_done lib/dns/zone.c:11428:15
#7 dump_quantum lib/dns/masterdump.c:1487:2
#8 dispatch lib/isc/task.c:1152:7
#9 run lib/isc/task.c:1344:2
Mutex M2 previously acquired by the same thread here:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 dump_done lib/dns/zone.c:11426:4
#3 dump_quantum lib/dns/masterdump.c:1487:2
#4 dispatch lib/isc/task.c:1152:7
#5 run lib/isc/task.c:1344:2
Thread T1 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:73:8
#2 isc_taskmgr_create lib/isc/task.c:1434:3
#3 create_managers bin/named/main.c:915:11
#4 setup bin/named/main.c:1223:11
#5 main bin/named/main.c:1523:2
Thread T2 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:73:8
#2 isc_taskmgr_create lib/isc/task.c:1434:3
#3 create_managers bin/named/main.c:915:11
#4 setup bin/named/main.c:1223:11
#5 main bin/named/main.c:1523:2
SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) in pthread_rwlock_rdlock
(cherry picked from commit c9dbad97b2
2020-09-17 18:26:27 +10:00
Mark Andrews
508bb0859c
Pause dbiterator to release rwlock to prevent lock-order-inversion.
...
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock)
Cycle in lock order graph: M1 (0x000000000000) => M2 (0x000000000000) => M1
Mutex M2 acquired here while holding mutex M1 in thread T1:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 getsigningtime lib/dns/rbtdb.c:8198:2
#3 dns_db_getsigningtime lib/dns/db.c:979:11
#4 set_resigntime lib/dns/zone.c:3887:11
#5 dns_zone_markdirty lib/dns/zone.c:11115:4
#6 update_action lib/ns/update.c:3376:3
#7 dispatch lib/isc/task.c:1152:7
#8 run lib/isc/task.c:1344:2
Mutex M1 previously acquired by the same thread here:
#0 pthread_mutex_lock <null>
#1 dns_zone_markdirty lib/dns/zone.c:11085:2
#2 update_action lib/ns/update.c:3376:3
#3 dispatch lib/isc/task.c:1152:7
#4 run lib/isc/task.c:1344:2
Mutex M1 acquired here while holding mutex M2 in thread T2:
#0 pthread_mutex_lock <null>
#1 zone_nsec3chain lib/dns/zone.c:8274:3
#2 zone_maintenance lib/dns/zone.c:11052:4
#3 zone_timer lib/dns/zone.c:14087:2
#4 dispatch lib/isc/task.c:1152:7
#5 run lib/isc/task.c:1344:2
Mutex M2 previously acquired by the same thread here:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 resume_iteration lib/dns/rbtdb.c:9357:2
#3 dbiterator_next lib/dns/rbtdb.c:9647:3
#4 dns_dbiterator_next lib/dns/dbiterator.c:87:10
#5 zone_nsec3chain lib/dns/zone.c:8412:13
#6 zone_maintenance lib/dns/zone.c:11052:4
#7 zone_timer lib/dns/zone.c:14087:2
#8 dispatch lib/isc/task.c:1152:7
#9 run lib/isc/task.c:1344:2
Thread T1 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:73:8
#2 isc_taskmgr_create lib/isc/task.c:1434:3
#3 create_managers bin/named/main.c:915:11
#4 setup bin/named/main.c:1223:11
#5 main bin/named/main.c:1523:2
Thread T2 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:73:8
#2 isc_taskmgr_create lib/isc/task.c:1434:3
#3 create_managers bin/named/main.c:915:11
#4 setup bin/named/main.c:1223:11
#5 main bin/named/main.c:1523:2
SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) in pthread_rwlock_rdlock
(cherry picked from commit 98025e15d0
2020-09-17 18:26:26 +10:00
Mark Andrews
36849cbfa7
Pause dbiterator to release rwlock to prevent lock-order-inversion.
...
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock)
Cycle in lock order graph: M1 (0x000000000001) => M2 (0x000000000002) => M3 (0x000000000000) => M1
Mutex M2 acquired here while holding mutex M1 in thread T1:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 findnodeintree lib/dns/rbtdb.c:2877:2
#3 findnode lib/dns/rbtdb.c:2941:10
#4 dns_db_findnode lib/dns/db.c:439:11
#5 copy_non_dnssec_records lib/dns/zone.c:16031:11
#6 receive_secure_db lib/dns/zone.c:16163:12
#7 dispatch lib/isc/task.c:1152:7
#8 run lib/isc/task.c:1344:2
Mutex M1 previously acquired by the same thread here:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 resume_iteration lib/dns/rbtdb.c:9357:2
#3 dbiterator_first lib/dns/rbtdb.c:9407:3
#4 dns_dbiterator_first lib/dns/dbiterator.c:43:10
#5 receive_secure_db lib/dns/zone.c:16160:16
#6 dispatch lib/isc/task.c:1152:7
#7 run lib/isc/task.c:1344:2
Mutex M3 acquired here while holding mutex M2 in thread T2:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 zone_sign lib/dns/zone.c:9244:3
#3 zone_maintenance lib/dns/zone.c:11044:4
#4 zone_timer lib/dns/zone.c:14087:2
#5 dispatch lib/isc/task.c:1152:7
#6 run lib/isc/task.c:1344:2
Mutex M2 previously acquired by the same thread here:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 resume_iteration lib/dns/rbtdb.c:9357:2
#3 dbiterator_next lib/dns/rbtdb.c:9647:3
#4 dns_dbiterator_next lib/dns/dbiterator.c:87:10
#5 zone_sign lib/dns/zone.c:9485:13
#6 zone_maintenance lib/dns/zone.c:11044:4
#7 zone_timer lib/dns/zone.c:14087:2
#8 dispatch lib/isc/task.c:1152:7
#9 run lib/isc/task.c:1344:2
Mutex M1 acquired here while holding mutex M3 in thread T3:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 findnodeintree lib/dns/rbtdb.c:2877:2
#3 findnode lib/dns/rbtdb.c:2941:10
#4 dns_db_findnode lib/dns/db.c:439:11
#5 zone_get_from_db lib/dns/zone.c:5602:11
#6 get_raw_serial lib/dns/zone.c:2520:12
#7 zone_gotwritehandle lib/dns/zone.c:2559:4
#8 dispatch lib/isc/task.c:1152:7
#9 run lib/isc/task.c:1344:2
Mutex M3 previously acquired by the same thread here:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 zone_gotwritehandle lib/dns/zone.c:2552:2
#3 dispatch lib/isc/task.c:1152:7
#4 run lib/isc/task.c:1344:2
Thread T1 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:73:8
#2 isc_taskmgr_create lib/isc/task.c:1434:3
#3 create_managers bin/named/main.c:915:11
#4 setup bin/named/main.c:1223:11
#5 main bin/named/main.c:1523:2
Thread T2 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:73:8
#2 isc_taskmgr_create lib/isc/task.c:1434:3
#3 create_managers bin/named/main.c:915:11
#4 setup bin/named/main.c:1223:11
#5 main bin/named/main.c:1523:2
Thread T3 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:73:8
#2 isc_taskmgr_create lib/isc/task.c:1434:3
#3 create_managers bin/named/main.c:915:11
#4 setup bin/named/main.c:1223:11
#5 main bin/named/main.c:1523:2
SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) in pthread_rwlock_rdlock
(cherry picked from commit e185e37137
2020-09-17 18:26:25 +10:00
Mark Andrews
7f6cddad0c
Address lock-order-inversion between the keytable and the db locks.
...
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock)
Cycle in lock order graph: M1 (0x000000000000) => M2 (0x000000000000) => M1
Mutex M2 acquired here while holding mutex M1 in thread T1:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 dns_keytable_find lib/dns/keytable.c:522:2
#3 sync_keyzone lib/dns/zone.c:4560:12
#4 dns_zone_synckeyzone lib/dns/zone.c:4635:11
#5 mkey_refresh bin/named/server.c:15423:2
#6 named_server_mkeys bin/named/server.c:15727:4
#7 named_control_docommand bin/named/control.c:236:12
#8 control_command bin/named/controlconf.c:365:17
#9 dispatch lib/isc/task.c:1152:7
#10 run lib/isc/task.c:1344:2
Mutex M1 previously acquired by the same thread here:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 resume_iteration lib/dns/rbtdb.c:9357:2
#3 dbiterator_first lib/dns/rbtdb.c:9407:3
#4 dns_dbiterator_first lib/dns/dbiterator.c:43:10
#5 dns_rriterator_first lib/dns/rriterator.c:71:15
#6 sync_keyzone lib/dns/zone.c:4543:16
#7 dns_zone_synckeyzone lib/dns/zone.c:4635:11
#8 mkey_refresh bin/named/server.c:15423:2
#9 named_server_mkeys bin/named/server.c:15727:4
#10 named_control_docommand bin/named/control.c:236:12
#11 control_command bin/named/controlconf.c:365:17
#12 dispatch lib/isc/task.c:1152:7
#13 run lib/isc/task.c:1344:2
Mutex M1 acquired here while holding mutex M2 in thread T1:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 zone_find lib/dns/rbtdb.c:4029:2
#3 dns_db_find lib/dns/db.c:500:11
#4 addifmissing lib/dns/zone.c:4481:11
#5 dns_keytable_forall lib/dns/keytable.c:786:4
#6 sync_keyzone lib/dns/zone.c:4586:2
#7 dns_zone_synckeyzone lib/dns/zone.c:4635:11
#8 mkey_refresh bin/named/server.c:15423:2
#9 named_server_mkeys bin/named/server.c:15727:4
#10 named_control_docommand bin/named/control.c:236:12
#11 control_command bin/named/controlconf.c:365:17
#12 dispatch lib/isc/task.c:1152:7
#13 run lib/isc/task.c:1344:2
Mutex M2 previously acquired by the same thread here:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 dns_keytable_forall lib/dns/keytable.c:770:2
#3 sync_keyzone lib/dns/zone.c:4586:2
#4 dns_zone_synckeyzone lib/dns/zone.c:4635:11
#5 mkey_refresh bin/named/server.c:15423:2
#6 named_server_mkeys bin/named/server.c:15727:4
#7 named_control_docommand bin/named/control.c:236:12
#8 control_command bin/named/controlconf.c:365:17
#9 dispatch lib/isc/task.c:1152:7
#10 run lib/isc/task.c:1344:2
Thread T1 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:73:8
#2 isc_taskmgr_create lib/isc/task.c:1434:3
#3 create_managers bin/named/main.c:915:11
#4 setup bin/named/main.c:1223:11
#5 main bin/named/main.c:1523:2
SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) in pthread_rwlock_rdlock
(cherry picked from commit 9e5f83c499
2020-09-17 18:26:22 +10:00
Tinderbox User and Michał Kępień
e7b0d5de15
prep 9.11.23
2020-09-16 23:00:15 +02:00
Mark Andrews
f30ec018f9
The runtime of rbt_test can exceed 5 minutes when running under TSAN testing.
...
Limit rbt_insert_and_remove subtest runtime to ~180 seconds which gives a total
runtime of ~270 seconds.
2020-09-15 02:15:27 +00:00
Evan Hunt
400171aee8
update all copyright headers to eliminate the typo
2020-09-14 17:00:40 -07:00
Mark Andrews
e075ae082c
LOCK should be UNLOCK
...
*** CID 306731: Program hangs (LOCK)
/lib/dns/resolver.c: 8943 in spillattimer_countdown()
8937 result = isc_timer_reset(res->spillattimer,
8938 isc_timertype_inactive, NULL,
8939 NULL, true);
8940 RUNTIME_CHECK(result == ISC_R_SUCCESS);
8941 }
8942 count = res->spillat;
CID 306731: Program hangs (LOCK)
"pthread_mutex_lock" locks "res->spill_lock" while it is locked.
8943 LOCK(&res->spill_lock);
8944 UNLOCK(&res->lock);
8945 if (logit)
8946 isc_log_write(dns_lctx, DNS_LOGCATEGORY_RESOLVER,
8947 DNS_LOGMODULE_RESOLVER, ISC_LOG_NOTICE,
8948 "clients-per-query decreased to %u", count);
2020-09-09 15:28:57 +10:00
Mark Andrews
90185b225f
Update 'init_count' atomically to silence tsan errors.
2020-09-08 17:41:33 +10:00
Mark Andrews
a1dcb73f67
The node lock was released too early.
...
NEGATIVE needs to be call with the node lock held.
WARNING: ThreadSanitizer: data race
Write of size 2 at 0x000000000001 by thread T1 (mutexes: write M1):
#0 mark_stale_header lib/dns/rbtdb.c:1802:21
#1 add32 lib/dns/rbtdb.c:6559:5
#2 addrdataset lib/dns/rbtdb.c:6975:12
#3 dns_db_addrdataset lib/dns/db.c:783:10
#4 cache_name lib/dns/resolver.c:5829:13
#5 cache_message lib/dns/resolver.c:5926:14
#6 resquery_response lib/dns/resolver.c:8618:12
#7 dispatch lib/isc/task.c:1157:7
#8 run lib/isc/task.c:1331:2
Previous read of size 2 at 0x000000000001 by thread T2:
#0 cache_findrdataset lib/dns/rbtdb.c:5932:6
#1 dns_db_findrdataset lib/dns/db.c:739:10
#2 query_addadditional2 bin/named/query.c:2196:11
#3 additionaldata_ns lib/dns/./rdata/generic/ns_2.c:198:10
#4 dns_rdata_additionaldata lib/dns/rdata.c:1246:2
#5 dns_rdataset_additionaldata lib/dns/rdataset.c:629:12
#6 query_addrdataset bin/named/query.c:2411:8
#7 query_addrrset bin/named/query.c:2802:2
#8 query_addbestns bin/named/query.c:3501:2
#9 query_find bin/named/query.c:9165:4
#10 query_resume bin/named/query.c:4164:12
#11 dispatch lib/isc/task.c:1157:7
#12 run lib/isc/task.c:1331:2
2020-09-08 17:41:33 +10:00
Mark Andrews
8f88bc61e9
Lock zone to read DNS_ZONEFLG_DIALREFRESH
2020-09-08 17:41:33 +10:00
Mark Andrews
7db778854e
Use a reference counter for zt
...
WARNING: ThreadSanitizer: data race
Write of size 8 at 0x000000000001 by thread T1 (mutexes: write M1):
#0 memset <null>
#1 mem_put lib/isc/mem.c:819
#2 isc___mem_free lib/isc/mem.c:1662
#3 isc__mem_free lib/isc/mem.c:3078
#4 isc___mem_putanddetach lib/isc/mem.c:1221
#5 isc__mem_putanddetach lib/isc/mem.c:3033
#6 zt_destroy lib/dns/zt.c:214
#7 doneloading lib/dns/zt.c:591
#8 zone_asyncload lib/dns/zone.c:2243
#9 dispatch lib/isc/task.c:1157
#10 run lib/isc/task.c:1331
#11 <null> <null>
Previous atomic read of size 8 at 0x000000000001 by thread T2:
#0 __tsan_atomic64_load <null>
#1 isc_rwlock_unlock lib/isc/rwlock.c:612
#2 doneloading lib/dns/zt.c:585
#3 zone_asyncload lib/dns/zone.c:2243
#4 dispatch lib/isc/task.c:1157
#5 run lib/isc/task.c:1331
#6 <null> <null>
Location is heap block of size 273 at 0x000000000015 allocated by thread T3:
#0 malloc <null>
#1 internal_memalloc lib/isc/mem.c:887
#2 mem_get lib/isc/mem.c:792
#3 mem_allocateunlocked lib/isc/mem.c:1545
#4 isc___mem_allocate lib/isc/mem.c:1566
#5 isc__mem_allocate lib/isc/mem.c:3048
#6 isc___mem_get lib/isc/mem.c:1304
#7 isc__mem_get lib/isc/mem.c:3012
#8 dns_zt_create lib/dns/zt.c:85
#9 dns_view_create lib/dns/view.c:126
#10 create_view server.c:5312
#11 load_configuration server.c:8101
#12 loadconfig server.c:9428
#13 ns_server_reconfigcommand server.c:9763
#14 ns_control_docommand bin/named/control.c:243
#15 control_recvmessage bin/named/controlconf.c:465
#16 dispatch lib/isc/task.c:1157
#17 run lib/isc/task.c:1331
#18 <null> <null>
2020-09-08 17:41:33 +10:00
Mark Andrews
7da21a6ff3
Lock zone before reading DNS_ZONEFLG_REFRESH
2020-09-08 09:25:43 +10:00
Mark Andrews
9e8e886fd6
Don't grab all the adb locks at once
...
FATAL: ThreadSanitizer CHECK failed: /build/llvm-toolchain-10-10.0.1~++20200708124224+ef32c611aa2/compiler-rt/lib/sanitizer_common/sanitizer_deadlock_detector.h:67 "((n_all_locks_)) < (((sizeof(all_locks_with_contexts_)/sizeof((all_locks_with_contexts_)[0]))))" (0x40, 0x40)
#0 __tsan::TsanCheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) <null> (named+0x4ad775)
#1 __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) <null> (named+0x4c429f)
#2 __sanitizer::DeadlockDetectorTLS<__sanitizer::TwoLevelBitVector<1ul, __sanitizer::BasicBitVector<unsigned long> > >::addLock(unsigned long, unsigned long, unsigned int) <null> (named+0x4b8a3c)
#3 __sanitizer::DD::MutexAfterLock(__sanitizer::DDCallback*, __sanitizer::DDMutex*, bool, bool) <null> (named+0x4b7fae)
#4 __tsan::MutexPostLock(__tsan::ThreadState*, unsigned long, unsigned long, unsigned int, int) <null> (named+0x4abc58)
#5 pthread_mutex_lock <null> (named+0x4643a8)
#6 dump_adb /builds/isc-projects/bind9/lib/dns/adb.c:3461:3 (libdns.so.1110+0x48be6)
#7 dns_adb_dump /builds/isc-projects/bind9/lib/dns/adb.c:3434:2 (libdns.so.1110+0x48738)
#8 dumpdone /builds/isc-projects/bind9/bin/named/./server.c:10149:4 (named+0x512022)
#9 dump_quantum /builds/isc-projects/bind9/lib/dns/masterdump.c:1424:2 (libdns.so.1110+0x9f253)
#10 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x50845)
#11 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d799)
#12 __tsan_thread_start_func <null> (named+0x446e3c)
#13 start_thread /build/glibc-vjB4T1/glibc-2.28/nptl/pthread_create.c:486:8 (libpthread.so.0+0x7fa2)
#14 clone /build/glibc-vjB4T1/glibc-2.28/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95 (libc.so.6+0xf94ce)
2020-09-08 09:25:43 +10:00
Mark Andrews
56ab2fc457
Lock access to state_mctx
...
WARNING: ThreadSanitizer: data race
Read of size 4 at 0x000000000001 by thread T1 (mutexes: write M1):
#0 isc___mem_allocate lib/isc/mem.c:1565:2
#1 isc__mem_allocate lib/isc/mem.c:3048:11
#2 isc___mem_get lib/isc/mem.c:1304:11
#3 isc__mem_get lib/isc/mem.c:3012:11
#4 set_state lib/dns/geoip2.c:174:29
#5 get_entry_for lib/dns/geoip2.c:236:11
#6 dns_geoip_match lib/dns/geoip2.c:383:10
#7 dns_aclelement_match2 lib/dns/acl.c:493:11
#8 dns_acl_match2 lib/dns/acl.c:298:7
#9 allowed bin/named/client.c:1866:11
#10 ns_client_isself bin/named/client.c:1931:7
#11 notify_isself lib/dns/zone.c:11025:11
#12 notify_send lib/dns/zone.c:11344:7
#13 notify_find_address lib/dns/zone.c:11148:2
#14 zone_notify lib/dns/zone.c:11586:3
#15 zone_maintenance lib/dns/zone.c:10261:4
#16 zone_timer lib/dns/zone.c:13106:2
#17 dispatch lib/isc/task.c:1157:7
#18 run lib/isc/task.c:1331:2
Previous write of size 8 at 0x000000000001 by thread T2 (mutexes: write M2, write M3):
#0 malloc <null>
#1 internal_memalloc lib/isc/mem.c:887:8
#2 isc_mem_createx2 lib/isc/mem.c:956:8
#3 isc_mem_create lib/isc/mem.c:2793:11
#4 state_key_init lib/dns/geoip2.c:119:14
#5 get_entry_for lib/dns/geoip2.c:214:11
#6 dns_geoip_match lib/dns/geoip2.c:383:10
#7 dns_aclelement_match2 lib/dns/acl.c:493:11
#8 dns_acl_match2 lib/dns/acl.c:298:7
#9 allowed bin/named/client.c:1866:11
#10 ns_client_isself bin/named/client.c:1931:7
#11 notify_isself lib/dns/zone.c:11025:11
#12 notify_send lib/dns/zone.c:11344:7
#13 notify_find_address lib/dns/zone.c:11148:2
#14 zone_notify lib/dns/zone.c:11586:3
#15 zone_maintenance lib/dns/zone.c:10261:4
#16 zone_timer lib/dns/zone.c:13106:2
#17 dispatch lib/isc/task.c:1157:7
#18 run lib/isc/task.c:1331:2
2020-09-08 09:25:43 +10:00
Mark Andrews
063a881a6c
Use atomics to update counters.
...
WARNING: ThreadSanitizer: data race
Write of size 4 at 0x000000000001 by thread T1:
#0 dns_acache_countquerymiss lib/dns/acache.c:1184:22
#1 rdataset_getadditional lib/dns/rbtdb.c:9875:3
#2 dns_rdataset_getadditional lib/dns/rdataset.c:711:11
#3 query_addadditional2 bin/named/query.c:1967:11
#4 additionaldata_ns lib/dns/./rdata/generic/ns_2.c:198:10
#5 dns_rdata_additionaldata lib/dns/rdata.c:1246:2
#6 dns_rdataset_additionaldata lib/dns/rdataset.c:629:12
#7 query_addrdataset bin/named/query.c:2411:8
#8 query_addrrset bin/named/query.c:2802:2
#9 query_find bin/named/query.c:9135:4
#10 query_resume bin/named/query.c:4164:12
#11 dispatch lib/isc/task.c:1157:7
#12 run lib/isc/task.c:1331:2
Previous write of size 4 at 0x000000000001 by thread T2:
#0 dns_acache_countquerymiss lib/dns/acache.c:1184:22
#1 rdataset_getadditional lib/dns/rbtdb.c:9875:3
#2 dns_rdataset_getadditional lib/dns/rdataset.c:711:11
#3 query_addadditional2 bin/named/query.c:1967:11
#4 additionaldata_ns lib/dns/./rdata/generic/ns_2.c:198:10
#5 dns_rdata_additionaldata lib/dns/rdata.c:1246:2
#6 dns_rdataset_additionaldata lib/dns/rdataset.c:629:12
#7 query_addrdataset bin/named/query.c:2411:8
#8 query_addrrset bin/named/query.c:2802:2
#9 query_find bin/named/query.c:9135:4
#10 query_resume bin/named/query.c:4164:12
#11 dispatch lib/isc/task.c:1157:7
#12 run lib/isc/task.c:1331:2
2020-09-08 09:25:43 +10:00
Mark Andrews
3d6dda54e7
state_key_initialized had a data race.
...
WARNING: ThreadSanitizer: data race (pid=25060)
Read of size 1 at 0x7fb4b1448098 by thread T12 (mutexes: write M802339438951732248):
#0 state_key_init /builds/isc-projects/bind9/lib/dns/geoip2.c:113:7 (libdns.so.1110+0x24204a)
#1 get_entry_for /builds/isc-projects/bind9/lib/dns/geoip2.c:214:11 (libdns.so.1110+0x241dc7)
#2 dns_geoip_match /builds/isc-projects/bind9/lib/dns/geoip2.c:383:10 (libdns.so.1110+0x2418a5)
#3 dns_aclelement_match2 /builds/isc-projects/bind9/lib/dns/acl.c:493:11 (libdns.so.1110+0x3f46b)
#4 dns_acl_match2 /builds/isc-projects/bind9/lib/dns/acl.c:298:7 (libdns.so.1110+0x3f236)
#5 allowed /builds/isc-projects/bind9/bin/named/client.c:1866:11 (named+0x4da797)
#6 ns_client_isself /builds/isc-projects/bind9/bin/named/client.c:1931:7 (named+0x4da688)
#7 notify_isself /builds/isc-projects/bind9/lib/dns/zone.c:11025:11 (libdns.so.1110+0x20f74a)
#8 notify_send /builds/isc-projects/bind9/lib/dns/zone.c:11344:7 (libdns.so.1110+0x20f397)
#9 notify_find_address /builds/isc-projects/bind9/lib/dns/zone.c:11148:2 (libdns.so.1110+0x20d9db)
#10 zone_notify /builds/isc-projects/bind9/lib/dns/zone.c:11586:3 (libdns.so.1110+0x204d98)
#11 zone_maintenance /builds/isc-projects/bind9/lib/dns/zone.c:10261:4 (libdns.so.1110+0x203d4d)
#12 zone_timer /builds/isc-projects/bind9/lib/dns/zone.c:13106:2 (libdns.so.1110+0x1e847a)
#13 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x50845)
#14 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d799)
Previous write of size 1 at 0x7fb4b1448098 by thread T11 (mutexes: write M818946462577691672, write M52428):
#0 state_key_init /builds/isc-projects/bind9/lib/dns/geoip2.c:129:27 (libdns.so.1110+0x242118)
#1 get_entry_for /builds/isc-projects/bind9/lib/dns/geoip2.c:214:11 (libdns.so.1110+0x241dc7)
#2 dns_geoip_match /builds/isc-projects/bind9/lib/dns/geoip2.c:383:10 (libdns.so.1110+0x2418a5)
#3 dns_aclelement_match2 /builds/isc-projects/bind9/lib/dns/acl.c:493:11 (libdns.so.1110+0x3f46b)
#4 dns_acl_match2 /builds/isc-projects/bind9/lib/dns/acl.c:298:7 (libdns.so.1110+0x3f236)
#5 allowed /builds/isc-projects/bind9/bin/named/client.c:1866:11 (named+0x4da797)
#6 ns_client_isself /builds/isc-projects/bind9/bin/named/client.c:1931:7 (named+0x4da688)
#7 notify_isself /builds/isc-projects/bind9/lib/dns/zone.c:11025:11 (libdns.so.1110+0x20f74a)
#8 notify_send /builds/isc-projects/bind9/lib/dns/zone.c:11344:7 (libdns.so.1110+0x20f397)
#9 notify_find_address /builds/isc-projects/bind9/lib/dns/zone.c:11148:2 (libdns.so.1110+0x20d9db)
#10 zone_notify /builds/isc-projects/bind9/lib/dns/zone.c:11586:3 (libdns.so.1110+0x204d98)
#11 zone_maintenance /builds/isc-projects/bind9/lib/dns/zone.c:10261:4 (libdns.so.1110+0x203d4d)
#12 zone_timer /builds/isc-projects/bind9/lib/dns/zone.c:13106:2 (libdns.so.1110+0x1e847a)
#13 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x50845)
#14 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d799)
Location is global 'state_key_initialized' of size 1 at 0x7fb4b1448098 (libdns.so.1110+0x0000002c6098)
2020-09-08 09:25:43 +10:00
Mark Andrews
80bf3f3852
Only test node->data if we care about whether data is present or not.
...
WARNING: ThreadSanitizer: data race (pid=28788)
Write of size 8 at 0x7b200002e060 by thread T1 (mutexes: write M2947):
#0 add32 /builds/isc-projects/bind9/lib/dns/rbtdb.c:6638:18 (libdns.so.1110+0xe7843)
#1 addrdataset /builds/isc-projects/bind9/lib/dns/rbtdb.c:6975:12 (libdns.so.1110+0xe4185)
#2 dns_db_addrdataset /builds/isc-projects/bind9/lib/dns/db.c:783:10 (libdns.so.1110+0x650ee)
#3 validated /builds/isc-projects/bind9/lib/dns/resolver.c:5140:11 (libdns.so.1110+0x1909f7)
#4 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507f5)
#5 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d749)
Previous read of size 8 at 0x7b200002e060 by thread T5 (mutexes: write M521146194917735760):
#0 dns_rbt_findnode /builds/isc-projects/bind9/lib/dns/rbt.c:1708:9 (libdns.so.1110+0xd910d)
#1 cache_find /builds/isc-projects/bind9/lib/dns/rbtdb.c:5098:11 (libdns.so.1110+0xe188e)
#2 dns_db_find /builds/isc-projects/bind9/lib/dns/db.c:554:11 (libdns.so.1110+0x642bb)
#3 dns_view_find2 /builds/isc-projects/bind9/lib/dns/view.c:1068:11 (libdns.so.1110+0x1cc2c4)
#4 dbfind_name /builds/isc-projects/bind9/lib/dns/adb.c:3714:11 (libdns.so.1110+0x46a4b)
#5 dns_adb_createfind2 /builds/isc-projects/bind9/lib/dns/adb.c:3133:12 (libdns.so.1110+0x45278)
#6 findname /builds/isc-projects/bind9/lib/dns/resolver.c:3166:11 (libdns.so.1110+0x1827f0)
#7 fctx_getaddresses /builds/isc-projects/bind9/lib/dns/resolver.c:3462:3 (libdns.so.1110+0x18032d)
#8 fctx_try /builds/isc-projects/bind9/lib/dns/resolver.c:3819:12 (libdns.so.1110+0x17e174)
#9 fctx_start /builds/isc-projects/bind9/lib/dns/resolver.c:4219:4 (libdns.so.1110+0x1787a3)
#10 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507f5)
#11 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d749)
2020-09-08 09:25:43 +10:00
Mark Andrews
674a219466
Address data race in dns_adbentry_overquota
...
Read of size 4 at 0x7b440003da70 by thread T8:
#0 dns_adbentry_overquota /builds/isc-projects/bind9/lib/dns/adb.c:4797:39 (libdns.so.1110+0x4da82)
#1 fctx_try /builds/isc-projects/bind9/lib/dns/resolver.c:3838:10 (libdns.so.1110+0x17e229)
#2 fctx_start /builds/isc-projects/bind9/lib/dns/resolver.c:4219:4 (libdns.so.1110+0x1787a3)
#3 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507f5)
#4 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d749)
Previous write of size 4 at 0x7b440003da70 by thread T12 (mutexes: write M549293692588722840):
#0 dns_adb_beginudpfetch /builds/isc-projects/bind9/lib/dns/adb.c:4811:21 (libdns.so.1110+0x4db82)
#1 fctx_query /builds/isc-projects/bind9/lib/dns/resolver.c:1901:3 (libdns.so.1110+0x1814ee)
#2 fctx_try /builds/isc-projects/bind9/lib/dns/resolver.c:3863:11 (libdns.so.1110+0x17e2fa)
#3 fctx_start /builds/isc-projects/bind9/lib/dns/resolver.c:4219:4 (libdns.so.1110+0x1787a3)
#4 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507f5)
#5 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d749)
2020-09-08 09:25:43 +10:00
Mark Andrews
121837aa75
increment header->count atomically
2020-09-08 09:25:43 +10:00
Mark Andrews
3a59243284
Address data race in dns_adb_detach.
...
Move INSIST inside of locked section.
2020-09-08 09:25:43 +10:00
Mark Andrews
505e338156
Address lock-order-inversion
...
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=12714)
Cycle in lock order graph: M100252 (0x7b7c00010a08) => M1171 (0x7b7400000dc8) => M100252
Mutex M1171 acquired here while holding mutex M100252 in thread T1:
#0 pthread_mutex_lock <null> (delv+0x4483a6)
#1 dns_resolver_createfetch3 /builds/isc-projects/bind9/lib/dns/resolver.c:9585:2 (libdns.so.1110+0x1769fd)
#2 dns_resolver_createfetch /builds/isc-projects/bind9/lib/dns/resolver.c:9504:10 (libdns.so.1110+0x174e17)
#3 create_fetch /builds/isc-projects/bind9/lib/dns/validator.c:1156:10 (libdns.so.1110+0x1c1e5f)
#4 validatezonekey /builds/isc-projects/bind9/lib/dns/validator.c:2124:13 (libdns.so.1110+0x1c3b6d)
#5 start_positive_validation /builds/isc-projects/bind9/lib/dns/validator.c:2301:10 (libdns.so.1110+0x1bfde9)
#6 validator_start /builds/isc-projects/bind9/lib/dns/validator.c:3647:12 (libdns.so.1110+0x1bef62)
#7 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507d5)
#8 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d729)
Mutex M100252 previously acquired by the same thread here:
#0 pthread_mutex_lock <null> (delv+0x4483a6)
#1 validator_start /builds/isc-projects/bind9/lib/dns/validator.c:3628:2 (libdns.so.1110+0x1bee31)
#2 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507d5)
#3 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d729)
Mutex M100252 acquired here while holding mutex M1171 in thread T1:
#0 pthread_mutex_lock <null> (delv+0x4483a6)
#1 dns_validator_destroy /builds/isc-projects/bind9/lib/dns/validator.c:3912:2 (libdns.so.1110+0x1bf788)
#2 validated /builds/isc-projects/bind9/lib/dns/resolver.c:4916:2 (libdns.so.1110+0x18fdfd)
#3 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507d5)
#4 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d729)
Mutex M1171 previously acquired by the same thread here:
#0 pthread_mutex_lock <null> (delv+0x4483a6)
#1 validated /builds/isc-projects/bind9/lib/dns/resolver.c:4907:2 (libdns.so.1110+0x18fc3d)
#2 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507d5)
#3 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d729)
Thread T1 'isc-worker0000' (tid=12729, running) created by main thread at:
#0 pthread_create <null> (delv+0x42afdb)
#1 isc_thread_create /builds/isc-projects/bind9/lib/isc/pthreads/thread.c:60:8 (libisc.so.1107+0x726d8)
#2 isc__taskmgr_create /builds/isc-projects/bind9/lib/isc/task.c:1468:7 (libisc.so.1107+0x4d635)
#3 isc_taskmgr_createinctx /builds/isc-projects/bind9/lib/isc/task.c:2091:11 (libisc.so.1107+0x4f4ac)
#4 main /builds/isc-projects/bind9/bin/delv/delv.c:1639:2 (delv+0x4b7f96)
SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) (/builds/isc-projects/bind9/bin/delv/.libs/delv+0x4483a6) in pthread_mutex_lock
2020-09-08 09:25:43 +10:00
Mark Andrews
3ed7746ec9
Address lock-order-inversion
...
Obtain references to view->redirect and view->managed_keys then
release view->lock so dns_zone_setviewcommit and dns_zone_setviewrevert
can obtain the view->lock while holding zone->lock.
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=9132)
Cycle in lock order graph: M987831431424375936 (0x000000000000) => M1012319771577875480 (0x000000000000) => M987831431424375936
Mutex M1012319771577875480 acquired here while holding mutex M987831431424375936 in thread T2:
#0 pthread_mutex_lock <null> (named+0x4642a6)
#1 dns_zone_setviewcommit /builds/isc-projects/bind9/lib/dns/zone.c:1571:2 (libdns.so.1110+0x1d74eb)
#2 dns_view_setviewcommit /builds/isc-projects/bind9/lib/dns/view.c:2388:3 (libdns.so.1110+0x1cfe29)
#3 load_configuration /builds/isc-projects/bind9/bin/named/./server.c:8188:3 (named+0x51eadd)
#4 loadconfig /builds/isc-projects/bind9/bin/named/./server.c:9438:11 (named+0x510c66)
#5 ns_server_reconfigcommand /builds/isc-projects/bind9/bin/named/./server.c:9773:2 (named+0x510b41)
#6 ns_control_docommand /builds/isc-projects/bind9/bin/named/control.c:243:12 (named+0x4e451a)
#7 control_recvmessage /builds/isc-projects/bind9/bin/named/controlconf.c:465:13 (named+0x4e9056)
#8 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507d5)
#9 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d729)
Mutex M987831431424375936 previously acquired by the same thread here:
#0 pthread_mutex_lock <null> (named+0x4642a6)
#1 dns_view_setviewcommit /builds/isc-projects/bind9/lib/dns/view.c:2382:2 (libdns.so.1110+0x1cfde7)
#2 load_configuration /builds/isc-projects/bind9/bin/named/./server.c:8188:3 (named+0x51eadd)
#3 loadconfig /builds/isc-projects/bind9/bin/named/./server.c:9438:11 (named+0x510c66)
#4 ns_server_reconfigcommand /builds/isc-projects/bind9/bin/named/./server.c:9773:2 (named+0x510b41)
#5 ns_control_docommand /builds/isc-projects/bind9/bin/named/control.c:243:12 (named+0x4e451a)
#6 control_recvmessage /builds/isc-projects/bind9/bin/named/controlconf.c:465:13 (named+0x4e9056)
#7 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507d5)
#8 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d729)
Mutex M987831431424375936 acquired here while holding mutex M1012319771577875480 in thread T7:
#0 pthread_mutex_lock <null> (named+0x4642a6)
#1 dns_view_findzonecut2 /builds/isc-projects/bind9/lib/dns/view.c:1300:2 (libdns.so.1110+0x1cc93a)
#2 dns_view_findzonecut /builds/isc-projects/bind9/lib/dns/view.c:1261:9 (libdns.so.1110+0x1cc864)
#3 fctx_create /builds/isc-projects/bind9/lib/dns/resolver.c:4459:13 (libdns.so.1110+0x1779d3)
#4 dns_resolver_createfetch3 /builds/isc-projects/bind9/lib/dns/resolver.c:9628:12 (libdns.so.1110+0x176cb6)
#5 dns_resolver_createfetch /builds/isc-projects/bind9/lib/dns/resolver.c:9504:10 (libdns.so.1110+0x174e17)
#6 zone_refreshkeys /builds/isc-projects/bind9/lib/dns/zone.c:10061:12 (libdns.so.1110+0x2055a5)
#7 zone_maintenance /builds/isc-projects/bind9/lib/dns/zone.c:10274:5 (libdns.so.1110+0x203a78)
#8 zone_timer /builds/isc-projects/bind9/lib/dns/zone.c:13106:2 (libdns.so.1110+0x1e815a)
#9 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507d5)
#10 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d729)
Mutex M1012319771577875480 previously acquired by the same thread here:
#0 pthread_mutex_lock <null> (named+0x4642a6)
#1 zone_refreshkeys /builds/isc-projects/bind9/lib/dns/zone.c:9951:2 (libdns.so.1110+0x204dc3)
#2 zone_maintenance /builds/isc-projects/bind9/lib/dns/zone.c:10274:5 (libdns.so.1110+0x203a78)
#3 zone_timer /builds/isc-projects/bind9/lib/dns/zone.c:13106:2 (libdns.so.1110+0x1e815a)
#4 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507d5)
#5 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d729)
Thread T2 'isc-worker0001' (tid=9163, running) created by main thread at:
#0 pthread_create <null> (named+0x446edb)
#1 isc_thread_create /builds/isc-projects/bind9/lib/isc/pthreads/thread.c:60:8 (libisc.so.1107+0x726d8)
#2 isc__taskmgr_create /builds/isc-projects/bind9/lib/isc/task.c:1468:7 (libisc.so.1107+0x4d635)
#3 isc_taskmgr_create /builds/isc-projects/bind9/lib/isc/task.c:2109:11 (libisc.so.1107+0x4f587)
#4 create_managers /builds/isc-projects/bind9/bin/named/./main.c:886:11 (named+0x4f1a97)
#5 setup /builds/isc-projects/bind9/bin/named/./main.c:1305:11 (named+0x4f05ee)
#6 main /builds/isc-projects/bind9/bin/named/./main.c:1556:2 (named+0x4ef12d)
Thread T7 'isc-worker0006' (tid=9168, running) created by main thread at:
#0 pthread_create <null> (named+0x446edb)
#1 isc_thread_create /builds/isc-projects/bind9/lib/isc/pthreads/thread.c:60:8 (libisc.so.1107+0x726d8)
#2 isc__taskmgr_create /builds/isc-projects/bind9/lib/isc/task.c:1468:7 (libisc.so.1107+0x4d635)
#3 isc_taskmgr_create /builds/isc-projects/bind9/lib/isc/task.c:2109:11 (libisc.so.1107+0x4f587)
#4 create_managers /builds/isc-projects/bind9/bin/named/./main.c:886:11 (named+0x4f1a97)
#5 setup /builds/isc-projects/bind9/bin/named/./main.c:1305:11 (named+0x4f05ee)
#6 main /builds/isc-projects/bind9/bin/named/./main.c:1556:2 (named+0x4ef12d)
SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) (/builds/isc-projects/bind9/bin/named/.libs/named+0x4642a6) in pthread_mutex_lock
2020-09-08 09:25:43 +10:00
Mark Andrews
d776a73362
Take complete ownership of *validatorp before calling destroy.
2020-09-08 09:25:43 +10:00
Mark Andrews
05cb799274
Take complete ownership or *keyp before calling tsigkey_free.
2020-09-08 09:25:43 +10:00
Mark Andrews
Michał Kępień
Tinderbox UserandMichał Kępień
Diego Fronza
Evan Hunt