Ondřej Surý
98def35e72
Handle the errors from sysconf() call in isc_meminfo_totalphys()
...
isc_meminfo_totalphys() would return invalid memory size when sysconf()
call would fail, because ((size_t)-1 * -1) is very large number.
(cherry picked from commit 79ca724d46
2020-09-21 10:59:36 +02:00
Ondřej Surý
fe738c7432
Exclude isc_mem_isovermem from ThreadSanitizer
...
The .is_overmem member of isc_mem_t structure is intentionally accessed
unlocked as 100% accuracy isn't necessary here.
Without the attribute, following TSAN warning would show up:
WARNING: ThreadSanitizer: data race
Write of size 1 at 0x000000000001 by thread T1 (mutexes: write M1, write M2):
#0 isc___mem_put lib/isc/mem.c:1119:19
#1 isc__mem_put lib/isc/mem.c:2439:2
#2 dns_rdataslab_fromrdataset lib/dns/rdataslab.c:327:2
#3 addrdataset lib/dns/rbtdb.c:6761:11
#4 dns_db_addrdataset lib/dns/db.c:719:10
#5 cache_name lib/dns/resolver.c:6538:13
#6 cache_message lib/dns/resolver.c:6628:14
#7 resquery_response lib/dns/resolver.c:7883:13
#8 dispatch lib/isc/task.c:1152:7
#9 run lib/isc/task.c:1344:2
Previous read of size 1 at 0x000000000001 by thread T2 (mutexes: write M3):
#0 isc_mem_isovermem lib/isc/mem.c:1553:15
#1 addrdataset lib/dns/rbtdb.c:6866:25
#2 dns_db_addrdataset lib/dns/db.c:719:10
#3 addoptout lib/dns/ncache.c:281:10
#4 dns_ncache_add lib/dns/ncache.c:101:10
#5 ncache_adderesult lib/dns/resolver.c:6668:12
#6 ncache_message lib/dns/resolver.c:6845:11
#7 rctx_ncache lib/dns/resolver.c:9174:11
#8 resquery_response lib/dns/resolver.c:7894:2
#9 dispatch lib/isc/task.c:1152:7
#10 run lib/isc/task.c:1344:2
Location is heap block of size 328 at 0x000000000020 allocated by thread T3:
#0 malloc <null>
#1 default_memalloc lib/isc/mem.c:713:8
#2 mem_create lib/isc/mem.c:763:8
#3 isc_mem_create lib/isc/mem.c:2425:2
#4 configure_view bin/named/server.c:4494:4
#5 load_configuration bin/named/server.c:9062:3
#6 run_server bin/named/server.c:9771:2
#7 dispatch lib/isc/task.c:1152:7
#8 run lib/isc/task.c:1344:2
[...]
SUMMARY: ThreadSanitizer: data race lib/isc/mem.c:1119:19 in isc___mem_put
(cherry picked from commit 0110d1ab17
2020-09-17 17:39:03 +02:00
Mark Andrews
267fe9a6b7
Pause dbiterator ealier to prevent lock-order-inversion
...
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock)
Cycle in lock order graph: M1 (0x000000000000) => M2 (0x000000000000) => M1
Mutex M2 acquired here while holding mutex M1 in thread T1:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 findnodeintree lib/dns/rbtdb.c:2877:2
#3 findnode lib/dns/rbtdb.c:2941:10
#4 dns_db_findnode lib/dns/db.c:439:11
#5 resume_addnsec3chain lib/dns/zone.c:3776:11
#6 rss_post lib/dns/zone.c:20659:3
#7 setnsec3param lib/dns/zone.c:20471:3
#8 dispatch lib/isc/task.c:1152:7
#9 run lib/isc/task.c:1344:2
Mutex M1 previously acquired by the same thread here:
#0 pthread_mutex_lock <null>
#1 rss_post lib/dns/zone.c:20658:3
#2 setnsec3param lib/dns/zone.c:20471:3
#3 dispatch lib/isc/task.c:1152:7
#4 run lib/isc/task.c:1344:2
Mutex M1 acquired here while holding mutex M2 in thread T2:
#0 pthread_mutex_lock <null>
#1 zone_nsec3chain lib/dns/zone.c:8666:5
#2 zone_maintenance lib/dns/zone.c:11063:4
#3 zone_timer lib/dns/zone.c:14098:2
#4 dispatch lib/isc/task.c:1152:7
#5 run lib/isc/task.c:1344:2
Mutex M2 previously acquired by the same thread here:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 resume_iteration lib/dns/rbtdb.c:9357:2
#3 dbiterator_next lib/dns/rbtdb.c:9647:3
#4 dns_dbiterator_next lib/dns/dbiterator.c:87:10
#5 zone_nsec3chain lib/dns/zone.c:8656:13
#6 zone_maintenance lib/dns/zone.c:11063:4
#7 zone_timer lib/dns/zone.c:14098:2
#8 dispatch lib/isc/task.c:1152:7
#9 run lib/isc/task.c:1344:2
(cherry picked from commit 9e584a4511
2020-09-17 18:27:14 +10:00
Mark Andrews
9781ce414a
Pause the database iterator to release rwlock
...
(cherry picked from commit 2e63de94aa
2020-09-17 18:26:28 +10:00
Mark Andrews
215526caf4
Pause dbiterator to release rwlock to prevent lock-order-inversion.
...
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock)
Cycle in lock order graph: M1 (0x000000000000) => M2 (0x000000000001) => M1
Mutex M2 acquired here while holding mutex M1 in thread T1:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 getsigningtime lib/dns/rbtdb.c:8198:2
#3 dns_db_getsigningtime lib/dns/db.c:979:11
#4 set_resigntime lib/dns/zone.c:3887:11
#5 dns_zone_markdirty lib/dns/zone.c:11119:4
#6 update_action lib/ns/update.c:3376:3
#7 dispatch lib/isc/task.c:1152:7
#8 run lib/isc/task.c:1344:2
Mutex M1 previously acquired by the same thread here:
#0 pthread_mutex_lock <null>
#1 dns_zone_markdirty lib/dns/zone.c:11089:2
#2 update_action lib/ns/update.c:3376:3
#3 dispatch lib/isc/task.c:1152:7
#4 run lib/isc/task.c:1344:2
Mutex M1 acquired here while holding mutex M2 in thread T1:
#0 pthread_mutex_lock <null>
#1 zone_nsec3chain lib/dns/zone.c:8502:3
#2 zone_maintenance lib/dns/zone.c:11056:4
#3 zone_timer lib/dns/zone.c:14091:2
#4 dispatch lib/isc/task.c:1152:7
#5 run lib/isc/task.c:1344:2
Mutex M2 previously acquired by the same thread here:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 resume_iteration lib/dns/rbtdb.c:9357:2
#3 dbiterator_current lib/dns/rbtdb.c:9695:3
#4 dns_dbiterator_current lib/dns/dbiterator.c:101:10
#5 zone_nsec3chain lib/dns/zone.c:8539:3
#6 zone_maintenance lib/dns/zone.c:11056:4
#7 zone_timer lib/dns/zone.c:14091:2
#8 dispatch lib/isc/task.c:1152:7
#9 run lib/isc/task.c:1344:2
Thread T1 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:73:8
#2 isc_taskmgr_create lib/isc/task.c:1434:3
#3 create_managers bin/named/main.c:915:11
#4 setup bin/named/main.c:1223:11
#5 main bin/named/main.c:1523:2
SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) in pthread_rwlock_rdlock
(cherry picked from commit fbed962204
2020-09-17 18:26:27 +10:00
Mark Andrews
38734d80b5
Pause dbiterator to release rwlock to prevent lock-order-inversion.
...
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock)
Cycle in lock order graph: M1 (0x000000000001) => M2 (0x000000000000) => M1
Mutex M2 acquired here while holding mutex M1 in thread T1:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 zone_sign lib/dns/zone.c:9247:3
#3 zone_maintenance lib/dns/zone.c:11047:4
#4 zone_timer lib/dns/zone.c:14090:2
#5 dispatch lib/isc/task.c:1152:7
#6 run lib/isc/task.c:1344:2
Mutex M1 previously acquired by the same thread here:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 resume_iteration lib/dns/rbtdb.c:9357:2
#3 dbiterator_next lib/dns/rbtdb.c:9647:3
#4 dns_dbiterator_next lib/dns/dbiterator.c:87:10
#5 zone_sign lib/dns/zone.c:9488:13
#6 zone_maintenance lib/dns/zone.c:11047:4
#7 zone_timer lib/dns/zone.c:14090:2
#8 dispatch lib/isc/task.c:1152:7
#9 run lib/isc/task.c:1344:2
Mutex M1 acquired here while holding mutex M2 in thread T2:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 findnodeintree lib/dns/rbtdb.c:2877:2
#3 findnode lib/dns/rbtdb.c:2941:10
#4 dns_db_findnode lib/dns/db.c:439:11
#5 dns_db_getsoaserial lib/dns/db.c:780:11
#6 dump_done lib/dns/zone.c:11428:15
#7 dump_quantum lib/dns/masterdump.c:1487:2
#8 dispatch lib/isc/task.c:1152:7
#9 run lib/isc/task.c:1344:2
Mutex M2 previously acquired by the same thread here:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 dump_done lib/dns/zone.c:11426:4
#3 dump_quantum lib/dns/masterdump.c:1487:2
#4 dispatch lib/isc/task.c:1152:7
#5 run lib/isc/task.c:1344:2
Thread T1 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:73:8
#2 isc_taskmgr_create lib/isc/task.c:1434:3
#3 create_managers bin/named/main.c:915:11
#4 setup bin/named/main.c:1223:11
#5 main bin/named/main.c:1523:2
Thread T2 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:73:8
#2 isc_taskmgr_create lib/isc/task.c:1434:3
#3 create_managers bin/named/main.c:915:11
#4 setup bin/named/main.c:1223:11
#5 main bin/named/main.c:1523:2
SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) in pthread_rwlock_rdlock
(cherry picked from commit c9dbad97b2
2020-09-17 18:26:27 +10:00
Mark Andrews
508bb0859c
Pause dbiterator to release rwlock to prevent lock-order-inversion.
...
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock)
Cycle in lock order graph: M1 (0x000000000000) => M2 (0x000000000000) => M1
Mutex M2 acquired here while holding mutex M1 in thread T1:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 getsigningtime lib/dns/rbtdb.c:8198:2
#3 dns_db_getsigningtime lib/dns/db.c:979:11
#4 set_resigntime lib/dns/zone.c:3887:11
#5 dns_zone_markdirty lib/dns/zone.c:11115:4
#6 update_action lib/ns/update.c:3376:3
#7 dispatch lib/isc/task.c:1152:7
#8 run lib/isc/task.c:1344:2
Mutex M1 previously acquired by the same thread here:
#0 pthread_mutex_lock <null>
#1 dns_zone_markdirty lib/dns/zone.c:11085:2
#2 update_action lib/ns/update.c:3376:3
#3 dispatch lib/isc/task.c:1152:7
#4 run lib/isc/task.c:1344:2
Mutex M1 acquired here while holding mutex M2 in thread T2:
#0 pthread_mutex_lock <null>
#1 zone_nsec3chain lib/dns/zone.c:8274:3
#2 zone_maintenance lib/dns/zone.c:11052:4
#3 zone_timer lib/dns/zone.c:14087:2
#4 dispatch lib/isc/task.c:1152:7
#5 run lib/isc/task.c:1344:2
Mutex M2 previously acquired by the same thread here:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 resume_iteration lib/dns/rbtdb.c:9357:2
#3 dbiterator_next lib/dns/rbtdb.c:9647:3
#4 dns_dbiterator_next lib/dns/dbiterator.c:87:10
#5 zone_nsec3chain lib/dns/zone.c:8412:13
#6 zone_maintenance lib/dns/zone.c:11052:4
#7 zone_timer lib/dns/zone.c:14087:2
#8 dispatch lib/isc/task.c:1152:7
#9 run lib/isc/task.c:1344:2
Thread T1 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:73:8
#2 isc_taskmgr_create lib/isc/task.c:1434:3
#3 create_managers bin/named/main.c:915:11
#4 setup bin/named/main.c:1223:11
#5 main bin/named/main.c:1523:2
Thread T2 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:73:8
#2 isc_taskmgr_create lib/isc/task.c:1434:3
#3 create_managers bin/named/main.c:915:11
#4 setup bin/named/main.c:1223:11
#5 main bin/named/main.c:1523:2
SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) in pthread_rwlock_rdlock
(cherry picked from commit 98025e15d0
2020-09-17 18:26:26 +10:00
Mark Andrews
36849cbfa7
Pause dbiterator to release rwlock to prevent lock-order-inversion.
...
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock)
Cycle in lock order graph: M1 (0x000000000001) => M2 (0x000000000002) => M3 (0x000000000000) => M1
Mutex M2 acquired here while holding mutex M1 in thread T1:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 findnodeintree lib/dns/rbtdb.c:2877:2
#3 findnode lib/dns/rbtdb.c:2941:10
#4 dns_db_findnode lib/dns/db.c:439:11
#5 copy_non_dnssec_records lib/dns/zone.c:16031:11
#6 receive_secure_db lib/dns/zone.c:16163:12
#7 dispatch lib/isc/task.c:1152:7
#8 run lib/isc/task.c:1344:2
Mutex M1 previously acquired by the same thread here:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 resume_iteration lib/dns/rbtdb.c:9357:2
#3 dbiterator_first lib/dns/rbtdb.c:9407:3
#4 dns_dbiterator_first lib/dns/dbiterator.c:43:10
#5 receive_secure_db lib/dns/zone.c:16160:16
#6 dispatch lib/isc/task.c:1152:7
#7 run lib/isc/task.c:1344:2
Mutex M3 acquired here while holding mutex M2 in thread T2:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 zone_sign lib/dns/zone.c:9244:3
#3 zone_maintenance lib/dns/zone.c:11044:4
#4 zone_timer lib/dns/zone.c:14087:2
#5 dispatch lib/isc/task.c:1152:7
#6 run lib/isc/task.c:1344:2
Mutex M2 previously acquired by the same thread here:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 resume_iteration lib/dns/rbtdb.c:9357:2
#3 dbiterator_next lib/dns/rbtdb.c:9647:3
#4 dns_dbiterator_next lib/dns/dbiterator.c:87:10
#5 zone_sign lib/dns/zone.c:9485:13
#6 zone_maintenance lib/dns/zone.c:11044:4
#7 zone_timer lib/dns/zone.c:14087:2
#8 dispatch lib/isc/task.c:1152:7
#9 run lib/isc/task.c:1344:2
Mutex M1 acquired here while holding mutex M3 in thread T3:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 findnodeintree lib/dns/rbtdb.c:2877:2
#3 findnode lib/dns/rbtdb.c:2941:10
#4 dns_db_findnode lib/dns/db.c:439:11
#5 zone_get_from_db lib/dns/zone.c:5602:11
#6 get_raw_serial lib/dns/zone.c:2520:12
#7 zone_gotwritehandle lib/dns/zone.c:2559:4
#8 dispatch lib/isc/task.c:1152:7
#9 run lib/isc/task.c:1344:2
Mutex M3 previously acquired by the same thread here:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 zone_gotwritehandle lib/dns/zone.c:2552:2
#3 dispatch lib/isc/task.c:1152:7
#4 run lib/isc/task.c:1344:2
Thread T1 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:73:8
#2 isc_taskmgr_create lib/isc/task.c:1434:3
#3 create_managers bin/named/main.c:915:11
#4 setup bin/named/main.c:1223:11
#5 main bin/named/main.c:1523:2
Thread T2 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:73:8
#2 isc_taskmgr_create lib/isc/task.c:1434:3
#3 create_managers bin/named/main.c:915:11
#4 setup bin/named/main.c:1223:11
#5 main bin/named/main.c:1523:2
Thread T3 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:73:8
#2 isc_taskmgr_create lib/isc/task.c:1434:3
#3 create_managers bin/named/main.c:915:11
#4 setup bin/named/main.c:1223:11
#5 main bin/named/main.c:1523:2
SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) in pthread_rwlock_rdlock
(cherry picked from commit e185e37137
2020-09-17 18:26:25 +10:00
Mark Andrews
7f6cddad0c
Address lock-order-inversion between the keytable and the db locks.
...
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock)
Cycle in lock order graph: M1 (0x000000000000) => M2 (0x000000000000) => M1
Mutex M2 acquired here while holding mutex M1 in thread T1:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 dns_keytable_find lib/dns/keytable.c:522:2
#3 sync_keyzone lib/dns/zone.c:4560:12
#4 dns_zone_synckeyzone lib/dns/zone.c:4635:11
#5 mkey_refresh bin/named/server.c:15423:2
#6 named_server_mkeys bin/named/server.c:15727:4
#7 named_control_docommand bin/named/control.c:236:12
#8 control_command bin/named/controlconf.c:365:17
#9 dispatch lib/isc/task.c:1152:7
#10 run lib/isc/task.c:1344:2
Mutex M1 previously acquired by the same thread here:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 resume_iteration lib/dns/rbtdb.c:9357:2
#3 dbiterator_first lib/dns/rbtdb.c:9407:3
#4 dns_dbiterator_first lib/dns/dbiterator.c:43:10
#5 dns_rriterator_first lib/dns/rriterator.c:71:15
#6 sync_keyzone lib/dns/zone.c:4543:16
#7 dns_zone_synckeyzone lib/dns/zone.c:4635:11
#8 mkey_refresh bin/named/server.c:15423:2
#9 named_server_mkeys bin/named/server.c:15727:4
#10 named_control_docommand bin/named/control.c:236:12
#11 control_command bin/named/controlconf.c:365:17
#12 dispatch lib/isc/task.c:1152:7
#13 run lib/isc/task.c:1344:2
Mutex M1 acquired here while holding mutex M2 in thread T1:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 zone_find lib/dns/rbtdb.c:4029:2
#3 dns_db_find lib/dns/db.c:500:11
#4 addifmissing lib/dns/zone.c:4481:11
#5 dns_keytable_forall lib/dns/keytable.c:786:4
#6 sync_keyzone lib/dns/zone.c:4586:2
#7 dns_zone_synckeyzone lib/dns/zone.c:4635:11
#8 mkey_refresh bin/named/server.c:15423:2
#9 named_server_mkeys bin/named/server.c:15727:4
#10 named_control_docommand bin/named/control.c:236:12
#11 control_command bin/named/controlconf.c:365:17
#12 dispatch lib/isc/task.c:1152:7
#13 run lib/isc/task.c:1344:2
Mutex M2 previously acquired by the same thread here:
#0 pthread_rwlock_rdlock <null>
#1 isc_rwlock_lock lib/isc/rwlock.c:48:3
#2 dns_keytable_forall lib/dns/keytable.c:770:2
#3 sync_keyzone lib/dns/zone.c:4586:2
#4 dns_zone_synckeyzone lib/dns/zone.c:4635:11
#5 mkey_refresh bin/named/server.c:15423:2
#6 named_server_mkeys bin/named/server.c:15727:4
#7 named_control_docommand bin/named/control.c:236:12
#8 control_command bin/named/controlconf.c:365:17
#9 dispatch lib/isc/task.c:1152:7
#10 run lib/isc/task.c:1344:2
Thread T1 (running) created by main thread at:
#0 pthread_create <null>
#1 isc_thread_create lib/isc/pthreads/thread.c:73:8
#2 isc_taskmgr_create lib/isc/task.c:1434:3
#3 create_managers bin/named/main.c:915:11
#4 setup bin/named/main.c:1223:11
#5 main bin/named/main.c:1523:2
SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) in pthread_rwlock_rdlock
(cherry picked from commit 9e5f83c499
2020-09-17 18:26:22 +10:00
Tinderbox User and Michał Kępień
e7b0d5de15
prep 9.11.23
2020-09-16 23:00:15 +02:00
Mark Andrews
f30ec018f9
The runtime of rbt_test can exceed 5 minutes when running under TSAN testing.
...
Limit rbt_insert_and_remove subtest runtime to ~180 seconds which gives a total
runtime of ~270 seconds.
2020-09-15 02:15:27 +00:00
Evan Hunt
400171aee8
update all copyright headers to eliminate the typo
2020-09-14 17:00:40 -07:00
Mark Andrews
87e8497b6a
Silence REVERSE_INULL warning (CID 306652)
...
(cherry picked from commit 584dbffab1
2020-09-10 07:57:20 +10:00
Mark Andrews
1d0c1e62b4
Silence Coverity missing lock warning.
...
Lock access to 'done' during initialisation in manytasks().
2020-09-09 21:02:51 +00:00
Mark Andrews
e075ae082c
LOCK should be UNLOCK
...
*** CID 306731: Program hangs (LOCK)
/lib/dns/resolver.c: 8943 in spillattimer_countdown()
8937 result = isc_timer_reset(res->spillattimer,
8938 isc_timertype_inactive, NULL,
8939 NULL, true);
8940 RUNTIME_CHECK(result == ISC_R_SUCCESS);
8941 }
8942 count = res->spillat;
CID 306731: Program hangs (LOCK)
"pthread_mutex_lock" locks "res->spill_lock" while it is locked.
8943 LOCK(&res->spill_lock);
8944 UNLOCK(&res->lock);
8945 if (logit)
8946 isc_log_write(dns_lctx, DNS_LOGCATEGORY_RESOLVER,
8947 DNS_LOGMODULE_RESOLVER, ISC_LOG_NOTICE,
8948 "clients-per-query decreased to %u", count);
2020-09-09 15:28:57 +10:00
Mark Andrews
90185b225f
Update 'init_count' atomically to silence tsan errors.
2020-09-08 17:41:33 +10:00
Mark Andrews
a1dcb73f67
The node lock was released too early.
...
NEGATIVE needs to be call with the node lock held.
WARNING: ThreadSanitizer: data race
Write of size 2 at 0x000000000001 by thread T1 (mutexes: write M1):
#0 mark_stale_header lib/dns/rbtdb.c:1802:21
#1 add32 lib/dns/rbtdb.c:6559:5
#2 addrdataset lib/dns/rbtdb.c:6975:12
#3 dns_db_addrdataset lib/dns/db.c:783:10
#4 cache_name lib/dns/resolver.c:5829:13
#5 cache_message lib/dns/resolver.c:5926:14
#6 resquery_response lib/dns/resolver.c:8618:12
#7 dispatch lib/isc/task.c:1157:7
#8 run lib/isc/task.c:1331:2
Previous read of size 2 at 0x000000000001 by thread T2:
#0 cache_findrdataset lib/dns/rbtdb.c:5932:6
#1 dns_db_findrdataset lib/dns/db.c:739:10
#2 query_addadditional2 bin/named/query.c:2196:11
#3 additionaldata_ns lib/dns/./rdata/generic/ns_2.c:198:10
#4 dns_rdata_additionaldata lib/dns/rdata.c:1246:2
#5 dns_rdataset_additionaldata lib/dns/rdataset.c:629:12
#6 query_addrdataset bin/named/query.c:2411:8
#7 query_addrrset bin/named/query.c:2802:2
#8 query_addbestns bin/named/query.c:3501:2
#9 query_find bin/named/query.c:9165:4
#10 query_resume bin/named/query.c:4164:12
#11 dispatch lib/isc/task.c:1157:7
#12 run lib/isc/task.c:1331:2
2020-09-08 17:41:33 +10:00
Mark Andrews
1f7a4f9ab2
Turn off TSAN for isc_log_wouldlog
2020-09-08 17:41:33 +10:00
Mark Andrews
8f88bc61e9
Lock zone to read DNS_ZONEFLG_DIALREFRESH
2020-09-08 17:41:33 +10:00
Mark Andrews
7db778854e
Use a reference counter for zt
...
WARNING: ThreadSanitizer: data race
Write of size 8 at 0x000000000001 by thread T1 (mutexes: write M1):
#0 memset <null>
#1 mem_put lib/isc/mem.c:819
#2 isc___mem_free lib/isc/mem.c:1662
#3 isc__mem_free lib/isc/mem.c:3078
#4 isc___mem_putanddetach lib/isc/mem.c:1221
#5 isc__mem_putanddetach lib/isc/mem.c:3033
#6 zt_destroy lib/dns/zt.c:214
#7 doneloading lib/dns/zt.c:591
#8 zone_asyncload lib/dns/zone.c:2243
#9 dispatch lib/isc/task.c:1157
#10 run lib/isc/task.c:1331
#11 <null> <null>
Previous atomic read of size 8 at 0x000000000001 by thread T2:
#0 __tsan_atomic64_load <null>
#1 isc_rwlock_unlock lib/isc/rwlock.c:612
#2 doneloading lib/dns/zt.c:585
#3 zone_asyncload lib/dns/zone.c:2243
#4 dispatch lib/isc/task.c:1157
#5 run lib/isc/task.c:1331
#6 <null> <null>
Location is heap block of size 273 at 0x000000000015 allocated by thread T3:
#0 malloc <null>
#1 internal_memalloc lib/isc/mem.c:887
#2 mem_get lib/isc/mem.c:792
#3 mem_allocateunlocked lib/isc/mem.c:1545
#4 isc___mem_allocate lib/isc/mem.c:1566
#5 isc__mem_allocate lib/isc/mem.c:3048
#6 isc___mem_get lib/isc/mem.c:1304
#7 isc__mem_get lib/isc/mem.c:3012
#8 dns_zt_create lib/dns/zt.c:85
#9 dns_view_create lib/dns/view.c:126
#10 create_view server.c:5312
#11 load_configuration server.c:8101
#12 loadconfig server.c:9428
#13 ns_server_reconfigcommand server.c:9763
#14 ns_control_docommand bin/named/control.c:243
#15 control_recvmessage bin/named/controlconf.c:465
#16 dispatch lib/isc/task.c:1157
#17 run lib/isc/task.c:1331
#18 <null> <null>
2020-09-08 17:41:33 +10:00
Mark Andrews
d6c727d669
ISC_QUEUE_POP is not tsan safe. Suppress warnings
...
ret->link.next is tested to see if it is NULL unlocked
to avoid obtaining taillock when it is not nexessary then
retested once the taillock is obtained.
2020-09-08 17:41:33 +10:00
Mark Andrews
72cbe648c4
Test if linked while holding the queue lock
...
WARNING: ThreadSanitizer: data race
Read of size 8 at 0x000000000001 by thread T1:
#0 client_shutdown bin/named/client.c:849:6
#1 dispatch lib/isc/task.c:1157:7
#2 run lib/isc/task.c:1331:2
Previous write of size 8 at 0x000000000001 by thread T2 (mutexes: write M1, write M2):
#0 client_shutdown bin/named/client.c:850:3
#1 dispatch lib/isc/task.c:1157:7
#2 run lib/isc/task.c:1331:2
2020-09-08 09:25:43 +10:00
Mark Andrews
7da21a6ff3
Lock zone before reading DNS_ZONEFLG_REFRESH
2020-09-08 09:25:43 +10:00
Mark Andrews
9e8e886fd6
Don't grab all the adb locks at once
...
FATAL: ThreadSanitizer CHECK failed: /build/llvm-toolchain-10-10.0.1~++20200708124224+ef32c611aa2/compiler-rt/lib/sanitizer_common/sanitizer_deadlock_detector.h:67 "((n_all_locks_)) < (((sizeof(all_locks_with_contexts_)/sizeof((all_locks_with_contexts_)[0]))))" (0x40, 0x40)
#0 __tsan::TsanCheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) <null> (named+0x4ad775)
#1 __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) <null> (named+0x4c429f)
#2 __sanitizer::DeadlockDetectorTLS<__sanitizer::TwoLevelBitVector<1ul, __sanitizer::BasicBitVector<unsigned long> > >::addLock(unsigned long, unsigned long, unsigned int) <null> (named+0x4b8a3c)
#3 __sanitizer::DD::MutexAfterLock(__sanitizer::DDCallback*, __sanitizer::DDMutex*, bool, bool) <null> (named+0x4b7fae)
#4 __tsan::MutexPostLock(__tsan::ThreadState*, unsigned long, unsigned long, unsigned int, int) <null> (named+0x4abc58)
#5 pthread_mutex_lock <null> (named+0x4643a8)
#6 dump_adb /builds/isc-projects/bind9/lib/dns/adb.c:3461:3 (libdns.so.1110+0x48be6)
#7 dns_adb_dump /builds/isc-projects/bind9/lib/dns/adb.c:3434:2 (libdns.so.1110+0x48738)
#8 dumpdone /builds/isc-projects/bind9/bin/named/./server.c:10149:4 (named+0x512022)
#9 dump_quantum /builds/isc-projects/bind9/lib/dns/masterdump.c:1424:2 (libdns.so.1110+0x9f253)
#10 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x50845)
#11 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d799)
#12 __tsan_thread_start_func <null> (named+0x446e3c)
#13 start_thread /build/glibc-vjB4T1/glibc-2.28/nptl/pthread_create.c:486:8 (libpthread.so.0+0x7fa2)
#14 clone /build/glibc-vjB4T1/glibc-2.28/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95 (libc.so.6+0xf94ce)
2020-09-08 09:25:43 +10:00
Mark Andrews
56ab2fc457
Lock access to state_mctx
...
WARNING: ThreadSanitizer: data race
Read of size 4 at 0x000000000001 by thread T1 (mutexes: write M1):
#0 isc___mem_allocate lib/isc/mem.c:1565:2
#1 isc__mem_allocate lib/isc/mem.c:3048:11
#2 isc___mem_get lib/isc/mem.c:1304:11
#3 isc__mem_get lib/isc/mem.c:3012:11
#4 set_state lib/dns/geoip2.c:174:29
#5 get_entry_for lib/dns/geoip2.c:236:11
#6 dns_geoip_match lib/dns/geoip2.c:383:10
#7 dns_aclelement_match2 lib/dns/acl.c:493:11
#8 dns_acl_match2 lib/dns/acl.c:298:7
#9 allowed bin/named/client.c:1866:11
#10 ns_client_isself bin/named/client.c:1931:7
#11 notify_isself lib/dns/zone.c:11025:11
#12 notify_send lib/dns/zone.c:11344:7
#13 notify_find_address lib/dns/zone.c:11148:2
#14 zone_notify lib/dns/zone.c:11586:3
#15 zone_maintenance lib/dns/zone.c:10261:4
#16 zone_timer lib/dns/zone.c:13106:2
#17 dispatch lib/isc/task.c:1157:7
#18 run lib/isc/task.c:1331:2
Previous write of size 8 at 0x000000000001 by thread T2 (mutexes: write M2, write M3):
#0 malloc <null>
#1 internal_memalloc lib/isc/mem.c:887:8
#2 isc_mem_createx2 lib/isc/mem.c:956:8
#3 isc_mem_create lib/isc/mem.c:2793:11
#4 state_key_init lib/dns/geoip2.c:119:14
#5 get_entry_for lib/dns/geoip2.c:214:11
#6 dns_geoip_match lib/dns/geoip2.c:383:10
#7 dns_aclelement_match2 lib/dns/acl.c:493:11
#8 dns_acl_match2 lib/dns/acl.c:298:7
#9 allowed bin/named/client.c:1866:11
#10 ns_client_isself bin/named/client.c:1931:7
#11 notify_isself lib/dns/zone.c:11025:11
#12 notify_send lib/dns/zone.c:11344:7
#13 notify_find_address lib/dns/zone.c:11148:2
#14 zone_notify lib/dns/zone.c:11586:3
#15 zone_maintenance lib/dns/zone.c:10261:4
#16 zone_timer lib/dns/zone.c:13106:2
#17 dispatch lib/isc/task.c:1157:7
#18 run lib/isc/task.c:1331:2
2020-09-08 09:25:43 +10:00
Mark Andrews
063a881a6c
Use atomics to update counters.
...
WARNING: ThreadSanitizer: data race
Write of size 4 at 0x000000000001 by thread T1:
#0 dns_acache_countquerymiss lib/dns/acache.c:1184:22
#1 rdataset_getadditional lib/dns/rbtdb.c:9875:3
#2 dns_rdataset_getadditional lib/dns/rdataset.c:711:11
#3 query_addadditional2 bin/named/query.c:1967:11
#4 additionaldata_ns lib/dns/./rdata/generic/ns_2.c:198:10
#5 dns_rdata_additionaldata lib/dns/rdata.c:1246:2
#6 dns_rdataset_additionaldata lib/dns/rdataset.c:629:12
#7 query_addrdataset bin/named/query.c:2411:8
#8 query_addrrset bin/named/query.c:2802:2
#9 query_find bin/named/query.c:9135:4
#10 query_resume bin/named/query.c:4164:12
#11 dispatch lib/isc/task.c:1157:7
#12 run lib/isc/task.c:1331:2
Previous write of size 4 at 0x000000000001 by thread T2:
#0 dns_acache_countquerymiss lib/dns/acache.c:1184:22
#1 rdataset_getadditional lib/dns/rbtdb.c:9875:3
#2 dns_rdataset_getadditional lib/dns/rdataset.c:711:11
#3 query_addadditional2 bin/named/query.c:1967:11
#4 additionaldata_ns lib/dns/./rdata/generic/ns_2.c:198:10
#5 dns_rdata_additionaldata lib/dns/rdata.c:1246:2
#6 dns_rdataset_additionaldata lib/dns/rdataset.c:629:12
#7 query_addrdataset bin/named/query.c:2411:8
#8 query_addrrset bin/named/query.c:2802:2
#9 query_find bin/named/query.c:9135:4
#10 query_resume bin/named/query.c:4164:12
#11 dispatch lib/isc/task.c:1157:7
#12 run lib/isc/task.c:1331:2
2020-09-08 09:25:43 +10:00
Mark Andrews
3d6dda54e7
state_key_initialized had a data race.
...
WARNING: ThreadSanitizer: data race (pid=25060)
Read of size 1 at 0x7fb4b1448098 by thread T12 (mutexes: write M802339438951732248):
#0 state_key_init /builds/isc-projects/bind9/lib/dns/geoip2.c:113:7 (libdns.so.1110+0x24204a)
#1 get_entry_for /builds/isc-projects/bind9/lib/dns/geoip2.c:214:11 (libdns.so.1110+0x241dc7)
#2 dns_geoip_match /builds/isc-projects/bind9/lib/dns/geoip2.c:383:10 (libdns.so.1110+0x2418a5)
#3 dns_aclelement_match2 /builds/isc-projects/bind9/lib/dns/acl.c:493:11 (libdns.so.1110+0x3f46b)
#4 dns_acl_match2 /builds/isc-projects/bind9/lib/dns/acl.c:298:7 (libdns.so.1110+0x3f236)
#5 allowed /builds/isc-projects/bind9/bin/named/client.c:1866:11 (named+0x4da797)
#6 ns_client_isself /builds/isc-projects/bind9/bin/named/client.c:1931:7 (named+0x4da688)
#7 notify_isself /builds/isc-projects/bind9/lib/dns/zone.c:11025:11 (libdns.so.1110+0x20f74a)
#8 notify_send /builds/isc-projects/bind9/lib/dns/zone.c:11344:7 (libdns.so.1110+0x20f397)
#9 notify_find_address /builds/isc-projects/bind9/lib/dns/zone.c:11148:2 (libdns.so.1110+0x20d9db)
#10 zone_notify /builds/isc-projects/bind9/lib/dns/zone.c:11586:3 (libdns.so.1110+0x204d98)
#11 zone_maintenance /builds/isc-projects/bind9/lib/dns/zone.c:10261:4 (libdns.so.1110+0x203d4d)
#12 zone_timer /builds/isc-projects/bind9/lib/dns/zone.c:13106:2 (libdns.so.1110+0x1e847a)
#13 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x50845)
#14 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d799)
Previous write of size 1 at 0x7fb4b1448098 by thread T11 (mutexes: write M818946462577691672, write M52428):
#0 state_key_init /builds/isc-projects/bind9/lib/dns/geoip2.c:129:27 (libdns.so.1110+0x242118)
#1 get_entry_for /builds/isc-projects/bind9/lib/dns/geoip2.c:214:11 (libdns.so.1110+0x241dc7)
#2 dns_geoip_match /builds/isc-projects/bind9/lib/dns/geoip2.c:383:10 (libdns.so.1110+0x2418a5)
#3 dns_aclelement_match2 /builds/isc-projects/bind9/lib/dns/acl.c:493:11 (libdns.so.1110+0x3f46b)
#4 dns_acl_match2 /builds/isc-projects/bind9/lib/dns/acl.c:298:7 (libdns.so.1110+0x3f236)
#5 allowed /builds/isc-projects/bind9/bin/named/client.c:1866:11 (named+0x4da797)
#6 ns_client_isself /builds/isc-projects/bind9/bin/named/client.c:1931:7 (named+0x4da688)
#7 notify_isself /builds/isc-projects/bind9/lib/dns/zone.c:11025:11 (libdns.so.1110+0x20f74a)
#8 notify_send /builds/isc-projects/bind9/lib/dns/zone.c:11344:7 (libdns.so.1110+0x20f397)
#9 notify_find_address /builds/isc-projects/bind9/lib/dns/zone.c:11148:2 (libdns.so.1110+0x20d9db)
#10 zone_notify /builds/isc-projects/bind9/lib/dns/zone.c:11586:3 (libdns.so.1110+0x204d98)
#11 zone_maintenance /builds/isc-projects/bind9/lib/dns/zone.c:10261:4 (libdns.so.1110+0x203d4d)
#12 zone_timer /builds/isc-projects/bind9/lib/dns/zone.c:13106:2 (libdns.so.1110+0x1e847a)
#13 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x50845)
#14 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d799)
Location is global 'state_key_initialized' of size 1 at 0x7fb4b1448098 (libdns.so.1110+0x0000002c6098)
2020-09-08 09:25:43 +10:00
Mark Andrews
fb8a3c9ab2
Remove optimisation on obtaining a headlock as it triggers a tsan.
...
WARNING: ThreadSanitizer: data race (pid=15898)
Write of size 8 at 0x7b6400011818 by thread T9 (mutexes: write M1597):
#0 get_client /builds/isc-projects/bind9/bin/named/client.c:3876:3 (named+0x4db171)
#1 ns_client_replace /builds/isc-projects/bind9/bin/named/client.c:3710:12 (named+0x4d737b)
#2 query_recurse /builds/isc-projects/bind9/bin/named/query.c:4325:13 (named+0x4ff469)
#3 query_find /builds/isc-projects/bind9/bin/named/query.c (named+0x4fb949)
#4 ns_query_start /builds/isc-projects/bind9/bin/named/query.c:9675:8 (named+0x4f37cb)
#5 client_request /builds/isc-projects/bind9/bin/named/client.c:3112:3 (named+0x4de9ef)
#6 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x50845)
#7 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d799)
Previous read of size 8 at 0x7b6400011818 by thread T2:
#0 exit_check /builds/isc-projects/bind9/bin/named/client.c:698:5 (named+0x4d5d22)
#1 ns_client_detach /builds/isc-projects/bind9/bin/named/client.c:3687:8 (named+0x4d7762)
#2 query_find /builds/isc-projects/bind9/bin/named/query.c (named+0x4f9021)
#3 query_resume /builds/isc-projects/bind9/bin/named/query.c:4164:12 (named+0x509b68)
#4 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x50845)
#5 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d799)
2020-09-08 09:25:43 +10:00
Mark Andrews
15ae4585d2
counter->used was read without the lock being held.
...
WARNING: ThreadSanitizer: data race (pid=11785)
Write of size 4 at 0x7b180001ba10 by thread T12 (mutexes: write M835834548863482336):
#0 isc_counter_increment /builds/isc-projects/bind9/lib/isc/counter.c:70:15 (libisc.so.1107+0x1dcb6)
#1 fctx_try /builds/isc-projects/bind9/lib/dns/resolver.c:3851:11 (libdns.so.1110+0x17e312)
#2 resume_dslookup /builds/isc-projects/bind9/lib/dns/resolver.c:7505:3 (libdns.so.1110+0x18ccf0)
#3 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507f5)
#4 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d749)
Previous read of size 4 at 0x7b180001ba10 by thread T7:
#0 isc_counter_used /builds/isc-projects/bind9/lib/isc/counter.c:82:19 (libisc.so.1107+0x1dd5f)
#1 fctx_try /builds/isc-projects/bind9/lib/dns/resolver.c:3798:6 (libdns.so.1110+0x17e0d1)
#2 fctx_start /builds/isc-projects/bind9/lib/dns/resolver.c:4219:4 (libdns.so.1110+0x178833)
#3 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507f5)
#4 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d749)
2020-09-08 09:25:43 +10:00
Mark Andrews
cf4a9d9ab8
It appears that you can't change what you are polling for while connecting.
...
WARNING: ThreadSanitizer: data race (pid=6465)
Read of size 8 at 0x7ba000002040 by thread T14:
#0 epoll_ctl <null> (named+0x44ccd2)
#1 watch_fd /builds/isc-projects/bind9/lib/isc/unix/socket.c:975:8 (libisc.so.1107+0x6bd90)
#2 wakeup_socket /builds/isc-projects/bind9/lib/isc/unix/socket.c:1164:11 (libisc.so.1107+0x7057d)
#3 process_ctlfd /builds/isc-projects/bind9/lib/isc/unix/socket.c:4258:3 (libisc.so.1107+0x6c308)
#4 process_fds /builds/isc-projects/bind9/lib/isc/unix/socket.c:4162:10 (libisc.so.1107+0x6bf74)
#5 watcher /builds/isc-projects/bind9/lib/isc/unix/socket.c:4401:10 (libisc.so.1107+0x64348)
Previous write of size 8 at 0x7ba000002040 by thread T9 (mutexes: write M81481868977181736):
#0 connect <null> (named+0x44b7e0)
#1 isc__socket_connect /builds/isc-projects/bind9/lib/isc/unix/socket.c:5902:7 (libisc.so.1107+0x67a79)
#2 isc_socket_connect /builds/isc-projects/bind9/lib/isc/unix/./../socket_api.c:169:11 (libisc.so.1107+0x6aa4b)
#3 resquery_send /builds/isc-projects/bind9/lib/dns/resolver.c:2573:13 (libdns.so.1110+0x18570b)
#4 fctx_query /builds/isc-projects/bind9/lib/dns/resolver.c:1903:12 (libdns.so.1110+0x1815a3)
#5 fctx_try /builds/isc-projects/bind9/lib/dns/resolver.c:3863:11 (libdns.so.1110+0x17e3a9)
#6 fctx_start /builds/isc-projects/bind9/lib/dns/resolver.c:4219:4 (libdns.so.1110+0x178833)
#7 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507f5)
#8 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d749)
Location is file descriptor 516 created by thread T9 at:
#0 connect <null> (named+0x44b7e0)
#1 isc__socket_connect /builds/isc-projects/bind9/lib/isc/unix/socket.c:5902:7 (libisc.so.1107+0x67a79)
#2 isc_socket_connect /builds/isc-projects/bind9/lib/isc/unix/./../socket_api.c:169:11 (libisc.so.1107+0x6aa4b)
#3 resquery_send /builds/isc-projects/bind9/lib/dns/resolver.c:2573:13 (libdns.so.1110+0x18570b)
#4 fctx_query /builds/isc-projects/bind9/lib/dns/resolver.c:1903:12 (libdns.so.1110+0x1815a3)
#5 fctx_try /builds/isc-projects/bind9/lib/dns/resolver.c:3863:11 (libdns.so.1110+0x17e3a9)
#6 fctx_start /builds/isc-projects/bind9/lib/dns/resolver.c:4219:4 (libdns.so.1110+0x178833)
#7 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507f5)
#8 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d749)
2020-09-08 09:25:43 +10:00
Mark Andrews
80bf3f3852
Only test node->data if we care about whether data is present or not.
...
WARNING: ThreadSanitizer: data race (pid=28788)
Write of size 8 at 0x7b200002e060 by thread T1 (mutexes: write M2947):
#0 add32 /builds/isc-projects/bind9/lib/dns/rbtdb.c:6638:18 (libdns.so.1110+0xe7843)
#1 addrdataset /builds/isc-projects/bind9/lib/dns/rbtdb.c:6975:12 (libdns.so.1110+0xe4185)
#2 dns_db_addrdataset /builds/isc-projects/bind9/lib/dns/db.c:783:10 (libdns.so.1110+0x650ee)
#3 validated /builds/isc-projects/bind9/lib/dns/resolver.c:5140:11 (libdns.so.1110+0x1909f7)
#4 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507f5)
#5 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d749)
Previous read of size 8 at 0x7b200002e060 by thread T5 (mutexes: write M521146194917735760):
#0 dns_rbt_findnode /builds/isc-projects/bind9/lib/dns/rbt.c:1708:9 (libdns.so.1110+0xd910d)
#1 cache_find /builds/isc-projects/bind9/lib/dns/rbtdb.c:5098:11 (libdns.so.1110+0xe188e)
#2 dns_db_find /builds/isc-projects/bind9/lib/dns/db.c:554:11 (libdns.so.1110+0x642bb)
#3 dns_view_find2 /builds/isc-projects/bind9/lib/dns/view.c:1068:11 (libdns.so.1110+0x1cc2c4)
#4 dbfind_name /builds/isc-projects/bind9/lib/dns/adb.c:3714:11 (libdns.so.1110+0x46a4b)
#5 dns_adb_createfind2 /builds/isc-projects/bind9/lib/dns/adb.c:3133:12 (libdns.so.1110+0x45278)
#6 findname /builds/isc-projects/bind9/lib/dns/resolver.c:3166:11 (libdns.so.1110+0x1827f0)
#7 fctx_getaddresses /builds/isc-projects/bind9/lib/dns/resolver.c:3462:3 (libdns.so.1110+0x18032d)
#8 fctx_try /builds/isc-projects/bind9/lib/dns/resolver.c:3819:12 (libdns.so.1110+0x17e174)
#9 fctx_start /builds/isc-projects/bind9/lib/dns/resolver.c:4219:4 (libdns.so.1110+0x1787a3)
#10 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507f5)
#11 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d749)
2020-09-08 09:25:43 +10:00
Mark Andrews
674a219466
Address data race in dns_adbentry_overquota
...
Read of size 4 at 0x7b440003da70 by thread T8:
#0 dns_adbentry_overquota /builds/isc-projects/bind9/lib/dns/adb.c:4797:39 (libdns.so.1110+0x4da82)
#1 fctx_try /builds/isc-projects/bind9/lib/dns/resolver.c:3838:10 (libdns.so.1110+0x17e229)
#2 fctx_start /builds/isc-projects/bind9/lib/dns/resolver.c:4219:4 (libdns.so.1110+0x1787a3)
#3 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507f5)
#4 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d749)
Previous write of size 4 at 0x7b440003da70 by thread T12 (mutexes: write M549293692588722840):
#0 dns_adb_beginudpfetch /builds/isc-projects/bind9/lib/dns/adb.c:4811:21 (libdns.so.1110+0x4db82)
#1 fctx_query /builds/isc-projects/bind9/lib/dns/resolver.c:1901:3 (libdns.so.1110+0x1814ee)
#2 fctx_try /builds/isc-projects/bind9/lib/dns/resolver.c:3863:11 (libdns.so.1110+0x17e2fa)
#3 fctx_start /builds/isc-projects/bind9/lib/dns/resolver.c:4219:4 (libdns.so.1110+0x1787a3)
#4 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507f5)
#5 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d749)
2020-09-08 09:25:43 +10:00
Mark Andrews
121837aa75
increment header->count atomically
2020-09-08 09:25:43 +10:00
Mark Andrews
3a59243284
Address data race in dns_adb_detach.
...
Move INSIST inside of locked section.
2020-09-08 09:25:43 +10:00
Mark Andrews
505e338156
Address lock-order-inversion
...
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=12714)
Cycle in lock order graph: M100252 (0x7b7c00010a08) => M1171 (0x7b7400000dc8) => M100252
Mutex M1171 acquired here while holding mutex M100252 in thread T1:
#0 pthread_mutex_lock <null> (delv+0x4483a6)
#1 dns_resolver_createfetch3 /builds/isc-projects/bind9/lib/dns/resolver.c:9585:2 (libdns.so.1110+0x1769fd)
#2 dns_resolver_createfetch /builds/isc-projects/bind9/lib/dns/resolver.c:9504:10 (libdns.so.1110+0x174e17)
#3 create_fetch /builds/isc-projects/bind9/lib/dns/validator.c:1156:10 (libdns.so.1110+0x1c1e5f)
#4 validatezonekey /builds/isc-projects/bind9/lib/dns/validator.c:2124:13 (libdns.so.1110+0x1c3b6d)
#5 start_positive_validation /builds/isc-projects/bind9/lib/dns/validator.c:2301:10 (libdns.so.1110+0x1bfde9)
#6 validator_start /builds/isc-projects/bind9/lib/dns/validator.c:3647:12 (libdns.so.1110+0x1bef62)
#7 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507d5)
#8 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d729)
Mutex M100252 previously acquired by the same thread here:
#0 pthread_mutex_lock <null> (delv+0x4483a6)
#1 validator_start /builds/isc-projects/bind9/lib/dns/validator.c:3628:2 (libdns.so.1110+0x1bee31)
#2 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507d5)
#3 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d729)
Mutex M100252 acquired here while holding mutex M1171 in thread T1:
#0 pthread_mutex_lock <null> (delv+0x4483a6)
#1 dns_validator_destroy /builds/isc-projects/bind9/lib/dns/validator.c:3912:2 (libdns.so.1110+0x1bf788)
#2 validated /builds/isc-projects/bind9/lib/dns/resolver.c:4916:2 (libdns.so.1110+0x18fdfd)
#3 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507d5)
#4 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d729)
Mutex M1171 previously acquired by the same thread here:
#0 pthread_mutex_lock <null> (delv+0x4483a6)
#1 validated /builds/isc-projects/bind9/lib/dns/resolver.c:4907:2 (libdns.so.1110+0x18fc3d)
#2 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507d5)
#3 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d729)
Thread T1 'isc-worker0000' (tid=12729, running) created by main thread at:
#0 pthread_create <null> (delv+0x42afdb)
#1 isc_thread_create /builds/isc-projects/bind9/lib/isc/pthreads/thread.c:60:8 (libisc.so.1107+0x726d8)
#2 isc__taskmgr_create /builds/isc-projects/bind9/lib/isc/task.c:1468:7 (libisc.so.1107+0x4d635)
#3 isc_taskmgr_createinctx /builds/isc-projects/bind9/lib/isc/task.c:2091:11 (libisc.so.1107+0x4f4ac)
#4 main /builds/isc-projects/bind9/bin/delv/delv.c:1639:2 (delv+0x4b7f96)
SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) (/builds/isc-projects/bind9/bin/delv/.libs/delv+0x4483a6) in pthread_mutex_lock
2020-09-08 09:25:43 +10:00
Mark Andrews
3ed7746ec9
Address lock-order-inversion
...
Obtain references to view->redirect and view->managed_keys then
release view->lock so dns_zone_setviewcommit and dns_zone_setviewrevert
can obtain the view->lock while holding zone->lock.
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=9132)
Cycle in lock order graph: M987831431424375936 (0x000000000000) => M1012319771577875480 (0x000000000000) => M987831431424375936
Mutex M1012319771577875480 acquired here while holding mutex M987831431424375936 in thread T2:
#0 pthread_mutex_lock <null> (named+0x4642a6)
#1 dns_zone_setviewcommit /builds/isc-projects/bind9/lib/dns/zone.c:1571:2 (libdns.so.1110+0x1d74eb)
#2 dns_view_setviewcommit /builds/isc-projects/bind9/lib/dns/view.c:2388:3 (libdns.so.1110+0x1cfe29)
#3 load_configuration /builds/isc-projects/bind9/bin/named/./server.c:8188:3 (named+0x51eadd)
#4 loadconfig /builds/isc-projects/bind9/bin/named/./server.c:9438:11 (named+0x510c66)
#5 ns_server_reconfigcommand /builds/isc-projects/bind9/bin/named/./server.c:9773:2 (named+0x510b41)
#6 ns_control_docommand /builds/isc-projects/bind9/bin/named/control.c:243:12 (named+0x4e451a)
#7 control_recvmessage /builds/isc-projects/bind9/bin/named/controlconf.c:465:13 (named+0x4e9056)
#8 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507d5)
#9 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d729)
Mutex M987831431424375936 previously acquired by the same thread here:
#0 pthread_mutex_lock <null> (named+0x4642a6)
#1 dns_view_setviewcommit /builds/isc-projects/bind9/lib/dns/view.c:2382:2 (libdns.so.1110+0x1cfde7)
#2 load_configuration /builds/isc-projects/bind9/bin/named/./server.c:8188:3 (named+0x51eadd)
#3 loadconfig /builds/isc-projects/bind9/bin/named/./server.c:9438:11 (named+0x510c66)
#4 ns_server_reconfigcommand /builds/isc-projects/bind9/bin/named/./server.c:9773:2 (named+0x510b41)
#5 ns_control_docommand /builds/isc-projects/bind9/bin/named/control.c:243:12 (named+0x4e451a)
#6 control_recvmessage /builds/isc-projects/bind9/bin/named/controlconf.c:465:13 (named+0x4e9056)
#7 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507d5)
#8 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d729)
Mutex M987831431424375936 acquired here while holding mutex M1012319771577875480 in thread T7:
#0 pthread_mutex_lock <null> (named+0x4642a6)
#1 dns_view_findzonecut2 /builds/isc-projects/bind9/lib/dns/view.c:1300:2 (libdns.so.1110+0x1cc93a)
#2 dns_view_findzonecut /builds/isc-projects/bind9/lib/dns/view.c:1261:9 (libdns.so.1110+0x1cc864)
#3 fctx_create /builds/isc-projects/bind9/lib/dns/resolver.c:4459:13 (libdns.so.1110+0x1779d3)
#4 dns_resolver_createfetch3 /builds/isc-projects/bind9/lib/dns/resolver.c:9628:12 (libdns.so.1110+0x176cb6)
#5 dns_resolver_createfetch /builds/isc-projects/bind9/lib/dns/resolver.c:9504:10 (libdns.so.1110+0x174e17)
#6 zone_refreshkeys /builds/isc-projects/bind9/lib/dns/zone.c:10061:12 (libdns.so.1110+0x2055a5)
#7 zone_maintenance /builds/isc-projects/bind9/lib/dns/zone.c:10274:5 (libdns.so.1110+0x203a78)
#8 zone_timer /builds/isc-projects/bind9/lib/dns/zone.c:13106:2 (libdns.so.1110+0x1e815a)
#9 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507d5)
#10 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d729)
Mutex M1012319771577875480 previously acquired by the same thread here:
#0 pthread_mutex_lock <null> (named+0x4642a6)
#1 zone_refreshkeys /builds/isc-projects/bind9/lib/dns/zone.c:9951:2 (libdns.so.1110+0x204dc3)
#2 zone_maintenance /builds/isc-projects/bind9/lib/dns/zone.c:10274:5 (libdns.so.1110+0x203a78)
#3 zone_timer /builds/isc-projects/bind9/lib/dns/zone.c:13106:2 (libdns.so.1110+0x1e815a)
#4 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x507d5)
#5 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d729)
Thread T2 'isc-worker0001' (tid=9163, running) created by main thread at:
#0 pthread_create <null> (named+0x446edb)
#1 isc_thread_create /builds/isc-projects/bind9/lib/isc/pthreads/thread.c:60:8 (libisc.so.1107+0x726d8)
#2 isc__taskmgr_create /builds/isc-projects/bind9/lib/isc/task.c:1468:7 (libisc.so.1107+0x4d635)
#3 isc_taskmgr_create /builds/isc-projects/bind9/lib/isc/task.c:2109:11 (libisc.so.1107+0x4f587)
#4 create_managers /builds/isc-projects/bind9/bin/named/./main.c:886:11 (named+0x4f1a97)
#5 setup /builds/isc-projects/bind9/bin/named/./main.c:1305:11 (named+0x4f05ee)
#6 main /builds/isc-projects/bind9/bin/named/./main.c:1556:2 (named+0x4ef12d)
Thread T7 'isc-worker0006' (tid=9168, running) created by main thread at:
#0 pthread_create <null> (named+0x446edb)
#1 isc_thread_create /builds/isc-projects/bind9/lib/isc/pthreads/thread.c:60:8 (libisc.so.1107+0x726d8)
#2 isc__taskmgr_create /builds/isc-projects/bind9/lib/isc/task.c:1468:7 (libisc.so.1107+0x4d635)
#3 isc_taskmgr_create /builds/isc-projects/bind9/lib/isc/task.c:2109:11 (libisc.so.1107+0x4f587)
#4 create_managers /builds/isc-projects/bind9/bin/named/./main.c:886:11 (named+0x4f1a97)
#5 setup /builds/isc-projects/bind9/bin/named/./main.c:1305:11 (named+0x4f05ee)
#6 main /builds/isc-projects/bind9/bin/named/./main.c:1556:2 (named+0x4ef12d)
SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) (/builds/isc-projects/bind9/bin/named/.libs/named+0x4642a6) in pthread_mutex_lock
2020-09-08 09:25:43 +10:00
Mark Andrews
d776a73362
Take complete ownership of *validatorp before calling destroy.
2020-09-08 09:25:43 +10:00
Mark Andrews
05cb799274
Take complete ownership or *keyp before calling tsigkey_free.
2020-09-08 09:25:43 +10:00
Mark Andrews
8b80f40993
Take complete ownership of *aclp before calling destroy.
2020-09-08 09:25:43 +10:00
Mark Andrews
bf0bac4068
isc_refcount_current should be memory_order_acquire
2020-09-08 09:25:43 +10:00
Mark Andrews
fffcfe8acc
To read locknum we need to obtain the lock identified by locknum
...
this is a catch 22. make dns_rbtnode_t:locknum independent
2020-09-08 09:25:43 +10:00
Mark Andrews
deb3bf845b
Address tsan error in view:destroy()
2020-09-08 09:25:43 +10:00
Mark Andrews
e17a750221
move resolver->spillat* under renamed spill_lock
2020-09-08 09:25:43 +10:00
Mark Andrews
52a715b122
isc_mutex_init_errcheck prototype should not be under ISC_MUTEX_PROFILE
2020-09-08 09:25:43 +10:00
Mark Andrews
054dc48a1f
Only read dns_master_indent and dns_master_indentstr in named
...
The old code was not thread safe w.r.t. to the use of these variable.
We now only set them at the start of execution and copy them to
the message structure so they can be safely updated. This is the
minimal change to make them thread safe.
2020-09-08 09:25:43 +10:00
Mark Andrews
1f7ccaba1b
Prevent loads_pending going to zero while kicking the loads.
2020-09-08 09:25:43 +10:00
Mark Andrews
16e7e27328
Address data races between socket bitfields
...
* address data race between sock->pending_recv and sock->connected
* address data race between sock->bound and sock->pending_recv
==================
WARNING: ThreadSanitizer: data race (pid=1985)
Read of size 2 at 0x7b54000c07c0 by thread T6:
#0 isc__socket_sendto /builds/isc-projects/bind9/lib/isc/unix/socket.c:5291:2 (libisc.so.1107+0x65a00)
#1 isc__socket_send /builds/isc-projects/bind9/lib/isc/unix/socket.c:5270:10 (libisc.so.1107+0x65944)
#2 isc_socket_send /builds/isc-projects/bind9/lib/isc/unix/./../socket_api.c:329:10 (libisc.so.1107+0x6b3c9)
#3 sendstream /builds/isc-projects/bind9/bin/named/xfrout.c:1548:3 (named+0x555038)
#4 ns_xfr_start /builds/isc-projects/bind9/bin/named/xfrout.c:1132:2 (named+0x553147)
#5 ns_query_start /builds/isc-projects/bind9/bin/named/query.c:9572:4 (named+0x4f3329)
#6 client_request /builds/isc-projects/bind9/bin/named/client.c:3115:3 (named+0x4de6af)
#7 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x50845)
#8 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d799)
Previous write of size 2 at 0x7b54000c07c0 by thread T14 (mutexes: write M57, write M855819529908651432):
#0 dispatch_recv /builds/isc-projects/bind9/lib/isc/unix/socket.c:3353:21 (libisc.so.1107+0x6c601)
#1 process_fd /builds/isc-projects/bind9/lib/isc/unix/socket.c:4048:5 (libisc.so.1107+0x6c1be)
#2 process_fds /builds/isc-projects/bind9/lib/isc/unix/socket.c:4161:3 (libisc.so.1107+0x6bfc0)
#3 watcher /builds/isc-projects/bind9/lib/isc/unix/socket.c:4407:10 (libisc.so.1107+0x64398)
2020-09-08 09:25:43 +10:00
Mark Andrews
bcfbc17384
Lock access to ctx->blocked as it is updated by multiple threads
2020-09-08 09:25:43 +10:00
Mark Andrews
ca0c1e5b4b
Update 'init_count' atomically to silence tsan errors.
2020-09-02 09:35:43 +00:00
Mark Andrews
d8fa989c29
Lock access to flags in dns__zone_loadpending
...
==================
WARNING: ThreadSanitizer: data race (pid=1938)
Read of size 4 at 0x7b7800000dd4 by main thread:
#0 dns__zone_loadpending /builds/isc-projects/bind9/lib/dns/zone.c:2308:10 (libdns.so.1110+0x1da342)
#1 asyncload_zone /builds/isc-projects/bind9/lib/dns/tests/zt_test.c:204:9 (zt_test+0x4b8303)
#2 <null> <null> (libcmocka.so.0+0x50d8)
#3 __libc_start_main /build/glibc-vjB4T1/glibc-2.28/csu/../csu/libc-start.c:308:16 (libc.so.6+0x2409a)
Previous write of size 4 at 0x7b7800000dd4 by thread T16 (mutexes: write M2181):
#0 zone_asyncload /builds/isc-projects/bind9/lib/dns/zone.c:2237:3 (libdns.so.1110+0x1da08d)
#1 dispatch /builds/isc-projects/bind9/lib/isc/task.c:1157:7 (libisc.so.1107+0x50845)
#2 run /builds/isc-projects/bind9/lib/isc/task.c:1331:2 (libisc.so.1107+0x4d799)
2020-09-02 09:35:43 +00:00
Ondřej Surý
Mark Andrews
Tinderbox UserandMichał Kępień
Evan Hunt