Commit Graph
3686 Commits
Author SHA1 Message Date
Evan Hunt 5234a8e00a count statistics in netmgr TCP code 2020-01-13 14:09:42 -08:00
Evan Hunt 90a1dabe74 count statistics in netmgr UDP code
- also restored a test in the statistics test which was changed when
  the netmgr was introduced because active sockets were not being
  counted.
2020-01-13 14:09:37 -08:00
Evan Hunt 80a5c9f5c8 associate socket stats counters with netmgr socket objects
- the socket stat counters have been moved from socket.h to stats.h.
- isc_nm_t now attaches to the same stats counter group as
  isc_socketmgr_t, so that both managers can increment the same
  set of statistics
- isc__nmsocket_init() now takes an interface as a paramter so that
  the address family can be determined when initializing the socket.
- based on the address family and socket type, a group of statistics
  counters will be associated with the socket - for example, UDP4Active
  with IPv4 UDP sockets and TCP6Active with IPv6 TCP sockets.  note
  that no counters are currently associated with TCPDNS sockets; those
  stats will be handled by the underlying TCP socket.
- the counters are not actually used by netmgr sockets yet; counter
  increment and decrement calls will be added in a later commit.
2020-01-13 14:05:02 -08:00
Witold Kręcicki 20c077afc5 Disable pktinfo for ipv6 on all unices
If pktinfo were supported then we could listen on :: for ipv6 and get
the information about the destination address from pktinfo structure passed
in recvmsg but this method is not portable and libuv doesn't support it - so
we need to listen on all interfaces.
We should verify that this doesn't impact performance (we already do it for
ipv4) and either remove all the ipv6pktinfo detection code or think of fixing
libuv.
2020-01-13 22:00:20 +01:00
Evan Hunt e38004457c netmgr fixes:
- use UV_{TC,UD}P_IPV6ONLY for IPv6 sockets, keeping the pre-netmgr
   behaviour.
 - add a new listening_error bool flag which is set if the child
   listener fails to start listening. This fixes a bug where named would
   hang if, e.g.,  we failed to bind to a TCP socket.
2020-01-13 10:54:17 -08:00
Witold KręcickiandEvan Hunt 67c1ca9a79 Use isc_uv_export() to pass bound TCP listening socket to child listeners.
For multithreaded TCP listening we need to pass a bound socket to all
listening threads. Instead of using uv_pipe handle passing method which
is quite complex (lots of callbacks, each of them with its own error
handling) we now use isc_uv_export() to export the socket, pass it as a
member of the isc__netievent_tcpchildlisten_t structure, and then
isc_uv_import() it in the child thread, simplifying the process
significantly.
2020-01-13 10:53:44 -08:00
Witold KręcickiandEvan Hunt c6c0a9fdba Add isc_uv_export()/isc_uv_import() functions to libuv compatibility layer.
These functions can be used to pass a uv handle between threads in a
safe manner. The other option is to use uv_pipe and pass the uv_handle
via IPC, which is way more complex.  uv_export() and uv_import() functions
existed in libuv at some point but were removed later. This code is
based on the original removed code.

The Windows version of the code uses two functions internal to libuv;
a patch for libuv is attached for exporting these functions.
2020-01-13 10:52:07 -08:00
Ondřej SurýandOndřej Surý afc4867e99 Remove use of PTHREAD_MUTEX_INITIALIZER in tests
Remove the pthread specific static initializer in favor of dynamic
initialization.
2020-01-13 09:09:03 +01:00
Ondřej SurýandOndřej Surý 4f7d1298a8 Use isc_threadresult_t instead of pthread specific void * return type
The ISC thread API already defines isc_threadresult_t type,
but we are using a pthread specific return type (void *).
2020-01-13 09:08:48 +01:00
Michal Nowak 640dd566e9 Add out-of-tree build to the CI
Fixes #1546.
2020-01-09 10:16:06 +01:00
Ondřej Surý 17deac8b8e Remove unused isc_log_get() function 2020-01-08 11:53:04 +01:00
Ondřej Surý 91e1981988 Add missing locks to isc_logconfig_get and disable thread sanitizer for isc_log_wouldlog 2020-01-08 11:53:04 +01:00
Ondřej Surý 255134166c Add conditional ISC_NO_SANITIZE macro to disable TSAN for function 2020-01-08 11:53:04 +01:00
Ondřej Surý 5746172da3 Convert task flags to C11 atomics 2019-12-13 07:10:25 +01:00
Tinderbox User e088272172 prep 9.15.7 2019-12-12 23:59:39 +00:00
Diego FronzaandEvan Hunt ed9853e739 Fix tcp-highwater stats updating
After the network manager rewrite, tcp-higwater stats was only being
updated when a valid DNS query was received over tcp.

It turns out tcp-quota is updated right after a tcp connection is
accepted, before any data is read, so in the event that some client
connect but don't send a valid query, it wouldn't be taken into
account to update tcp-highwater stats, that is wrong.

This commit fix tcp-highwater to update its stats whenever a tcp connection
is established, independent of what happens after (timeout/invalid
request, etc).
2019-12-12 11:23:10 -08:00
Ondřej Surý d5b6db3b09 Additionally lock accessing the ISC_LISTs in free_socket() 2019-12-12 13:08:34 +01:00
Ondřej Surý d35739d516 Add missing isc_refcount_destroy and lock the socket ISC_LISTS in destroy() 2019-12-12 12:59:39 +01:00
Mark Andrews ad12c2f3b0 address lock order inversion 2019-12-12 17:43:03 +11:00
Ondřej Surý 1fa0deb4ea Add isc_refcount_destroy() call to nm_handle_free() 2019-12-10 13:43:18 +01:00
Ondřej Surý 71fe7d3c25 Add isc_refcount_destroy() call to nm_destroy() 2019-12-10 13:43:18 +01:00
Ondřej Surý 3248de7785 Correct the DbC check order in isc__nm_async_tcpchildstop() 2019-12-10 13:43:18 +01:00
Witold Kręcicki ccd44b69e5 Fix a potential lock-order-inversion in tcp listening code 2019-12-10 10:05:15 +01:00
Evan HuntandWitold Kręcicki 31b3980ef0 shorten some names
reduce line breaks and general unwieldiness by changing some
function, type, and parameter names.
2019-12-09 21:44:04 +01:00
Evan HuntandWitold Kręcicki 8c0792723d style nits 2019-12-09 21:44:04 +01:00
Witold Kręcicki a34ced776e Remove read callback before detaching from inner socket in tcpdns 2019-12-09 21:44:04 +01:00
Witold Kręcicki 86a847314a Fix a race in socket destruction - we need to remove handle from socket in async close callback or we might race between destruction in the callback and in the original nmhandle_unref 2019-12-09 21:44:04 +01:00
Witold Kręcicki b0779cc429 netmgr: Add more DbC checks for asynchronous calls. 2019-12-09 21:44:04 +01:00
Witold Kręcicki ef2dff5c7a pause and unpause netmgr in isc_nm_destroy to flush all events from worker queues 2019-12-09 21:44:04 +01:00
Evan HuntandWitold Kręcicki c7b86d1cac Style fixes 2019-12-09 21:44:03 +01:00
Witold Kręcicki 3e66b7ba1c Fix a race in tcpdns close with uv_close on timer
stop timers before closing

netmgr: tcpdns_close needs to be asynchronous, it manipulates sock->timer
2019-12-09 21:43:45 +01:00
Witold Kręcicki 23ab349bbd netmgr: fix a race in socket destruction, happening if we close the socket
externally and, at the same time, a timeout timer callback was called.
2019-12-09 21:43:45 +01:00
Witold Kręcicki 0bf74ac792 netmgr:
- make tcp listening IPC pipe name saner
 - put the pipe in /tmp on unices
 - add pid to the pipe name to avoid conflicts between processes
 - fsync directory in which the pipe resides to make sure that the
   child threads will see it and be able to open it
2019-12-09 21:43:45 +01:00
Evan Hunt b05194160b style, comments 2019-12-09 11:15:27 -08:00
Witold KręcickiandEvan Hunt 8c5aaacbef - Add separate priority event queue for events that must be processed
even when worker is paused (e.g. interface reconfiguration). This is
  needed to prevent deadlocks when reconfiguring interfaces - as network
  manager is paused then, but we still need to stop/start listening.

- Proper handling of TCP listen errors in netmgr - bind to the socket first,
  then return the error code.
2019-12-09 11:15:27 -08:00
Witold KręcickiandEvan Hunt 5a65ec0aff Add uv_handle_{get,set}_data functions that's absent in pre-1.19 libuv to make code clearer.
This might be removed when we stop supporting older libuv versions.
2019-12-09 11:15:27 -08:00
Witold KręcickiandEvan Hunt bc5aae1579 netmgr: make tcp listening multithreaded.
When listening for TCP connections we create a socket, bind it
and then pass it over IPC to all threads - which then listen on
in and accept connections. This sounds broken, but it's the
official way of dealing with multithreaded TCP listeners in libuv,
and works on all platforms supported by libuv.
2019-12-09 11:15:27 -08:00
Ondřej Surý b1a7ec7481 Remove isc_thread_key API in favor of ISC_THREAD_LOCAL variables
For BIND 9.16+, TLS aware compiler is required, and using
ISC_THREAD_LOCAL is preferred way of using Thread Local Storage.  The
isc_thread_key API is no longer used anywhere and hence was removed from
BIND 9.
2019-12-04 14:17:19 +01:00
Ondřej SurýandOndřej Surý 01731d4b1b Add and use ISC_THREAD_LOCAL macro
The new ISC_THREAD_LOCAL macro unifies usage of platform dependent
Thread Local Storage definition thread_local vs __thread vs
__declspec(thread) to a single macro.

The commit also unifies the required level of support for TLS as for
some parts of the code it was mandatory and for some parts of the code
it wasn't.
2019-12-03 16:27:24 +01:00
Mark Andrews 912ce87479 Make fctx->attributes atomic.
FCTX_ATTR_SHUTTINGDOWN needs to be set and tested while holding the node
lock but the rest of the attributes don't as they are task locked. Making
fctx->attributes atomic allows both behaviours without races.
2019-12-03 08:58:53 +11:00
Witold KręcickiandOndřej Surý f01d739968 Use clock_realtime_fast where available 2019-11-28 12:36:55 +01:00
Ondřej Surý c5b18d3dcb Use ATOMIC_VAR_INIT to initialize tid_v_base to 0 2019-11-28 12:06:10 +01:00
Ondřej Surý 9e8feec7d3 Improve the atomics usage in isc_counter API 2019-11-28 11:21:52 +01:00
Witold Kręcicki bf2cc19b04 - Access netmgr worker->finished under worker lock only
- Join network worker thread to silence tsan
2019-11-26 15:21:25 +01:00
Ondřej SurýandOndřej Surý b9f4ba19a6 Use integer fast type for mutexatomic shim type 2019-11-26 13:07:12 +01:00
Ondřej Surý 3ce6708be2 Add missing header guard to new header files 2019-11-25 09:10:29 +01:00
Evan Hunt 00333a5c97 netmgr: add shutdown function
- new function isc_nm_shutdown() shuts down all active TCP connections,
  but does not destroy the netmgr.
2019-11-22 16:46:32 -08:00
Witold KręcickiandEvan Hunt d6c5052f7e netmgr: actively close all sockets when shutting down server
without this change, named could sometimes lag for a while on shutdown
while it waited for open TCP connections to time out.
2019-11-22 16:46:32 -08:00
Witold KręcickiandEvan Hunt 37354ee225 netmgr: fix TCP backlog and client quota count
- add support for TCP backlog, using the value provided by config.
 - don't attach to TCP client quota for listening sockets, only
   connected sockets.
2019-11-22 16:46:32 -08:00
Evan Hunt c4ad0466d6 netmgr: log TCP connection errors 2019-11-22 16:46:32 -08:00