dig was not working with a server with offloaded TLS;
this turned out to be because it was sending the URI as
"https://<address>/dns-query", not "https://<name>/dns-query".
using userarg instead of servname fixes the problem. (incidentally
also corrected logged output to use the hostname when available.)
calling recv_done() with ISC_R_CANCELED prevents dig from cleaning
up fully from the query, because when that result code is used
it means some resources are already gone. changing the result
to ISC_R_UNEXPECTED prevents the hang.
- a bug that was already fixed in tcpdns.c and tlsdns.c had been
left unfixed in tcp.c: we needed to check whether the socket() call
succeeds before attempting any asynchronous connection code.
this corrects a problem that occurred when uv__socket() failed
with too many open files.
- also corrected the behavior of nm_connectcb(). previously, it was
forcing synchronous execution when force_async was false. what we
want is to force asynchronous execution if force_async is true. if
force_async is false, we want it to decide based on whether we're in
the network thread.
isc_nm_httprequest() is retained for use by doh_test, but has
been renamed isc__nm_http_request() and is no longer part of
the main API.
eventually doh_test should be changed to use isc_nm_read() and
isc_nm_send(), and this function should be removed.
we don't want to call isc__nm_http_cleanup_data() for TLS sockets
or isc__nm_tls_cleanup_data() for HTTP sockets, they can step on
each other and cause a crash or deadlock.
this isn't working yet, but it's a step in the right direction:
we need to call the connect callback from isc_nm_httpconnect()
asynchronously, after the connect has already returned. this
will enable dig to unlock the lookup before tcp_connected()
locks it, as it does when using tcpdns or tlsdns sockets.
this commit also removes the http_connect_t structure and stores
the connection data in a newly created http socket instead. (since
the socket structure was going to be needed to call isc__nm_connectcb()
anyway, it seemed like a good idea to make use of it to simplify
the connection code.)
in tls_call_connect_cb(), we want to update the result for
the TLS socket. however, in some error conditions on connect,
the handle we're passing is for the TCP socket instead. this
was causing the wrong socket's result code to be updated, and
the thread that was waiting for the result to be updated never
woke up.
(note: while the fix in this commit causes doh_test to stop
deadlocking, I'm not sure whether I've actually fixed the bug.
in other places, we create a new 'tlshandle' to pass into
tls_call_connect_cb(). perhaps we should be doing that in
all cases?)
incidentally, I also renamed tls_connect_cb() to tcp_connected(),
because the handle passed in is for the TCP socket, and I kept
being confused by the name having "tls" in it.
the goal here is to have the interface to http sockets be
the same as other types of netmgr socket, so that (for
example) dig can open an http connection and then just work
with no further code change, the same as it does with tcpdns
or tlsdns.
but the supporting code for https seems to be written with
the assumption that there will be a tight connection between
a request and a response, so the send and read functions have been
compressed into a single 'httprequest' function call. in an effort
to make the normal send and read semantics work correctly, we're
trying to set it up so that read sets us up to listen for a
response, send sets up the query we're sending, and whichever one
comes second triggers the transaction.
currently this is working in doh_test, but dig is failing due to a
deadlock; isc_http_connect() needs to return so launch_next_query()
can unlock the lookup before tcp_connected() runs.
instead of setting up ALPN in isc_tlsctx_createclient(), we now
have a function isc_tlsctx_enable_http2client_alpn() that can be
run from isc_nm_httpconnect(). isc_tlsctx_enable_http2server_alpn()
can similarly be run from isc_nm_httpconnect().
- style, cleanup, and removal of unnecessary code
- combine isc_nm_http_add_endpoint() and isc_nm_http_add_doh_endpoint()
into one function, renamed isc_http_endpoint()
- move isc_nm_http_connect_send_request() into doh_test.c as a helper
function; remove it from the public API
- renamed isc_http2 and isc_nm_http2 entities to just isc_http and
isc_nm_http, for consistency with other existing names
- shortened a number of long names
- the caller is now responsible for determining the peer address
in isc_nm_httpconnect(); this eliminates the need to parse the URI
and the dependency on an external resolver. the caller is also now
responsible for creating the SSL client context, for consistency with
isc_nm_tlsdnsconnect().
This commit contains fixes to unit tests to make them work well on
various platforms (in particular ones shipping old versions of
OpenSSL) and for different configurations.
It also updates the generated manpage to include DoH configuration
options.
This commit completes the support for DNS-over-HTTP(S) built on top of
nghttp2 and plugs it into the BIND. Support for both GET and POST
requests is present, as required by RFC8484.
Both encrypted (via TLS) and unencrypted HTTP/2 connections are
supported. The latter are mostly there for debugging/troubleshooting
purposes and for the means of encryption offloading to third-party
software (as might be desirable in some environments to simplify TLS
certificates management).
This commit includes work-in-progress implementation of
DNS-over-HTTP(S).
Server-side code remains mostly untested, and there is only support
for POST requests.
This commit resurrects the old TLS code from
8f73c70d23.
It also includes numerous stability fixes and support for
isc_nm_cancelread() for the TLS layer.
The code was resurrected to be used for DoH.
The BIND 9 libraries are considered to be internal only and hence the
API and ABI changes a lot. Keeping track of the API/ABI changes takes
time and it's a complicated matter as the safest way to make everything
stable would be to bump any library in the dependency chain as in theory
if libns links with libdns, and a binary links with both, and we bump
the libdns SOVERSION, but not the libns SOVERSION, the old libns might
be loaded by binary pulling old libdns together with new libdns loaded
by the binary. The situation gets even more complicated with loading
the plugins that have been compiled with few versions old BIND 9
libraries and then dynamically loaded into the named.
We are picking the safest option possible and usable for internal
libraries - instead of using -version-info that has only a weak link to
BIND 9 version number, we are using -release libtool option that will
embed the corresponding BIND 9 version number into the library name.
That means that instead of libisc.so.1701 (as an example) the library
will now be named libisc-9.17.10.so.
* Following the example set in 634bdfb16d, the tlsdns netmgr
module now uses libuv and SSL primitives directly, rather than
opening a TLS socket which opens a TCP socket, as the previous
model was difficult to debug. Closes#2335.
* Remove the netmgr tls layer (we will have to re-add it for DoH)
* Add isc_tls API to wrap the OpenSSL SSL_CTX object into libisc
library; move the OpenSSL initialization/deinitialization from dstapi
needed for OpenSSL 1.0.x to the isc_tls_{initialize,destroy}()
* Add couple of new shims needed for OpenSSL 1.0.x
* When LibreSSL is used, require at least version 2.7.0 that
has the best OpenSSL 1.1.x compatibility and auto init/deinit
* Enforce OpenSSL 1.1.x usage on Windows
* Added a TLSDNS unit test and implemented a simple TLSDNS echo
server and client.
On Windows, we were limiting the number of listening children to just 1,
but we were then iterating on mgr->nworkers. That lead to scheduling
more async_*listen() than actually allocated and out-of-bound read-write
operation on the heap.
When we were in nmthread, the isc__nm_async_<proto>connect() function
executes in the same thread as the isc__nm_<proto>connect() and on a
failure, it would block indefinitely because the failure branch was
setting sock->active to false before the condition around the wait had a
chance to skip the WAIT().
This also fixes the zero system test being stuck on FreeBSD 11, so we
re-enable the test in the commit.
On FreeBSD, the option to configure connection timeout is called
TCP_KEEPINIT, use it to configure the connection timeout there.
This also fixes the dangling socket problems in the unit test, so
re-enable them.
On platforms without load-balancing socket all the queries would be
handle by a single thread. Currently, the support for load-balanced
sockets is present in Linux with SO_REUSEPORT and FreeBSD 12 with
SO_REUSEPORT_LB.
This commit adds workaround for such platforms that:
1. setups single shared listening socket for all listening nmthreads for
UDP, TCP and TCPDNS netmgr transports
2. Calls uv_udp_bind/uv_tcp_bind on the underlying socket just once and
for rest of the nmthreads only copy the internal libuv flags (should
be just UV_HANDLE_BOUND and optionally UV_HANDLE_IPV6).
3. start reading on UDP socket or listening on TCP socket
The load distribution among the nmthreads is uneven, but it's still
better than utilizing just one thread for processing all the incoming
queries
On FreeBSD, the stack is destroyed more aggressively than on Linux and
that revealed a bug where we were allocating the 16-bit len for the
TCPDNS message on the stack and the buffer got garbled before the
uv_write() sendback was executed. Now, the len is part of the uvreq, so
we can safely pass it to the uv_write() as the req gets destroyed after
the sendcb is executed.
Evan Hunt
Artem Boldariev
Ondřej Surý
Witold KręcickiandOndřej Surý
Mark Andrews
Ondřej SurýandMichał Kępień
