This commit adds a set of functions which are aimed at simplification
of ngtcp2 integration in the BIND's code-base. As such, it complements
the ngtcp2 crypto API integration library.
This commit adds code that implements an interface between ngtcp2 and
a crypto library like OpenSSL/LibreSSL/QuicTLS.
The code mostly follows the structure of the original ngtcp2 crypto
library and shares significant amount of code with it, in particular
the portable its part and the part related to QuicTLS. That is done to
make it easier to update the code from time to time when there are
significant updates to either QUIC or TLS specifications. However,
there are many differences:
1. The code works not only on QuicTLS or LibreSSL, but also on OpenSSL
which, unlike the former two, does not (and, likely, will not) provide
native BoringSSL-style QUIC integration API;
2. The public interface of the library (API) has been significantly
simplified and reduced without sacrificing any functionality;
3. The code avoids using any "magic" constants as most of them can be
derived in runtime. That makes the code *much* easier to follow;
4. A lot of code was significantly simplified as we have some
middleware code that makes a lot of things much easier to do in a
secure way (like 'isc_buffer_t');
5. The code is better hardened and contains more security-related
checks;
6. The code is FIPS compliant. The original code is not suitable for
use on FIPS-certified systems and would fail to initialise on them.
That provides a foundation to implement QUIC networking code across
the whole range of platforms we support and in a unified way without
depending on OpenSSL-specific (or, rather any crypto-library specific)
support for QUIC. Any versions of the crypto-libraries we support
should be sufficient for that provided that they have TLSv1.3 support.
This commit adds a unit test specifically intended to verify that a
QUIC client and server could complete a handshake using the provided
wrapping code. It only simulates data exchanges via networking stack,
no networking or multi-threading are used, making a it helpful for
debugging QUIC-related handshake code.
Moreover, there are check that ensure that TLS session resumption
works over TLS too.
When QuicTLS is used, we ensure the portable OpenSSL QUIC
compatibility code (which is compatible with QuicTLS) interoperability
against a native QUIC API provided by the library. That ensure
compatibility with this widely deployed library without a need to
write networking code at all.
This commit adds a QUIC integration interface implementation that is
implemented in such a way that allows it to work on OpenSSL and its
highly compatible forks that may not have native BoringSSL-style QUIC
integration API.
The code is know to not work on LibreSSL (due to deliberately broken
functionality within the library), but works on QuicTLS and
OpenSSL (1.1.1+).
This commit adds a set of QUIC/TLS related cryptography functions
which is intended to be used by both OpenSSL QUIC compatibility code
and ngtcp2 crypto API implementation.
These can be considered a lowest level QUIC cryptography building
blocks.
This commit provides an interface for interacting with a QUIC
integration TLS functionality. The interface is designed in such a way
that allows multiple QUIC integration APIs co-exist in one
codebase. It is modelled after the QUIC integration API found in
BoringSSL/LibreSSL/QuicTLS and thus makes it easy to make an
implementation of the interface for these libraries.
Two implementations are planned:
1. A "native" one that uses the QUIC integration API provided by
crypto libraries (LibreSSL/QuicTLS).
2. A portable compatibility layer.
An API that allows switching between the two at runtime is meant to
aid in unit and interoperability testing.
This merge request addresses several key performance bottlenecks in the DoH (DNS over HTTPS) implementation by introducing significant optimizations and improvements.
### Key Improvements
1. **Simplification and Optimisation of `http_do_bio()` Function**:
- The code flow in the `http_do_bio()` function has been significantly simplified.
2. **Flushing HTTP Write Buffer on Outgoing DNS Messages**:
- The buffer is flushed and a send operation is performed when there is an outgoing DNS message.
3. **Bumping Active Streams Processing Limit**:
- The total number of active streams has been increased to 60% of the total streams limit.
These changes collectively enhance the performance and reliability of the DoH implementation, making it more efficient and robust for handling high-load scenarios, particularly noticeable in long runs (>= 1h) of `stress:long:rpz:doh+udp:linux:*` tests. It improves perf. for tests for BIND 9.18, but it likely will have a positive but less pronounced effect on newer versions as well.
In essence, the merge request fixes three bottlenecks stacked upon each other.
*It is a logical continuation of the merge requests !10109.* !10109, unfortunately, did not completely [address the performance drop in 9.18](https://gitlab.isc.org/isc-projects/bind9/-/pipelines/221545) for longer runs of the stress test. This merge request [addresses that](https://gitlab.isc.org/isc-projects/bind9/-/pipelines/223661).
**P.S.**
The origin of the fixes is, in fact, the branch in !10193. So this MR is a ... *forward port* of them.
Merge branch 'artem-doh-performance-drop-post-fix' into 'main'
See merge request isc-projects/bind9!10192
This commit bumps the total number of active streams (= the opened
streams for which a request is received, but response is not ready) to
60% of the total streams limit.
The previous limit turned out to be too tight as revealed by
longer (≥1h) runs of "stress:long:rpz:doh+udp:linux:*" tests.
Previously, the code would try to avoid sending any data regardless of
what it is unless:
a) The flush limit is reached;
b) There are no sends in flight.
This strategy is used to avoid too numerous send requests with little
amount of data. However, it has been proven to be too aggressive and,
in fact, harms performance in some cases (e.g., on longer (≥1h) runs
of "stress:long:rpz:doh+udp:linux:*").
Now, additionally to the listed cases, we also:
c) Flush the buffer and perform a send operation when there is an
outgoing DNS message passed to the code (which is indicated by the
presence of a send callback).
That helps improve performance for "stress:long:rpz:doh+udp:linux:*"
tests.
Previously, a function for continuing IO processing on the next UV
tick was introduced (http_do_bio_async()). The intention behind this
function was to ensure that http_do_bio() is eventually called at
least once in the future. However, the current implementation allows
queueing multiple such delayed requests needlessly. There is currently
no need for these excessive requests as http_do_bio() can requeue them
if needed. At the same time, each such request can lead to a memory
allocation, particularly in BIND 9.18.
This commit ensures that the number of enqueued delayed IO processing
requests never exceeds one in order to avoid potentially bombarding IO
threads with the delayed requests needlessly.
This commit significantly simplifies the code flow in the
http_do_bio() function, which is responsible for processing incoming
and outgoing HTTP/2 data. It seems that the way it was structured
before was indirectly caused by the presence of the missing callback
calls bug, fixed in 8b8f4d500d.
The change introduced by this commit is known to remove a bottleneck
and allows reproducible and measurable performance improvement for
long runs (>= 1h) of "stress:long:rpz:doh+udp:linux:*" tests.
Additionally, it fixes a similar issue with potentially missing send
callback calls processing and hardens the code against use-after-free
errors related to the session object (they can potentially occur).
Cleanup short list macros from <isc/util.h>, remove two unused headers, move locking macros to respective headers and use only the C11 static assertion.
Merge branch 'ondrej/cleanup-short-macros' into 'main'
See merge request isc-projects/bind9!10196
Previously, a gcc < 4.6 shim for _Static_assert() was included. Such an
old compiler is not supported now anyway, so the macro variant has been
removed in favor of a single definition using _Static_assert().
Previously, the LOCK()/UNLOCK() and friends macros were defined in the
isc/util.h header. Those macros were moved to their respective headers
as those would have to be included anyway if that particular lock was in
use.
Formerly, isc/util.h would pull a few extra headers (isc/list.h,
isc/attributes.h, isc/result.h and errno.h). These includes were
removed in favor of including them directly when used.
The short convenience list macros were used very sparingly and
inconsistenly in the code base. As the consistency is prefered over
the convenience, all shortened list macro were removed in favor of
their ISC_LIST API targets.
When querying zone transfers information from the statistics channel there was a rare possibility that `named` could terminate unexpectedly if a zone transfer was in a state when transferring from all the available primary servers had failed earlier. This has been fixed.
Closes#5198
Merge branch '5198-dns_remote_curraddr-bug-fix' into 'main'
See merge request isc-projects/bind9!10182
When all the addresses were already iterated over, the
dns_remote_curraddr() function asserts. So before calling it,
dns_zone_getprimaryaddr() now checks the address list using the
dns_remote_done() function. This also means that instead of
returning 'isc_sockaddr_t' it now returns 'isc_result_t' and
writes the primary's address into the provided pointer only when
returning success.
Logging initialization check is now redundant as there is a default global log context created during libisc's constructor.
`isc_log` calls can safely be made at any time outside libisc's constructor.
Merge branch 'aydin/remove-log-check' into 'main'
See merge request isc-projects/bind9!10186
Disabling dynamic tags ensures the Clang symbolizer creates a valid TSAN
report. For consistency, also add the option to gcc:tsan so they are
both on the same footing.
Closes#5149
Merge branch '5149-fix-tsan-flags' into 'main'
See merge request isc-projects/bind9!10185
Disabling new dynamic ELF tags ensures the Clang symbolizer creates
valid TSAN reports. For consistency, also add the option to gcc:tsan so
they are both on the same footing.
The fix in !10172 was incomplete; there were still some code paths in the resolver that could set the dns_fetchresponse `result` field to the wrong value when the rdataset type was CNAME or DNAME.
Closes#5201
Merge branch '5201-eresult-fix' into 'main'
See merge request isc-projects/bind9!10178
