Commit Graph

37172 Commits

Author SHA1 Message Date
Arаm Sаrgsyаn
a21562c6cf Merge branch 'aram/dns_dispatch_createtcp-bugfix-v9_18' into 'v9_18'
[9.18] Don't pass a NULL pointer to isc_sockaddr_format()

See merge request isc-projects/bind9!7278
2022-12-28 14:54:16 +00:00
Aram Sargsyan
35b5ba2ff4 Don't pass a NULL pointer to isc_sockaddr_format()
The 'localaddr' pointer can be NULL, which causes an assertion failure.

Use '&disp->local' instead when printing a debug log message.

(cherry picked from commit 41ca9d419e)
2022-12-28 14:19:37 +00:00
Tom Krizek
5102e676e7 Merge branch 'tkrizek/gitignore-fixes-v9_18' into 'v9_18'
[9.18] Don't track gitignored files in the repo

See merge request isc-projects/bind9!7274
2022-12-23 14:14:28 +00:00
Michał Kępień
df5c90843b Detect "ignored yet tracked" files in GitLab CI
Ensure that no .gitignore file in the tree contains patterns matching
files tracked by Git by adding a simple script that detects such
situations and running that script as part of the "misc" job in the
"precheck" stage of every GitLab CI pipeline.

(cherry picked from commit 5b28da7ea0)
2022-12-23 14:29:48 +01:00
Tom Krizek
ee4a18ee23 Remove obsolete gitignore file in mysqldyn
The Makefile used to be autogenerated prior to
67f76b1269 which is the reason why the
.gitignore existed in the first place. Since then, a static Makefile is
used and is supposed to be tracked in the git repo.

(cherry picked from commit a02da6cf21)
2022-12-23 14:29:47 +01:00
Tom Krizek
6b56dad1cd Make gitignore matching in rpzrecurse more precise
The pre-defined test cases use named.$TESTCASE.conf naming convention,
where TESTCASE is a human readable name contaning actual word(s). The
autogenerated test cases' names always start with a number from 1 to 6.

(cherry picked from commit e03daf22ec)
2022-12-23 14:29:47 +01:00
Tom Krizek
b9065fb187 Remove ans.pl system test files from gitignore
The ans*.pl scripts are part of system tests and should be part of the
repository. The gitignore entires for these files have been removed.

(cherry picked from commit b8616e457f)
2022-12-23 14:29:47 +01:00
Tom Krizek
bff1f5af2b Rename dig reference files in the rrsetorder test
bin/tests/system/rrsetorder/dig.out* files match a gitignore expression
present in bin/tests/system/.gitignore.  Since these are meant to be
reference files that are compared to the files generated when the
"rrsetorder" system test is run, rename them to avoid listing tracked
files in .gitignore files.

(cherry picked from commit c5708044e7)
2022-12-23 14:29:44 +01:00
Artem Boldariev
5a4b3491f9 Merge branch '3748-rename-tls-caches-creation-functions-v9-18' into 'v9_18'
[9.18] Rename isc_tlsctx_cache_new() to isc_tlsctx_cache_create(), tlsctx_client_session_cache_new() to tlsctx_client_session_create()

See merge request isc-projects/bind9!7272
2022-12-23 12:43:55 +00:00
Artem Boldariev
bccbf28249 tlsctx_client_session_cache_new() -> tlsctx_client_session_create()
Additionally to renaming, it changes the function definition so that
it accepts a pointer to pointer instead of returning a pointer to the
new object.

It is mostly done to make it in line with other functions in the
module.

(cherry picked from commit 7962e7f575)
2022-12-23 13:58:14 +02:00
Artem Boldariev
b05b05cb48 Rename isc_tlsctx_cache_new() -> isc_tlsctx_cache_create()
Additionally to renaming, it changes the function definition so that
it accepts a pointer to pointer instead of returning a pointer to the
new object.

It is mostly done to make it in line with other functions in the
module.

(cherry picked from commit f102df96b8)
2022-12-23 13:58:14 +02:00
Arаm Sаrgsyаn
3922728175 Merge branch '3752-resolver-adb-quota-bug-v9_18' into 'v9_18'
[9.18] Fix an ADB quota management bug in the resolver

See merge request isc-projects/bind9!7273
2022-12-23 10:44:19 +00:00
Aram Sargsyan
0e849a99cf Add CHANGES and release notes for [GL #3752]
(cherry picked from commit a4def095f3)
2022-12-23 10:08:00 +00:00
Aram Sargsyan
926f0323b6 Fix an ADB quota management error in the resolver
Normally, when a 'resquery_t' object is created in fctx_query(),
we call dns_adb_beginudpfetch() (which increases the ADB quota)
only if it's a UDP query. Then, in fctx_cancelquery(), we call
dns_adb_endudpfetch() to decreases back the ADB quota, again only
if it's a UDP query.

The problem is that a UDP query can become a TCP query, preventing
the quota from adjusting back in fctx_cancelquery() later.

Call dns_adb_beginudpfetch() also when switching the query type
from UDP to TCP.

(cherry picked from commit 53afe1f978)
2022-12-23 10:08:00 +00:00
Aram Sargsyan
650f3098e1 INSIST that active quota is 0 in destroy_adbentry()
This should catch ADB quota management errors in the resolver.

(cherry picked from commit c7ba26c3d6)
2022-12-23 10:08:00 +00:00
Tom Krizek
02f775a447 Merge branch 'tkrizek/dangerfile-backport-tweaks-v9_18' into 'v9_18'
[9.18] danger CI: tweak backport check and add Affects label check

See merge request isc-projects/bind9!7270
2022-12-23 08:57:53 +00:00
Tom Krizek
893d6dc98f danger: check the Affects labels are set
Unless the MR is a backport, the Affects labels should be used to
indicate which versions are affected by the issue that prompted the MR.

(cherry picked from commit 64d71a1f5f)
2022-12-23 09:52:40 +01:00
Tom Krizek
8bc806a9f2 danger: check version in MR title
Enforce the version indicator to be at the start of the MR title.

(cherry picked from commit d1172e011c)
2022-12-23 09:52:32 +01:00
Tom Krizek
2c2148fb23 Merge tag 'v9_18_10' into v9_18
BIND 9.18.10
2022-12-22 10:06:46 +01:00
Arаm Sаrgsyаn
fc122887f8 Merge branch '3750-legacy-system-test-dig-timed-out-fix-v9_18' into 'v9_18'
[9.18] Fix DiG "timed out" message check in "legacy" system test

See merge request isc-projects/bind9!7260
2022-12-21 15:09:50 +00:00
Aram Sargsyan
5e57ffbd51 Fix DiG "timed out" message check in "legacy" system test
The test expects a "connection timed out" message from DiG when it
experiences a timeout, while the current version of DiG prints just
a "timed out" message, like below:

    ;; communications error to 10.53.0.1#11314: timed out
    ;; communications error to 10.53.0.1#11314: timed out
    ;; communications error to 10.53.0.1#11314: timed out

    ; <<>> DiG 9.19.9-dev <<>> -p 11314 +tries +time +tcp +tries +time @10.53.0.1 dropedns. TXT
    ; (1 server found)
    ;; global options: +cmd
    ;; no servers could be reached

Change the expected string to match the current DiG output.

Use the '-F' switch for "grep" for matching a fixed string.

(cherry picked from commit 35988748f3)
2022-12-21 14:32:18 +00:00
Tom Krizek
e64dbb578b Merge branch 'tkrizek/system-tests-conventions-v9_18' into 'v9_18'
[9.18] Unify system tests naming conventions

See merge request isc-projects/bind9!7257
2022-12-21 13:44:34 +00:00
Tom Krizek
3c29317ad3 Update tests gitignore file
The .mypy_cache was added to the system test dir gitignore, since it
should be ignored.

(cherry picked from commit 4145068bfa)
2022-12-21 14:02:47 +01:00
Tom Krizek
873feb0c53 Rename engine_pkcs11 system test
In order to have a common naming convention for system tests, rename the
only outlier "engine_pkcs11" to "enginepkcs11", which was the only
system test using an underscore in its name.

The only allowed word separators for system test names are either dash
or no separator.

(cherry picked from commit 62eea511db)
2022-12-21 14:02:47 +01:00
Tom Krizek
4652da6caf Use common name convention for pytest files
It is better to use consistent file names to avoid issue with sorting
etc.

Using underscore in filenames as opposed to dash was chosen because it
seems more common in pytest/python to use underscore for filenames.

Also rename the bin/tests/system/timeouts/tests-tcp.py file to
bin/tests/system/timeouts/tests_tcp_timeouts.py to avoid pytest name
collision (there can't be two files named tests_tcp.py).

(cherry picked from commit f6409ee6ac)
2022-12-21 14:02:45 +01:00
Ondřej Surý
6fd6686865 Merge branch '3178-dispatch-race-v9_18' into 'v9_18'
[9.18] Fix the thread safety in the dns_dispatch unit

See merge request isc-projects/bind9!7251
2022-12-21 12:41:22 +00:00
Ondřej Surý
3492dc2f0e Add CHANGES and release note for [GL #3178] and [GL #3636]
(cherry picked from commit 2df311eb21)
2022-12-21 12:41:15 +00:00
Ondřej Surý
be3cf85cfe Call the connected dns_dispatch callback asynchronously
The dns_request code is very sensitive about calling the connected and
deadlocks when the timing is "right" in several places.  Move the call
to the connected callback to the (udp|tcp)_connected() functions, so
they are called asynchronously instead of directly from
the (udp|tcp)_dispentry_cancel() functions.

(cherry picked from commit 9dd8deaf01)
2022-12-21 12:41:15 +00:00
Ondřej Surý
87ad3ecaf0 Ignore TCP dispatches that have zero references
The TCP dispatches are removed from the dispatchmgr->list in the
dispatch_destroy() and there's a brief period of time where
dns_dispatch_gettcp() can find a dispatch in connected state that's
being destroyed.

Set the dispatch state to DNS_DISPATCHSTATE_NONE in the TCP connection
callback if there are no responses waiting, and ignore TCP dispatches
with zero references in dns_dispatch_gettcp().

(cherry picked from commit 3fac4ca57e)
2022-12-21 12:41:15 +00:00
Ondřej Surý
3f4970da1a Fix assignment vs comparison typo in tcp_connected()
In tcp_connected() a typo has turned a DbC check into an assignment
breaking the state machine and making the dns_dispatch_gettcp() try to
attach to dispatch in process of destruction.
2022-12-21 12:41:15 +00:00
Ondřej Surý
b714033731 Don't remove dispatches in CANCELED state from the list
In dns_dispatch_gettcp(), we can't remove canceled dispatches from the
mgr->list because ISC_LIST_NEXT() would fail in the next iteration.
2022-12-21 12:41:15 +00:00
Ondřej Surý
3edccaf0c7 Ignore TCP dispatches in DNS_DISPATCHSTATE_NONE state
The TCP dispatches in DNS_DISPATCHSTATE_NONE could be either very
fresh or those could be dispatches that failed connecting to the
destination.  Ignore them when trying to connect to an existing
TCP dispatch via dns_dispatch_gettcp().
2022-12-21 12:41:15 +00:00
Ondřej Surý
5cc12ab92c Fix the thread safety in the dns_dispatch unit
The dispatches are not thread-bound, and used freely between various
threads (see the dns_resolver and dns_request units for details).

This refactoring make sure that all non-const dns_dispatch_t and
dns_dispentry_t members are accessed under a lock, and both object now
track their internal state (NONE, CONNECTING, CONNECTED, CANCELED)
instead of guessing the state from the state of various struct members.

During the refactoring, the artificial limit DNS_DISPATCH_SOCKSQUOTA on
UDP sockets per dispatch was removed as the limiting needs to happen and
happens on in dns_resolver and limiting the number of UDP sockets
artificially in dispatch could lead to unpredictable behaviour in case
one dispatch has the limit exhausted by others are idle.

The TCP artificial limit of DNS_DISPATCH_MAXREQUESTS makes even less
sense as the TCP connections are only reused in the dns_request API
that's not a heavy user of the outgoing connections.

As a side note, the fact that UDP and TCP dispatch pretends to be same
thing, but in fact the connected UDP is handled from dns_dispentry_t and
dns_dispatch_t acts as a broker, but connected TCP is handled from
dns_dispatch_t and dns_dispatchmgr_t acts as a broker doesn't really
help the clarity of this unit.

This refactoring kept to API almost same - only dns_dispatch_cancel()
and dns_dispatch_done() were merged into dns_dispatch_done() as we need
to cancel active netmgr handles in any case to not leave dangling
connections around.  The functions handling UDP and TCP have been mostly
split to their matching counterparts and the dns_dispatch_<function>
functions are now thing wrappers that call <udp|tcp>_dispatch_<function>
based on the socket type.

More debugging-level logging was added to the unit to accomodate for
this fact.

(cherry picked from commit 6f317f27ea)
2022-12-21 12:41:15 +00:00
Ondřej Surý
24847cfd3e Backport ISC_REFCOUNT{,_TRACE}_{DECL,IMPL} macros
Backport macros that can be used to implement generic attach, detach,
ref, and unref functions, so they don't have to be repeated over and
over in each unit that uses reference counting.
2022-12-21 12:41:15 +00:00
Tom Krizek
7009aa9044 Merge branch 'tkrizek/default-alg-fixups-v9_18' into 'v9_18'
[9.18] Fix minor issues with DEFAULT_ALGORITHM selection in system tests

See merge request isc-projects/bind9!7255
2022-12-21 11:43:02 +00:00
Tom Krizek
45b003316f Make Python interpreter required for system tests
This introduces a Python dependency for running system tests. It is
needed in order to:
  - write new test control scripts in Python
  - gradually rewrite old Perl scripts into Python if needed
  - eventually introduce pytest as the new test runner framework

This commit is not intended to be backported to 9.16.

(cherry picked from commit 56416ebd65)
2022-12-21 10:49:33 +01:00
Tom Krizek
efa09a37f3 Make Perl interpreter required for system tests
This change has no practical impact, as Perl was already required for
all system tests, this check only makes it more explicit.

(cherry picked from commit 084d72d1d5)
2022-12-21 10:49:33 +01:00
Tom Krizek
0079745f97 Ensure test interpreters are defined before common config
Nothing from conf.sh.common is required to set these values. On the
contrary, a Python interpreter needs to be set in order to randomize the
algorithm set (which happens in conf.sh.common).

(cherry picked from commit 492992dca8)
2022-12-21 10:49:33 +01:00
Tom Krizek
54a877ef47 Force quiet mode when using testcrypto.sh directly
When testcrypto.sh is used as a standalone script, always use quiet mode
to avoid using undefined commands (such as echo_i) which require
inclusion of the entire conf.sh machinery.

(cherry picked from commit ba35a6df9c)
2022-12-21 10:42:10 +01:00
Ondřej Surý
3a78f58c55 Merge branch '3739-adb-cleans-overzealously-under-memory-pressure-v9_18' into 'v9_18'
Exclude the ADB hashmaps from ADB overmem accounting [v9.18]

See merge request isc-projects/bind9!7248
2022-12-19 08:58:27 +00:00
Ondřej Surý
c3f1a0dfff Add CHANGES and release note for [GL #3739]
(cherry picked from commit 11df7f02fd)
2022-12-19 09:26:59 +01:00
Ondřej Surý
146f93ff46 Don't expire fresh ADB names and entries
The overmem cleaning in ADB could become overzealous and clean fresh ADB
names and entries.  Add a safety check to not clean any ADB names and
entries that are below ADB_CACHE_MINIMUM threshold.

(cherry picked from commit 0b661b6f95)
2022-12-19 09:26:58 +01:00
Ondřej Surý
ecde82689e Exclude the ADB hashmaps from ADB overmem accounting
The ADB overmem accounting would include the memory used by hashtables
thus vastly reducing the space that can be used for ADB names and
entries when the hashtables would grow.  Create own memory context for
the ADB names and entries hash tables.

(cherry picked from commit 59dee0b078)
2022-12-16 19:22:43 +01:00
Ondřej Surý
25201365a6 Merge branch '3637-try-next-server-on-resolver-timeout-v9_18' into 'v9_18'
Try next server on resolver timeout [v9.18]

See merge request isc-projects/bind9!7249
2022-12-16 18:22:39 +00:00
Ondřej Surý
6ca03d1012 Add CHANGES and release notes for [GL #3637]
(cherry picked from commit d1f1db2f5a)
2022-12-16 18:38:02 +01:00
Ondřej Surý
2cb6001a2e Add comment why statistics/ns3 has misconfigure root.hints
To count pending queries in the statistics, we need the ns3 to be
misconfigured.  Document the fact in the statistics/ns3/root.hints.

(cherry picked from commit 32c2acf6fc)
2022-12-16 18:37:22 +01:00
Ondřej Surý
9e8c23be7e Revert the statistics system test change after we fixed the resolver
When the resolver was refactored, the statistics system test had to be
adjusted in c6b4d82557.  Unfortunately,
this change had to be done because of an error in the resolver
refactoring where timeout would not retry next server, but keep trying
the same server.  As we have now fixed this bug, revert the change to
the test back to the previous state.

(cherry picked from commit b679640950)
2022-12-16 18:37:22 +01:00
Ondřej Surý
095f634f48 Try next server on resolver timeout
Instead of resending to the same server on the (dispatch) timeout in the
resolver, try the next server.

(cherry picked from commit 5466a48fc9)
2022-12-16 18:37:22 +01:00
Tom Krizek
773517960f Merge branch 'tkrizek/danger-backports-v9_18' into 'v9_18'
Check backport workflow in danger CI [v9_18]

See merge request isc-projects/bind9!7245
2022-12-16 13:39:32 +00:00
Tom Krizek
a11bcfa8ba danger: remove obsolete check for cherry pick msg
With proper backport commit detection, this check has been made
redundant.

(cherry picked from commit e8a5ebaee5)
2022-12-16 14:21:10 +01:00