Commit Graph
39646 Commits
Author SHA1 Message Date
Tom Krizek 355dc73391 .gitignore temporary directories and symlinks in system test dir 2023-08-21 15:54:40 +02:00
Tom Krizek d66ff81543 Add clean-local target to clean pytest runner artifacts
The command finds all directories in bin/tests/system which contain an
underscore. Underscore indicates either a temporary directory (_tmp_), a
symlink to test artifacts (TESTNAME_MODULENAME), or a python-related
cache. Using underscore for a system test name is invalid and a hyphen
must be used instead.
2023-08-21 15:54:40 +02:00
Tom Krizek 83ddca7690 Silence pylint's refactoring suggestions for system_test_dir()
While it'd be fairly easy to split the function up into smaller ones,
the readability wouldn't be improved in this case. Silence the
suggestions instead.
2023-08-21 15:54:40 +02:00
Tom Krizek f91d0b13e8 Improve tempdir logging for pytest runner
At the end of the test, display the symlink path to the artifact
directory in case it's preserved. Log the full tempdir name in debug
log.
2023-08-21 15:54:40 +02:00
Tom Krizek e1ca5c8071 Create symlinks to test artifacts for pytest runner
While temporary directories are useful for test execution to keep
everything clean, they are difficult to work with manually. Create a
symlink for each test artifact directory with a stable and predictable
path. The symlink always either points to the latest artifacts, or is
missing in case the last run succeeded.

Ensure these symlinked directories aren't detected as test suites by the
pytest runner.
2023-08-21 15:54:38 +02:00
Ondřej Surý 838dc9b053 Merge branch '4125-release-notes' into 'main'
Add release note for [GL #4125]

Closes #4125

See merge request isc-projects/bind9!8216
2023-08-21 13:19:00 +00:00
Ondřej Surý 58a8de5246 Add release note for [GL #4125] 2023-08-21 15:16:41 +02:00
Tom Krizek f52dd73cac Merge branch '4246-remove-legacy-test-runner-from-ci' into 'main'
Use pytest runner for out-of-tree and OpenBSD system tests in CI

Closes #4246

See merge request isc-projects/bind9!8193
2023-08-21 13:13:33 +00:00
Tom Krizek b1a9d1fe25 Improve pytest message when sanitizer report is found 2023-08-21 15:12:53 +02:00
Tom Krizek bc66bf146f ci: remove legacy system test runner template 2023-08-21 15:12:53 +02:00
Tom Krizek d49d9ec80e ci: run out-of-tree system tests with pytest runner
Out-of-tree builds are built in a directory that is different from
source directory. The build directory doesn't contain the non-compiled
test files from bin/tests/system which are the test cases required by
the pytest runner.

In order to run the system tests for out-of-tree build, copy over the
contents (tests) of bin/tests/system/ from the source directory into the
build directory. Then, it is possible to invoke the pytest runner inside
the build directory.
2023-08-21 15:12:53 +02:00
Tom Krizek 8846877dfe ci: use pytest system test runner on OpenBSD
A supported pytest version finally made it to OpenBSD repositories,
allowing us to run system tests with the pytest runner.
2023-08-21 15:12:51 +02:00
Ondřej Surý f9657267af Merge branch '4125-statschannel-content-length-overflow' into 'main'
Parse statschannel Content-Length: more carefully

Closes #4125

See merge request isc-projects/bind9!8018
2023-08-21 13:01:28 +00:00
Tony FinchandOndřej Surý 1ba538f32f Add CHANGES entry for [GL #4125] 2023-08-21 14:14:18 +02:00
Tony FinchandOndřej Surý 26e10e8fb5 Parse statschannel Content-Length: more carefully
A negative or excessively large Content-Length could cause a crash
by making `INSIST(httpd->consume != 0)` fail.
2023-08-21 14:14:18 +02:00
Michał Kępień 159c880240 Merge branch 'michal/post-release-tweaks' into 'main'
Post-release tweaks (cross-version testing cleanup, release checklist tweaks)

See merge request isc-projects/bind9!8210
2023-08-17 15:31:51 +00:00
Michał Kępień 45bf6123ef Update release checklist
Update the release checklist with links to recently implemented tools
for automating certain tasks.
2023-08-17 17:30:37 +02:00
Michał Kępień 96e4139fb8 Revert "Exclude dupsigs and keymgr2kasp from cross-version-config-tests"
This reverts commit 4eac32fe23.

With the v9.19.16 release tag merged, the "cross-version-config-tests"
GitLab CI job will no longer fail due to the two relevant system tests
being absent from the development branch.  This makes the pytest
filtering expression added to work around that issue unnecessary, so
remove it.
2023-08-17 17:30:37 +02:00
Michal Nowak 4970d0ca38 Merge branch '4139-test_send_timeout-check-can-be-unstable' into 'main'
Make the test_send_timeout check more stable

Closes #4139

See merge request isc-projects/bind9!8168
2023-08-17 13:11:04 +00:00
Michal Nowak 681b23c398 Mark test_send_timeout as flaky
In some cases, BIND is not fast enough to fill the send buffer and
manages to answer all queries, contrary to what the test expects.
Repeat the check up to 3 times to limit this test instability.
2023-08-17 14:13:59 +02:00
Tom KrizekandMichal Nowak 5b703de733 Add custom flaky decorator to handle unstable tests
If the flaky plugin for pytest is available, use its decorator to
support re-running unstable tests. In case the package is missing,
execute the test as usual without attempts to re-run it in case of
failure.

This is mostly intended to increase the test stability in CI. Using a
custom decorator enables us to keep the flaky package as an optional
dependency.
2023-08-17 14:13:59 +02:00
Michał Kępień e27a33e621 Merge tag 'v9.19.16' 2023-08-17 14:10:53 +02:00
Michal Nowak b88f914c79 Merge branch 'mnowak/clean-leftover-files' into 'main'
Clean leftover files in autosign and masterformat

See merge request isc-projects/bind9!8167
2023-08-16 12:45:24 +00:00
Michal Nowak 8a0da13479 Clean leftover files in autosign and masterformat
The following files were reported in CI by the legacy system test runner
and prevented job to pass. They should be removed.

    $ if git rev-parse > /dev/null 2>&1; then ( ! grep "^I:.*:file.*not removed$" *.log ); fi
    autosign.log:I:autosign:file autosign/ns3/kskonly.example.db.jbk not removed
    autosign.log:I:autosign:file autosign/ns3/optout.example.db.jbk not removed
    autosign.log:I:autosign:file autosign/ns3/reconf.example.db.jbk not removed
    masterformat.log:I:masterformat:file masterformat/ns1/signed.db.raw.jbk not removed
    masterformat.log:I:masterformat:file masterformat/ns1/signed.db.raw.signed not removed
    masterformat.log:I:masterformat:file masterformat/ns1/signed.db.raw.signed.jnl not removed

Don't print an error when the ns*/inactive directory is not
present:

    rmdir: ns*/inactive: No such file or directory

Remove nsupdate.out.test file instead of nsupdate.out, as the latter
does not exist.
2023-08-16 12:01:11 +02:00
Ondřej Surý 2c51e936e4 Merge branch '4228-fix-heap-use-after-free-in-dns_dispatch_createtcp' into 'main'
Attach to the dns_dispatchmgr in the dns_view object

Closes #4228

See merge request isc-projects/bind9!8203
2023-08-16 07:22:23 +00:00
Ondřej SurýandEvan Hunt ce1b1b4058 Add CHANGES note for [GL #4228] 2023-08-15 10:26:02 -07:00
Ondřej SurýandEvan Hunt d76ab69772 Attach to the dns_dispatchmgr in the dns_view object
The dns_dispatchmgr object was only set in the dns_view object making it
prone to use-after-free in the dns_xfrin unit when shutting down named.

Remove dns_view_setdispatchmgr() and optionally pass the dispatchmgr
directly to dns_view_create() when it is attached and not just assigned,
so the dns_dispatchmgr doesn't cease to exist too early.

The dns_view_getdnsdispatchmgr() is now protected by the RCU lock, the
dispatchmgr reference is incremented, so the caller needs to detach from
it, and the function can return NULL in case the dns_view has been
already shut down.
2023-08-15 10:25:37 -07:00
Evan Hunt cfad194e1d Merge branch 'each-qp-nta' into 'main'
Use a QP trie for the NTA table, forwarders and key tables

See merge request isc-projects/bind9!7811
2023-08-15 17:16:21 +00:00
Ondřej Surý bf1223bf13 Add CHANGES note for [GL !7811] 2023-08-15 17:34:20 +02:00
Evan HuntandOndřej Surý b466439437 use a qp-trie for the keytable
Instead of an RBT for the trust anchor tables, use a QP-trie.
2023-08-15 14:25:24 +02:00
Evan HuntandOndřej Surý dea79e7053 use a qp-trie for the forwarders table
Instead of an RBT for the forwarders table, use a QP trie.

We now use reference counting for dns_forwarders_t. When a forwarders
object is retrieved by dns_fwdtable_find(), it must now be explicitly
detached by the caller afterward.

QP tries require stored objects to include their names, so the
the forwarders object now has that. This obviates the need to
pass back a separate 'foundname' value from dns_fwdtable_find().
2023-08-15 14:25:24 +02:00
Evan HuntandOndřej Surý 5768dd96ea clean up some unused functions
dns_fwdtable_delete() was only used by dns_client_clearservers(),
which in turn was never used. both functions have now been deleted.
2023-08-15 14:24:46 +02:00
Evan HuntandOndřej Surý aff01bda54 use a qp-trie for the NTA table
replace the red-black tree used by the negative trust anchor table
with a QP trie.

because of this change, dns_ntatable_init() can no longer fail, and
neither can dns_view_initntatable(). these functions have both been
changed to type void.
2023-08-15 14:24:46 +02:00
Evan HuntandOndřej Surý 06216f4f90 rename dns_qp_findname_parent() to _findname_ancestor()
this function finds the closest matching ancestor, but the function
name could be read to imply that it returns the direct parent node;
this commit suggests a slightly less misleading name.
2023-08-15 14:24:46 +02:00
Tony FinchandOndřej Surý b38c71961d Improve qp-trie leaf return values
Make the `pval_r` and `ival_r` out arguments optional.

Add `pval_r` and `ival_r` out arguments to `dns_qp_deletekey()`
and `dns_qp_deletename()`, to return the deleted leaf.
2023-08-15 14:24:39 +02:00
Ondřej Surý 8a2763487d Merge branch 'fanf/add-SET_IF_NOT_NULL-macro' into 'main'
A SET_IF_NOT_NULL() macro for optional return values

See merge request isc-projects/bind9!8201
2023-08-15 11:23:27 +00:00
Tony FinchandOndřej Surý c622b349e4 Apply the SET_IF_NOT_NULL() semantic patch
spatch --sp-file cocci/set_if_not_null.spatch --use-gitgrep --dir "." --include-headers --in-place
2023-08-15 12:21:41 +02:00
Tony FinchandOndřej Surý 0d6dcd217d A SET_IF_NOT_NULL() macro for optional return values
The SET_IF_NOT_NULL() macro avoids a fair amount of tedious boilerplate,
checking pointer parameters to see if they're non-NULL and updating
them if they are.  The macro was already in the dns_zone unit, and this
commit moves it to the <isc/util.h> header.

I have included a Coccinelle semantic patch to use SET_IF_NOT_NULL()
where appropriate. The patch needs an #include in `openssl_shim.c`
in order to work.
2023-08-15 12:04:29 +02:00
Mark Andrews 5a36bebfce Merge branch '4259-statschannel-leftovers' into 'main'
Resolve "Statschannel leftovers"

Closes #4259

See merge request isc-projects/bind9!8199
2023-08-15 04:21:33 +00:00
Mark Andrews 7da47fd858 Remove bind9.xsl.1, bind9.xsl.2 and bind9.xsl.3 2023-08-15 10:34:19 +10:00
Mark Andrews cc491f539b Merge branch '4238-the-mkeys-system-test-can-update-the-root-zone-too-fast' into 'main'
Resolve "The mkeys system test can update the root zone too fast"

Closes #4238

See merge request isc-projects/bind9!8161
2023-08-14 23:23:33 +00:00
Mark Andrews 40e3529379 Add sleeps so that the modification time changes
The mkeys system test could fail because root zone was resigned
within the same second as it was previously signed causing reloads
to fail.  Add delays to the test to prevent this.
2023-08-14 22:49:16 +00:00
Ondřej Surý dadc214b0f Merge branch '4124-check-statschannel-if-modified-since' into 'main'
Fix a stack buffer overflow in the statistics channel

Closes #4124

See merge request isc-projects/bind9!8195
2023-08-14 10:59:02 +00:00
Ondřej Surý 57c8bdaff5 Add CHANGES and release notes for [GL #4124] 2023-08-14 11:30:45 +02:00
Tony FinchandOndřej Surý b22c87ca61 Fix a stack buffer overflow in the statistics channel
A long timestamp in an If-Modified-Since header could overflow a
fixed-size buffer.
2023-08-14 11:30:24 +02:00
Mark Andrews bafb3f97c2 Merge branch 'marka-fix-formatting-errors' into 'main'
Follow-up: Fixes to provider/engine based ECDSA key handling

See merge request isc-projects/bind9!8186
2023-08-09 23:49:25 +00:00
Matthijs MekkingandMark Andrews fa108db279 Revert a337dbef
Revert commit that always uses OpenSSL 3.0 API when available,
the new APIs should work always, but OpenSSL has non-obvious
omissions in the automatic mappings it provides.
2023-08-09 23:48:17 +00:00
Mark Andrews 00a09e0d35 Only set key->engine if engine != NULL 2023-08-09 23:48:17 +00:00
Mark Andrews d527ae11c9 Fix clang formatting 2023-08-09 23:48:17 +00:00
Mark Andrews 3907de98ca Fix line wrap 2023-08-09 23:48:17 +00:00