Commit Graph

38216 Commits

Author SHA1 Message Date
Petr Špaček
b12e5bc84d Merge branch 'pspacek/doc-and-build-tweaks-9.18' into 'bind-9.18'
[9.18] Describe BIND threat model

See merge request isc-projects/bind9!8379
2023-10-13 07:54:26 +00:00
Petr Špaček
43f39a266a Describe BIND threat model
Basically all local data is considered trusted, and proper ACLs and
limits need to be explicitly configured. We are also free to let
protocol non-compliant servers burn in flames.

(cherry picked from commit fc907baa7f)
2023-10-13 09:26:57 +02:00
Michał Kępień
ee3ac71622 Merge branch 'michal/install-pdf-dependencies-on-demand-in-gitlab-ci' into 'bind-9.18'
Install PDF dependencies on demand in GitLab CI

See merge request isc-projects/bind9!8351
2023-10-12 13:09:15 +00:00
Michał Kępień
29cba33d44 Install PDF dependencies on demand in GitLab CI
Building the PDF version of the BIND 9 ARM requires TeX Live to be
present on the build host.  A TeX Live installation takes up several
gigabytes of disk space.  This significantly increases the size of the
Debian Docker images that include that toolchain, even though only two
GitLab CI jobs actually use it.

Instead of including TeX Live in the Docker image itself, install the
former on demand in a new GitLab CI job that only tests building the PDF
version of the BIND 9 ARM.  Do the same for qpdf, a tool used for
checking the PDF output produced by TeX Live.  This enables the size of
the "base" Docker image (which a lot of GitLab CI jobs need to pull) to
remain within reasonable limits.  As downloading and installing TeX Live
takes a significant amount of time, only run the new job in scheduled
pipelines and for tags.  Adjust job dependencies so that the "release"
job continues to work.
2023-10-12 14:27:35 +02:00
Michał Kępień
02df1a681e Merge branch 'michal/move-linux-stress-tests-to-autoscaled-instances-9.18' into 'bind-9.18'
[9.18] Move Linux "stress" tests to autoscaled instances

See merge request isc-projects/bind9!8362
2023-10-06 11:12:06 +00:00
Michał Kępień
1f9d59f726 Move Linux "stress" tests to autoscaled instances
The autoscaling GitLab CI runners currently used for most GitLab CI jobs
spin up AWS EC2 instances that are at least as powerful as the dedicated
instances used for running "stress" tests.  Move all Linux-based
"stress" tests to autoscaling GitLab CI runners to enable deprovisioning
Linux AWS instances reserved for running "stress" tests.  Leave FreeBSD
"stress" tests intact as there is currently no support for autoscaling
BSD instances.

(cherry picked from commit 12ea994680)
2023-10-06 13:10:09 +02:00
Michal Nowak
0de0cc0206 Merge branch 'mnowak/monitor-stuck-system-tests-9.18' into 'bind-9.18'
[9.18] Report hung system tests

See merge request isc-projects/bind9!8360
2023-10-06 07:20:44 +00:00
Michal Nowak
28bae02a96 Report hung system tests
At times, a problem might occur where a test is not responding,
especially in the CI, determining the specific test responsible can be
difficult. Fortunately, when running tests with the pytest runner,
pytest sets the PYTEST_CURRENT_TEST environment variable to the current
test nodeid and stage. Afterward, the variable can be examined to
identify the test that has stopped responding.

The monitoring script needs to be started in the background. Still, the
shell executor used for BSD and FIPS testing can't handle the background
process cleanly, and the script step will wait for the background
process for the entire duration of the background process (currently
3000 seconds). Therefore, run the monitoring script only when the Docker
executor is used where this is not a problem.

(cherry picked from commit 35792b1700)
2023-10-05 19:44:41 +02:00
Petr Špaček
3ebcca683f Merge branch 'pspacek/cross-version-tests-junit-9.18' into 'bind-9.18'
[9.18] Generate and capture JUnit XML output from cross-version-config-tests

See merge request isc-projects/bind9!8358
2023-10-04 12:52:10 +00:00
Petr Špaček
fccfc99b44 Generate and capture JUnit XML output from cross-version-config-tests
(cherry picked from commit f2a3eb00b9)
2023-10-04 14:51:31 +02:00
Petr Špaček
9b617a3649 Merge branch 'pspacek/fix-no-case-compression-docs-9.18' into 'bind-9.18'
[9.18] Fix no-case-compress description in the ARM

See merge request isc-projects/bind9!8355
2023-10-03 13:09:34 +00:00
Petr Špaček
241de55536 Fix no-case-compress description in the ARM
We confused ourselves, it seems.

(cherry picked from commit 6451462a93)
2023-10-03 14:37:16 +02:00
Arаm Sаrgsyаn
fb8e80a6b9 Merge branch '4343-cid-465861-unnecessary-null-check-in-ns__client_setup-9.18' into 'bind-9.18'
[9.18] Remove unnecessary NULL-checks in ns__client_setup()

See merge request isc-projects/bind9!8353
2023-10-02 11:29:53 +00:00
Aram Sargsyan
c061b90cc6 Remove unnecessary NULL-checks in ns__client_setup()
All these pointers are guaranteed to be non-NULL.

Additionally, update a comment to remove obviously outdated
information about the function's requirements.

(cherry picked from commit b970556f21)
2023-10-02 10:04:56 +00:00
Arаm Sаrgsyаn
4779eccdea Merge branch '4331-resolver.c-assert-uninitialized-link-9.18' into 'bind-9.18'
[9.18] Don't use an uninitialized link on an error path

See merge request isc-projects/bind9!8346
2023-09-28 11:32:25 +00:00
Aram Sargsyan
197d032aa8 Add a CHANGES note for [GL #4331]
(cherry picked from commit 9c545c3513)
2023-09-28 10:31:43 +00:00
Aram Sargsyan
92e5173a9f Don't use an uninitialized link on an error path
Move the block on the error path, where the link is checked, to a place
where it makes sense, to avoid accessing an unitialized link when
jumping to the 'cleanup_query' label from 4 different places. The link
is initialized only after those jumps happen.

In addition, initilize the link when creating the object, to avoid
similar errors.

(cherry picked from commit fb7bbbd1be)
2023-09-28 10:30:42 +00:00
Mark Andrews
87b857fad3 Merge branch '4316-dynamic-update-refused-shortly-after-zone-was-thawed-bind-9.18' into 'bind-9.18'
[9.18] Resolve "dynamic update refused shortly after zone was thawed"

See merge request isc-projects/bind9!8336
2023-09-26 04:37:26 +00:00
Mark Andrews
745db8775b Document that reloading happens asynchronously
(cherry picked from commit e33dbd0cbd)
2023-09-26 14:11:30 +10:00
Mark Andrews
de94bee5f3 Wait for the test zone to finish re-loading
'rndc thaw' initiates asynchrous loading of all the zones
similar to 'rndc load'.  Wait for the test zone's load to
complete before testing that it is updatable again.

(cherry picked from commit 5b3238aa85)
2023-09-26 14:11:30 +10:00
Ondřej Surý
357584abc6 Merge branch '4327-minor-warning-about-ctype-h-function-9.18' into 'bind-9.18'
[9.18] Add semantic patch to explicitly cast chars to unsigned for ctype.h

See merge request isc-projects/bind9!8332
2023-09-22 15:41:39 +00:00
Ondřej Surý
cb57e77c68 Add CHANGES note for [GL #4327]
(cherry picked from commit 0e49a8422f)
2023-09-22 17:02:17 +02:00
Ondřej Surý
818f4dc3a7 Explicitly cast chars to unsigned chars for <ctype.h> functions
Apply the semantic patch to catch all the places where we pass 'char' to
the <ctype.h> family of functions (isalpha() and friends, toupper(),
tolower()).

(cherry picked from commit 29caa6d1f0)
2023-09-22 17:01:59 +02:00
Ondřej Surý
d9b8412f08 Add semantic patch to explicitly cast chars to unsigned for ctype.h
Add a semantic patch to catch all the places where we pass 'char' to the
<ctype.h> family of functions (isalpha() and friends, toupper(),
tolower()).  While it generally works because the way how these
functions are constructed in the libc, it's safer to do the explicit
cast.

(cherry picked from commit 5ec65ab5d0)
2023-09-22 17:01:18 +02:00
Michał Kępień
7ff22f5961 Merge branch 'mnowak/add-cve-ref-role-9.18' into 'bind-9.18'
[9.18] Add a Sphinx role for linking CVEs to the ISC Knowledgebase

See merge request isc-projects/bind9!8326
2023-09-21 12:37:29 +00:00
Michal Nowak
5477f97643 Add a Sphinx role for linking CVEs to the ISC Knowledgebase
The new :cve: Sphinx role takes a CVE number as an argument and creates
a hyperlink to the relevant ISC Knowledgebase document that might have
more up-to-date or verbose information than the relevant release note.
This makes reaching ISC Knowledgebase pages directly from the release
notes easier.

Make all CVE references in the release notes use the new Sphinx role.

(cherry picked from commit 41b857e567)
2023-09-21 14:24:24 +02:00
Michał Kępień
56ae3eba80 Merge branch 'michal/update-sphinx-and-sphinx_rtd_theme-9.18' into 'bind-9.18'
[9.18] Update Sphinx and sphinx_rtd_theme

See merge request isc-projects/bind9!8323
2023-09-20 15:25:01 +00:00
Michał Kępień
fa7fc9084f Update Sphinx and sphinx_rtd_theme
Update Sphinx-related Python packages to their current versions pulled
in by "pip install sphinx-rtd-theme" run in a fresh virtual environment.

(cherry picked from commit 2f879cdec3)
2023-09-20 17:21:50 +02:00
Michał Kępień
2f08b622bf Merge tag 'v9.18.19' into bind-9.18 2023-09-20 16:52:16 +02:00
Tom Krizek
9a97f6f70a Merge branch '4312-pytest-symlink-name-collision-9.18' into 'bind-9.18'
[9.18] Rename convenience symlink to pytest artifacts

See merge request isc-projects/bind9!8318
2023-09-20 14:36:27 +00:00
Tom Krizek
2ad197b37d Rename convenience symlink to pytest artifacts
The previous symlink name convention was prone to name collisions If a
system test contained both a shell test and a pytest module of the same
name (e.g. dnstap test has both tests.sh and tests_dnstap.py), then
these would have the same convenience symlink, which could cause test
setup issues as well as confusion when examining test artifacts.

Update the naming convention to include the full pytest module name.
This results in a slightly more verbose names for shell tests (e.g.
dnstap_sh_dnstap instead of the previous dnstap_dnstap), but it removes
the chance of a collision.

(cherry picked from commit fd13dfb097)
2023-09-20 16:04:40 +02:00
Tom Krizek
08647118ce Merge branch '4250-remove-legacy-runner-support-for-python-tests-9.18' into 'bind-9.18'
[9.18] remove support for running python system tests with legacy test runner

See merge request isc-projects/bind9!8317
2023-09-20 13:52:38 +00:00
Tom Krizek
a5eb779c94 Update system test runner doc with pytest glue files
Document that pytest requires a glue file for each shell tests in order
to detect it.

(cherry picked from commit d050843a30)
2023-09-20 14:51:49 +02:00
Tom Krizek
f318cc081b Add CHANGES note for [GL #4250]
(cherry picked from commit 363677bde7)
2023-09-20 14:51:47 +02:00
Tom Krizek
60c4356554 Use integers for ports fixtures in pytest
Reorganize individual port fixtures and re-use the ports fixture to
obtain their number. Store it as integer and only cast it to string when
setting it as environment variable.

(cherry picked from commit 8deb026748)
2023-09-20 14:51:05 +02:00
Tom Krizek
0ac22ebc95 Reformat conftest.py with black
(cherry picked from commit 197dac55a2)
2023-09-20 14:51:05 +02:00
Tom Krizek
7a28a76885 Remove legacy runner support from conftest.py
Remove code fork for legacy runner, reorganize imports and move a
pylint-silencing snippet to the top of the file. The rest of the code
was just unindented.

(cherry picked from commit d16e5fd5e5)
2023-09-20 14:51:02 +02:00
Tom Krizek
98191e6371 Remove pytest invocation from legacy runner
In order to python system tests, pytest (runner) has to be used
directly. This makes it possible to simplify the pytest runner and make
its behavior simpler and easier to extend.

The legacy runner can still be used to run shell system tests.

(cherry picked from commit 4af9bafb4e)
2023-09-20 14:50:18 +02:00
Tom Krizek
9759d8ee28 Merge branch '4262-make-check-pytest-9.18' into 'bind-9.18'
[9.18] Use pytest runner for make check

See merge request isc-projects/bind9!8316
2023-09-20 12:38:34 +00:00
Tom Krizek
8c1a1b304c Use 0 exit code for skipped tests in legacy runner
Since the legacy runner is no longer used in the automake test suite,
don't use the special GNU exit code indicating a skipped tests. Instead,
use 0 to avoid considering skipped tests as failed when using simpler
mechanism (such as xargs -P) to run the tests with the legacy runner.
2023-09-20 13:45:41 +02:00
Tom Krizek
20c8fab299 ci: make sure to use legacy test runner on EL7
EL7 doesn't have the required dependencies for the newer pytest runner.
Since make check now invokes the pytest runner, ensure that the legacy
runner will be used instead.
2023-09-20 13:45:32 +02:00
Tom Krizek
01aada479c Remove make check invocation from legacy.run.sh
The legacy runner no longer uses make check. Ensure the legacy runner
script doesn't interact with that automake target in any way. The legacy
runner script remains available to execute the legacy runner, but there
is no out-of-the box support for running tests in parallel. Other tools
such as xargs can be utilized for that.

(cherry picked from commit 0a9ee342e9)
2023-09-20 10:35:27 +02:00
Tom Krizek
a4edb7db29 ci: switch OpenBSD job to use make check
Invoking pytest directly provides a better formatted output and more
flexibility. However, it's prudent to verify that `make check` keeps
working as expected. Use it in the OpenBSD job which isn't executed as
frequently and its output is of least concern.

(cherry picked from commit f1c631f448)
2023-09-20 10:33:42 +02:00
Tom Krizek
ca392c6f64 Modify custom-test-driver to interpret JUnit results
Pytest provides JUnit output and uses different exit codes from
Automake. Use the conversion script to interpret the JUnit test results
from python rather than relying on the status code.

(cherry picked from commit 295890a16b)
2023-09-20 10:33:42 +02:00
Tom Krizek
fce6fa0fc4 Convert JUnit XML from pytest into Automake .trs files
It's important to parse the JUnit result file rather than relying on the
exit code from pytest, which has a different meaning. Include a .trs test
result for each test case and set an exit code which is most appropriate
as the aggregate result (e.g. it will be set to 77 (SKIP) if there's at
least one test case that was skipped).

(cherry picked from commit b96c8e8062)
2023-09-20 10:33:42 +02:00
Tom Krizek
68f186dd82 Prioritize long-running system tests for legacy runner
Synchronize the test priority with the pytest runner (which handles the
prioritization in conftest.py).

(cherry picked from commit 15fa8f1506)
2023-09-20 10:33:38 +02:00
Tom Krizek
a28529c11d Use prereq.sh for rpzrecurse system test
(cherry picked from commit 0538b7e0da)
2023-09-20 10:32:11 +02:00
Tom Krizek
3fb1f2e851 Use prereq.sh for serve-stale system test
(cherry picked from commit c3abedc0a2)
2023-09-20 10:32:11 +02:00
Tom Krizek
27c3e576fd Use prereq.sh for chain system test
(cherry picked from commit 5d9a09c086)
2023-09-20 10:32:11 +02:00
Tom Krizek
74def754f9 Use prereq.sh for digdelv, dnssec, forward system tests
(cherry picked from commit 00b92bd064)
2023-09-20 10:32:11 +02:00