Commit Graph

36431 Commits

Author SHA1 Message Date
Michał Kępień
2cd20ee370 Clean up convert-trs-to-junit.py invocations
- Use absolute paths when invoking the convert-trs-to-junit.py script
    so that it also works correctly for out-of-tree and tarball-based
    test jobs.

  - Quote the variables used in convert-trs-to-junit.py invocations to
    future-proof the code.

  - Use "&&" instead of ";" in shell pipelines invoking the
    convert-trs-to-junit.py script in order to prevent "source" errors
    from being silently ignored.

  - Ensure convert-trs-to-junit.py is invoked from the correct directory
    for out-of-tree and tarball-based unit test jobs by adding
    appropriate "cd" invocations.

  - Ensure the convert-trs-to-junit.py invocations are always the last
    step in each 'after_script', in order to run that script from the
    correct directory for out-of-tree and tarball-based system test jobs
    and to ensure that any potential errors in that script do not
    prevent more important steps in the 'after_script' from being
    executed.
2022-06-22 12:59:33 +02:00
Michał Kępień
e6aebf10cc Move out-of-tree workspace back to $CI_PROJECT_DIR
Out-of-tree build & test jobs currently defined in GitLab CI use
/tmp/out_of_tree_workspace as the working directory.  This requires
juggling that directory around as it gets passed from the build job to
the test jobs and then again after the test jobs are finished, so that
artifacts can be collected for the purpose of investigating test
failures.  The original intention of doing this was to ensure that
bin/tests/system/run.sh does not rely on being executed from within a
Git working copy (which happens e.g. if the out-of-tree workspace is a
subdirectory of $CI_PROJECT_DIR, i.e. the path into which GitLab
Runner clones the project in each job).

However, even with these complications in place, not all possible
scenarios that should be handled properly by the system test framework
(e.g. invoking a given test one time after another from the same
out-of-tree build directory) are tested in GitLab CI anyway.  Meanwhile,
the requirement for moving the out-of-tree workspace into
$CI_PROJECT_DIR in the 'after_script' for each out-of-tree job makes
these jobs less robust than they could be; for example, if any step in
the 'after_script' returns a non-zero exit code, the job's artifacts
will not include the out-of-tree workspace, hindering troubleshooting.

Simplify job definitions in .gitlab-ci.yml by moving the workspace used
by out-of-tree build & test jobs back to a subdirectory of
$CI_PROJECT_DIR.  Whether the out-of-tree workspace exists within a Git
working copy or not does not matter for Autotools, so this is considered
to be a reasonable trade-off in terms of test coverage.
2022-06-22 12:59:33 +02:00
Michal Nowak
cc6dc3c73e Merge branch 'mnowak/ubuntu-22.04-jammy-jellyfish' into 'main'
Add Ubuntu 22.04 LTS (Jammy Jellyfish)

See merge request isc-projects/bind9!6446
2022-06-22 08:57:40 +00:00
Michal Nowak
06c591667e Do not run Ubuntu 18.04 jobs in MR-triggered pipelines
With the addition of Ubuntu 22.04 three more CI jobs were added. To
compensate for that, move Ubuntu 18.04 jobs out of MR-triggered
pipelines to schedule-triggered ones.

Also, move --disable-geoip ./configure options from Ubuntu 18.04 to
Ubuntu 20.04 jobs to keep these options in the more frequent
MR-triggered pipelines.
2022-06-22 10:56:50 +02:00
Michal Nowak
4c2af3bdfa Add Ubuntu 22.04 LTS (Jammy Jellyfish) 2022-06-22 10:56:50 +02:00
Matthijs Mekking
c31d2628b6 Merge branch '3412-system-test-passthru-virtualenv-perl5lib' into 'main'
System tests pass through virtualenv and perlmods

Closes #3412

See merge request isc-projects/bind9!6458
2022-06-22 08:32:48 +00:00
Matthijs Mekking
53eb127be1 System tests pass through virtualenv and perlmods
Have system tests use required Python (dnspython) and PERL modules
(Digest::HMAC and Net::DNS) from user directories, avoid using
privileges.

Note: The pythonenv must be setup to use the same version of python as
the test uses, for example /usr/bin/python.

Thanks to Stacey Marshall.
2022-06-22 09:39:51 +02:00
Matthijs Mekking
95c32b9a54 Merge branch '3416-document-key-lifetime-min-length' into 'main'
Document what is a too short key lifetime

Closes #3416

See merge request isc-projects/bind9!6460
2022-06-21 15:34:23 +00:00
Matthijs Mekking
c47735b86b Document what is a too short key lifetime
To give a hint to users that get an error that the key lifetime is
shorter than the time it takes to do a rollover.
2022-06-21 16:18:49 +02:00
Matthijs Mekking
cc7f132ff8 Merge branch '3397-document-interaction-dnssec-update-config-options' into 'main'
Document where updates and DNSSEC records are stored

Closes #3397

See merge request isc-projects/bind9!6457
2022-06-20 12:12:50 +00:00
Matthijs Mekking
c80531758c Add comment on DNSSEC signing zone configuration
I was slightly confused here, so IMO it can use a comment.
2022-06-20 11:18:46 +02:00
Matthijs Mekking
8860f6b4ff Document where updates and DNSSEC records are stored
Make clear that inline-signing stores DNSSEC records in a signed
version of the zone, using the zone's filename plus ".signed" extension.

Tell that dynamic zones store updates in the zone's filename.

DNSSEC records for dynamic zones also go in the zone's filename, unless
inline-signing is enabled.

Then, dnssec-policy assumes inline-signing, but only if the zone is
not dynamic.
2022-06-20 11:18:46 +02:00
Michal Nowak
84854b3f22 Merge branch 'mnowak/bump-clang-to-version-14' into 'main'
Update clang to version 14

See merge request isc-projects/bind9!6368
2022-06-16 16:14:32 +00:00
Michal Nowak
bd17dec045 Extract Clang version to $CLANG_VERSION
Extracting Clang version helps with updating Clang version in one place
and not missing one of its instances.
2022-06-16 17:21:12 +02:00
Michal Nowak
e98b9c5009 Fix external_symbolizer_path on Fedora
The external_symbolizer_path was wrong on Fedora since moving gcc:tsan
jobs from Debian to Fedora.
2022-06-16 17:21:12 +02:00
Michal Nowak
1c45a9885a Update clang to version 14 2022-06-16 17:21:11 +02:00
Michal Nowak
51ebc6e1e8 Merge branch 'mnowak/checkds-pylint-implicit-str-concat-fixes' into 'main'
Fix implicit string concatenation in tests-checkds.py

See merge request isc-projects/bind9!6452
2022-06-16 11:40:27 +00:00
Michal Nowak
831ac8add1 Fix implicit string concatenation in tests-checkds.py
pylint 2.14.2 reports the following warnings:

    bin/tests/system/checkds/tests-checkds.py:265:0: W1404: Implicit string concatenation found in call (implicit-str-concat)
    bin/tests/system/checkds/tests-checkds.py:273:0: W1404: Implicit string concatenation found in call (implicit-str-concat)
2022-06-16 11:42:29 +02:00
Tom Krizek
636d561926 Merge branch 'tkrizek/pytest-skipped-reason' into 'main'
Report reasons for skipped/xfailed system pytests

See merge request isc-projects/bind9!6441
2022-06-16 07:17:37 +00:00
Tom Krizek
f6d368167a Report reasons for skipped/xfailed system pytests
If skip/xfail is used in pytest, it can have a reason string associated
with it. When evaluating these tests, it can be useful to be able to
differentiate the reason why the test was skipped/xfailed/xpassed,
because there might be multiple possible reasons for that.

The extra options passed to pytest ensure that the string with the
reason appears in the test summary and thus we're able to find the
string with the reason in the log output.

See https://docs.pytest.org/en/7.1.x/how-to/skipping.html for more info
2022-06-16 09:13:53 +02:00
Petr Špaček
a1cbe71ab2 Merge branch 'pspacek/nsec3-doc-update' into 'main'
Update NSEC3 guidance to match draft-ietf-dnsop-nsec3-guidance-10

See merge request isc-projects/bind9!6413
2022-06-15 15:56:53 +00:00
Petr Špaček
2ee3f4e6c8 Update NSEC3 guidance to match draft-ietf-dnsop-nsec3-guidance-10
https://datatracker.ietf.org/doc/html/draft-ietf-dnsop-nsec3-guidance-10
is on it's way to become RFC, so let's update our recommendations in the
docs to be in line with it.
2022-06-15 17:53:33 +02:00
Michał Kępień
529a7c0c1a Merge tag 'v9_19_2'
BIND 9.19.2
2022-06-15 15:53:59 +02:00
Artem Boldariev
f3cb438091 Merge branch 'artem-tlsdns-do-not-call-acceptcb-twice' into 'main'
TLS DNS: do not call accept callback twice

See merge request isc-projects/bind9!6439
2022-06-15 12:29:35 +00:00
Artem Boldariev
e616d7f240 TLS DNS: do not call accept callback twice
Before the changes from this commit were introduced, the accept
callback function will get called twice when accepting connection
during two of these stages:

* when accepting the TCP connection;
* when handshake has completed.

That is clearly an error, as it should have been called only once. As
far as I understand it the mistake is a result of TLS DNS transport
being essentially a fork of TCP transport, where calling the accept
callback immediately after accepting TCP connection makes sense.

This commit fixes this mistake. It did not have any very serious
consequences because in BIND the accept callback only checks an ACL
and updates stats.
2022-06-15 14:21:11 +03:00
Petr Špaček
f397803ec4 Merge branch 'pspacek/doc-cleanup-dnssec-prereq' into 'main'
Update intro texts in the DNSSEC Guide

See merge request isc-projects/bind9!6418
2022-06-14 16:11:54 +00:00
Petr Špaček
fd3a2c7854 Update FAQ in the DNSSEC Guide
Mention DoT/DoH, update stats, remove mentions of early stages of
deployment.
2022-06-14 18:08:23 +02:00
Petr Špaček
635885afe6 Update Authoritative Server Hardware requirements in DNSSEC Guide
Based on measurements done on BIND v9_19_2 using bank. TLD and a
synthetitc fullly signed zone, using RSASHA256 and ECDSAP256SHA256
algorithms with NSEC and NSEC3 without opt-out.
2022-06-14 18:08:22 +02:00
Petr Špaček
832c172985 Update DNSSEC validation deployment stats in DNSSEC Guide 2022-06-14 18:08:22 +02:00
Petr Špaček
6cf8066b9c Rewrite Recursive Server Hardware requirements in DNSSEC Guide
This section was completely out of date. Current measurements on dataset
Telco EU 2022-02 and BIND 9.19.1 indicate absolutely different results
than described in the old version of the text.
2022-06-14 18:08:22 +02:00
Petr Špaček
6e79877759 Remove outdated software requirements from DNSSEC Guide
Guide in this repo is tied to latest version anyway, so let's not even
mention ancient versions of BIND.

This also solves the OpenSSL question because it is now mandatory for
build, which subsequently removes the entropy problem - so let's not
mention it either.
2022-06-14 18:08:15 +02:00
Arаm Sаrgsyаn
6923f7973b Merge branch '3401-race-condition-in-route_connected' into 'main'
Resolve "race condition in route_connected() leads to crash on startup/shutdown"

Closes #3401

See merge request isc-projects/bind9!6429
2022-06-14 15:31:40 +00:00
Aram Sargsyan
4b0e7e41fc Add CHANGES note for [GL #3401] 2022-06-14 14:31:55 +00:00
Aram Sargsyan
f6e729635f Fix a race condition between shutdown and route_connected()
When shutting down, the interface manager can be destroyed
before the `route_connected()` callback is called, which is
unexpected for the latter and can cause a crash.

Move the interface manager attachment code from the callback
to the place before the callback is registered using
`isc_nm_routeconnect()` function, which will make sure that
the interface manager will live at least until the callback
is called.

Make sure to detach the interface manager if the
`isc_nm_routeconnect()` function is not implemented, or when
the callback is called with a result value which differs from
`ISC_R_SUCCESS`.
2022-06-14 14:31:24 +00:00
Aram Sargsyan
1d93fe973b Do not use the interface manager until it is ready
The `ns_interfacemgr_create()` function, when calling
`isc_nm_routeconnect()`, uses the newly created `ns_interfacemgr_t`
instance before initializing its reference count and the magic value.

Defer the `isc_nm_routeconnect()` call until the initializations
are complete.
2022-06-14 14:31:24 +00:00
Arаm Sаrgsyаn
8f4b11773b Merge branch '3207-dig-nssearch-crash-fix' into 'main'
Resolve "dig +nssearch org crashes when network is unreachable"

Closes #3207

See merge request isc-projects/bind9!6343
2022-06-14 14:03:04 +00:00
Aram Sargsyan
0450c9bd32 Add CHANGES note for [GL #3207] 2022-06-14 13:20:55 +00:00
Aram Sargsyan
1290863c22 Fix a crash in dig NS search mode
In special NS search mode, after the initial lookup, dig starts the
followup lookup with discovered NS servers in the queries list. If one
of those queries then fail, dig, as usual, tries to start the next query
in the list, which results in a crash, because the NS search mode is
special in a way that the queries are running in parallel, so the next
query is usually already started.

Apply some special logic in `recv_done()` function to deal with the
described situation when handling the query result for the NS search
mode. Particularly, print a warning message for the failed query,
and do not try to start the next query in the list. Also, set a non-zero
exit code if all the queries in the followup lookup fail.
2022-06-14 13:20:25 +00:00
Michal Nowak
99baab6268 Merge branch '3138-capture-scripts-to-coverity-scan' into 'main'
Capture scripts for Coverity Scan analysis

Closes #3138

See merge request isc-projects/bind9!5984
2022-06-14 12:18:12 +00:00
Michal Nowak
b4a2674d98 Capture scripts for Coverity Scan analysis
With the recent Coverity Scan 2021.12 version, Python 3 scripts are
being analyzed in addition to C files. The --fs-capture-search option
scripts for Coverity Scan analysis should be added to leverage this
feature.
2022-06-14 14:05:13 +02:00
Michal Nowak
1333bdf67e Download Coverity Scan analysis tool to /tmp
Downloading and unpacking Coverity Scan analysis tool tarball
(cov-analysis-linux64.tgz) to $CI_PROJECT_DIR interferes with the
execution of the analysis tool when the --fs-capture-search option is
used because the tool starts to analyze some of its Javascript files.
(There's the --fs-capture-search-exclude-regex <path> option, but I
failed to find a way to make it work.)
2022-06-14 14:05:13 +02:00
Michal Nowak
c966304e90 Drop coverity cache feature
The coverity CI job cache feature is used to ensure that the 1 GB
cov-analysis-linux64.tgz file is being cached on GitLab CI runner, where
it was downloaded in the past. This feature does not seem to work
anymore; given that the proper solution to creating distributed cache is
complicated, better to drop the feature altogether.
2022-06-14 14:05:12 +02:00
Michał Kępień
8b48172d82 Merge branch '3168-5-tidy-check_recursionquota' into 'main'
[5/5] Tidy check_recursionquota()

Closes #3168

See merge request isc-projects/bind9!5886
2022-06-14 11:44:56 +00:00
Michał Kępień
296587903a Merge branch '3168-4-refactor-recursion-quota-handling' into 'main'
[4/5] Refactor recursion quota handling

See merge request isc-projects/bind9!5885
2022-06-14 11:43:14 +00:00
Michał Kępień
a4e92e3d0c Merge branch '3168-3-separate-all-recursions' into 'main'
[3/5] Separate all recursions

See merge request isc-projects/bind9!5884
2022-06-14 11:42:21 +00:00
Michał Kępień
7611b05e4f Merge branch '3168-2-untangle-prefetch-and-rpz' into 'main'
[2/5] Untangle prefetch and RPZ code

See merge request isc-projects/bind9!5883
2022-06-14 11:41:09 +00:00
Michał Kępień
b7285e4fee Merge branch '3168-1-add-recursionquota-helper-functions' into 'main'
[1/5] Add recursionquota_*() helper functions

See merge request isc-projects/bind9!5882
2022-06-14 11:39:15 +00:00
Michał Kępień
0d7e5d513c Assert on unknown isc_quota_attach() return values
The only values that the isc_quota_attach() function (called from
check_recursionquota() via recursionquotatype_attach_soft()) can
currently return are: ISC_R_SUCCESS, ISC_R_SOFTQUOTA, and ISC_R_QUOTA.
Instead of just propagating any other (unexpected) error up the call
stack, assert immediately, so that if the isc_quota_* API gets updated
in the future to return values currently matching the "default"
statement, check_recursionquota() can be promptly updated to handle such
new return values as desired.
2022-06-14 13:13:32 +02:00
Michał Kępień
cd8a687e5a Add CHANGES entry for GL #3168 2022-06-14 13:13:32 +02:00
Michał Kępień
95e703121d Ensure ns_query_cancel() handles all recursions
Previously, multiple code paths reused client->query.fetch, so it was
enough for ns_query_cancel() to issue a single call to
dns_resolver_cancelfetch() with that fetch as an argument.  Now, since
each slot in the 'recursions' array can hold a reference to a separate
resolver fetch, ns_query_cancel() needs to handle all of them, so that
all recursion callbacks get a chance to clean up the associated
resources when a query is canceled.
2022-06-14 13:13:32 +02:00