The "checkds" system test contains a lot of duplicated code despite
carrying out the same set of actions for every tested scenario
(zone_check() → wait for logs to appear → keystate_check()). Extract
the parts of the code shared between all tests into a new function,
test_checkds(), and use pytest's test parametrization capabilities to
pass distinct sets of test parameters to this new function, in an
attempt to cleanly separate the fixed parts of this system test from the
variable ones. Replace format() calls with f-strings.
(cherry picked from commit aa31a872d0)
The "checkds" system test only uses dns.resolver.Resolver objects to
access their 'nameservers' and 'port' attributes. Instances of the
NamedInstance class also expose that information via their attributes,
so only pass NamedInstance objects around instead of needlessly
depending on dns.resolver.Resolver.
(cherry picked from commit cf338a7ca3)
Make log file watching in Python-based system tests consistent by
employing the helper Python classes designed for that purpose. Drop the
custom code currently used.
(cherry picked from commit 2c35b839a5)
All changes in this commit were automated using the command:
shfmt -w -i 2 -ci -bn . $(find . -name "*.sh.in")
By default, only *.sh and files without extension are checked, so
*.sh.in files have to be added additionally. (See mvdan/sh#944)
(manually replayed commit 4cb8b13987)
The old name "common" clashes with the convention of system test
directory naming. It appears as a system test directory, but it only
contains helper files.
To reduce confusion and to allow automatic detection of issues with
possibly missing test files, rename the helper directory to "_common".
The leading underscore indicates the directory is different and the its
name can no longer be confused with regular system test directories.
(cherry picked from commit 168dba163c)
the default value of dnssec-validation is 'auto', which causes
a server to send a key refresh query to the root zone when starting
up. this is undesirable behavior in system tests, so this commit
sets dnssec-validation to either 'yes' or 'no' in all tests where
it had not previously been set.
this change had the mostly-harmless side effect of changing the cached
trust level of unvalidated answer data from 'answer' to 'authanswer',
which caused a few test cases in which dumped cache data was examined in
the serve-stale system test to fail. those test cases have now been
updated to expect 'authanswer'.
(cherry picked from commit 0b09ee8cdc)
Add a test case for a server that uses a resolver as an parental-agent.
We need two root servers, ns1 and ns10, one that delegates to the
'checkds' tld with the DS published (ns2), and one that delegates to
the 'checkds' tld with the DS removed (ns5). Both root zones are
being setup in the 'ns1/setup.sh' script.
We also need two resolvers, ns3 and ns8, that use different root hints
(one uses ns1 address as a hint, the other uses ns10).
Then add the checks to test_checkds.py is similar to the existing tests.
Update 'types' because for zones that have the DS withdrawn (or to be
withdrawn), the CDS and CDNSKEY records should not be published and
thus should not be in the NSEC bitmap.
(cherry picked from commit 0b9a9f9955)
It is better to use consistent file names to avoid issue with sorting
etc.
Using underscore in filenames as opposed to dash was chosen because it
seems more common in pytest/python to use underscore for filenames.
Also rename the bin/tests/system/timeouts/tests-tcp.py file to
bin/tests/system/timeouts/tests_tcp_timeouts.py to avoid pytest name
collision (there can't be two files named tests_tcp.py).
(cherry picked from commit f6409ee6ac)
The bad2-dswithdrawn.checkds tests were missing call to the
zone_checks() contributing to intermittent timing failures of the
checkds system test.
(cherry picked from commit 718831bfcc)
We no longer accept copying DNSSEC records from the raw zone to
the secure zone, so update the kasp system test that relies on this
accordingly.
Also add more debugging and store the dnssec-verify results in a file.
(cherry picked from commit 57ea9e08c6)
pylint 2.14.2 reports the following warnings:
bin/tests/system/checkds/tests-checkds.py:265:0: W1404: Implicit string concatenation found in call (implicit-str-concat)
bin/tests/system/checkds/tests-checkds.py:273:0: W1404: Implicit string concatenation found in call (implicit-str-concat)
(cherry picked from commit 831ac8add1)
Ensure all "import dns.*" statements are always placed after
pytest.importorskip('dns') calls, in order to allow the latter to
fulfill their purpose. Explicitly import all dnspython modules used by
each dnspython-based test to avoid relying on nested imports. Replace
function-scoped imports with global imports to reduce code duplication.
(cherry picked from commit 49312d6bb2)
The intended purpose of the @pytest.mark.dnspython{,2} decorators was to
cause dnspython-based tests to be skipped if dnspython is not available
(or not recent enough). However, a number of system tests employing
those decorators contain global "import dns.resolver" statements which
trigger ImportError exceptions during test initialization if dnspython
is not available. In other words, the @pytest.mark.dnspython{,2}
decorators serve no useful purpose.
Currently, whenever a Python-based test requires dnspython, that
requirement applies to all tests in a given *.py file. Given that,
employ global pytest.importorskip() calls to ensure dnspython-based
parts of various system tests are skipped when dnspython is not
available. Remove all occurrences of the @pytest.mark.dnspython{,2}
decorators (and all associated code) to prevent confusion.
(cherry picked from commit 05c97f2329)
Most Python-based system tests need to know which ports were assigned to
a given test by bin/tests/system/get_ports.sh. This is currently
handled by inspecting the values of various environment variables (set
by bin/tests/system/run.sh) and passing the port numbers to Python
scripts via pytest fixtures. However, this glue code has so far been
copy-pasted into each system test using it, rather than reused.
Since pytest also looks for conftest.py files in parent directories,
move commonly used fixtures to bin/tests/system/conftest.py. Set the
scope of all the moved fixtures to "session" as their return values are
only based on environment variables, so there is no point in recreating
them for every test requesting them. Adjust test code accordingly.
(cherry picked from commit 53ef8835c1)
- Removed all code that only runs under CYGWIN, and made all
code that doesn't run under CYGWIN non-optional.
- Removed the $TP variable which was used to add optional
trailing dots to filenames; they're no longer optional.
- Removed references to pssuspend and dos2unix.
- No need to use environment variables for diff and kill.
- Removed uses of "tr -d '\r'"; this was a workaround for
a cygwin regex bug that is no longer needed.
(cherry picked from commit 1d706f328c)
This commit converts the license handling to adhere to the REUSE
specification. It specifically:
1. Adds used licnses to LICENSES/ directory
2. Add "isc" template for adding the copyright boilerplate
3. Changes all source files to include copyright and SPDX license
header, this includes all the C sources, documentation, zone files,
configuration files. There are notes in the doc/dev/copyrights file
on how to add correct headers to the new files.
4. Handle the rest that can't be modified via .reuse/dep5 file. The
binary (or otherwise unmodifiable) files could have license places
next to them in <foo>.license file, but this would lead to cluttered
repository and most of the files handled in the .reuse/dep5 file are
system test files.
Address the following warnings reported by PyLint 2.10.2:
************* Module tests-checkds
bin/tests/system/checkds/tests-checkds.py:70:9: W1514: Using open without explicitly specifying an encoding (unspecified-encoding)
bin/tests/system/checkds/tests-checkds.py:120:13: W1514: Using open without explicitly specifying an encoding (unspecified-encoding)
bin/tests/system/checkds/tests-checkds.py:206:17: W1514: Using open without explicitly specifying an encoding (unspecified-encoding)
************* Module yamlget
bin/tests/system/digdelv/yamlget.py:22:5: W1514: Using open without explicitly specifying an encoding (unspecified-encoding)
************* Module stress_http_quota
bin/tests/system/doth/stress_http_quota.py:131:13: W1514: Using open without explicitly specifying an encoding (unspecified-encoding)
************* Module tests-rpz-passthru-logging
bin/tests/system/rpzextra/tests-rpz-passthru-logging.py:40:9: W1514: Using open without explicitly specifying an encoding (unspecified-encoding)
bin/tests/system/rpzextra/tests-rpz-passthru-logging.py:44:9: W1514: Using open without explicitly specifying an encoding (unspecified-encoding)
The function 'private_type_record()' is now used in multiple system
setup scripts and should be moved to the common configuration script
conf.sh.common.
Add a Pytest based system test for the 'checkds' feature. There is
one nameserver (ns9, because it should be started the latest) that
has configured several zones with dnssec-policy. The zones are set
in such a state that they are waiting for DS publication or DS
withdrawal.
Then several other name servers act as parent servers that either have
the DS for these published, or not. Also one server in the mix is
to test a badly configured parental-agent.
There are tests for DS publication, DS publication error handling,
DS withdrawal and DS withdrawal error handling.
The tests ensures that the zone is DNSSEC valid, and that the
DSPublish/DSRemoved key metadata is set (or not in case of the error
handling).
It does not test if the rollover continues, this is already tested in
the kasp system test (that uses 'rndc -dnssec checkds' to set the
DSPublish/DSRemoved key metadata).
With the introduction of dnssec-policy, the aforementioned tools were
either rendered obsolete, or they will be replaced with dnssec-policy
based tools. Remove the tools and the requirement to have Python
installed. Python 3 is still being used for tests, so keep the autoconf
test, but make it much simpler.
The first step in all existing setup.sh scripts is to call clean.sh. To
reduce code duplication and ensure all system tests added in the future
behave consistently with existing ones, invoke clean.sh from run.sh
before calling setup.sh.
This changes the behaviour so that it explicitly lists DS records that
are present in the parent but do not have keys in the child. Any
inconsistency is reported as an error, which is somewhat stricter than
before.
This is for conformance with the DS/CDS algorithm requirements in
https://tools.ietf.org/html/draft-ietf-dnsop-algorithm-update
- removed a few remaing places where output wasn't being passed
through echo_i or cat_i
- added a "digcomp" function to conf.sh.in to send digcomp.pl output
through cat_i and return the correct exit value
- set SYSTESTDIR when calling echo_i from nsX directories, so that
the test name will always be printed correctly
- fixed a test name typo in conf.sh.in
- add CHANGES note
- update copyrights and license headers
- add -j to the make commands in .gitlab-ci.yml to take
advantage of parallelization in the gitlab CI process
Conflicts:
bin/tests/system/conf.sh.in
lib/dns/win32/libdns.def.in
lib/isc/win32/file.c
The merge also needed to update files in legacy and tcp system tests
(newly introduced in master after branch was created) to introduce use
of lockfile.
