Commit Graph
60 Commits
Author SHA1 Message Date
Michał Kępień 268b4392ba Wait until fstrm_capture is ready
The fstrm_capture utility is started in the background during the
"dnstap" system test.  Consequently, "rndc dnstap-reopen" and similar
commands may be executed before fstrm_capture starts listening on the
Unix domain socket it is configured to receive dnstap data on.  This
results in the dnstap data sent to that socket in the meantime to be
lost; while the fstrm writer thread is able to recover from such a
scenario within a couple of seconds (by reopening the configured dnstap
destination itself), only one write attempt is made for data
successfully queued to the writer thread, so dnstap frames can still be
lost in the process.  This may happen during the "dnstap" system test,
leading to the dnstap output file being empty, which in turn causes the
test to fail.

Fix by waiting until fstrm_capture starts listening on the Unix domain
socket it is configured to use before asking named to reopen the
configured dnstap destination.  Since various fstrm_capture versions log
different messages when the listening socket is set up, wait for a
common string that works for all fstrm_capture versions released to
date.  Add a few extra debug messages indicating test progress and make
the test fail if the expected fstrm_capture log message is not generated
within 10 seconds.

(cherry picked from commit 26d3d97f12)
2023-08-07 14:01:51 +02:00
Michał Kępień c630efb2f1 Capture all fstrm_capture output
The fstrm_capture.out file is overwritten when the fstrm_capture utility
is restarted during the "dnstap" system test.  Use a separate output
file for each fstrm_capture instance to ensure all output produced by
that tool during the "dnstap" system test is preserved for forensic
purposes.

(cherry picked from commit bd2941fc72)
2023-08-07 14:01:51 +02:00
Tom Krizek 0374c27fc5 Check for proper file size output in dnstap test
Previously, the first check silently failed, as 450 is apparently (in
the CI) the minimum output size for the dnstap output, rather than
470 which the test was expecting. Effectively, the check served as a 5
second sleep rather than waiting for the proper file size.

Additionally, check the expected file sizes and fail if expectations
aren't met.

(manually picked from commit 5f809e50b6)

On main, the minimum file size seems to 454 bytes, while on some
platforms in our CI setup for the 9.16 branch, it appears to be 450
instead.
2023-06-26 14:33:43 +02:00
Aram Sargsyan 299c023024 Improve dnstap system test reliability
The dnstap system test fails intermittently, and it appears to be
a timing issue - adding a short delay after running 'fstrm_capture',
and before running 'dnstap -reopen' improves the situation from
50% failures (5 out of 10 times) to 0% failures (0 out of 20 times),
tested locally.

The reason is that 'fstrm_capture' is executed in the background,
and due to OS scheduling and other factors, the listener socket
may not be ready when the following command runs and tells 'named'
to (re)open it.

(cherry picked from commit fa686fcea5)
2023-03-13 11:57:43 +00:00
Tom Krizek 76d88b7f04 Add dnstap prerequisite for dnstap system test
(cherry picked from commit 9846c920c3)
2022-12-02 11:09:28 +01:00
Tom Krizek 172826bfa8 Simplify start/stop helper func in system tests
The system test should never attempt to start or stop any other server
than those that belong to that system test. Therefore, it is not
necessary to specify the system test name in function calls.

Additionally, this makes it possible to run the test inside a
differently named directory, as its name is automatically detected with
the $SYSTESTDIR variable. This enables running the system tests inside a
temporary directory.

Direct use of stop.pl was replaced with a more systematic approach to
use stop_servers helper function.

(cherry picked from commit c100308b7d)
2022-11-25 13:14:45 +01:00
Mark AndrewsandMichał Kępień f271667a1b Check that the UDP destination port is logged via dnstap
(cherry picked from commit 8d8396c3a7)
2022-06-22 13:52:08 +02:00
Tom Krizek b3c7bd1c04 Auto-format Python files with black
This patch is strictly the result of:
$ black $(git ls-files '*.py' '*.py.in')

There have been no manual changes.
2022-06-08 13:34:19 +02:00
Ondřej Surý 2bf7921c7e Update the copyright information in all files in the repository
This commit converts the license handling to adhere to the REUSE
specification.  It specifically:

1. Adds used licnses to LICENSES/ directory

2. Add "isc" template for adding the copyright boilerplate

3. Changes all source files to include copyright and SPDX license
   header, this includes all the C sources, documentation, zone files,
   configuration files.  There are notes in the doc/dev/copyrights file
   on how to add correct headers to the new files.

4. Handle the rest that can't be modified via .reuse/dep5 file.  The
   binary (or otherwise unmodifiable) files could have license places
   next to them in <foo>.license file, but this would lead to cluttered
   repository and most of the files handled in the .reuse/dep5 file are
   system test files.

(cherry picked from commit 58bd26b6cf)
2022-01-11 12:22:09 +01:00
Evan HuntandMichał Kępień 7b2880d191 further tidying of primary/secondary terminology in system tests
this changes most visble uses of master/slave terminology in tests.sh
and most uses of 'type master' or 'type slave' in named.conf files.
files in the checkconf test were not updated in order to confirm that
the old syntax still works. rpzrecurse was also left mostly unchanged
to avoid interference with DNSRPS.

(cherry picked from commit e43b3c1fa1)
2021-01-12 15:21:14 +01:00
Diego Fronza 64ae91c62a Fix dnstap system test on FreeBSD
This commit ensures that dnstap output files captured
by fstrm_capture are properly flushed before any attempt
on reading them with dnstap-read is done.

By reading fstrm-capture source code it was noticed that
signal SIGHUP is used to flush the capture file.
2020-10-20 10:22:50 -03:00
Evan Hunt df698d73f4 update all copyright headers to eliminate the typo 2020-09-14 16:50:58 -07:00
Michał Kępień 894b7a8345 Use "-T maxcachesize=2097152" in all system tests
In order to lower the amount of memory allocated at startup by named
instances used in the BIND system test suite, set the default value of
"max-cache-size" for these to 2 megabytes.  The purpose of this change
is to prevent named instances (or even entire virtual machines) from
getting killed by the operating system on the test host due to excessive
memory use.

Remove all "max-cache-size" statements from named configuration files
used in system tests ("checkconf" notwithstanding) to prevent confusion
as the "-T maxcachesize=..." command line option takes precedence over
configuration files.

(cherry picked from commit dad6572093)
2020-08-31 23:42:38 +02:00
Michał KępieńandMark Andrews 62f631f798 Silence PyYAML warning
Make yaml.load_all() use yaml.SafeLoader to address a warning currently
emitted when bin/tests/system/dnstap/ydump.py is run:

    ydump.py:28: YAMLLoadWarning: calling yaml.load_all() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
      for l in yaml.load_all(f.stdout):
2020-06-30 11:19:47 +10:00
Mark Andrews bf205b00c3 Check that 'rndc dnstap -roll <value>' works
(cherry picked from commit a289a57c7f)
2020-06-29 22:30:01 +00:00
Mark AndrewsandOndřej Surý 2a45b67ee1 Silence: E741 ambiguous variable name 'l'
(cherry picked from commit 189d21fb96)
2020-05-12 14:06:52 +02:00
Michał KępieńandOndřej Surý 299954d006 Make dnstap work reliably with netmgr
The introduction of netmgr doubled the number of threads from which
dnstap data may be logged: previously, it could only happen from within
taskmgr worker threads; with netmgr, it can happen both from taskmgr
worker threads and from network threads.  Since the argument passed to
fstrm_iothr_options_set_num_input_queues() was not updated to reflect
this change, some calls to fstrm_iothr_get_input_queue() can now return
NULL, effectively preventing some dnstap data from being logged.
Whether this bug is triggered or not depends on thread scheduling order
and packet distribution between network threads, but will almost
certainly be triggered on any recursive resolver sooner or later.  Fix
by requesting the correct number of dnstap input queues to be allocated.

(cherry picked from commit 77dc091855)
2020-05-01 16:29:18 +02:00
Ondřej SurýandOndřej Surý 4c4af11e5b Make the python code pylint and flake8 compliant
(cherry picked from commit 1e4ff9d485)
2020-04-15 07:43:35 +02:00
Mark Andrews 9b6df37303 wait for the ./NS lookup to complete 2020-01-23 19:16:05 +00:00
Mark Andrews 4a992c7a18 check that all servers have finished loading before beginging tests 2020-01-23 19:16:05 +00:00
Michał Kępień d8905b7a9c Automatically run clean.sh from run.sh
The first step in all existing setup.sh scripts is to call clean.sh.  To
reduce code duplication and ensure all system tests added in the future
behave consistently with existing ones, invoke clean.sh from run.sh
before calling setup.sh.
2019-12-06 14:11:01 +01:00
Mark Andrews f3d53630c3 add more dnstap/dnstap-output combinations 2019-10-21 11:06:21 +11:00
Evan Hunt 33887dd941 add digdelv +yaml system tests 2019-08-25 16:41:38 -07:00
Mark Andrews a4f38bec6a named-checkconf failed to report dnstap-output missing
from named.conf when dnstap was specified
2019-07-23 05:09:01 +10:00
Michał Kępień 3549abe81d dnstap-read: clear buffer before expanding it
When printing a packet, dnstap-read checks whether its text form takes
up more than the 2048 bytes allocated for the output buffer by default.
If that is the case, the output buffer is automatically expanded, but
the truncated output is left in the buffer, resulting in malformed data
being printed.  Clear the output buffer before expanding it to prevent
this issue from occurring.
2019-06-28 12:43:51 +02:00
Evan Hunt 1f578cdb12 dnstap: if recursion is not available, log queries as AQ instead of CQ 2019-04-11 15:13:13 -07:00
Evan Hunt fc99651d7d disable QNAME minimization in the dnstap test so packet counts will be correct 2018-11-02 11:34:27 -07:00
Evan Hunt ae8026171b update system test to add a check for UPDATE messages 2018-10-03 01:03:56 -07:00
Evan Hunt 0d6f504ba3 specify "dnssec-validation yes" instead of default "auto" 2018-08-23 13:52:57 -04:00
Evan HuntandOndřej Surý a7a2fa296a update system tests so validation won't fail when using IANA key
- all tests with "recursion yes" now also specify "dnssec-validation yes",
  and all tests with "recursion no" also specify "dnssec-validation no".
  this must be maintained in all new tests, or else validation will fail
  when we use local root zones for testing.
- clean.sh has been modified where necessary to remove managed-keys.bind
  and viewname.mkeys files.
2018-05-31 18:22:33 +02:00
Tony FinchandEvan Hunt 286a7b6b9a Allow TTL values when configuring TTLs and time intervals.
Options updated to use ttlvals: max-cache-ttl, max-ncache-ttl,
max-policy-ttl, fstrm-set-reopen-interval, interface-interval, and
min-update-interval.
2018-04-13 11:46:06 -07:00
Ondřej Surý 843d389661 Update license headers to not include years in copyright in all applicable files 2018-02-23 10:12:02 +01:00
Evan Hunt 0c559199bf final cleanup
- add CHANGES note
- update copyrights and license headers
- add -j to the make commands in .gitlab-ci.yml to take
  advantage of parallelization in the gitlab CI process
2018-02-22 22:58:15 -08:00
Evan Hunt c032c54dda parallelize most system tests 2018-02-22 15:29:02 -08:00
Mark Andrews f5d429e7c3 save why ydump.py failed 2017-12-22 16:02:39 +11:00
Evan Hunt 49740fb0f2 [master] add I: when echoing information about failed tests 2017-09-18 14:15:37 -07:00
Tinderbox User 12ed5d34b9 update copyright notice / whitespace 2017-08-04 23:46:19 +00:00
Evan Hunt 61367c604c [master] refactor resquery_response() and related functions
4669.	[func]		Iterative query logic in resolver.c has been
			refactored into smaller functions and commented,
			for improved readability, maintainability and
			testability. [RT #45362]
2017-08-04 16:08:11 -07:00
Evan Hunt 6ce8a05f6c [master] update copyrights that had been missed recently 2017-04-23 17:06:00 -07:00
Evan Hunt f5c39b072c [master] hex output mode for dnstap-read
4594.	[func]		"dnstap-read -x" prints a hex dump of the wire
			format of each logged DNS message. [RT #44816]
2017-04-20 20:22:19 -07:00
Evan Hunt 6d19d975c6 [master] python 3 compatibility
4591.	[port]		Addressed some python 3 compatibility issues.
			Thanks to Ville Skytta. [RT #44955] [RT #44956]
2017-04-20 17:30:35 -07:00
Evan Hunt 612b2e2c0d [master] timestamp suffixes for log files
4579.	[func]		Logging channels and dnstap output files can now
			be configured with a "suffix" option, set to
			either "increment" or "timestamp", indicating
			whether to use incrementing numbers or timestamps
			as the file suffix when rolling over a log file.
			[RT #42838]
2017-03-08 23:20:40 -08:00
Tinderbox User f929677ed8 update copyright notice / whitespace 2017-02-08 23:45:32 +00:00
Evan Hunt d8339932af [master] fix memory leak from dnstap-output parsing 2017-02-06 22:13:51 -08:00
Evan Hunt c4e4bd6a09 [master] dnstap size and versions options
4572.	[func]		The "dnstap-output" option can now take "size" and
			"versions" parameters to indicate the maximum size
			a dnstap log file can grow before rolling to a new
			file, and how many old files to retain. [RT #44502]
2017-02-06 16:34:58 -08:00
Mark Andrews 813ff2d277 remove #!/usr/bin/python 2016-12-29 23:37:53 +11:00
Mark Andrews da40ddaeca add copyright notice 2016-12-29 23:10:37 +11:00
wpk e910d18007 4545. [func] Make dnstap-read output more functionally usable.
[RT #43642]

4544.	[func]		Add message/payload size to dnstap-read YAML output.
			[RT #43622]
2016-12-28 11:57:28 +01:00
Mark Andrews ed7097fc00 4463. [bug] The dnstap system test failed on some systems.
[RT #43129]
2016-09-08 11:40:24 +10:00
Tinderbox User 4c60000629 update copyright notice / whitespace 2016-08-30 23:45:30 +00:00