The current script used ephemeral port range which clashed with the
ports used by the tools (dig, ...), and the range always started with
the first port and there was 100 ports allocated for each system test.
In this commit, the first port has been randomized, the get_ports.sh
script outputs the variables (the output has to be eval'ed from run.sh)
and there's less waste in the port range.
There are several improvements over the default/previous behaviour of
the test log driver and log compiler:
* The system-test-driver.sh was dropped (it was used incorrectly)
* The run.sh script is now both log compiler and cli script to run
individual tests
* The custom-test-driver was added as extended version of the automake
test-driver with capability to tee the test output to stdout when
`--verbose yes` is passed to it (you can use LOG_DRIVER_FLAGS to
add the option by default)
* Makefile.am has been extended to honor V=1 for the system tests
test-driver (e.g. V=1 adds `--verbose yes` to AM_LOG_DRIVER_FLAGS)
The bin/tests/wire_test helper program is currently not included in any
Makefile.am file. Move its source code to bin/tests/system and build it
along other helper tools when dnstap support is requested as the
"dnstap" system test needs this tool in order to pass.
The libirs contained own re-implementations of the getaddrinfo,
getnameinfo and gai_strerror + irs_context and irs_dnsconf API that was
unused anywhere in the BIND 9.
Keep just the irs_resonf API that is being extensively used to parse
/etc/resolv.conf by several of BIND 9 tools.
The 'ephemeral' database implementation was used to provide a
lightweight database implemenation that doesn't cache results, and the
only place where it was really use is "samples" because delv is
overriding this to use "rbtdb" instead. Otherwise it was completely
unused.
* The 'ephemeral' cache DB (ecdb) implementation. An ecdb just provides
* temporary storage for ongoing name resolution with the common DB interfaces.
* It actually doesn't cache anything. The implementation expects any stored
* data is released within a short period, and does not care about the
* scalability in terms of the number of nodes.
The rewrite of BIND 9 build system is a large work and cannot be reasonable
split into separate merge requests. Addition of the automake has a positive
effect on the readability and maintainability of the build system as it is more
declarative, it allows conditional and we are able to drop all of the custom
make code that BIND 9 developed over the years to overcome the deficiencies of
autoconf + custom Makefile.in files.
This squashed commit contains following changes:
- conversion (or rather fresh rewrite) of all Makefile.in files to Makefile.am
by using automake
- the libtool is now properly integrated with automake (the way we used it
was rather hackish as the only official way how to use libtool is via
automake
- the dynamic module loading was rewritten from a custom patchwork to libtool's
libltdl (which includes the patchwork to support module loading on different
systems internally)
- conversion of the unit test executor from kyua to automake parallel driver
- conversion of the system test executor from custom make/shell to automake
parallel driver
- The GSSAPI has been refactored, the custom SPNEGO on the basis that
all major KRB5/GSSAPI (mit-krb5, heimdal and Windows) implementations
support SPNEGO mechanism.
- The various defunct tests from bin/tests have been removed:
bin/tests/optional and bin/tests/pkcs11
- The text files generated from the MD files have been removed, the
MarkDown has been designed to be readable by both humans and computers
- The xsl header is now generated by a simple sed command instead of
perl helper
- The <irs/platform.h> header has been removed
- cleanups of configure.ac script to make it more simpler, addition of multiple
macros (there's still work to be done though)
- the tarball can now be prepared with `make dist`
- the system tests are partially able to run in oot build
Here's a list of unfinished work that needs to be completed in subsequent merge
requests:
- `make distcheck` doesn't yet work (because of system tests oot run is not yet
finished)
- documentation is not yet built, there's a different merge request with docbook
to sphinx-build rst conversion that needs to be rebased and adapted on top of
the automake
- msvc build is non functional yet and we need to decide whether we will just
cross-compile bind9 using mingw-w64 or fix the msvc build
- contributed dlz modules are not included neither in the autoconf nor automake
With the introduction of dnssec-policy, the aforementioned tools were
either rendered obsolete, or they will be replaced with dnssec-policy
based tools. Remove the tools and the requirement to have Python
installed. Python 3 is still being used for tests, so keep the autoconf
test, but make it much simpler.
Our python code didn't adhere to any coding standard. In this commit, we add
flame8 (https://pypi.org/project/flake8/), and pylint (https://www.pylint.org/).
There's couple of exceptions:
- ans.py scripts are not checked, nor fixed as part of this MR
- pylint's missing-*-docstring and duplicate-code checks have
been disabled via .pylintrc
Both exceptions should be removed in due time.
We introduce a isc_quota_attach_cb function - if ISC_R_QUOTA is returned
at the time the function is called, then a callback will be called when
there's quota available (with quota already attached). The callbacks are
organized as a LIFO queue in the quota structure.
It's needed for TCP client quota - with old networking code we had one
single place where tcp clients quota was processed so we could resume
accepting when the we had spare slots, but it's gone with netmgr - now
we need to notify the listener/accepter that there's quota available so
that it can resume accepting.
Remove unused isc_quota_force() function.
The isc_quote_reserve and isc_quota_release were used only internally
from the quota.c and the tests. We should not expose API we are not
using.
The two "functions" that isc/safe.h declared before were actually simple
defines to matching OpenSSL functions. The downside of the approach was
enforcing all users of the libisc library to explicitly list the include
path to OpenSSL and link with -lcrypto. By hiding the specific
implementation into the private namespace changing the defines into
simple functions, we no longer enforce this. In the long run, this
might also allow us to switch cryptographic library implementation
without affecting the downstream users.
The kasp system test is timing critical. The test passes on all
Linux based machines, but fails frequently on Windows. The test
takes a lot more time on Windows and at the final checks fail
because the expected next key event is too far off. For example:
I:kasp:check next key event for zone step2.algorithm-roll.kasp (570)
I:kasp:error: bad next key event time 20909 for zone \
step2.algorithm-roll.kasp (expect 21600)
I:kasp:failed
This is because the kasp system test calculates the time when the
next key event should occur based on the policy. This assumes that
named is able to do key management within a minute. But starting,
named, doing key management for other zones, and reconfiguring takes
much more time on Windows and thus the next key event on Windows is
much shorter than anticipated.
That this happens is a good thing because this means that the
correct next key event is used, but is not so nice for testing, as
it is hard to determine how much time named needed before finishing
the current key event.
Disable the kasp test on Windows now because it is blocking the
release. We know the cause of these test failures, and it is clear
that this is a fault in the test, not the code. Therefore we feel
comfortable disabling the test right now and work on a fix while
unblocking the release.
ABI checker tools generate HTML and TXT API compatibility reports of
BIND libraries. Comparison is being done between two bind source trees
which hold built BIND.
In the CI one version is the reference version defined by
BIND_BASELINE_VERSION variable, the latter one is the HEAD of branch
under test.
Add a test case for algorithm rollover. This is triggered by
changing the dnssec-policy. A new nameserver ns6 is introduced
for tests related to dnssec-policy changes.
This requires a slight change in check_next_key_event to only
check the last occurrence. Also, change the debug log message in
lib/dns/zone.c to deal with checks when no next scheduled key event
exists (and default to loadkeys interval 3600).
- Define the SLOT environment variable before starting the test. This
variable defaults to 0 and that does not work with SoftHSM 2.
- The system test expects the PIN environment variable to be set to
"1234" while bin/tests/prepare-softhsm2.sh sets it to "0000".
Update bin/tests/prepare-softhsm2.sh so that it sets the PIN to
"1234".
- Move contents of bin/tests/system/pkcs11/prereq.sh to
bin/tests/system/pkcs11/setup.sh as the former was creating a file
called "supported" that was getting removed by the latter before
bin/tests/system/pkcs11/tests.sh could access it.
- Fix typo in "have_ecx".
- add util/cformat.sh, which runs clang-format on all C files with
the default .clang-format, and on all header files with a slightly
modified version.
- use correct bracing after multi-line control statements
- stop aligning variable declarations to avoid problems with pointer
alignment, but retain aligned declarations in header files so that
struct definitions look cleaner.
- static function prototypes in C files can skip the line break after
the return type, but function prototypes in header files still have
the line break.
- don't break-before-brace in function definitions. ISC style calls
for braces on the same line when function parameters fit on a single
line, and a line break if they don't, but clang-format doesn't yet
support that distinction. one-line function definitions are about
four times more common than multi-line, so let's use the option that
deviates less.
- Add quotes before and after zone name when generating "addzone"
input so avoid "unexpected token" errors.
- Use a hex digest for zone filenames when the zone or view name
contains a slash.
- Test with a domain name containing a slash.
- Incidentally added 'catzhash.py' to contrib/scripts to generate
hash labels for catalog zones, as it was needed to write the test.
These functions can be used to pass a uv handle between threads in a
safe manner. The other option is to use uv_pipe and pass the uv_handle
via IPC, which is way more complex. uv_export() and uv_import() functions
existed in libuv at some point but were removed later. This code is
based on the original removed code.
The Windows version of the code uses two functions internal to libuv;
a patch for libuv is attached for exporting these functions.
The util/prepare-softhsm2.sh script is useful for initializing a working
SoftHSM environment which can be used by unit tests and system tests.
However, since it is a test-specific script, it does not really belong
in the util/ subdirectory which is mostly pruned during the BIND source
tarball creation process. Move the prepare-softhsm2.sh script to
bin/tests/ so that its location is more appropriate for its purpose and
also so that it does not get removed during the BIND source tarball
creation process, allowing it to be used for setting up test
environments for tarball-based builds.
The first step in all existing setup.sh scripts is to call clean.sh. To
reduce code duplication and ensure all system tests added in the future
behave consistently with existing ones, invoke clean.sh from run.sh
before calling setup.sh.
Since the role of the bin/tests/system/clean.sh script has now been
reduced to calling a given system test's clean.sh script, remove the
former altogether and replace its only use with a direct invocation of
the latter.
The "runtime" system test currently fails on Windows because it waits
for named to log a message indicating successful startup ("running"),
but that never happens since named on Windows fails to open the
configuration file as its path includes control characters.
Instead of putting control characters in directory names, put them in
the value of the -D command line switch passed to named, which is used
for identifying an instance of named in a process listing and whose
value is completely ignored by named, but still logged.
While a similar check using special characters appears to be working
fine on Windows for the time being, modify it in the same way to avoid
potential future problems on other platforms and make the test cleaner.
