Commit Graph
30598 Commits
Author SHA1 Message Date
Michał Kępień 5a4a6b5e91 Add a job creating a release tarball to GitLab CI
Add a GitLab CI job (which is run only if all other jobs in a pipeline
succeed) that builds a BIND release tarball, i.e. fetches the source
tarball from the tarball building job, creates Windows zips, puts
certain parts of BIND documentation into the appropriate places, and
packs it all up into a single tarball whose contents can be subsequently
signed and published.
2019-12-11 12:04:29 +01:00
Michał Kępień 2b1c8c54d1 Add a Windows debug system test job to GitLab CI
Add a system test job for binaries created by Visual Studio in the
"Debug" build configuration to GitLab CI so that they can be tested
along their "Release" counterparts when necessary.
2019-12-11 12:04:29 +01:00
Michał Kępień 12564928a7 Add a Windows debug build job to GitLab CI
Add a Visual Studio build job using the "Debug" build configuration to
GitLab CI without enabling it for every pipeline as it takes about twice
as long to complete as its "Release" counterpart.
2019-12-11 12:04:29 +01:00
Michał Kępień 8d56749046 Create and test BIND source tarballs in GitLab CI
Add a set of jobs to GitLab CI that create a BIND source tarball and
then build and test its contents.  Run those extra jobs only when a tag
is pushed to the Git repository as they are only meant to be sanity
checks of BIND source tarball contents.
2019-12-11 12:04:29 +01:00
Michał Kępień c0be772ebc Include prepare-softhsm2.sh in source tarballs
The util/prepare-softhsm2.sh script is useful for initializing a working
SoftHSM environment which can be used by unit tests and system tests.
However, since it is a test-specific script, it does not really belong
in the util/ subdirectory which is mostly pruned during the BIND source
tarball creation process.  Move the prepare-softhsm2.sh script to
bin/tests/ so that its location is more appropriate for its purpose and
also so that it does not get removed during the BIND source tarball
creation process, allowing it to be used for setting up test
environments for tarball-based builds.
2019-12-11 12:04:29 +01:00
Michał Kępień 925ecb0aae List paths which should be excluded from tarballs
Convert the logic (currently present in the form of "rm -rf" calls in
util/kit.sh) for removing files and directories which are tracked by Git
but redundant in release tarballs into a set of .gitattributes rules
which allow the same effect to be achieved using "git archive".
2019-12-11 12:04:29 +01:00
Michał Kępień eee162257c Merge branch 'michal/fix-the-forward-system-test-on-windows' into 'master'
Fix the "forward" system test on Windows

See merge request isc-projects/bind9!2750
2019-12-11 09:37:35 +00:00
Michał Kępień 075613aea4 Fix the "forward" system test on Windows
Make sure carriage return characters are stripped from sed input to
enable the "forward" system test to pass on Windows.
2019-12-11 09:44:20 +01:00
Mark Andrews 48ece3bb9d Merge branch '1479-_wait_for_rcode-adds-extraneous-query' into 'master'
Resolve "_wait_for_rcode adds extraneous query"

Closes #1479

See merge request isc-projects/bind9!2747
2019-12-11 00:25:07 +00:00
Mark Andrews 0ee0580fc9 consume all arguments we have processed in shift 2019-12-11 00:01:08 +00:00
Mark Andrews 37fe7c5269 Merge branch '1411-threadsanitizer-data-race-resolver-c-2153-in-fctx_query' into 'master'
Resolve "ThreadSanitizer: data race resolver.c:2153 in fctx_query"

Closes #1411

See merge request isc-projects/bind9!2675
2019-12-10 23:53:13 +00:00
Mark Andrews 13aaeaa06f Note bucket lock requirements and move REQUIRE inside locked section. 2019-12-10 22:16:15 +00:00
Mark Andrews 5589748eca lock access to fctx->nqueries 2019-12-10 22:16:15 +00:00
Ondřej Surý 03ed64c251 Merge branch '1473-threadsanitizer-data-race-home-ondrej-projects-bind9-lib-isc-netmgr-netmgr-c-1027-in' into 'master'
Resolve "ThreadSanitizer: data race /home/ondrej/Projects/bind9/lib/isc/netmgr/netmgr.c:1027 in nmhandle_free"

Closes #1473

See merge request isc-projects/bind9!2739
2019-12-10 13:00:17 +00:00
Ondřej Surý 1fa0deb4ea Add isc_refcount_destroy() call to nm_handle_free() 2019-12-10 13:43:18 +01:00
Ondřej Surý 71fe7d3c25 Add isc_refcount_destroy() call to nm_destroy() 2019-12-10 13:43:18 +01:00
Ondřej Surý 3248de7785 Correct the DbC check order in isc__nm_async_tcpchildstop() 2019-12-10 13:43:18 +01:00
Mark Andrews 977f334648 Merge branch '1441-threadsanitizer-lock-order-inversion-potential-deadlock-usr-lib-x86_64-linux-gnu-libtsan-so-0' into 'master'
Resolve "ThreadSanitizer: lock-order-inversion (potential deadlock) (/usr/lib/x86_64-linux-gnu/libtsan.so.0+0x2cf99) in pthread_rwlock_rdlock"

Closes #1441

See merge request isc-projects/bind9!2734
2019-12-10 12:35:20 +00:00
Mark Andrews fd52417f71 address deadlock introduced in cd2469d3cd 2019-12-10 12:08:57 +00:00
Michał Kępień ac564683cf Merge branch '1465-fix-idna-system-test' into 'master'
Fix the "idna" system test

See merge request isc-projects/bind9!2740
2019-12-10 10:57:02 +00:00
Michał Kępień 2ee7ff23ce Only use LC_ALL=C where intended
The LC_ALL=C assignments in the "idna" system test, which were only
meant to affect a certain subset of checks, in fact persist throughout
all the subsequent checks in that system test.  That affects the test's
behavior and is misleading.

When the "VARIABLE=value command ..." syntax is used in a shell script,
in order for the variable assignment to only apply to "command", the
latter must be an external binary; otherwise, the VARIABLE=value
assignment persists for all subsequent commands in a script:

    $ cat foo.sh
    #!/bin/sh

    foo() {
        /bin/sh bar.sh
    }

    BAR="baz0"
    BAR="baz1" /bin/sh bar.sh
    echo "foo: BAR=${BAR}"
    BAR="baz2" foo
    echo "foo: BAR=${BAR}"

    $ cat bar.sh
    #!/bin/sh

    echo "bar: BAR=${BAR}"

    $ /bin/sh foo.sh
    bar: BAR=baz1
    foo: BAR=baz0
    bar: BAR=baz2
    foo: BAR=baz2
    $

Fix by saving the value of LC_ALL before the relevant set of checks in
the "idna" system test, restoring it afterwards, and dropping the
"LC_ALL=C command ..." syntax.
2019-12-10 10:56:19 +01:00
Ondřej Surý dbbfcdc1f7 Merge branch 'ondrej/run-full-pipeline-on-schedule' into 'master'
Run all jobs on scheduled builds (including OpenBSD and Windows)

See merge request isc-projects/bind9!2736
2019-12-10 09:30:26 +00:00
Ondřej Surý 52773e226a Run all jobs on scheduled builds (including OpenBSD and Windows) 2019-12-10 09:29:12 +00:00
Witold Krecicki ec80c7b576 Merge branch '1469-lock-order-inversion-tcp-listening' into 'master'
Fix a potential lock-order-inversion in tcp listening code

Closes #1469

See merge request isc-projects/bind9!2737
2019-12-10 09:21:03 +00:00
Witold Kręcicki ccd44b69e5 Fix a potential lock-order-inversion in tcp listening code 2019-12-10 10:05:15 +01:00
Witold Krecicki 01481dee1c Merge branch 'wpk/tcp-multithreaded' into 'master'
netmgr: make tcp listening multithreaded.

See merge request isc-projects/bind9!2659
2019-12-09 21:20:40 +00:00
Witold Kręcicki 83e54f906d CHANGES entry 2019-12-09 21:44:04 +01:00
Evan HuntandWitold Kręcicki 31b3980ef0 shorten some names
reduce line breaks and general unwieldiness by changing some
function, type, and parameter names.
2019-12-09 21:44:04 +01:00
Evan HuntandWitold Kręcicki 8c0792723d style nits 2019-12-09 21:44:04 +01:00
Witold Kręcicki 35679aef9b unittest: Allow for 32 (not 16) mock nmhandles in ns tests 2019-12-09 21:44:04 +01:00
Witold Kręcicki a34ced776e Remove read callback before detaching from inner socket in tcpdns 2019-12-09 21:44:04 +01:00
Witold Kręcicki 86a847314a Fix a race in socket destruction - we need to remove handle from socket in async close callback or we might race between destruction in the callback and in the original nmhandle_unref 2019-12-09 21:44:04 +01:00
Witold Kręcicki b804d3a395 always return true in ns_interfacemgr_listeningon if interfacemgr is shutting down
to avoid deadlocks on shutdown.
2019-12-09 21:44:04 +01:00
Witold Kręcicki b0779cc429 netmgr: Add more DbC checks for asynchronous calls. 2019-12-09 21:44:04 +01:00
Witold Kręcicki ef2dff5c7a pause and unpause netmgr in isc_nm_destroy to flush all events from worker queues 2019-12-09 21:44:04 +01:00
Evan HuntandWitold Kręcicki c7b86d1cac Style fixes 2019-12-09 21:44:03 +01:00
Witold Kręcicki 3e66b7ba1c Fix a race in tcpdns close with uv_close on timer
stop timers before closing

netmgr: tcpdns_close needs to be asynchronous, it manipulates sock->timer
2019-12-09 21:43:45 +01:00
Witold Kręcicki 23ab349bbd netmgr: fix a race in socket destruction, happening if we close the socket
externally and, at the same time, a timeout timer callback was called.
2019-12-09 21:43:45 +01:00
Witold Kręcicki 0bf74ac792 netmgr:
- make tcp listening IPC pipe name saner
 - put the pipe in /tmp on unices
 - add pid to the pipe name to avoid conflicts between processes
 - fsync directory in which the pipe resides to make sure that the
   child threads will see it and be able to open it
2019-12-09 21:43:45 +01:00
Evan Hunt b05194160b style, comments 2019-12-09 11:15:27 -08:00
Witold KręcickiandEvan Hunt 8c5aaacbef - Add separate priority event queue for events that must be processed
even when worker is paused (e.g. interface reconfiguration). This is
  needed to prevent deadlocks when reconfiguring interfaces - as network
  manager is paused then, but we still need to stop/start listening.

- Proper handling of TCP listen errors in netmgr - bind to the socket first,
  then return the error code.
2019-12-09 11:15:27 -08:00
Witold KręcickiandEvan Hunt 5a65ec0aff Add uv_handle_{get,set}_data functions that's absent in pre-1.19 libuv to make code clearer.
This might be removed when we stop supporting older libuv versions.
2019-12-09 11:15:27 -08:00
Witold KręcickiandEvan Hunt bc5aae1579 netmgr: make tcp listening multithreaded.
When listening for TCP connections we create a socket, bind it
and then pass it over IPC to all threads - which then listen on
in and accept connections. This sounds broken, but it's the
official way of dealing with multithreaded TCP listeners in libuv,
and works on all platforms supported by libuv.
2019-12-09 11:15:27 -08:00
Ondřej Surý 09c2dbffb5 Merge branch '1443-threadsanitizer-data-race-lib-dns-rbtdb-c-1960-in-decrement_reference-2' into 'master'
Resolve "ThreadSanitizer: data race lib/dns/rbtdb.c:1960 in decrement_reference"

Closes #1443

See merge request isc-projects/bind9!2703
2019-12-09 18:48:35 +00:00
Mark AndrewsandOndřej Surý c6efc0e50f Add is_leaf and send_to_prune_tree.
Add is_leaf and send_to_prune_tree to make the logic easier
to understand in cleanup_dead_nodes and decrement_reference.
2019-12-09 17:43:54 +00:00
Mark AndrewsandOndřej Surý 176b23b6cd Testing node->down requires the tree lock to be held.
In decrement_reference only test node->down if the tree lock
is held.  As node->down is not always tested in
decrement_reference we need to test that it is non NULL in
cleanup_dead_nodes prior to removing the node from the rbt
tree.  Additionally it is not always possible to aquire the
node lock and reactivate a node when adding parent nodes.
Reactivate such nodes in cleanup_dead_nodes if required.
2019-12-09 17:43:54 +00:00
Ondřej Surý 23e29b17db Merge branch '1453-the-zero-system-test-timeouts-intermittently' into 'master'
Bail-out early if dig fails to finish successfully or takes too long

Closes #1453

See merge request isc-projects/bind9!2712
2019-12-09 17:41:25 +00:00
Ondřej Surý eb8007a5ba Merge branch '1458-intermittent-failure-in-the-forward-system-test' into 'master'
Resolve "Intermittent failure in the forward system test"

Closes #1458

See merge request isc-projects/bind9!2716
2019-12-09 17:15:53 +00:00
Ondřej SurýandOndřej Surý 2a65a47f39 Bail-out early if dig fails to finish successfully or takes too long
Before, the zero system test could get stuck almost infinitely, because
the first test sends > 300 queries with 5 seconds timeout on each in
each pass.  If named crashed early, it would took the test more than 4
hours to properly timeout.

This commit introduces a "watchdog" on the dig commands running in the
background and failing the test on timeout, failing any test if any dig
command fails to return successfully, and making the tests.sh script
shellcheck clean.
2019-12-09 18:15:18 +01:00
Ondřej Surý fb03edacd8 Wait for named to forward the question before testing the validity 2019-12-09 17:30:37 +01:00