Commit Graph

351 Commits

Author SHA1 Message Date
Tinderbox User
152ff84f79 prep 9.16.2 2020-04-16 23:07:40 +02:00
Ondřej Surý
8b84fb4f42 Disable MSB8028 warning
All our MSVS Project files share the same intermediate directory.  We
know that this doesn't cause any problems, so we can just disable the
detection in the project files.

Example of the warning:

  warning MSB8028: The intermediate directory (.\Release\) contains files shared from another project (dnssectool.vcxproj).  This can lead to incorrect clean and rebuild behavior.

(cherry picked from commit b6c2012d93)
2020-04-16 07:56:38 +02:00
Ondřej Surý
820b9ba38a Set WarningLevel to Level1 for Release, treat warnings as errors
Our vcxproj files set the WarningLevel to Level3, which is too verbose
for a code that needs to be portable.  That basically leads to ignoring
all the errors that MSVC produces.  This commits downgrades the
WarningLevel to Level1 and enables treating warnings as errors for
Release builds.  For the Debug builds the WarningLevel got upgraded to
Level4, and treating warnings as errors is explicitly disabled.

We should eventually make the code clean of all MSVC warnings, but it's
a long way to go for Level4, so it's more reasonable to start at Level1.

For reference[1], these are the warning levels as described by MSVC
documentation:

  * /W0 suppresses all warnings. It's equivalent to /w.
  * /W1 displays level 1 (severe) warnings. /W1 is the default setting
    in the command-line compiler.
  * /W2 displays level 1 and level 2 (significant) warnings.
  * /W3 displays level 1, level 2, and level 3 (production quality)
    warnings. /W3 is the default setting in the IDE.
  * /W4 displays level 1, level 2, and level 3 warnings, and all level 4
    (informational) warnings that aren't off by default. We recommend
    that you use this option to provide lint-like warnings. For a new
    project, it may be best to use /W4 in all compilations. This option
    helps ensure the fewest possible hard-to-find code defects.
  * /Wall displays all warnings displayed by /W4 and all other warnings
    that /W4 doesn't include — for example, warnings that are off by
    default.
  * /WX treats all compiler warnings as errors. For a new project, it
    may be best to use /WX in all compilations; resolving all warnings
    ensures the fewest possible hard-to-find code defects.

1. https://docs.microsoft.com/en-us/cpp/build/reference/compiler-option-warning-level?view=vs-2019

(cherry picked from commit 789d253e3d)
2020-04-16 07:55:40 +02:00
Ondřej Surý
cacaa94350 Fix the statistic counter underflow in ns_client_t
In case of normal fetch, the .recursionquota is attached and
ns_statscounter_recursclients is incremented when the fetch is created.  Then
the .recursionquota is detached and the counter decremented in the
fetch_callback().

In case of prefetch or rpzfetch, the quota is attached, but the counter is not
incremented.  When we reach the soft-quota, the function returns early but don't
detach from the quota, and it gets destroyed during the ns_client_endrequest(),
so no memory was leaked.

But because the ns_statscounter_recursclients is only incremented during the
normal fetch the counter would be incorrectly decremented on two occassions:

1) When we reached the softquota, because the quota was not properly detached
2) When the prefetch or rpzfetch was cancelled mid-flight and the callback
   function was never called.

(cherry picked from commit 78886d4bed)
2020-04-03 20:22:56 +02:00
Witold Kręcicki
365636dbc9 netmgr refactoring: use generic functions when operating on sockets.
tcpdns used transport-specific functions to operate on the outer socket.
Use generic ones instead, and select the proper call in netmgr.c.
Make the missing functions (e.g. isc_nm_read) generic and add type-specific
calls (isc__nm_tcp_read). This is the preparation for netmgr TLS layer.

(cherry picked from commit 5fedd21e16)
2020-04-03 13:44:28 +02:00
Tinderbox User
aed7d77c97 prep 9.16.1
Updated version and CHANGES files with new release number.

Check the API files:
- lib/bind9/api:
  Source code changes, but no interface changes: increment
  LIBREVISION.
- lib/dns/api:
  Function dns_acl_match changed, struct dns_badcache changed,
  function dns_badcache_add changed, function dns_clent_startupdate
  changed, struct dns_compress changed, struct dns_resolver changed,
  rwlock size changed. This means a LIBINTERFACE increment.
- lib/irs/api:
  Source code changes, but no interface changes: increment
  LIBREVISION.
- lib/isc/api:
  The structs isc__networker and isc_nmsocket changed. This means
  increment LIBINTERFACE.  The functions isc_uv_export and
  isc_uv_import are removed, so LIBAGE must beq zero.
- lib/isccc/api:
  Source code changes, but no interface changes: increment
  LIBREVISION.
- lib/isccfg/api:
  Source code changes, but no interface changes: increment
  LIBREVISION.
- lib/ns/api:
  Function ns_clientmgr_create, ns_interfacemgr_create, and
  structs ns_clientmgr, ns_interface, ns_interfacemgr changed:
  increment LIBINTERFACE.

No need to update README or release notes.

Updated CHANGES: Add GitLab MR reference to entry 5357. Remove
merge conflict gone wrong ("max-ixfr-ratio" is not in 9.16).

Add /util/check-make-install.in to .gitattributes.
2020-03-20 11:47:01 +01:00
Mark Andrews
af14091f65 Refactor the isc_log API so it cannot fail on memory failures
The isc_mem API now crashes on memory allocation failure, and this is
the next commit in series to cleanup the code that could fail before,
but cannot fail now, e.g. isc_result_t return type has been changed to
void for the isc_log API functions that could only return ISC_R_SUCCESS.

(cherry picked from commit 0b793166d0)
2020-03-18 11:44:18 +01:00
Ondřej Surý
f3c2274479 Use the new sorting rules to regroup #include headers 2020-03-11 08:55:12 +00:00
Witold Kręcicki
aec3dd28d6 Destroy query in killoldestquery under a lock.
Fixes a race between ns_client_killoldestquery and ns_client_endrequest -
killoldestquery takes a client from `recursing` list while endrequest
destroys client object, then killoldestquery works on a destroyed client
object. Prevent it by holding reclist lock while cancelling query.

(cherry picked from commit df3dbdff81)
2020-03-05 23:55:42 +00:00
Michał Kępień
69974ea278 Add ZLIB_LIBS to ISCLIBS
When --with-zlib is passed to ./configure (or when the latter
autodetects zlib's presence), libisc uses certain zlib functions and
thus libisc's users should be linked against zlib in that case.  Adjust
Makefile variables appropriately to prevent shared build failures caused
by underlinking.

(cherry picked from commit fc967ba092)
2020-02-28 15:22:59 +01:00
Evan Hunt
d794d85ce1 comments
(cherry picked from commit 0b76d8a490)
2020-02-28 10:05:25 +01:00
Witold Kręcicki
e11a8e832e Don't define NS_CLIENT_TRACE by default
(cherry picked from commit 4b6a064972)
2020-02-28 10:05:25 +01:00
Witold Kręcicki
0edf2e290b Remove some stale fields from ns_client_t; make sendbuf allocated on heap
(cherry picked from commit 8c6c07286f)
2020-02-28 10:05:25 +01:00
Witold Kręcicki
d0b8c6a378 Don't check if the client is on recursing list (requires locking) if it's not RECURSING
(cherry picked from commit 938b61405b)
2020-02-28 10:05:25 +01:00
Witold Kręcicki
c8dd7e07cd Don't issue ns_client_endrequest on a NS_CLIENTSTATE_READY client.
Fix a potential assertion failure on shutdown in ns__client_endrequest.
Scenario:
1. We are shutting down, interface->clientmgr is gone.
2. We receive a packet, it gets through ns__client_request
3. mgr == NULL, return
4. isc_nmhandle_detach calls ns_client_reset_cb
5. ns_client_reset_cb calls ns_client_endrequest
6. INSIST(client->state == NS_CLIENTSTATE_WORKING ||
          client->state == NS_CLIENTSTATE_RECURSING) is not met
   - we haven't started processing this packet so
   client->state == NS_CLIENTSTATE_READY.
As a solution - don't do anything in ns_client_reset_cb if the client
is still in READY state.

(cherry picked from commit b0888ff039)
2020-02-27 12:08:02 +00:00
Evan Hunt
f0972d5bbd some Fossies-reported spelling errors were accidentally left unfixed
(cherry picked from commit a2aa29ed4f)
2020-02-21 14:15:18 -08:00
Evan Hunt
11a0d771f9 fix spelling errors reported by Fossies.
(cherry picked from commit ba0313e649)
2020-02-21 07:05:31 +00:00
Witold Kręcicki
1b9df6779d Use thread-friendly mctxpool and taskpool in ns_client.
Make ns_client mctxpool more thread-friendly by sharding it by
netmgr threadid, use task pool also sharded by thread id to avoid
lock contention.
2020-02-18 11:57:46 +01:00
Evan Hunt
25496b2299 adjust the clang-format penalties to reduce string breaking
this corrects some style glitches such as:
```
        long_function_call(arg, arg2, arg3, arg4, arg5, "str"
                                                        "ing");
```
...by adjusting the penalties for breaking strings and call
parameter lists.

(cherry picked from commit 0002377dca)
2020-02-17 14:43:46 -08:00
Ondřej Surý
829b461c54 Merge branch '46-enforce-clang-format-rules' into 'master'
Start enforcing the clang-format rules on changed files

Closes #46

See merge request isc-projects/bind9!3063

(cherry picked from commit a04cdde45d)

d2b5853b Start enforcing the clang-format rules on changed files
618947c6 Switch AlwaysBreakAfterReturnType from TopLevelDefinitions to All
654927c8 Add separate .clang-format files for headers
5777c44a Reformat using the new rules
60d29f69 Don't enforce copyrights on .clang-format
2020-02-14 08:45:59 +00:00
Ondřej Surý
cdef20bb66 Merge branch 'each-style-tweak' into 'master'
adjust clang-format options to get closer to ISC style

See merge request isc-projects/bind9!3061

(cherry picked from commit d3b49b6675)

0255a974 revise .clang-format and add a C formatting script in util
e851ed0b apply the modified style
2020-02-14 05:35:29 +00:00
Ondřej Surý
2e55baddd8 Merge branch '46-add-curly-braces' into 'master'
Add curly braces using uncrustify and then reformat with clang-format back

Closes #46

See merge request isc-projects/bind9!3057

(cherry picked from commit 67b68e06ad)

36c6105e Use coccinelle to add braces to nested single line statement
d14bb713 Add copy of run-clang-tidy that can fixup the filepaths
056e133c Use clang-tidy to add curly braces around one-line statements
2020-02-13 21:28:35 +00:00
Tinderbox User
29696e495f prep v9.16.0 2020-02-12 20:03:16 +00:00
Ondřej Surý
c931d8e417 Merge branch '46-just-use-clang-format-to-reformat-sources' into 'master'
Reformat source code with clang-format

Closes #46

See merge request isc-projects/bind9!2156

(cherry picked from commit 7099e79a9b)

4c3b063e Import Linux kernel .clang-format with small modifications
f50b1e06 Use clang-format to reformat the source files
11341c76 Update the definition files for Windows
df6c1f76 Remove tkey_test (which is no-op anyway)
2020-02-12 14:51:18 +00:00
Ondřej Surý
bc1d4c9cb4 Clear the pointer to destroyed object early using the semantic patch
Also disable the semantic patch as the code needs tweaks here and there because
some destroy functions might not destroy the object and return early if the
object is still in use.
2020-02-09 18:00:17 -08:00
Ondřej Surý
41fe9b7a14 Formatting issues found by local coccinelle run 2020-02-08 03:12:09 -08:00
Ondřej Surý
c73e5866c4 Refactor the isc_buffer_allocate() usage using the semantic patch
The isc_buffer_allocate() function now cannot fail with ISC_R_MEMORY.
This commit removes all the checks on the return code using the semantic
patch from previous commit, as isc_buffer_allocate() now returns void.
2020-02-03 08:29:00 +01:00
Witold Kręcicki
b5cfc1c056 Get rid of the remains of -Tdelay option 2020-01-22 12:16:59 +01:00
Witold Kręcicki
8d6dc8613a clean up some handle/client reference counting errors in error cases.
We weren't consistent about who should unreference the handle in
case of network error. Make it consistent so that it's always the
client code responsibility to unreference the handle - either
in the callback or right away if send function failed and the callback
will never be called.
2020-01-20 22:28:36 +01:00
Witold Kręcicki
dcc0835a3a cleanup properly if we fail to initialize ns_client structure
If taskmgr is shutting down ns_client_setup will fail to create
a task for the newly created client, we weren't cleaning up already
created/attached things (memory context, server, clientmgr).
2020-01-20 22:28:36 +01:00
Tinderbox User
05f2241fcb prep 9.15.8 2020-01-16 08:01:20 +00:00
Witold Kręcicki
eda4300bbb netmgr: have a single source of truth for tcpdns callback
We pass interface as an opaque argument to tcpdns listening socket.
If we stop listening on an interface but still have in-flight connections
the opaque 'interface' is not properly reference counted, and we might
hit a dead memory. We put just a single source of truth in a listening
socket and make the child sockets use that instead of copying the
value from listening socket. We clean the callback when we stop listening.
2020-01-15 17:22:13 +01:00
Evan Hunt
7fdf40770f remove all code that uses non-DS trust anchors
as initial-key and static-key trust anchors will now be stored as a
DS rrset, code referencing keynodes storing DNSKEY trust anchors will
no longer be reached.
2020-01-14 09:24:13 -08:00
Ondřej Surý
6afa99362a Remove duplicate INSIST checks for isc_refcount API
This commits removes superfluous checks when using the isc_refcount API.

Examples of superfluous checks:

1. The isc_refcount_decrement function ensures there was not underflow,
   so this check is superfluous:

    INSIST(isc_refcount_decrement(&r) > 0);

2 .The isc_refcount_destroy() includes check whether the counter
   is zero, therefore this is superfluous:

    INSIST(isc_refcount_decrement(&r) == 1 && isc_refcount_destroy(&r));
2020-01-14 13:12:13 +01:00
Ondřej Surý
7c3e342935 Use isc_refcount_increment0() where appropriate 2020-01-14 13:12:13 +01:00
Ondřej Surý
fbf9856f43 Add isc_refcount_destroy() as appropriate 2020-01-14 13:12:13 +01:00
Michał Kępień
ea7bddb4ca Bind to random port numbers in unit tests
Some unit tests need various managers to be created before they are run.
The interface manager spawned during libns tests listens on a fixed port
number, which causes intermittent issues when multiple tests using an
interface manager are run concurrently.  Make the interface manager
listen on a randomized port number to greatly reduce the risk of
multiple unit tests using the same port concurrently.
2020-01-09 09:32:25 +00:00
Tinderbox User
e088272172 prep 9.15.7 2019-12-12 23:59:39 +00:00
Diego Fronza
ed9853e739 Fix tcp-highwater stats updating
After the network manager rewrite, tcp-higwater stats was only being
updated when a valid DNS query was received over tcp.

It turns out tcp-quota is updated right after a tcp connection is
accepted, before any data is read, so in the event that some client
connect but don't send a valid query, it wouldn't be taken into
account to update tcp-highwater stats, that is wrong.

This commit fix tcp-highwater to update its stats whenever a tcp connection
is established, independent of what happens after (timeout/invalid
request, etc).
2019-12-12 11:23:10 -08:00
Diego Fronza
ead7b3dc53 Fix tcp-highwater initial value
During BIND startup it scans for network interfaces available, in this
process it ensures that for every interface it will bind and listen to,
at least one socket will be always available accepting connections on
that interface, this way avoiding some DOS attacks that could exploit
tcp quota on some interface and make others unavailable.

In the previous network implementation this initial "reserved" tcp-quota
used by BIND was already been added to the tcp-highwater stats, but with
the new network code it was necesary to add this workaround to ensure
tcp-highwater stats reflect the tcp-quota used by BIND after startup.
2019-12-12 11:23:10 -08:00
Witold Kręcicki
35679aef9b unittest: Allow for 32 (not 16) mock nmhandles in ns tests 2019-12-09 21:44:04 +01:00
Witold Kręcicki
b804d3a395 always return true in ns_interfacemgr_listeningon if interfacemgr is shutting down
to avoid deadlocks on shutdown.
2019-12-09 21:44:04 +01:00
Ondřej Surý
edd97cddc1 Refactor dns_name_dup() usage using the semantic patch 2019-11-29 14:00:37 +01:00
Evan Hunt
715afa9c57 add a stats counter for clients dropped due to recursive-clients limit 2019-11-26 17:55:06 +00:00
Witold Kręcicki
37354ee225 netmgr: fix TCP backlog and client quota count
- add support for TCP backlog, using the value provided by config.
 - don't attach to TCP client quota for listening sockets, only
   connected sockets.
2019-11-22 16:46:32 -08:00
Evan Hunt
199bd6b623 netmgr: make TCP timeouts configurable
- restore support for tcp-initial-timeout, tcp-idle-timeout,
  tcp-keepalive-timeout and tcp-advertised-timeout configuration
  options, which were ineffective previously.
2019-11-22 16:46:31 -08:00
Tinderbox User
767a2aef43 prep 9.15.6 2019-11-17 18:59:41 -08:00
Evan Hunt
123ee350dc place a limit on pipelined queries that can be processed simultaneously
when the TCPDNS_CLIENTS_PER_CONN limit has been exceeded for a TCP
DNS connection, switch to sequential mode to ensure that memory cannot
be exhausted by too many simultaneous queries.
2019-11-17 18:59:39 -08:00
Ondřej Surý
e95af30b23 Make lib/ns Thread Sanitizer clean 2019-11-17 17:42:41 -08:00
Mark Andrews
00605058b4 conditionally test based on USE_LIBTOOL or LD_WRAP 2019-11-16 11:46:06 +08:00