Hold a weak reference to the view so that it can't go away while
nta is performing its lookups. Cancel nta timers once all external
references to the view have gone to prevent them triggering new work.
(cherry picked from commit 0b2555e8cf)
Add a new 'rndc' command 'dnssec -checkds' that allows the user to
signal named that a new DS record has been seen published in the
parent, or that an existing DS record has been withdrawn from the
parent.
Upon the 'checkds' request, 'named' will write out the new state for
the key, updating the 'DSPublish' or 'DSRemoved' timing metadata.
This replaces the "parent-registration-delay" configuration option,
this was unreliable because it was purely time based (if the user
did not actually submit the new DS to the parent for example, this
could result in an invalid DNSSEC state).
Because we cannot rely on the parent registration delay for state
transition, we need to replace it with a different guard. Instead,
if a key wants its DS state to be moved to RUMOURED, the "DSPublish"
time must be set and must not be in the future. If a key wants its
DS state to be moved to UNRETENTIVE, the "DSRemoved" time must be set
and must not be in the future.
By default, with '-checkds' you set the time that the DS has been
published or withdrawn to now, but you can set a different time with
'-when'. If there is only one KSK for the zone, that key has its
DS state moved to RUMOURED. If there are multiple keys for the zone,
specify the right key with '-key'.
(cherry picked from commit 04d8fc0143)
There were several problems with rbt hashtable implementation:
1. Our internal hashing function returns uint64_t value, but it was
silently truncated to unsigned int in dns_name_hash() and
dns_name_fullhash() functions. As the SipHash 2-4 higher bits are
more random, we need to use the upper half of the return value.
2. The hashtable implementation in rbt.c was using modulo to pick the
slot number for the hash table. This has several problems because
modulo is: a) slow, b) oblivious to patterns in the input data. This
could lead to very uneven distribution of the hashed data in the
hashtable. Combined with the single-linked lists we use, it could
really hog-down the lookup and removal of the nodes from the rbt
tree[a]. The Fibonacci Hashing is much better fit for the hashtable
function here. For longer description, read "Fibonacci Hashing: The
Optimization that the World Forgot"[b] or just look at the Linux
kernel. Also this will make Diego very happy :).
3. The hashtable would rehash every time the number of nodes in the rbt
tree would exceed 3 * (hashtable size). The overcommit will make the
uneven distribution in the hashtable even worse, but the main problem
lies in the rehashing - every time the database grows beyond the
limit, each subsequent rehashing will be much slower. The mitigation
here is letting the rbt know how big the cache can grown and
pre-allocate the hashtable to be big enough to actually never need to
rehash. This will consume more memory at the start, but since the
size of the hashtable is capped to `1 << 32` (e.g. 4 mio entries), it
will only consume maximum of 32GB of memory for hashtable in the
worst case (and max-cache-size would need to be set to more than
4TB). Calling the dns_db_adjusthashsize() will also cap the maximum
size of the hashtable to the pre-computed number of bits, so it won't
try to consume more gigabytes of memory than available for the
database.
FIXME: What is the average size of the rbt node that gets hashed? I
chose the pagesize (4k) as initial value to precompute the size of
the hashtable, but the value is based on feeling and not any real
data.
For future work, there are more places where we use result of the hash
value modulo some small number and that would benefit from Fibonacci
Hashing to get better distribution.
Notes:
a. A doubly linked list should be used here to speedup the removal of
the entries from the hashtable.
b. https://probablydance.com/2018/06/16/fibonacci-hashing-the-optimization-that-the-world-forgot-or-a-better-alternative-to-integer-modulo/
(cherry picked from commit e24bc324b4)
Implement the 'rndc dnssec -status' command that will output
some information about the key states, such as which policy is
used for the zone, what keys are in use, and when rollover is
scheduled.
Add loose testing in the kasp system test, the actual times are
already tested via key file inspection.
(cherry picked from commit 19ce9ec1d4)
All our MSVS Project files share the same intermediate directory. We
know that this doesn't cause any problems, so we can just disable the
detection in the project files.
Example of the warning:
warning MSB8028: The intermediate directory (.\Release\) contains files shared from another project (dnssectool.vcxproj). This can lead to incorrect clean and rebuild behavior.
(cherry picked from commit b6c2012d93)
Our vcxproj files set the WarningLevel to Level3, which is too verbose
for a code that needs to be portable. That basically leads to ignoring
all the errors that MSVC produces. This commits downgrades the
WarningLevel to Level1 and enables treating warnings as errors for
Release builds. For the Debug builds the WarningLevel got upgraded to
Level4, and treating warnings as errors is explicitly disabled.
We should eventually make the code clean of all MSVC warnings, but it's
a long way to go for Level4, so it's more reasonable to start at Level1.
For reference[1], these are the warning levels as described by MSVC
documentation:
* /W0 suppresses all warnings. It's equivalent to /w.
* /W1 displays level 1 (severe) warnings. /W1 is the default setting
in the command-line compiler.
* /W2 displays level 1 and level 2 (significant) warnings.
* /W3 displays level 1, level 2, and level 3 (production quality)
warnings. /W3 is the default setting in the IDE.
* /W4 displays level 1, level 2, and level 3 warnings, and all level 4
(informational) warnings that aren't off by default. We recommend
that you use this option to provide lint-like warnings. For a new
project, it may be best to use /W4 in all compilations. This option
helps ensure the fewest possible hard-to-find code defects.
* /Wall displays all warnings displayed by /W4 and all other warnings
that /W4 doesn't include — for example, warnings that are off by
default.
* /WX treats all compiler warnings as errors. For a new project, it
may be best to use /WX in all compilations; resolving all warnings
ensures the fewest possible hard-to-find code defects.
1. https://docs.microsoft.com/en-us/cpp/build/reference/compiler-option-warning-level?view=vs-2019
(cherry picked from commit 789d253e3d)
The first attempt to add DNSSEC sign statistics was naive: for each
zone we allocated 64K counters, twice. In reality each zone has at
most four keys, so the new approach only has room for four keys per
zone. If after a rollover more keys have signed the zone, existing
keys are rotated out.
The DNSSEC sign statistics has three counters per key, so twelve
counters per zone. First counter is actually a key id, so it is
clear what key contributed to the metrics. The second counter
tracks the number of generated signatures, and the third tracks
how many of those are refreshes.
This means that in the zone structure we no longer need two separate
references to DNSSEC sign metrics: both the resign and refresh stats
are kept in a single dns_stats structure.
Incrementing dnssecsignstats:
Whenever a dnssecsignstat is incremented, we look up the key id
to see if we already are counting metrics for this key. If so,
we update the corresponding operation counter (resign or
refresh).
If the key is new, store the value in a new counter and increment
corresponding counter.
If all slots are full, we rotate the keys and overwrite the last
slot with the new key.
Dumping dnssecsignstats:
Dumping dnssecsignstats is no longer a simple wrapper around
isc_stats_dump, but uses the same principle. The difference is that
rather than dumping the index (key tag) and counter, we have to look
up the corresponding counter.
(cherry picked from commit 705810d577)
"max-journal-size" is set by default to twice the size of the zone
database. however, the calculation of zone database size was flawed.
- change the size calculations in dns_db_getsize() to more accurately
represent the space needed for a journal file or *XFR message to
contain the data in the database. previously we returned the sizes
of all rdataslabs, including header overhead and offset tables,
which resulted in the database size being reported as much larger
than the equivalent journal transactions would have been.
- map files caused a particular problem here: the full name can't be
determined from the node while a file is being deserialized, because
the uppernode pointers aren't set yet. so we store "full name length"
in the dns_rbtnode structure while serializing, and clear it after
deserialization is complete.
adjust clang-format options to get closer to ISC style
See merge request isc-projects/bind9!3061
(cherry picked from commit d3b49b6675)
0255a974 revise .clang-format and add a C formatting script in util
e851ed0b apply the modified style
Reformat source code with clang-format
Closes#46
See merge request isc-projects/bind9!2156
(cherry picked from commit 7099e79a9b)
4c3b063e Import Linux kernel .clang-format with small modifications
f50b1e06 Use clang-format to reformat the source files
11341c76 Update the definition files for Windows
df6c1f76 Remove tkey_test (which is no-op anyway)
When you do a restart or reconfig of named, or rndc loadkeys, this
triggers the key manager to run. The key manager will check if new
keys need to be created. If there is an active key, and key rollover
is scheduled far enough away, no new key needs to be created.
However, there was a bug that when you just start to sign your zone,
it takes a while before the KSK becomes an active key. An active KSK
has its DS submitted or published, but before the key manager allows
that, the DNSKEY needs to be omnipresent. If you restart named
or rndc loadkeys in quick succession when you just started to sign
your zone, new keys will be created because the KSK is not yet
considered active.
Fix is to check for introducing as well as active keys. These keys
all have in common that their goal is to become omnipresent.
it now removes matching trust anchors from from the dslist while leaving
the other trust anchors in place.
also cleaned up the API to remove functions that were never being used.
NOTE: the keytable test is still failing because dns_keytable_deletekey()
is looking for exact matches in keynodes containing dst_key objects,
which no keynode has anymore.
the internal keytable structure has not yet been changed, but
insertion of DS anchors is the only method now available.
NOTE: the keytable unit test is currently failing because of tests
that expect individual keynode objects to contain single DST key
objects.
as initial-key and static-key trust anchors will now be stored as a
DS rrset, code referencing keynodes storing DNSKEY trust anchors will
no longer be reached.
this function is used by dns_view_untrust() to handle revoked keys, so
it will still be needed after the keytable/validator refactoring is
complete, even though the keytable will be storing DS trust anchors
instead of keys. to simplify the way it's called, it now takes a DNSKEY
rdata struct instead of a DST key.
Previously, the dns_geoip API used isc_thread_key API for TLS, which is
fairly complicated and requires initialization of memory contexts, etc.
This part of code was refactored to use a ISC_THREAD_LOCAL pointer which
greatly simplifies the whole code related to storing TLS variables, and
creating the local memory context was moved to named and stored in the
named_g_geoip global context.
Previously, the dns_dt API used isc_thread_key API for TLS, which is
fairly complicated and requires initialization of memory contexts, etc.
This part of code was refactored to use a ISC_THREAD_LOCAL pointer which
greatly simplifies the whole code related to storing TLS variables.
Previously, the dns_name API used isc_thread_key API for TLS, which is
fairly complicated and requires initialization of memory contexts, etc.
This part of code was refactored to use a ISC_THREAD_LOCAL pointer which
greatly simplifies the whole code related to storing TLS variables.
note: this is a frankensteinian kluge which needs further refactoring.
the keytable started as an RBT where the node->data points to a list of
dns_keynode structures, each of which points to a single dst_key.
later it was modified so that the list could instead point to a single
"null" keynode structure, which does not reference a key; this means
a trust anchor has been configured but the RFC 5011 refresh failed.
in this branch it is further updated to allow the first keynode in
the list to point to an rdatalist of DS-style trust anchors. these will
be used by the validator to populate 'val->dsset' when validating a zone
key.
a DS style trust anchor can be updated as a result of RFC 5011
processing to contain DST keys instead; this results in the DS list
being freed. the reverse is not possible; attempting to add a DS-style
trust anchor if a key-style trust anchor is already in place results
in an error.
later, this should be refactored to use rdatalists for both DS-style
and key-style trust anchors, but we're keeping the existing code for
old-style trust anchors for now.
Update dns_dnssec_get_hints and dns_dnssec_keyactive to use dst_key
functions and thus if dnssec-policy/KASP is used the key states are
being considered.
Add a new variable to 'struct dns_dnsseckey' to signal whether this
key is a zone-signing key (it is no longer true that ksk == !zsk).
Also introduce a hint for revoke.
Update 'dns_dnssec_findzonekeys' and 'dns_dnssec_findmatchingkeys'
to also read the key state file, if available.
Remove 'allzsk' from 'dns_dnssec_updatekeys' as this was only a
hint for logging.
Also make get_hints() (now dns_dnssec_get_hints()) public so that
we can use it in the key manager.
Add a key manager to named. If a 'dnssec-policy' is set, 'named'
will run a key manager on the matching keys. This will do a couple
of things:
1. Create keys when needed (in case of rollover for example)
according to the set policy.
2. Retire keys that are in excess of the policy.
3. Maintain key states according to "Flexible and Robust Key
Rollover" [1]. After key manager ran, key files will be saved to
disk.
[1] https://matthijsmekking.nl/static/pdf/satin2012-Schaeffer.pdf
KEY GENERATION
Create keys according to DNSSEC policy. Zones configured with
'dnssec-policy' will allow 'named' to create DNSSEC keys (similar
to dnssec-keymgr) if not available.
KEY ROLLOVER
Rather than determining the desired state from timing metadata,
add a key state goal. Any keys that are created or picked from the
key ring and selected to be a successor has its key state goal set
to OMNIPRESENT (this key wants to be signing!). At the same time,
a key that is being retired has its key state goal set to HIDDEN.
The keymgr state machine with the three rules will make sure no
introduction or withdrawal of DNSSEC records happens too soon.
KEY TIMINGS
All timings are based on RFC 7583.
The keymgr will return when the next action is happening so
that the zone can set the proper rekey event. Prior to this change
the rekey event will run every hour by default (configurable),
but with kasp we can determine exactly when we need to run again.
The prepublication time is derived from policy.
Add a couple of dst_key functions for determining hints that
consider key states if they are available.
- dst_key_is_unused:
A key has no timing metadata set other than Created.
- dst_key_is_published:
A key has publish timing metadata <= now, DNSKEY state in
RUMOURED or OMNIPRESENT.
- dst_key_is_active:
A key has active timing metadata <= now, RRSIG state in
RUMOURED or OMNIPRESENT.
- dst_key_is_signing:
KSK is_signing and is_active means different things than
for a ZSK. A ZSK is active means it is also signing, but
a KSK always signs its DNSKEY RRset but is considered
active if its DS is present (rumoured or omnipresent).
- dst_key_is_revoked:
A key has revoke timing metadata <= now.
- dst_key_is_removed:
A key has delete timing metadata <= now, DNSKEY state in
UNRETENTIVE or HIDDEN.
When doing rollover in a timely manner we need to have access to the
relevant kasp configured durations.
Most of these are simple get functions, but 'dns_kasp_signdelay'
will calculate the maximum time that is needed with this policy to
resign the complete zone (taking into account the refresh interval
and signature validity).
Introduce parent-propagation-delay, parent-registration-delay,
parent-ds-ttl, zone-max-ttl, zone-propagation-delay.
Introduce a new option '-s' for dnssec-settime that when manipulating
timing metadata, it also updates the key state file.
For testing purposes, add options to dnssec-settime to set key
states and when they last changed.
The dst code adds ways to write and read the new key states and
timing metadata. It updates the parsing code for private key files
to not parse the newly introduced metadata (these are for state
files only).
Introduce key goal (the state the key wants to be in).
When reading a key from file, you can set the DST_TYPE_STATE option
to also read the key state.
This expects the Algorithm and Length fields go above the metadata,
so update the write functionality to do so accordingly.
Introduce new DST metadata types for KSK, ZSK, Lifetime and the
timing metadata used in state files.
Write functions to access various elements of the kasp structure,
and the kasp keys. This in preparation of code in dnssec-keygen,
dnssec-settime, named...
Add a number of metadata variables (lifetime, ksk and zsk role).
For the roles we add a new type of metadata (booleans).
Add a function to write the state of the key to a separate file.
Only write out known metadata to private file. With the
introduction of the numeric metadata "Lifetime", adjust the write
private key file functionality to only write out metadata it knows
about.
This stores the dnssec-policy configuration and adds methods to
create, destroy, and attach/detach, as well as find a policy with
the same name in a list.
Also, add structures and functions for creating and destroying
kasp keys.
The dns_name_copy() function followed two different semanitcs that was driven
whether the last argument was or wasn't NULL. This commit splits the function
in two where now third argument to dns_name_copy() can't be NULL and
dns_name_copynf() doesn't have third argument.
Until now, the build process for BIND on Windows involved upgrading the
solution file to the version of Visual Studio used on the build host.
Unfortunately, the executable used for that (devenv.exe) is not part of
Visual Studio Build Tools and thus there is no clean way to make that
executable part of a Windows Server container.
Luckily, the solution upgrade process boils down to just adding XML tags
to Visual Studio project files and modifying certain XML attributes - in
files which we pregenerate anyway using win32utils/Configure. Thus,
extend win32utils/Configure with three new command line parameters that
enable it to mimic what "devenv.exe bind9.sln /upgrade" does. This
makes the devenv.exe build step redundant and thus facilitates building
BIND in Windows Server containers.
No function called dns_dnssecsignstats_decrement() actually exists.
Putting it into lib/dns/win32/libdns.def.in breaks at least some Windows
builds. Remove the nonexistent function from that file.
In addition to gather how many times signatures are created per
key in a zone, also count how many of those signature creations are
because of DNSSEC maintenance. These maintenance counters are
incremented if a signature is refreshed (but the RRset did not
changed), when the DNSKEY RRset is changed, and when that leads
to additional RRset / RRSIG updates (for example SOA, NSEC).
Since 2008, the cleaning-interval timer has been documented as
"effectively obsolete" and disabled in the default configuration with
a comment saying "now meaningless".
This change deletes all the code that implements the cleaning-interval
timer, except for the config parser in whcih it is now explicitly
marked as obsolete.
I have verified (using the deletelru and deletettl cache stats) that
named still cleans the cache after this change.
