Add to the keymgr a function that will schedule a rollover. This
basically means setting the time when the key needs to retire,
and updating the key lifetime, then update the state file. The next
time that named runs the keymgr the new lifetime will be taken into
account.
(cherry picked from commit df8276aef0)
This commit will be used as a base for the next code updates in
order to have a better control of dns_message_t objects' lifetime.
(cherry picked from commit 12d6d13100)
The message buffer passed to ns__client_request is only valid for
the life of the the ns__client_request call. Save a copy of it
when we recurse or process a update as ns__client_request will
return before those operations complete.
(cherry picked from commit f0d9bf7c30)
Hold a weak reference to the view so that it can't go away while
nta is performing its lookups. Cancel nta timers once all external
references to the view have gone to prevent them triggering new work.
(cherry picked from commit 0b2555e8cf)
Add a new 'rndc' command 'dnssec -checkds' that allows the user to
signal named that a new DS record has been seen published in the
parent, or that an existing DS record has been withdrawn from the
parent.
Upon the 'checkds' request, 'named' will write out the new state for
the key, updating the 'DSPublish' or 'DSRemoved' timing metadata.
This replaces the "parent-registration-delay" configuration option,
this was unreliable because it was purely time based (if the user
did not actually submit the new DS to the parent for example, this
could result in an invalid DNSSEC state).
Because we cannot rely on the parent registration delay for state
transition, we need to replace it with a different guard. Instead,
if a key wants its DS state to be moved to RUMOURED, the "DSPublish"
time must be set and must not be in the future. If a key wants its
DS state to be moved to UNRETENTIVE, the "DSRemoved" time must be set
and must not be in the future.
By default, with '-checkds' you set the time that the DS has been
published or withdrawn to now, but you can set a different time with
'-when'. If there is only one KSK for the zone, that key has its
DS state moved to RUMOURED. If there are multiple keys for the zone,
specify the right key with '-key'.
(cherry picked from commit 04d8fc0143)
There were several problems with rbt hashtable implementation:
1. Our internal hashing function returns uint64_t value, but it was
silently truncated to unsigned int in dns_name_hash() and
dns_name_fullhash() functions. As the SipHash 2-4 higher bits are
more random, we need to use the upper half of the return value.
2. The hashtable implementation in rbt.c was using modulo to pick the
slot number for the hash table. This has several problems because
modulo is: a) slow, b) oblivious to patterns in the input data. This
could lead to very uneven distribution of the hashed data in the
hashtable. Combined with the single-linked lists we use, it could
really hog-down the lookup and removal of the nodes from the rbt
tree[a]. The Fibonacci Hashing is much better fit for the hashtable
function here. For longer description, read "Fibonacci Hashing: The
Optimization that the World Forgot"[b] or just look at the Linux
kernel. Also this will make Diego very happy :).
3. The hashtable would rehash every time the number of nodes in the rbt
tree would exceed 3 * (hashtable size). The overcommit will make the
uneven distribution in the hashtable even worse, but the main problem
lies in the rehashing - every time the database grows beyond the
limit, each subsequent rehashing will be much slower. The mitigation
here is letting the rbt know how big the cache can grown and
pre-allocate the hashtable to be big enough to actually never need to
rehash. This will consume more memory at the start, but since the
size of the hashtable is capped to `1 << 32` (e.g. 4 mio entries), it
will only consume maximum of 32GB of memory for hashtable in the
worst case (and max-cache-size would need to be set to more than
4TB). Calling the dns_db_adjusthashsize() will also cap the maximum
size of the hashtable to the pre-computed number of bits, so it won't
try to consume more gigabytes of memory than available for the
database.
FIXME: What is the average size of the rbt node that gets hashed? I
chose the pagesize (4k) as initial value to precompute the size of
the hashtable, but the value is based on feeling and not any real
data.
For future work, there are more places where we use result of the hash
value modulo some small number and that would benefit from Fibonacci
Hashing to get better distribution.
Notes:
a. A doubly linked list should be used here to speedup the removal of
the entries from the hashtable.
b. https://probablydance.com/2018/06/16/fibonacci-hashing-the-optimization-that-the-world-forgot-or-a-better-alternative-to-integer-modulo/
(cherry picked from commit e24bc324b4)
Implement the 'rndc dnssec -status' command that will output
some information about the key states, such as which policy is
used for the zone, what keys are in use, and when rollover is
scheduled.
Add loose testing in the kasp system test, the actual times are
already tested via key file inspection.
(cherry picked from commit 19ce9ec1d4)
The first attempt to add DNSSEC sign statistics was naive: for each
zone we allocated 64K counters, twice. In reality each zone has at
most four keys, so the new approach only has room for four keys per
zone. If after a rollover more keys have signed the zone, existing
keys are rotated out.
The DNSSEC sign statistics has three counters per key, so twelve
counters per zone. First counter is actually a key id, so it is
clear what key contributed to the metrics. The second counter
tracks the number of generated signatures, and the third tracks
how many of those are refreshes.
This means that in the zone structure we no longer need two separate
references to DNSSEC sign metrics: both the resign and refresh stats
are kept in a single dns_stats structure.
Incrementing dnssecsignstats:
Whenever a dnssecsignstat is incremented, we look up the key id
to see if we already are counting metrics for this key. If so,
we update the corresponding operation counter (resign or
refresh).
If the key is new, store the value in a new counter and increment
corresponding counter.
If all slots are full, we rotate the keys and overwrite the last
slot with the new key.
Dumping dnssecsignstats:
Dumping dnssecsignstats is no longer a simple wrapper around
isc_stats_dump, but uses the same principle. The difference is that
rather than dumping the index (key tag) and counter, we have to look
up the corresponding counter.
(cherry picked from commit 705810d577)
"max-journal-size" is set by default to twice the size of the zone
database. however, the calculation of zone database size was flawed.
- change the size calculations in dns_db_getsize() to more accurately
represent the space needed for a journal file or *XFR message to
contain the data in the database. previously we returned the sizes
of all rdataslabs, including header overhead and offset tables,
which resulted in the database size being reported as much larger
than the equivalent journal transactions would have been.
- map files caused a particular problem here: the full name can't be
determined from the node while a file is being deserialized, because
the uppernode pointers aren't set yet. so we store "full name length"
in the dns_rbtnode structure while serializing, and clear it after
deserialization is complete.
Reformat source code with clang-format
Closes#46
See merge request isc-projects/bind9!2156
(cherry picked from commit 7099e79a9b)
4c3b063e Import Linux kernel .clang-format with small modifications
f50b1e06 Use clang-format to reformat the source files
11341c76 Update the definition files for Windows
df6c1f76 Remove tkey_test (which is no-op anyway)
When you do a restart or reconfig of named, or rndc loadkeys, this
triggers the key manager to run. The key manager will check if new
keys need to be created. If there is an active key, and key rollover
is scheduled far enough away, no new key needs to be created.
However, there was a bug that when you just start to sign your zone,
it takes a while before the KSK becomes an active key. An active KSK
has its DS submitted or published, but before the key manager allows
that, the DNSKEY needs to be omnipresent. If you restart named
or rndc loadkeys in quick succession when you just started to sign
your zone, new keys will be created because the KSK is not yet
considered active.
Fix is to check for introducing as well as active keys. These keys
all have in common that their goal is to become omnipresent.
it now removes matching trust anchors from from the dslist while leaving
the other trust anchors in place.
also cleaned up the API to remove functions that were never being used.
NOTE: the keytable test is still failing because dns_keytable_deletekey()
is looking for exact matches in keynodes containing dst_key objects,
which no keynode has anymore.
the internal keytable structure has not yet been changed, but
insertion of DS anchors is the only method now available.
NOTE: the keytable unit test is currently failing because of tests
that expect individual keynode objects to contain single DST key
objects.
as initial-key and static-key trust anchors will now be stored as a
DS rrset, code referencing keynodes storing DNSKEY trust anchors will
no longer be reached.
this function is used by dns_view_untrust() to handle revoked keys, so
it will still be needed after the keytable/validator refactoring is
complete, even though the keytable will be storing DS trust anchors
instead of keys. to simplify the way it's called, it now takes a DNSKEY
rdata struct instead of a DST key.
Previously, the dns_geoip API used isc_thread_key API for TLS, which is
fairly complicated and requires initialization of memory contexts, etc.
This part of code was refactored to use a ISC_THREAD_LOCAL pointer which
greatly simplifies the whole code related to storing TLS variables, and
creating the local memory context was moved to named and stored in the
named_g_geoip global context.
Previously, the dns_dt API used isc_thread_key API for TLS, which is
fairly complicated and requires initialization of memory contexts, etc.
This part of code was refactored to use a ISC_THREAD_LOCAL pointer which
greatly simplifies the whole code related to storing TLS variables.
Previously, the dns_name API used isc_thread_key API for TLS, which is
fairly complicated and requires initialization of memory contexts, etc.
This part of code was refactored to use a ISC_THREAD_LOCAL pointer which
greatly simplifies the whole code related to storing TLS variables.
note: this is a frankensteinian kluge which needs further refactoring.
the keytable started as an RBT where the node->data points to a list of
dns_keynode structures, each of which points to a single dst_key.
later it was modified so that the list could instead point to a single
"null" keynode structure, which does not reference a key; this means
a trust anchor has been configured but the RFC 5011 refresh failed.
in this branch it is further updated to allow the first keynode in
the list to point to an rdatalist of DS-style trust anchors. these will
be used by the validator to populate 'val->dsset' when validating a zone
key.
a DS style trust anchor can be updated as a result of RFC 5011
processing to contain DST keys instead; this results in the DS list
being freed. the reverse is not possible; attempting to add a DS-style
trust anchor if a key-style trust anchor is already in place results
in an error.
later, this should be refactored to use rdatalists for both DS-style
and key-style trust anchors, but we're keeping the existing code for
old-style trust anchors for now.
Update dns_dnssec_get_hints and dns_dnssec_keyactive to use dst_key
functions and thus if dnssec-policy/KASP is used the key states are
being considered.
Add a new variable to 'struct dns_dnsseckey' to signal whether this
key is a zone-signing key (it is no longer true that ksk == !zsk).
Also introduce a hint for revoke.
Update 'dns_dnssec_findzonekeys' and 'dns_dnssec_findmatchingkeys'
to also read the key state file, if available.
Remove 'allzsk' from 'dns_dnssec_updatekeys' as this was only a
hint for logging.
Also make get_hints() (now dns_dnssec_get_hints()) public so that
we can use it in the key manager.
Add a key manager to named. If a 'dnssec-policy' is set, 'named'
will run a key manager on the matching keys. This will do a couple
of things:
1. Create keys when needed (in case of rollover for example)
according to the set policy.
2. Retire keys that are in excess of the policy.
3. Maintain key states according to "Flexible and Robust Key
Rollover" [1]. After key manager ran, key files will be saved to
disk.
[1] https://matthijsmekking.nl/static/pdf/satin2012-Schaeffer.pdf
KEY GENERATION
Create keys according to DNSSEC policy. Zones configured with
'dnssec-policy' will allow 'named' to create DNSSEC keys (similar
to dnssec-keymgr) if not available.
KEY ROLLOVER
Rather than determining the desired state from timing metadata,
add a key state goal. Any keys that are created or picked from the
key ring and selected to be a successor has its key state goal set
to OMNIPRESENT (this key wants to be signing!). At the same time,
a key that is being retired has its key state goal set to HIDDEN.
The keymgr state machine with the three rules will make sure no
introduction or withdrawal of DNSSEC records happens too soon.
KEY TIMINGS
All timings are based on RFC 7583.
The keymgr will return when the next action is happening so
that the zone can set the proper rekey event. Prior to this change
the rekey event will run every hour by default (configurable),
but with kasp we can determine exactly when we need to run again.
The prepublication time is derived from policy.
Add a couple of dst_key functions for determining hints that
consider key states if they are available.
- dst_key_is_unused:
A key has no timing metadata set other than Created.
- dst_key_is_published:
A key has publish timing metadata <= now, DNSKEY state in
RUMOURED or OMNIPRESENT.
- dst_key_is_active:
A key has active timing metadata <= now, RRSIG state in
RUMOURED or OMNIPRESENT.
- dst_key_is_signing:
KSK is_signing and is_active means different things than
for a ZSK. A ZSK is active means it is also signing, but
a KSK always signs its DNSKEY RRset but is considered
active if its DS is present (rumoured or omnipresent).
- dst_key_is_revoked:
A key has revoke timing metadata <= now.
- dst_key_is_removed:
A key has delete timing metadata <= now, DNSKEY state in
UNRETENTIVE or HIDDEN.
When doing rollover in a timely manner we need to have access to the
relevant kasp configured durations.
Most of these are simple get functions, but 'dns_kasp_signdelay'
will calculate the maximum time that is needed with this policy to
resign the complete zone (taking into account the refresh interval
and signature validity).
Introduce parent-propagation-delay, parent-registration-delay,
parent-ds-ttl, zone-max-ttl, zone-propagation-delay.
Introduce a new option '-s' for dnssec-settime that when manipulating
timing metadata, it also updates the key state file.
For testing purposes, add options to dnssec-settime to set key
states and when they last changed.
The dst code adds ways to write and read the new key states and
timing metadata. It updates the parsing code for private key files
to not parse the newly introduced metadata (these are for state
files only).
Introduce key goal (the state the key wants to be in).
When reading a key from file, you can set the DST_TYPE_STATE option
to also read the key state.
This expects the Algorithm and Length fields go above the metadata,
so update the write functionality to do so accordingly.
Introduce new DST metadata types for KSK, ZSK, Lifetime and the
timing metadata used in state files.
Write functions to access various elements of the kasp structure,
and the kasp keys. This in preparation of code in dnssec-keygen,
dnssec-settime, named...
Add a number of metadata variables (lifetime, ksk and zsk role).
For the roles we add a new type of metadata (booleans).
Add a function to write the state of the key to a separate file.
Only write out known metadata to private file. With the
introduction of the numeric metadata "Lifetime", adjust the write
private key file functionality to only write out metadata it knows
about.
This stores the dnssec-policy configuration and adds methods to
create, destroy, and attach/detach, as well as find a policy with
the same name in a list.
Also, add structures and functions for creating and destroying
kasp keys.
The dns_name_copy() function followed two different semanitcs that was driven
whether the last argument was or wasn't NULL. This commit splits the function
in two where now third argument to dns_name_copy() can't be NULL and
dns_name_copynf() doesn't have third argument.
No function called dns_dnssecsignstats_decrement() actually exists.
Putting it into lib/dns/win32/libdns.def.in breaks at least some Windows
builds. Remove the nonexistent function from that file.
In addition to gather how many times signatures are created per
key in a zone, also count how many of those signature creations are
because of DNSSEC maintenance. These maintenance counters are
incremented if a signature is refreshed (but the RRset did not
changed), when the DNSKEY RRset is changed, and when that leads
to additional RRset / RRSIG updates (for example SOA, NSEC).
Since 2008, the cleaning-interval timer has been documented as
"effectively obsolete" and disabled in the default configuration with
a comment saying "now meaningless".
This change deletes all the code that implements the cleaning-interval
timer, except for the config parser in whcih it is now explicitly
marked as obsolete.
I have verified (using the deletelru and deletettl cache stats) that
named still cleans the cache after this change.
Use a zone's 'type' field instead of the value of its DNS_ZONEOPT_MIRROR
option for checking whether it is a mirror zone. This makes said zone
option and its associated helper function, dns_zone_mirror(), redundant,
so remove them. Remove a check specific to mirror zones from
named_zone_reusable() since another check in that function ensures that
changing a zone's type prevents it from being reused during
reconfiguration.
Since BIND libraries are no longer considered public and
dns_rdataslab_tordataset() is not used anywhere in the tree, remove the
latter and its associated dns_rdatasetmethods_t callbacks from
lib/dns/rdataslab.c.
