fix: usr: Checking whether a EDDSA key was private or not was broken

Checking whether a EDDSA key was private or not was broken could lead to
attempting to sign records with a public key and this could cause a
segmentation failure (read of a NULL pointer) within OpenSSL.

Closes #4855

Merge branch '4855-openssleddsa_isprivate-needs-to-supply-a-buffer' into 'bind-9.18'

See merge request isc-projects/bind9!9329
This commit is contained in:
Mark Andrews
2024-08-20 00:16:34 +00:00

View File

@@ -349,13 +349,20 @@ err:
static bool
openssleddsa_isprivate(const dst_key_t *key) {
EVP_PKEY *pkey = key->keydata.pkey;
size_t len;
unsigned char buf[DNS_KEY_ED448SIZE];
size_t len = sizeof(buf);
STATIC_ASSERT(sizeof(buf) >= DNS_KEY_ED448SIZE,
"increase size of 'buf'");
STATIC_ASSERT(sizeof(buf) >= DNS_KEY_ED25519SIZE,
"increase size of 'buf'");
if (pkey == NULL) {
return (false);
}
if (EVP_PKEY_get_raw_private_key(pkey, NULL, &len) == 1 && len > 0) {
/* Must have a buffer to actually check if there is a private key. */
if (EVP_PKEY_get_raw_private_key(pkey, buf, &len) == 1) {
return (true);
}
/* can check if first error is EC_R_INVALID_PRIVATE_KEY */