[gnutls-devel] GnuTLS | Unable to use RSA key with OAEP metadata for signature (#1734)

[Read-only notification of GnuTLS library development activities]

Adrien Béraud created an issue: https://gitlab.com/gnutls/gnutls/-/issues/1734



## Description of problem:
We are transitioning from RSA PKCSv1.5 to OAEP encryption, and are currently facing a possible limitation in the GnuTLS API.
To my knowledge, OAEP is an encryption padding scheme which doesn't prevent or forbid the use of an RSA private key for signing.

However, using the latest version of GnuTLS, I've been unable to perform signing operation using RSA private keys with OAEP information.

## Version of gnutls used:
Any version since 3.8.4.

## How reproducible:
```c
gnutls_x509_privkey_t key;
gnutls_x509_privkey_init(&key);
gnutls_x509_privkey_generate(key, GNUTLS_PK_RSA, 4096, 0);

gnutls_x509_spki_t spki;
gnutls_x509_spki_init(&spki);
gnutls_x509_spki_set_rsa_oaep_params(spki, GNUTLS_DIG_SHA512, nullptr);
gnutls_x509_privkey_set_spki(key, spki, 0);
gnutls_x509_spki_deinit(spki);

gnutls_datum_t dat = { (unsigned char *)"Hello, World!", 13 };
uint8_t sig[4096/8];
size_t sig_size = sizeof(sig);
int err = gnutls_x509_privkey_sign_data(key, GNUTLS_DIG_SHA512, 0, &dat, sig, &sig_size);
printf("gnutls_x509_privkey_sign_data returned %d: %s\n", err, gnutls_strerror(err));
gnutls_x509_privkey_deinit(key);
```

## Actual results:
This fails with `-50 The request is invalid`.
Using the abstract API gives the same result.

## Expected results:
It would be useful to be able to use OAEP keys for signing.

The GnuTLS doesn't seem to provide any easy way to circumvent this, as it doesn't allow to build an spki of type GNUTLS_PK_RSA to set on the private key, and doesn't provide an API to remove spki information form a private key. The only way would seem to export and import raw key parameters, which doesn't feel proper.

-- 
Reply to this email directly or view it on GitLab: https://gitlab.com/gnutls/gnutls/-/issues/1734
You're receiving this email because of your account on gitlab.com.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.gnupg.org/pipermail/gnutls-devel/attachments/20250909/51530fce/attachment-0001.html>