[gnutls-devel] GnuTLS | gnutls client should not negotiate TLS 1.3 if the private key from PKCS#11 does not support RSA-PSS nor raw-RSA (#730)

[Development of GNU's TLS library]

New Issue was created.

Issue 730: https://gitlab.com/gnutls/gnutls/issues/730
Author:    Anderson Sasaki
Assignee:  

## Description of problem:
When applications using GnuTLS with client certificate key in smart card connect using TLS, the library should check the PKCS#11 module capabilities before negotiating the TLS 1.3 and failing later without any mechanisms to try

originally reported in: https://bugzilla.redhat.com/show_bug.cgi?id=1681006

## Version of gnutls used:
3.6.5

## Distributor of gnutls (e.g., Ubuntu, Fedora, RHEL)
RHEL

## How reproducible:
always

Steps to Reproduce:

 * Prerequisites:
   * PKCS#11 module without support for RSA-PSS and raw-RSA mechanisms
   * The server requests TLS client authentication
 * Start a server requesting TLS client authentication:
```
$ gnutls-serv --http --require-client-cert --x509cafile ca.pem -d 9 --x509certfile cert.pem --x509keyfile key.pem
```
 * Connect to the server using wget (or any other application) compiled against GnuTLS with private key in the PKCS#11 module:
```
$ GNUTLS_DEBUG_LEVEL=9 wget --no-check-certificate --certificate="pkcs11:token=SomeDevice;object=cert;type=cert" --private-key="pkcs11:token=SomeDevice;object=key;type=private?pin-value=111111" --debug --tries 1 https://localhost:5556/
```

## Actual results:
The TLS 1.3 connection fails:
```
[...]
gnutls[4]: checking cert compat with RSA-PSS-SHA512
gnutls[4]: checking cert compat with RSA-PSS-RSAE-SHA512
gnutls[4]: checking cert compat with ECDSA-SECP521R1-SHA512
gnutls[4]: cannot use privkey of RSA with ECDSA-SECP521R1-SHA512
gnutls[4]: checking cert compat with RSA-SHA1
gnutls[3]: ASSERT: signature.c[_gnutls_session_sign_algo_enabled]:365
gnutls[4]: Signature algorithm RSA-SHA1 is not enabled
gnutls[4]: checking cert compat with ECDSA-SHA1
gnutls[4]: cannot use privkey of RSA with ECDSA-SHA1
gnutls[3]: ASSERT: tls13/certificate_verify.c[_gnutls13_send_certificate_verify]:192
gnutls[3]: ASSERT: handshake-tls13.c[_gnutls13_handshake_client]:178
GnuTLS: The signature is incompatible with the public key.
gnutls[5]: REC[0x55bab44e9510]: Start of epoch cleanup
gnutls[5]: REC[0x55bab44e9510]: Epoch #0 freed
gnutls[5]: REC[0x55bab44e9510]: End of epoch cleanup
gnutls[5]: REC[0x55bab44e9510]: Epoch #1 freed
Closed fd 4
Unable to establish SSL connection.
```

## Expected results:
The TLS connection should be downgraded to TLS 1.2

-- 
Reply to this email directly or view it on GitLab: https://gitlab.com/gnutls/gnutls/issues/730
You're receiving this email because of your account on gitlab.com.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.gnupg.org/pipermail/gnutls-devel/attachments/20190314/bbb4e15c/attachment.html>