[gnutls-help] Bug#964284: guile-gnutls: update to use guile 3.0

Ludovic Courtès ludo at gnu.org
Tue Dec 22 10:45:04 CET 2020 

Hi,

Ludovic Courtès <ludo at gnu.org> skribis:

> [10219|3] Peer requested CA: CN=GNUTLS TEST SERVER,OU=GNUTLS,O=FSF,C=GR
> [10219|3] Peer requested CA: CN=GNUTLS INTERMEDIATE TEST CA,OU=GNUTLS,O=FSF,C=GR
> [10219|4] checking cert compat with RSA-SHA256
> [10219|3] ASSERT: ../../../lib/ext/signature.c[_gnutls_session_sign_algo_enabled]:433
> [10219|4] Signature algorithm RSA-SHA256 is not enabled
> [10219|4] checking cert compat with RSA-PSS-SHA256
> [10219|4] 
> throw to `gnutls-error' with args (#<gnutls-error-enum Public key signing has failed.> read_from_session_record_port)
> checking cert compat with RSA-PSS-RSAE-SHA256
> [10219|4] HSK[0x55c505382030]: CERTIFICATE was queued [693 bytes]
> [10219|5] REC[0x55c505382030]: Preparing Packet Handshake(22) with length: 693 and min pad: 0
> [10219|5] REC[0x55c505382030]: Sent Packet[8906] Handshake(22) in epoch 2 and length: 715
> [10219|4] HSK[0x55c505382030]: signing TLS 1.3 handshake data: using RSA-PSS-RSAE-SHA256 and PRF: SHA384
> [10219|3] ASSERT: ../../../lib/nettle/pk.c[_rsa_pss_sign_digest_tr]:805

[...]

> In all cases, it’s the client failing to reauthenticate.  One example is
> ‘_nettle_rsa_sec_compute_root_tr’ returning 0 and thus leading to
> GNUTLS_E_PK_SIGN_FAILED as we see above.

In the end, it turns out that GMP bignums as used by Nettle are being
overwritten when GC runs.  Here’s an example of key material being
overwritten:

--8<---------------cut here---------------start------------->8---
rsa_sec_check_root (m=0x7f3ecbc6d100, x=0x7f3ecbc6d080, pub=0x7ffd2ca5ac90) at rsa-sign-tr.c:257
257	in rsa-sign-tr.c
(rr) p pub->n._mp_d
$12 = (mp_limb_t *) 0x7f3ecbc6db00
(rr) p *pub->n._mp_d
$13 = 139907683506816
(rr) p *pub->e._mp_d
$14 = 65537
(rr) watch *$12
Hardware watchpoint 5: *$12
(rr) rc
Continuing.

Thread 14 hit Hardware watchpoint 5: *$12

Old value = 139907683506816
New value = 13832525101577573263
0x00007f3ecf963785 in GC_reclaim_generic ()
   from /gnu/store/zg126cjicrpm2p6zc08ra5vh4ddag7ww-libgc-8.0.4/lib/libgc.so.1
(rr) bt
#0  0x00007f3ecf963785 in GC_reclaim_generic ()
   from /gnu/store/zg126cjicrpm2p6zc08ra5vh4ddag7ww-libgc-8.0.4/lib/libgc.so.1
#1  0x00007f3ecf976fef in GC_generic_malloc_many ()
   from /gnu/store/zg126cjicrpm2p6zc08ra5vh4ddag7ww-libgc-8.0.4/lib/libgc.so.1
#2  0x00007f3ecf9776b3 in GC_malloc_kind ()
   from /gnu/store/zg126cjicrpm2p6zc08ra5vh4ddag7ww-libgc-8.0.4/lib/libgc.so.1
#3  0x00007f3ecbcf98d2 in _nettle_gmp_alloc (n=n at entry=128) at gmp-glue.c:345
#4  0x00007f3ecbcf454f in rsa_sec_blind (mn=16, m=0x7f3ecbc6d180, ri=0x7f3ecbc6d000, c=0x7f3ecbc6d100, 
    random=0x7f3ecbeb61e0 <rnd_nonce_func>, random_ctx=0x0, pub=0x7ffd2ca5ac90) at rsa-sign-tr.c:175
#5  _nettle_rsa_sec_compute_root_tr (pub=pub at entry=0x7ffd2ca5ac90, key=key at entry=0x7ffd2ca5acc0, 
    random_ctx=random_ctx at entry=0x0, random=random at entry=0x7f3ecbeb61e0 <rnd_nonce_func>, x=x at entry=0x7f3ecbc6d100, 
    m=0x7f3ecbc6d180, mn=16) at rsa-sign-tr.c:330
#6  0x00007f3ecbcf4a74 in nettle_rsa_compute_root_tr (pub=pub at entry=0x7ffd2ca5ac90, key=key at entry=0x7ffd2ca5acc0, 
    random_ctx=random_ctx at entry=0x0, random=random at entry=0x7f3ecbeb61e0 <rnd_nonce_func>, x=x at entry=0x7ffd2ca5ac80, 
    m=m at entry=0x7ffd2ca5abf0) at rsa-sign-tr.c:365
#7  0x00007f3ecbcf5de0 in nettle_rsa_pss_sha256_sign_digest_tr (pub=0x7ffd2ca5ac90, key=0x7ffd2ca5acc0, 
    random_ctx=0x0, random=0x7f3ecbeb61e0 <rnd_nonce_func>, salt_length=<optimized out>, 
    salt=0xcf1890 "\003\365\204\222t\363B\334\316~S\227Q\tB\217\037\016\314\326\247U9\360\v\214\313\344`\263\212\316\230\305\313", 
    digest=0xcf1860 "q.\272\251\204\064X\302\fH4\262\263\363\024V5F\225\274\223\360U\fE\237\305$t\004\350u\230", 
    s=0x7ffd2ca5ac80) at rsa-pss-sha256-sign-tr.c:59
#8  0x00007f3ecbeb8709 in _rsa_pss_sign_digest_tr (rnd_ctx=0x0, rnd_func=0x7f3ecbeb61e0 <rnd_nonce_func>, 
    s=0x7ffd2ca5ac80, digest=<optimized out>, salt_size=<optimized out>, priv=0x7ffd2ca5acc0, pub=0x7ffd2ca5ac90, 
    dig=<optimized out>) at pk.c:807
#9  _wrap_nettle_pk_sign (algo=<optimized out>, signature=0x7ffd2ca5af30, vdata=<optimized out>, 
    pk_params=<optimized out>, sign_params=<optimized out>) at pk.c:1175
#10 0x00007f3ecbe08374 in privkey_sign_and_hash_data (signer=0xce1990, se=0x7f3ecbf50a20 <sign_algorithms+224>, 
    data=<optimized out>, signature=0x7ffd2ca5af30, params=0x7ffd2ca5ade0) at privkey.c:1296
#11 0x00007f3ecbe08658 in gnutls_privkey_sign_data2 (signer=signer at entry=0xce1990, algo=<optimized out>, 
    flags=flags at entry=0, data=data at entry=0x7ffd2ca5ae50, signature=signature at entry=0x7ffd2ca5af30) at privkey.c:1194
#12 0x00007f3ecbe1e203 in _gnutls13_handshake_sign_data (session=session at entry=0xcebd40, cert=<optimized out>, 
    pkey=0xce1990, context=0x7f3ecbf46210 <srv_ctx>, signature=signature at entry=0x7ffd2ca5af30, 
    se=se at entry=0x7f3ecbf50a20 <sign_algorithms+224>) at tls13-sig.c:207
#13 0x00007f3ecbe1d8db in _gnutls13_send_certificate_verify (session=session at entry=0xcebd40, again=<optimized out>)
    at tls13/certificate_verify.c:206
#14 0x00007f3ecbdd2fc1 in _gnutls13_handshake_server (session=session at entry=0xcebd40) at handshake-tls13.c:464
#15 0x00007f3ecbdde10d in handshake_server (session=<optimized out>) at handshake.c:3381
#16 gnutls_handshake (session=0xcebd40) at handshake.c:2783
#17 0x00007f3ecbf63e82 in scm_gnutls_handshake (session=<optimized out>) at core.c:196
--8<---------------cut here---------------end--------------->8---

This is because Guile >= 3.0.1 and >= 2.2.7 changes the GMP allocation
functions such that they go through libgc¹.  As a result, libgc may
reuse that memory when it becomes unreachable from its point of view; in
this case, since GnuTLS structures are not scanned by libgc, libgc
doesn’t “see” pointers to those bignums and thus considers they are no
longer reachable.

Where to go from here?  Here are options that come to mind:

  • Configure Nettle with ‘--enable-mini-gmp’.  However, the manual
    mentions that it’s “slower” and “more likely to leak side-channel
    information” (info "(nettle) Installation").

  • Have Guile use mini-GMP; this is not implemented yet.

  • In Guile-GnuTLS, arrange so that GnuTLS allocations are made through
    libgc.  Unfortunately, ‘gnutls_global_set_mem_functions’ was
    deprecated in GnuTLS 3.3.0 so this doesn’t look like an option.

  • Build Guile with ‘scm_install_gmp_memory_functions = 0’.  This would
    have a negative impact on the performance of bignum-heavy workloads
    such as the compiler itself.

I can’t think of a good workaround.  Thoughts?

Ludo’.

¹ https://git.savannah.gnu.org/cgit/guile.git/commit/?id=00fbdfa7345765168e14438eed0b0b8c64c27ab9

More information about the Gnutls-help mailing list