gnutls 2.11.1

[Ludovic Courtès]

Hi,

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

> On 09/15/2010 11:59 AM, Ludovic Courtès wrote:
>> Hi,
>> 
>> Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:
>> 
>>> On 09/14/2010 11:05 PM, Arfrever Frehtes Taifersar Arahesis wrote:
>>>> When GnuTLS 2.11.1 has been built with support for Nettle, then 1 test fails:
>>>> PASS: openpgp-keyring.scm
>>>> /bin/sh: line 5: 23938 Segmentation fault      GUILE_AUTO_COMPILE=0 ../../guile/pre-inst-guile -L . ${dir}$tst
>>>> FAIL: openpgp-auth.scm
>>> [...]
>>>> This problem doesn't occur when GnuTLS 2.11.1 has been built with support for libgcrypt.

OK, I’ve reproduced it.  The ‘openpgp-auth.scm’ programs spawns a client
and server that authenticate using OpenPGP keys.  The server either
hangs or segfaults this way:

--8<---------------cut here---------------start------------->8---
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff762956c in __gmpz_set () from /nix/store/00fbkpgn2g1xrp39crrvnyjb2kj3m6js-gmp-4.3.2/lib/libgmp.so.3
(gdb) bt
#0  0x00007ffff762956c in __gmpz_set () from /nix/store/00fbkpgn2g1xrp39crrvnyjb2kj3m6js-gmp-4.3.2/lib/libgmp.so.3
#1  0x00007ffff76230dd in __gmpz_invert () from /nix/store/00fbkpgn2g1xrp39crrvnyjb2kj3m6js-gmp-4.3.2/lib/libgmp.so.3
#2  0x00007ffff5bbda41 in wrap_nettle_pk_fixup (algo=<value optimized out>, direction=<value optimized out>, params=0x7fffffffc200) at pk.c:564
#3  0x00007ffff5ba05ea in _gnutls_pk_fixup (pkey=<value optimized out>, keyid=<value optimized out>, params=0x7fffffffc2d0, params_size=0x7fffffffc36c) at ../gnutls_pk.h:42
#4  _gnutls_openpgp_privkey_get_mpis (pkey=<value optimized out>, keyid=<value optimized out>, params=0x7fffffffc2d0, params_size=0x7fffffffc36c) at privkey.c:765
#5  0x00007ffff5ba37c9 in gnutls_openpgp_privkey_sign_hash (key=0x6666f0, hash=0x7fffffffc4c0, signature=0x7fffffffc530) at gnutls_openpgp.c:885
#6  0x00007ffff5b56729 in _gnutls_handshake_sign_data (session=0x6754f0, cert=0x676430, pkey=0x669680, params=<value optimized out>, signature=0x7fffffffc530, sign_algo=<value optimized out>) at gnutls_sig.c:221
#7  0x00007ffff5b579be in gen_dhe_server_kx (session=0x6754f0, data=0x7fffffffc5a8) at auth_dhe.c:151
#8  0x00007ffff5b44a8e in _gnutls_send_server_kx_message (session=0x6754f0, again=<value optimized out>) at gnutls_kx.c:180
#9  0x00007ffff5b41f1f in _gnutls_handshake_server (session=0x6754f0) at gnutls_handshake.c:2971
#10 0x00007ffff5b4253d in gnutls_handshake (session=0x6754f0) at gnutls_handshake.c:2634
#11 0x00007ffff5e191c2 in scm_gnutls_handshake (session=<value optimized out>) at core.c:161
#12 0x00007ffff7b3ef98 in vm_debug_engine (vm=0x6bc370, program=0x1, argv=0x6be320, nargs=0) at vm-i-system.c:860
#13 0x00007ffff7b3ef98 in vm_debug_engine (vm=0x6bc370, program=0x1, argv=0x6be260, nargs=0) at vm-i-system.c:860
#14 0x00007ffff7ac8fa3 in scm_primitive_eval (exp=0x95c6e0) at eval.c:844
#15 0x00007ffff7ac9003 in scm_eval (exp=0x95c6e0, module_or_state=0x6bd090) at eval.c:878
#16 0x00007ffff7b0d3af in scm_shell (argc=2, argv=0x7fffffffcca8) at script.c:760
#17 0x00007ffff7ae0ecf in invoke_main_func (body_data=<value optimized out>) at init.c:383
#18 0x00007ffff7abfb2a in c_body (d=0x7fffffffcad0) at continuations.c:473
#19 0x00007ffff7b3ed1b in vm_debug_engine (vm=0x6bc370, program=0x8e3520, argv=0x6be0b8, nargs=1) at vm-i-system.c:928
#20 0x00007ffff7ac8e63 in scm_call_4 (proc=0x659090, arg1=<value optimized out>, arg2=<value optimized out>, arg3=<value optimized out>, arg4=<value optimized out>) at eval.c:582
#21 0x00007ffff7ac0173 in scm_i_with_continuation_barrier (body=0x7ffff7abfb20 <c_body>, body_data=0x7fffffffcad0, handler=0x7ffff7abfb40 <c_handler>, handler_data=0x7fffffffcad0, 
    pre_unwind_handler=<value optimized out>, pre_unwind_handler_data=<value optimized out>) at continuations.c:450
#22 0x00007ffff7ac0210 in scm_c_with_continuation_barrier (func=<value optimized out>, data=<value optimized out>) at continuations.c:491
#23 0x00007ffff7b30822 in scm_i_with_guile_and_parent (func=0x7ffff7ae0eb0 <invoke_main_func>, data=0x7fffffffcba0, parent=<value optimized out>) at threads.c:741
#24 0x00007ffff7ae0f85 in scm_boot_guile (argc=<value optimized out>, argv=<value optimized out>, main_func=<value optimized out>, closure=<value optimized out>) at init.c:366
#25 0x0000000000400c70 in main (argc=<value optimized out>, argv=<value optimized out>) at guile.c:70
(gdb) frame 5
#5  0x00007ffff5ba37c9 in gnutls_openpgp_privkey_sign_hash (key=0x6666f0, hash=0x7fffffffc4c0, signature=0x7fffffffc530) at gnutls_openpgp.c:885
885           result = _gnutls_openpgp_privkey_get_mpis (key, NULL,
(gdb) p *key
$1 = {knode = 0x6670d0, preferred_keyid = "\000\000\000\000\000\000\000", preferred_set = 0}
(gdb) p *key->knode
$2 = {next = 0x6671d0, pkt = 0x679780, is_deleted = 0, is_cloned = 0}
--8<---------------cut here---------------end--------------->8---

This Guile test is the only unit test for OpenPGP authentication in
GnuTLS, and I feel that the issue is more closely related to the
CDK/Nettle integration than to Guile.

So, here’s a patch that adds a similar unit test in C.  The test hangs
while the server generates DH parameters, apparently stuck in the second
loop of ‘gen_prime’.  After some time the stack looks scrambled:

--8<---------------cut here---------------start------------->8---
(gdb) bt
#0  0x00007ffff78eef73 in __gmpn_redc_1 () from /nix/store/ms38kzmhai2xfzvzyvzdfdcpdays942x-user-environment/lib/libgmp.so.3
#1  0x000000000000000e in ?? ()
#2  0x000000000000000b in ?? ()
#3  0x000000000000000e in ?? ()
#4  0x0000000000000008 in ?? ()
#5  0x0000000000000011 in ?? ()
#6  0x00007fffffffcf50 in ?? ()
#7  0x00007ffff78c85ac in __gmpz_powm () from /nix/store/ms38kzmhai2xfzvzyvzdfdcpdays942x-user-environment/lib/libgmp.so.3
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
--8<---------------cut here---------------end--------------->8---

Unfortunately Valgrind doesn’t reveal anything interesting.

Any idea what to look at now?

Thanks,
Ludo’.

PS: The code in lib/nettle ought to go through ‘indent’ IMO.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: </pipermail/attachments/20100918/89715fb3/attachment.pgp>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Add-an-OpenPGP-authentication-unit-test.patch
Type: text/x-patch
Size: 10149 bytes
Desc: the patch
URL: </pipermail/attachments/20100918/89715fb3/attachment.bin>