[gnutls-help] SSL priority and handshake error

[Michal Suchanek]

On 13 July 2016 at 16:20, Michal Suchanek <hramrach at gmail.com> wrote:
> Hello,
>
> I tried to dust off a piece of old GnuTLS code and I get this error:
>
> ./httpfs2-ssl: SSL init: loaded 173 CA certificate(s).
> Thread main initializing SSL socket.
> ./httpfs2-ssl: invalid SSL priority
>  NORMAL
>  ^
> ./httpfs2-ssl: sourceforge.net:443 - ./httpfs2-ssl: SSL connection
> failed: -12 Handshake failed.
>
> The code was developed with older GnuTLS on Debian 7. The requirements
> say GnuTLS >= 2.10.
>
> It is difficult to install such an old version of GnuTLS without
> breaking my system.
>
> However, Debian still carries 2.12.20-8+deb7u5 which gives different error:
>
> ./httpfs2-ssl: SSL init: loaded 173 CA certificate(s).
> Thread main initializing SSL socket.
> ./httpfs2-ssl: sourceforge.net:443 - ./httpfs2-ssl: SSL connection
> failed: -12 Handshake failed.
>
> 3.x version available in Debian (3.5.2-1, 3.4.14-1, 3.4.13-1,
> 3.3.8-6+deb8u3) all have this issue.
>
> The problem is that the r = gnutls_priority_set_direct(url->ss, ps,
> &errp); line sets errp even when no error is reported.
>
> Extra line if (!r) errp = NULL; is required to provide correct
> diagnostics in GnuTLS 3.x.
>
> So now that this turns out to be a hanshake error how do you diagnose these?
>
> Is there some sample code for printing intelligible diagnostic in the
> event handshake fails?

Upping the debug level shows that server does not like the handshake :/
./httpfs2-ssl: SSL init: loaded 173 CA certificate(s).
REC[0xd05650]: Allocating epoch #0
ASSERT: gnutls_constate.c:695
REC[0xd05650]: Allocating epoch #1
HSK[0xd05650]: Keeping ciphersuite: DHE_RSA_AES_128_CBC_SHA1
HSK[0xd05650]: Keeping ciphersuite: DHE_RSA_AES_128_CBC_SHA256
HSK[0xd05650]: Keeping ciphersuite: DHE_RSA_CAMELLIA_128_CBC_SHA1
HSK[0xd05650]: Keeping ciphersuite: DHE_RSA_AES_256_CBC_SHA1
HSK[0xd05650]: Keeping ciphersuite: DHE_RSA_AES_256_CBC_SHA256
HSK[0xd05650]: Keeping ciphersuite: DHE_RSA_CAMELLIA_256_CBC_SHA1
HSK[0xd05650]: Keeping ciphersuite: DHE_RSA_3DES_EDE_CBC_SHA1
HSK[0xd05650]: Keeping ciphersuite: DHE_DSS_AES_128_CBC_SHA1
HSK[0xd05650]: Keeping ciphersuite: DHE_DSS_AES_128_CBC_SHA256
HSK[0xd05650]: Keeping ciphersuite: DHE_DSS_CAMELLIA_128_CBC_SHA1
HSK[0xd05650]: Keeping ciphersuite: DHE_DSS_AES_256_CBC_SHA1
HSK[0xd05650]: Keeping ciphersuite: DHE_DSS_AES_256_CBC_SHA256
HSK[0xd05650]: Keeping ciphersuite: DHE_DSS_CAMELLIA_256_CBC_SHA1
HSK[0xd05650]: Keeping ciphersuite: DHE_DSS_3DES_EDE_CBC_SHA1
HSK[0xd05650]: Keeping ciphersuite: DHE_DSS_ARCFOUR_SHA1
HSK[0xd05650]: Keeping ciphersuite: RSA_AES_128_CBC_SHA1
HSK[0xd05650]: Keeping ciphersuite: RSA_AES_128_CBC_SHA256
HSK[0xd05650]: Keeping ciphersuite: RSA_CAMELLIA_128_CBC_SHA1
HSK[0xd05650]: Keeping ciphersuite: RSA_AES_256_CBC_SHA1
HSK[0xd05650]: Keeping ciphersuite: RSA_AES_256_CBC_SHA256
HSK[0xd05650]: Keeping ciphersuite: RSA_CAMELLIA_256_CBC_SHA1
HSK[0xd05650]: Keeping ciphersuite: RSA_3DES_EDE_CBC_SHA1
HSK[0xd05650]: Keeping ciphersuite: RSA_ARCFOUR_SHA1
HSK[0xd05650]: Keeping ciphersuite: RSA_ARCFOUR_MD5
EXT[0xd05650]: Sending extension SAFE RENEGOTIATION (1 bytes)
EXT[SIGA]: sent signature algo (4.2) DSA-SHA256
EXT[SIGA]: sent signature algo (4.1) RSA-SHA256
EXT[SIGA]: sent signature algo (2.1) RSA-SHA1
EXT[SIGA]: sent signature algo (2.2) DSA-SHA1
EXT[0xd05650]: Sending extension SIGNATURE ALGORITHMS (10 bytes)
HSK[0xd05650]: CLIENT HELLO was sent [112 bytes]
REC[0xd05650]: Sending Packet[0] Handshake(22) with length: 112
REC[0xd05650]: Sent Packet[1] Handshake(22) with length: 117
REC[0xd05650]: Expected Packet[0] Handshake(22) with length: 1
REC[0xd05650]: Received Packet[0] Alert(21) with length: 2
REC[0xd05650]: Decrypted Packet[0] Alert(21) with length: 2
REC[0xd05650]: Alert[2|40] - Handshake failed - was received
ASSERT: gnutls_record.c:726
ASSERT: gnutls_record.c:1122
ASSERT: gnutls_handshake.c:2762

$ ldd ./httpfs2-ssl
    linux-vdso.so.1 (0x00007ffcad175000)
    libfuse.so.2 => /lib/x86_64-linux-gnu/libfuse.so.2 (0x00007fe20ff38000)
    libgnutls.so.26 => /usr/lib/x86_64-linux-gnu/libgnutls.so.26
(0x00007fe20fc78000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe20f8d4000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fe20f6d0000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
(0x00007fe20f4b3000)
    libtasn1.so.3 => /usr/lib/x86_64-linux-gnu/libtasn1.so.3
(0x00007fe20f2a2000)
    libgcrypt.so.11 => /lib/x86_64-linux-gnu/libgcrypt.so.11
(0x00007fe20f023000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fe20ee08000)
    libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0
(0x00007fe20eba3000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fe210176000)
    libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0
(0x00007fe20e98f000)
    libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007fe20e786000)