[gnutls-devel] Bug: gnutls_dh_get_group prepends a zero byte to prime

[Thomas Klute]

Hello,

I believe I have found a bug in gnutls_dh_get_group: It returns the
prime with an extra zero byte at the beginning.

The small program at [1] tries a handshake with DHE and then compares
the parameters with those read from a PEM encoded PKCS3 file (assuming
it's the same one used by the server). The prime as read from the file
matches the one in the Server Key Exchange packet (checked with
Wireshark), but the gnutls_datum_t filled by gnutls_dh_get_group
contains the extra byte. With the one byte offset in lines 240 and 241
[2], the primes match.

You can try my example code as follows:

Compile:
$ gcc --std=gnu11 -o dh_check dh_check.c `pkg-config --libs --cflags gnutls`

Start gnutls-serv with an X.509 key/cert and custom DH params from
dhfile.pem (or store the default params in that file) and run the example:
$ ./dh_check -c dhfile.pem

By default, dh_check tries to connect to localhost:443, you can use -h
HOST and -p PORT to connect to somewhere else.

The handshake works as expected, so I guess the bug is just in the code
that retrieves the prime from the session. ;-)

Kind regards,
Thomas


[1] https://gist.github.com/airtower-luna/5a62fd9356a19157471c
[2]
https://gist.github.com/airtower-luna/5a62fd9356a19157471c#file-dh_check-c-L240-L241