Buffer Overflow in gnutls_pk.c/_gnutls_pkcs1_rsa_decrypt

[Nikos Mavrogiannopoulos]

2012/1/10 Michal Ambroz <rebus at seznam.cz>:
> Hi Nikos,
> more info on https://bugzilla.redhat.com/show_bug.cgi?id=747167
> <  I would be curious on how you reached the buffer overflow. This is an
> < internal function and its input is controlled by its callers.
> Server gets there by calling gnutls_handshake when new client connects.
> I was hunting this one for quite some time.
> It works like that:
> 1) The code is/was wrongly used in openvas-libraries - use after free condition.
> During the inicialization of the gnutls credentials the code in openvas-library loaded private key from file,
> initialized credentials then freed the memory of pk structure (which is obviously wrong).
> 2) when new socket connection is initialized the function gnutls_handshake is called
> 3) this calls the _gnutls_pkcs1_rsa_decrypt
> It seems that on other operating systems/versions this race condition is hidden. Between inicialization and
> use there is usually not enough time to overwrite the data in the memory of former PK so nobody notices.
> In Fedora 16 it nearly always wins the race condition. tls_cred.pkey.key.509.params_size gets usually set with some high value and it smashes whole stack so it is flagrantly visible.
> The fact that it doesn't demonstrate in other operating systems/versions doesn't mean it is not there and can't be exploited.

Of course such issues are important, but except for sanity checks
gnutls can't really help if the memory of a program is corrupt. Sanity
checks might warn you or prevent a crash, but a malicious memory
modification would be able to overcome them. Think as an extreme
example a function that accepts a function pointer as a parameter. No
sanity checks would prevent a pointer to point to malicious code in
the heap.

For the specific issue, I've added a check in gnutls 2.12.x branch for
the size of the parameters.

regards,
Nikos