some strange decryption errors + one enhancement notice

[Miroslav Kratochvil]

Hi,

I'm using GnuTLS library to develop a small and secure networking
tool, see [1]. Recently I came to a very strange problem:

My application is used for data transfer, and all sockets used are
non-blocking. When there's too much of data to send, it usually comes
to the state that the application needs to write, but socket is full
(resulting in returning a 'would block' from gnutls_record_send() and
waiting for a writeable socket). In these cases, the application
usually starts to randomly fail in this way:

1] the second peer (the one that didn't fill up the send queue)
reports gnutls error "Decryption failed" thrown by a call to
gnutls_record_recv()
2] the first peer then (as usual) fails with "TLS packet with
unexpected length was received", which is most probably a result of
having the other connection endpoint closed.

My little debugging search led to gnutls_cipher.c, where the
"decryption failed" error seems to originate. It is detected in line
535 (below the comment "Check padding bytes, TLS 1.x") and returned
after 'if (pad_failed!=0)' on line 572  (please note that this return
probably misses gnutls_assert() call, it can be helpful on debugging..
or at least it helped me after i added it;) )

Now about my question: As a reliable transport (TCP) is used to
transfer the packets, how is it possible that receiving end detects
corrupted data?

I guess the possible causes can be those:

a] I'm somehow using gnutls_record_send in a very bad manner,
especially when it returns after non-successful nonblocking write;
which causes some data corruption of outgoing data, which is then
detected by the other communication peer. Is it possible?

b] Improbable bug in GnuTLS padding-creating function.

c] Very improbable bug in GnuTLS padding-checking function.

d] Extremely improbable: TCP stack somehow fails, or "Decryption
failed" error is caused by connection problem.

I expect that it's a]. Could someone comment on correct send retrying
(gnutls manual isn't very talkative at that section)? If I'm somehow
wrong, please correct me. If you want to see my source, please see
[2].

Thanks in advance,
Mirek Kratochvil

[1] http://e-x-a.org/?view=cloudvpn
[2] http://e-x-a.org/repos/view.cgi/cloudvpn/tree/src/cloud/comm.cpp#n683