gnutsl_cipher.c

[Nikos Mavrogiannopoulos]

On 11/11/2010 10:15 PM, HOY Mike wrote:
> Hello Nikos, We have been having decryption problems depending upon 
> file size.  We are talking to a specialized camera with TLS 1.0, 1.1
>  and TLS 1.2. Under each we had file transfers that failed 
> decryption. All with varying file sizes. After trouble shooting we 
> found that in gnutls_cipher.c in the function 
> _gnutls_ciphertext2compressed the local variable pad is defined as 
> uint8_t.  While this seemed fine we found it rolled over to an 
> illegal value of 0 for the pad.  Our fix was to change pad to an int 
> and get a pad value of 256, instead of zero.

Value 0 is not illegal. In TLS CBC padding can be from zero to 255. 256
is not a legal value for padding since it has to be stored in a byte.
Could you provide more information on when this problem occurs? Does the
 TLS implementation of the camera expect 256 bytes of padding if it
encounters the padding value zero? Or it does give 256 bytes of padding
and says it's zero?

In both cases it seems that TLS implementation is seriously broken
(unless the problem is something else). Does switching to a stream
cipher like arcfour solve your problem?

regards,
Nikos