Problem using the server name extension

[Sam Varshavchik]

Simon Josefsson writes:

> Sam Varshavchik <mrsam at courier-mta.com> writes:
> 
>> My client is compiled against gnutls 2.8.5. I am connecting to a
>> server that's built against OpenSSL 1.0.0.
>>
>> The OpenSSL server is failing the handshake with the following error
>> message:
>>
>> error:1408A0E3:SSL routines:SSL3_GET_CLIENT_HELLO:parse tlsext
>>
>> After some Googling around, I remove my client's call to
>> gnutls_server_name_set( .. GNUTLS_NAME_DNS .. ), and that makes
>> OpenSSL happy.
>>
>> If I do not invoke gnutls_server_name_set(), we have a happy
>> conversation. If I invoke gnutls_server_name_set(), OpenSSL bombs out
>> during the handshake.
>>
>> Has anyone seen this before?
> 
> We've seen it for very old implementations, notably some IBM-derived
> variant of OpenSSL, that cannot handle any extensions.  But it is very
> surprising to see it for a recent OpenSSL.  Are you sure OpenSSL 1.0.0
> is used?  Can you reproduce this using 'openssl s_server'?  Maybe the
> application server is requesting SSLv2 from OpenSSL?

The application is the client, and since the application is GnuTLS, it can't 
be asking for SSLv2.

Yes, Fedora 12, OpenSSL 1.0.0 is the server side. It's configured to accept 
all protocols (SSLv23_method() in OpenSSL's API), but I also tried 
TLSv1_method() as well, no difference.

On the GnuTLS client side, I'm specifying  GNUTLS_TLS1_1, GNUTLS_TLS1_0, and 
GNUTLS_SSL3 in that order. This is not a direct SSL/TLS connection, this is 
IMAP STARTTLS, so I can't easily drop in s_server in the server's place.

I'll explore what debugging messages are available on the OpenSSL side. I 
gave up on the debugger. Debugging optimized code, on either the server or 
the client side, just doesn't work very well.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: </pipermail/attachments/20100429/794d8476/attachment.pgp>