[WIP] DTLS 1.0 preliminary patches

[Simon Josefsson]

Jonathan Bastien-Filiatrault <joe at x2a.org> writes:

> Hello,
>
> Being interested in DTLS and GnuTLS I have decided to try to implement
> DTLS in the GnuTLS library.

Great!

> I have managed to send a valid DTLS ClientHello using a modified GnuTLS
> in a relatively non-intrusive way (but which may break the ABI since it
> messes with existing enum values). The OpenSSL implementation responds
> to this ClientHello with a HelloVerifyMessage and Wireshark considers
> the packet valid DTLS.
>
> You may find my patches at this URL: http://x2a.org/pub/dtls/

Even better.

If you feel it would help you, you could have git access so you could
push your work to a special branch.  You need a savannah.gnu.org account
and sign FSF papers though.

> Unfortunately the lower end of the record layer and buffer/transport
> layer seems rather messy to my untrained eye. I am having trouble
> imagining implementing UDP buffering easely. I would need to buffer the
> whole packet then iterate over the records contained within the packet.
>
> The main problem seems to be layering violations between the handshake,
> record and buffer layers. Would it be better if _gnutls_{recv,send}_int
> dealt with whole records (and possibly return prematurely if more data
> or buffer space is required) ? _gnutls_{recv,send}_int could also deal
> with the SSLv2.0 record encapsulation. The handhake layer would
> therefore deal with those two functions for sending/receiving from the
> lower layer. The handshake layer buffering would also be moved to
> gnutls_handshake.c.
>
> Am I making any sense ?

I agree that code is hairy, and I'm hoping Nikos can give you some
advice.  I don't feel strongly about changing this code, but it will
need significant testing before we can feel comfortable with the
changes.  Maybe as a bonus side-effect, the code will become more
readable... ;)

Re 0002-Add-DTLS1.0-protocol-entry.patch: This breaks the API.  Can you
re-order the DTLS addition so it is after GNUTLS_TLS1_2 and add a '=
100' after it so there is room for TLS 1.3 etc?  Also, please drop the
GNUTLS_DTLS1 mapping, I think it helps to be specific about version
numbers at all places.  I think this patch could be added quickly
without problem.

Re 0003-Add-DTLS-state.patch: do you think there will be more DTLS
additions?  I suggest putting the DTLS stuff into a separate struct, if
you think it is OK.

Re 0004-Add-gnutls_session_datagram-function.patch: this just toggles
one way.  DTLS is really a completely new protocol, not just a different
transport method for TLS.  So maybe there should really be a new
function that replaces gnutls_init?  How about gnutls_init_dtls?  It
would return a gnutls_session_t for DTLS.

Re last part of 0005-Make-version-lookup-transport-dependent.patch: Is
this a good idea?  Running DTLS over TCP won't work, will it?  But I
understand why you need this.  Hm.  Maybe the default priorities needs
to be DTLS-specific -- this also argues for having gnutls_init_dtls
function so that the priority setting functions know for certain that
the session is a DTLS or non-DTLS session, and that won't change later.

Generally, I like your incremental approach since it helps to review and
merge the patches.  I definitely think we should create a branch for
your DTLS work.

/Simon