[gnutls-devel] GnuTLS | The GNUTLS Release 3.3.16 has a bug in the DTLS Non-Blocking logic, bug located at gnutls-3.6.16/lib/record.c in function _gnutls_recv_in_buffers at lines 1307 and 1322 (#1413)

[Read-only notification of GnuTLS library development activities]

Chuck Wanner created an issue: https://gitlab.com/gnutls/gnutls/-/issues/1413



## Description of problem:
We are currently using the GNUTLS Library for a DTLS Server and Client.   Our design requires the GNUTLS_NONBLOCK flag to be set.  We wrote our DTLS Software back in 2018 and 2019.  We were using GNUTLS version 3.3.26 without any issues.   We are stepping up to Redhat 8 which is using GNUTLS Version 3.6.16.

We configure the DTLS Client and Server to be non-blocking, but our GNUTLS Pull Timeout function is never invoked.  Only the GNUTLS Pull Function is invoked.

I believe the issue that we are encountering is a logic error in gnutls-3.6.16/lib/record.c in function  _gnutls_recv_in_buffers at lines 1307 and 1322.

For example at line 1307:
	ret = recv_headers(session, record_params, type, htype, &record, (!(session->internals.flags & GNUTLS_NONBLOCK))?&ms:0);
When the GNUTLS_NONBLOCK flag is set, the recv_headers will expect the pointer to the timeout value to be set.  But this line will always pass NULL when the GNUTLS_NONBLOCK flag is set.  I believe the solution to the problem should be remove the not logic.

The fix for line 1307:
ret = recv_headers(session, record_params, type, htype, &record, (session->internals.flags & GNUTLS_NONBLOCK)?&ms:0);

The same issue exists at line 1322:
	/* Read the packet data and insert it to record_recv_buffer.
	 */
	ret =
	    _gnutls_io_read_buffered(session, record.packet_size,
				     record.type, (!(session->internals.flags & GNUTLS_NONBLOCK))?&ms:0);
The fix for line 1322:
	/* Read the packet data and insert it to record_recv_buffer.
	 */
	ret =
	    _gnutls_io_read_buffered(session, record.packet_size,
				     record.type, (session->internals.flags & GNUTLS_NONBLOCK)?&ms:0);

Note: There are a number of locations where the GNUTLS_NONBLOCK flag is used.  I did not review the other locations to determine if there is an issue.

Note: I did verify the same logic still exists in GNUTLS Version 3.7.8.

Below is some of our code sections initializing the GNUTLS Library.

At the start we invoke the following:

    /*
    ** Initialize the GNU TLS DTLS Session
    */
    gnutls_ret_code = gnutls_init(&mpGnuTlsSession, GNUTLS_SERVER | GNUTLS_DATAGRAM | GNUTLS_NONBLOCK);

We setup private data, pull functions, and push functions:
        /*
        ** Provide the UDP Socket to the GNU TLS DTLS Server
        **
        ** Note: No return value
        */
        gnutls_transport_set_ptr(this->mpGnuTlsSession, &this->mPrivateData);
        gnutls_transport_set_push_function(this->mpGnuTlsSession, gnutlsPushFunction);
        gnutls_transport_set_pull_function(this->mpGnuTlsSession, gnutlsPullFunction);
        gnutls_transport_set_pull_timeout_function(this->mpGnuTlsSession,
                                                   gnutlsPullTimeoutFunction);


After the Handshake is complete, we invoke the following to configure the Timeout:
            /*
            ** Set the Receive Timeout for the Application Data
            **
            ** Note: No return value
            */
            gnutls_record_set_timeout(this->mpGnuTlsSession, cDtlsTimeoutMs);

We invoke the following to process the application data sent from the DTLS Client or Server:

    /*
    ** Check to see if message is available
    */
    gnutls_recv_status = gnutls_record_recv_seq(this->mpGnuTlsSession, (void *)mRcvBuffer, cDtlsRecvMaxBufferSize, sequence);

We verified that the GNUTLS Pull Timeout function is never invoked.  Only the GNUTLS Pull function is invoked.

## Version of gnutls used:
Version 3.3.16

## Distributor of gnutls (e.g., Ubuntu, Fedora, RHEL)
RHEL 8

## How reproducible:
Configure the DTLS Server or Client with the GNUTLS_NONBLOCK flag.
Configure the DTLS Server or Client with the pull and pull timeout functions with log statements to verify when they are called.
Set the timeout with function gnutls_record_set_timeout.
After the handshake is complete between the server and client, invoke gnutls_record_recv_seq. 

Steps to Reproduce:

 * one
 * two
 * three

## Actual results:
With the logging in the Pull functions, you will only see the GNUTLS Pull function invoked.  The GNUTLS Pull Timeout function is never invoked.

## Expected results:

With the logging in the Pull functions, you should see the GNUTLS Pull Timeout function invoked.  The GNUTLS Pull function should be invoked if the GNUTLS Pull Timeout has determine data is available.

-- 
Reply to this email directly or view it on GitLab: https://gitlab.com/gnutls/gnutls/-/issues/1413
You're receiving this email because of your account on gitlab.com.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.gnupg.org/pipermail/gnutls-devel/attachments/20221012/0c63f205/attachment-0001.html>