[gnutls-dev] Re: Gnutls4win: Problem with custom push/pull functions, errno and Visual Studio (Workaround included)

[Simon Josefsson]

Tim Kosse <tim.kosse at filezilla-project.org> writes:

> I encountered some problems while trying to use custom push/pull
> functions in a program compiled with Visual Studio.
>
> Just setting errno to EAGAIN and returning -1 caused gnutls to fail with
> GNUTLS_E_PUSH_ERROR.
>
> I found out that libgnutls-13.dll is linked against msvcrt.dll, whereas
> visual studio programs use a different runtime library (msvcr71.dll for
> example). These libraries bring along their own version of errno.
>
> So the errno inside the program is different from the one used in GnuTLS.

Ouch.  So it seems our POSIX dependencies actually are send + recv +
errno, and it is only possible to replace send/recv with new push/pull
functions.  We should definitely fix this.  Grep'ing the code reveals
that fortunately the only use is in gnutls_buffers.c, after calling
send or recv.

> A workaround is to load the correct errno from msvcrt.dll:
>
> 	HMODULE crtdll = LoadLibrary(_T("msvcrt.dll"));
> 	typedef int* (*t_errno)(void);
> 	t_errno pErrno = (t_errno)GetProcAddress(crtdll, "_errno");
> 	*pErrno() = EAGAIN;
> 	FreeLibrary(crtdll);

Nice.

> Would it be possible to add a function like for example void
> gnutls_transport_set_errno(int), which does nothing else than setting
> errno to the provided value?

I'm considering installing the patch below.  What do you think?

It makes gnutls_transport_set_errno set a session-specific errno
variable, in case there are multiple threads setting and reading from
the global errno variable.  I know some Unix libraries make errno a
thread-local variable, so that gnutls_transport_set_global_errno is
actually a safe function, but it seems unsafe to assume this holds for
all systems.  In particular, I wonder whether 'errno' is a
thread-local variable under Windows.

Thanks,
Simon

--- gnutls_int.h	13 Aug 2006 17:08:15 +0200	2.369
+++ gnutls_int.h	05 Nov 2006 09:41:09 +0100	
@@ -598,6 +598,13 @@
    */
   gnutls_datum_t recv_buffer;
 
+  /* To avoid using global variables, and especially on Windows where
+   * the application may use a different errno variable than GnuTLS,
+   * it is possible to use gnutls_transport_set_errno to set a
+   * session-specific errno variable in the user-replaceable push/pull
+   * functions.  This value is used by the send/recv functions. */
+  int errno;
+
   /* If you add anything here, check _gnutls_handshake_internal_state_clear().
    */
 } internals_st;
--- gnutls_buffers.c	13 Aug 2006 17:05:52 +0200	2.128
+++ gnutls_buffers.c	05 Nov 2006 09:48:35 +0100	
@@ -77,6 +77,54 @@
 # include <io_debug.h>
 #endif
 
+/**
+ * gnutls_transport_set_errno:
+ * @session: is a #gnutls_session_t structure.
+ * @err: error value to store in session-specific errno variable.
+ *
+ * Store @err in the session-specific errno variable.
+ *
+ * This function is useful in replacement push/pull functions set by
+ * gnutls_transport_set_push_function and
+ * gnutls_transport_set_pullpush_function under Windows, where the
+ * replacement push/pull may not have access to the same @errno
+ * variable that is used by GnuTLS (e.g., the application is linked to
+ * msvcr71.dll and gnutls is linked to msvcrt.dll).
+ *
+ * If you don't have the @session variable easily accessible from the
+ * push/pull function, and don't worry about thread conflicts, you can
+ * also use gnutls_transport_set_global_errno().
+ **/
+void
+gnutls_transport_set_errno (gnutls_session_t session, int err)
+{
+  session->errno = err;
+}
+
+/**
+ * gnutls_transport_set_global_errno:
+ * @err: error value to store in global errno variable.
+ *
+ * Store @err in the global errno variable.
+ *
+ * This function is useful in replacement push/pull functions set by
+ * gnutls_transport_set_push_function and
+ * gnutls_transport_set_pullpush_function under Windows, where the
+ * replacement push/pull may not have access to the same @errno
+ * variable that is used by GnuTLS (e.g., the application is linked to
+ * msvcr71.dll and gnutls is linked to msvcrt.dll).
+ *
+ * Whether this function is thread safe or not depends on whether the
+ * global variable errno is thread safe, some system libraries make it
+ * a thread-local variable.  When feasible, using the guaranteed
+ * thread-safe gnutls_transport_set_errno() may be better.
+ **/
+void
+gnutls_transport_set_global_errno (int err)
+{
+  errno = err;
+}
+
 /* Buffers received packets of type APPLICATION DATA and
  * HANDSHAKE DATA.
  */
@@ -264,6 +312,8 @@
   while (left > 0)
     {
 
+      session->errno = 0;
+
       if (session->internals._gnutls_pull_func == NULL)
 	i = recv (GNUTLS_POINTER_TO_INT(fd), &ptr[sizeOfPtr - left],
 		  left, flags);
@@ -274,10 +324,12 @@
 
       if (i < 0)
 	{
-	  _gnutls_read_log ("READ: %d returned from %d, errno=%d\n", i,
-			    fd, errno);
+	  int err = session->errno ? session->errno : errno;
+
+	  _gnutls_read_log ("READ: %d returned from %d, errno=%d gerrno=%d\n",
+			    i, fd, errno, session->errno);
 
-	  if (errno == EAGAIN || errno == EINTR)
+	  if (err == EAGAIN || err == EINTR)
 	    {
 	      if (sizeOfPtr - left > 0)
 		{
@@ -289,7 +341,7 @@
 		}
 	      gnutls_assert ();
 
-	      return RET (errno);
+	      return RET (err);
 	    }
 	  else
 	    {
@@ -701,6 +753,8 @@
   while (left > 0)
     {
 
+      session->errno = 0;
+
       if (session->internals._gnutls_push_func == NULL)
 	i = send (GNUTLS_POINTER_TO_INT(fd), &ptr[n - left], left, 0);
       else
@@ -708,7 +762,9 @@
 
       if (i == -1)
 	{
-	  if (errno == EAGAIN || errno == EINTR)
+	  int err = session->errno ? session->errno : errno;
+
+	  if (err == EAGAIN || err == EINTR)
 	    {
 	      session->internals.record_send_buffer_prev_size += n - left;
 
@@ -726,7 +782,7 @@
 		("WRITE: Interrupted. Stored %d bytes to buffer. Already sent %d bytes.\n",
 		 left, n - left);
 
-	      retval = RET (errno);
+	      retval = RET (err);
 
 	      return retval;
 	    }