Possible flaw in MPI code w.r.t. sign flag

[Werner Koch]

On Fri, 10 May 2019 21:00, HeikoStamer at gmx.net said:

> gcry_mpi_set_ui(b, 0UL); // a = -0, b = +0
> assert(gcry_mpi_cmp(a, b)); // SHOULD be 0, but returns -1
> gcry_mpi_release(a), gcry_mpi_release(b);
>
> Any suggestions?

We need to handle the +-0 case first.  What about this:

--8<---------------cut here---------------start------------->8---
diff --git a/mpi/mpi-cmp.c b/mpi/mpi-cmp.c
index 66e09612..0c349b96 100644
--- a/mpi/mpi-cmp.c
+++ b/mpi/mpi-cmp.c
@@ -89,6 +89,10 @@ do_mpi_cmp (gcry_mpi_t u, gcry_mpi_t v, int absmode)
       usign = absmode? 0 : u->sign;
       vsign = absmode? 0 : v->sign;
 
+      /* Special treatment for +0 == -0 */
+      if (!usize && !vsize)
+        return 0;
+
       /* Compare sign bits.  */
 
       if (!usign && vsign)
--8<---------------cut here---------------end--------------->8---


Shalom-Salam,

   Werner


-- 
Die Gedanken sind frei.  Ausnahmen regelt ein Bundesgesetz.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <https://lists.gnupg.org/pipermail/gcrypt-devel/attachments/20190513/0d4d88a4/attachment.sig>