CMAC + SERPENT/IDEA/RC2 buffer overflow/crash with oversized key

[Guido Vranken]

In the case of IDEA, an oversized key only appears to cause a crash
(assertion failure, so no memory corruption).
In the case of SERPENT, it appears to overwrite a stack buffer. That makes
it unlikely to be exploitable when stack protector is enabled (which is the
case for the binaries of most Linux distro's I think).
In the case of RC2, it appears to overwrite a heap buffer. That makes it
more likely to be exploitable but you probably still have to deal with ASLR.

In addition to that the host application needs to allow setting oversized
keys but applications usually only allow hardcoded, legal key lengths.

With that said, exploitation might be possible in specific circumstances.


On Fri, Apr 2, 2021 at 6:00 PM Andreas Metzler <ametzler at bebt.de> wrote:

> On 2021-03-31 Guido Vranken via Gcrypt-devel <gcrypt-devel at gnupg.org>
> wrote:
> > In the program below, each of three calls to cmac() causes a different
> > crash (use AddressSanitizer to be sure). I think the correct approach is
> to
> > make gcry_mac_setkey() return an error code if the key has an
> inappropriate
> > size.
> [...]
>
> Is this exploitable?
>
> cu Andreas
> --
> `What a good friend you are to him, Dr. Maturin. His other friends are
> so grateful to you.'
> `I sew his ears on from time to time, sure'
>
> _______________________________________________
> Gcrypt-devel mailing list
> Gcrypt-devel at gnupg.org
> http://lists.gnupg.org/mailman/listinfo/gcrypt-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.gnupg.org/pipermail/gcrypt-devel/attachments/20210402/696d21f4/attachment.html>