[gnutls-devel] GnuTLS | Cannot use gnutls_pkcs7_sign() with GNUTLS_DIG_SHAKE_256 (#1719)

[Read-only notification of GnuTLS library development activities]

Richard Hughes created an issue: https://gitlab.com/gnutls/gnutls/-/issues/1719



Hi all,

I'm building PQC into libjcat, using GnuTLS. I'm loading a ML-DSA87 PKCS#7 certificate (and private key) and then signing some data using `gnutls_pkcs7_sign()`. This works wonderfully, until I add `GNUTLS_PKCS7_INCLUDE_TIME` which causes the sign process to abort with `GNUTLS_E_INVALID_REQUEST`.

The backtrace explains a little what's happening:

    #1  0x00007ffff784c0ad in gnutls_hash_fast (algorithm=GNUTLS_DIG_UNKNOWN, ptext=0x41fe68 <payload_str>, ptext_len=14, digest=digest at entry=0x7fffffffcdc0) at ../../lib/crypto-api.c:968
    #2  0x00007ffff78eb8b9 in write_attributes (root=0x7ffff79ba1c3 "signerInfos.?LAST.signedAttrs", c2=0x461740, data=0x7fffffffd010, me=0x7ffff7a74eb0 <hash_algorithms+1456>, other_attrs=<optimized out>, flags=2) at ../../../lib/x509/pkcs7.c:2273
    #3  gnutls_pkcs7_sign (pkcs7=0x452d50, signer=0x45dfe0, signer_key=0x458ed0, data=0x7fffffffd010, signed_attrs=<optimized out>, unsigned_attrs=<optimized out>, dig=GNUTLS_DIG_SHAKE_256, flags=2) at ../../../lib/x509/pkcs7.c:2452

So, we're calling `gnutls_pkcs7_sign(dig: gnutls_digest_algorithm_t=GNUTLS_DIG_SHAKE_256)`, `write_attributes(me: mac_entry_st)` which when calls `gnutls_hash_fast(MAC_TO_DIG(me->id))` -- `MAC_TO_DIG()` being an alias to `_gnutls_mac_to_dig()`.

Looking at the source code for that, I see:

    inline static gnutls_digest_algorithm_t
    _gnutls_mac_to_dig(gnutls_mac_algorithm_t mac)
    {
    	if (unlikely(mac >= GNUTLS_MAC_AEAD))
    		return GNUTLS_DIG_UNKNOWN;
    	return (gnutls_digest_algorithm_t)mac;
    }

...and sure enough, `GNUTLS_MAC_SHAKE_256` is indeed larger than `GNUTLS_MAC_AEAD` as defined in `gnutls_mac_algorithm_t` from `lib/includes/gnutls/gnutls.h.in`

I'm happy to write a merge request to fix this, but I'm not sure what the correct fix would be. Moving `GNUTLS_MAC_SHAKE_256` to be a lower number than `GNUTLS_MAC_AEAD` would work -- i.e. just after `GNUTLS_MAC_STREEBOG_512` -- but that's probably an API break.

I personally think adding the two constants to `_gnutls_mac_to_dig()` would be best. Thanks!

-- 
Reply to this email directly or view it on GitLab: https://gitlab.com/gnutls/gnutls/-/issues/1719
You're receiving this email because of your account on gitlab.com.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.gnupg.org/pipermail/gnutls-devel/attachments/20250627/8eda67c5/attachment-0001.html>