[gnutls-devel] GnuTLS | gnutls-cli on macOS aborts with "Curve 1.3.36.3.3.2.8.1.1.7 is not supported" and assertions when server cert uses brainpoolP256r1 (#1767)

[Read-only notification of GnuTLS library development activities]

ma ma created an issue: https://gitlab.com/gnutls/gnutls/-/issues/1767



Summary When connecting with gnutls-cli on macOS to a server whose X.509 certificate uses an EC public key with the Brainpool curve (brainpoolP256r1, OID 1.3.36.3.3.2.8.1.1.7), gnutls fails while parsing the certificate. The client prints "Curve ... is not supported" and a chain of ASSERTs, then aborts the handshake with a fatal certificate error.

Expected behavior gnutls-cli should either successfully import the certificate (if brainpool is supported) or return a clear, non‑assertive error such as "unsupported curve OID". It should not hit internal ASSERTs and crash/abort.

Actual behavior gnutls aborts the handshake. Relevant log excerpts:

|\<2\>| Curve 1.3.36.3.3.2.8.1.1.7 is not supported 

|\<3\>| ASSERT: key_decode.c\[\_gnutls_x509_read_ecc_params\]:274 

|\<3\>| ASSERT: mpi.c\[\_gnutls_get_asn_mpis\]:155 ... \*\*\* Fatal error: Fehler im Zertifikat.

<figure>

Reproduction steps

1. Run (example on macOS): gnutls-cli -p 443 -d 25 --x509certfile=pki/server.crt.pem --x509keyfile=pki/server.key.pem 192.168.165.101
2. Observe the debug output; the certificate parsing fails with the messages shown above.

Technical notes / hypothesis

* The logs indicate gnutls does not recognize the brainpool OID and therefore cannot decode the ECC parameters from the certificate. This leads to unexpected NULL/invalid values while parsing MPIs and triggers internal ASSERTs.
* Likely causes:
  * The crypto backend (nettle/libgcrypt) used by this gnutls build does not expose the Brainpool curve OID/parameters.
  * The macOS build may be missing OID-to-curve mapping or configuration to register brainpool curves.
* Request: even if brainpool support is not desired, the library should handle unsupported curves gracefully (return a proper error), instead of aborting with assertions.

Requested assistance

1. Can maintainers confirm whether gnutls (which versions) is expected to support RFC‑5639 Brainpool curves?
2. If yes: what exact build dependencies and configure flags are required to enable brainpool support on macOS?
3. If this is a bug: please consider a patch to avoid ASSERTs and return a clean error path when encountering unknown curve OIDs in certificates.

</figure>

-- 
Reply to this email directly or view it on GitLab: https://gitlab.com/gnutls/gnutls/-/issues/1767
You're receiving this email because of your account on gitlab.com.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.gnupg.org/pipermail/gnutls-devel/attachments/20251124/5480fff2/attachment.html>