[gnutls-devel] GnuTLS | Problems related to the suite after the fips mode is turned off (#1485)

[Read-only notification of GnuTLS library development activities]

wang cheng created an issue: https://gitlab.com/gnutls/gnutls/-/issues/1485



## Description of problem:
The fips mode of the system will generate /etc/system-fips and /proc/sys/crypto/fips_enabled files. Gnutls will use these two files to handle fips-related tasks in global init. But there is a bug in _gnutls_global_init:
```
#ifdef ENABLE_FIPS140
	res = _gnutls_fips_mode_enabled();
	/* res == 1 -> fips140-2 mode enabled
	 * res == 2 -> only self checks performed - but no failure
	 * res == not in fips140 mode
	 */
	if (res != 0) {
		_gnutls_debug_log("FIPS140-2 mode: %d\n", res);
		_gnutls_priority_update_fips();

		/* first round of self checks, these are done on the
		 * nettle algorithms which are used internally */
		ret = _gnutls_fips_perform_self_checks1();
		if (res != 2) {
			if (ret < 0) {
				gnutls_assert();
				goto out;
			}
		}
	}
#endif
```
When the system disables fips mode, the result of `_gnutls_fips_mode_enabled` function is res=2. As mentioned in the above code, only self checks will perform when res=2. But `_gnu_tls_priority update fips` is called, which caused the changing of algorithm suite selection. This is inconsistent with actual fips status.

So a if syntax should be added before the `_gnutls_priority_update_fips` function.

-- 
Reply to this email directly or view it on GitLab: https://gitlab.com/gnutls/gnutls/-/issues/1485
You're receiving this email because of your account on gitlab.com.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.gnupg.org/pipermail/gnutls-devel/attachments/20230605/fe7cbd6d/attachment.html>