From n.mavrogiannopoulos at gmail.com Thu Aug 10 10:31:28 2017 From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos) Date: Thu, 10 Aug 2017 10:31:28 +0200 Subject: [gnutls-devel] symbol and library versioning Message-ID: Hi, With mind in the upcoming 3.6.0 release I tried to codify/clarify the library and symbol versioning rules that are considered best practice today. The output is at: https://gitlab.com/gnutls/gnutls/blob/master/CONTRIBUTING.md#symbol-and-library-versioning My take-away from it, is that the complexity of libtool rules is unnecessary and in fact all that is being used from it is the major soname version (thus the minor and patch versions set via libtool are being used by no software). The versioning that matters is via the linker script, which is then used by packaging software like rpm (I guess possibly dpkg as well), to detect proper dependencies. Did I miss something in that? Are there are best practices I missed, or any issues in the rules I set above? regards, Nikos From berrange at redhat.com Thu Aug 10 11:14:26 2017 From: berrange at redhat.com (Daniel P. Berrange) Date: Thu, 10 Aug 2017 10:14:26 +0100 Subject: [gnutls-devel] symbol and library versioning In-Reply-To: References: Message-ID: <20170810091426.GA12980@redhat.com> On Thu, Aug 10, 2017 at 10:31:28AM +0200, Nikos Mavrogiannopoulos wrote: > Hi, > With mind in the upcoming 3.6.0 release I tried to codify/clarify the > library and symbol versioning rules that are considered best practice > today. > > The output is at: > https://gitlab.com/gnutls/gnutls/blob/master/CONTRIBUTING.md#symbol-and-library-versioning > > My take-away from it, is that the complexity of libtool rules is > unnecessary and in fact all that is being used from it is the major > soname version (thus the minor and patch versions set via libtool are > being used by no software). Yes, libtool versioning is not really useful, especially for a library that intends to maintain API/ABI compat long term / forever. > The versioning that matters is via the linker script, which is then > used by packaging software like rpm (I guess possibly dpkg as well), > to detect proper dependencies. > > Did I miss something in that? Are there are best practices I missed, > or any issues in the rules I set above? I believe your backporting rule is a bad idea. "if symbol gnutls_xyz with version GNUTLS_3_6_3 is backported on gnutls 3.3.15, it should use version GNUTLS_3_3_15." This creates ABI breakage when you upgrade gnutls in the distro. eg An application which links to your gnutls 3.3.15 version and uses symbol gnutls_xyz will get a embedded symbol reference of gnutls_xyz at GNUTLS_3_3_15. When you then update to gnutls 3.6.3 in the distro, the linker will be unable to resolve gnutls_xyz at GNUTLS_3_3_15, because the library will now be exporting gnutls_xyz at GNUTLS_3_6_3 instead. There's a few other options for backporting, but they are all awful in their own ways: https://www.berrange.com/posts/2011/01/13/versioning-in-the-libvirt-library/ so for libvirt we simply refuse to ever backport public APIs to older versions. Backports are for bug fixes only, never features. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| From n.mavrogiannopoulos at gmail.com Thu Aug 10 13:41:55 2017 From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos) Date: Thu, 10 Aug 2017 13:41:55 +0200 Subject: [gnutls-devel] symbol and library versioning In-Reply-To: <20170810091426.GA12980@redhat.com> References: <20170810091426.GA12980@redhat.com> Message-ID: On Thu, Aug 10, 2017 at 11:14 AM, Daniel P. Berrange wrote: > On Thu, Aug 10, 2017 at 10:31:28AM +0200, Nikos Mavrogiannopoulos wrote: >> Hi, >> With mind in the upcoming 3.6.0 release I tried to codify/clarify the >> library and symbol versioning rules that are considered best practice >> today. >> >> The output is at: >> https://gitlab.com/gnutls/gnutls/blob/master/CONTRIBUTING.md#symbol-and-library-versioning >> >> My take-away from it, is that the complexity of libtool rules is >> unnecessary and in fact all that is being used from it is the major >> soname version (thus the minor and patch versions set via libtool are >> being used by no software). > > Yes, libtool versioning is not really useful, especially for a library > that intends to maintain API/ABI compat long term / forever. I wonder whether there is a reason not to keep the libtool numbers fixed, and follow its versioning requirements. >> The versioning that matters is via the linker script, which is then >> used by packaging software like rpm (I guess possibly dpkg as well), >> to detect proper dependencies. >> >> Did I miss something in that? Are there are best practices I missed, >> or any issues in the rules I set above? > > I believe your backporting rule is a bad idea. > > "if symbol gnutls_xyz with version GNUTLS_3_6_3 is backported on > gnutls 3.3.15, it should use version GNUTLS_3_3_15." > > This creates ABI breakage when you upgrade gnutls in the distro. eg > > An application which links to your gnutls 3.3.15 version and uses > symbol gnutls_xyz will get a embedded symbol reference of > gnutls_xyz at GNUTLS_3_3_15. When you then update to gnutls 3.6.3 in > the distro, the linker will be unable to resolve gnutls_xyz at GNUTLS_3_3_15, > because the library will now be exporting gnutls_xyz at GNUTLS_3_6_3 > instead. Right. Note however that in gnutls we haven't kept ABI compatibility over the years and this statement applies to backporting to an ABI-incompatible version. Backporting to ABI-compatible versions is explicitly disallowed to avoid the issues you describe in your post. It is my intention to try keeping a single supported stable tree with a given ABI (and hopefully I succeed in that). > There's a few other options for backporting, but they are all awful > in their own ways: > > https://www.berrange.com/posts/2011/01/13/versioning-in-the-libvirt-library/ > so for libvirt we simply refuse to ever backport public APIs to older > versions. Backports are for bug fixes only, never features. Thank you, that's a nice read. I should have read that years ago. regards, Nikos From berrange at redhat.com Thu Aug 10 14:20:01 2017 From: berrange at redhat.com (Daniel P. Berrange) Date: Thu, 10 Aug 2017 13:20:01 +0100 Subject: [gnutls-devel] symbol and library versioning In-Reply-To: References: <20170810091426.GA12980@redhat.com> Message-ID: <20170810122001.GI12980@redhat.com> On Thu, Aug 10, 2017 at 01:41:55PM +0200, Nikos Mavrogiannopoulos wrote: > On Thu, Aug 10, 2017 at 11:14 AM, Daniel P. Berrange > wrote: > > On Thu, Aug 10, 2017 at 10:31:28AM +0200, Nikos Mavrogiannopoulos wrote: > >> Hi, > >> With mind in the upcoming 3.6.0 release I tried to codify/clarify the > >> library and symbol versioning rules that are considered best practice > >> today. > >> > >> The output is at: > >> https://gitlab.com/gnutls/gnutls/blob/master/CONTRIBUTING.md#symbol-and-library-versioning > >> > >> My take-away from it, is that the complexity of libtool rules is > >> unnecessary and in fact all that is being used from it is the major > >> soname version (thus the minor and patch versions set via libtool are > >> being used by no software). > > > > Yes, libtool versioning is not really useful, especially for a library > > that intends to maintain API/ABI compat long term / forever. > > I wonder whether there is a reason not to keep the libtool numbers > fixed, and follow its versioning requirements. AFAIK, the per-symbol versions offer a superset of the functionality provided by libtool soname versioning, at least on ELF platforms. If you care about Windows DLLs or OS-X dynlibs, then perhaps the libtool versions would have some small benefit, but I don't think its worth it personally. > >> The versioning that matters is via the linker script, which is then > >> used by packaging software like rpm (I guess possibly dpkg as well), > >> to detect proper dependencies. > >> > >> Did I miss something in that? Are there are best practices I missed, > >> or any issues in the rules I set above? > > > > I believe your backporting rule is a bad idea. > > > > "if symbol gnutls_xyz with version GNUTLS_3_6_3 is backported on > > gnutls 3.3.15, it should use version GNUTLS_3_3_15." > > > > This creates ABI breakage when you upgrade gnutls in the distro. eg > > > > An application which links to your gnutls 3.3.15 version and uses > > symbol gnutls_xyz will get a embedded symbol reference of > > gnutls_xyz at GNUTLS_3_3_15. When you then update to gnutls 3.6.3 in > > the distro, the linker will be unable to resolve gnutls_xyz at GNUTLS_3_3_15, > > because the library will now be exporting gnutls_xyz at GNUTLS_3_6_3 > > instead. > > Right. Note however that in gnutls we haven't kept ABI compatibility > over the years and this statement applies to backporting to an > ABI-incompatible version. Backporting to ABI-compatible versions is > explicitly disallowed to avoid the issues you describe in your post. Ok, yes, if you have an .so ELF version change between those two versions, then apps need a rebuild already, so the problem I describe wouldn't hit. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| From ametzler at bebt.de Thu Aug 10 14:21:29 2017 From: ametzler at bebt.de (Andreas Metzler) Date: Thu, 10 Aug 2017 14:21:29 +0200 Subject: [gnutls-devel] symbol and library versioning In-Reply-To: References: Message-ID: <20170810122129.pntvzk325aeelfuo@argenau.bebt.de> On 2017-08-10 Nikos Mavrogiannopoulos wrote: > With mind in the upcoming 3.6.0 release I tried to codify/clarify the > library and symbol versioning rules that are considered best practice > today. > The output is at: > https://gitlab.com/gnutls/gnutls/blob/master/CONTRIBUTING.md#symbol-and-library-versioning > My take-away from it, is that the complexity of libtool rules is > unnecessary and in fact all that is being used from it is the major > soname version (thus the minor and patch versions set via libtool are > being used by no software). Hello, yes and no. ;-) Afaiui libtool versioning is an abstraction level. On e.g. Linux the only important result is that a soname bump happens when necessary. On other more exotic platforms libtool might (need to) do something different. > The versioning that matters is via the linker script, which is then > used by packaging software like rpm (I guess possibly dpkg as well), > to detect proper dependencies. rpm and dpkg work differently here. rpm looks at the library at build time and scans it for versioned symbol definitions. If the library exports GNUTLS_3_4_0 and GNUTLS_3_4_1 the resulting library package provides libgnutls.so.30 libgnutls.so.30 (GNUTLS_3_4_0) libgnutls.so.30 (GNUTLS_3_4_1) And a package linking against gnutls which uses foo at GNUTLS_3_4_1 will get a dependency against "libgnutls.so.30 (GNUTLS_3_4_1)". Debian works a little bit differently. There is no benefit for dpkg based systems in using the fine grained symbol version approach. The gnutls development package comes with a list of exported symbols[1] and the respective version that a using package needs to depend on: gnutls_idna_reverse_map at GNUTLS_3_4 3.5.9 gnutls_init at GNUTLS_3_4 3.5.6 gnutls_key_generate at GNUTLS_3_4 3.5.0 If a binary uses both gnutls_key_generate and gnutls_init it will depend on gnutls >= 3.5.6 (the maximum of 3.5.6 and 3.5.0). There is usually a manual review process involved in updating this list of symbols. This is evident by the gnutls_init() dependency info. gnutls_init was introduced ages ago, however the GNUTLS_NO_TICKETS argument was only added in 3.5.6. > Did I miss something in that? Are there are best practices I missed, > or any issues in the rules I set above? | Symbol versioning as provided by libgnutls.map have several advantages. | 1 they allow for symbol clashing between different gnutls library | versions being in the same address space. | 2 they allow installers to detect the library version used for an | application utilizing a specific symbol | 3 the allow introducing multiple versions of a symbol a la libc, | keeping the semantics of old functions while introducing new. 1 is what GnuTLS symbol-versioning in released versions (all symbols versioned with a unified sonamed-specific label) currently accomplishes. 2 seems to be what you are aiming for, enhancing RedHat's package dependencies. 3 Is this a goal you are envisioning of following? If not it would clear things up to clearly state that GnuTLS is not (currently) using symbol versioning for this purpose. (On a sidenote: Is there anybody except glibc doing this?) I think there is typo in the following text, the numbers seem to be off. s/GNUTLS_3_6_2/GNUTLS_3_6_3/ | As such for every symbol introduced on a particular version, we create | an entry in libgnutls.map based on the version and containing the new | symbols. For example, if in version 3.6.3 we introduce symbol | gnutls_xyz, the entry would be: | | GNUTLS_3_6_2 { global: gnutls_xyz; } GNUTLS_3_6_1; | | where GNUTLS_3_6_1 is the last version that symbols were introduced, and | indicates a dependency of 3.6.2 to symbols of 3.6.1. This has not been a problem with the old approach. | Backporting new symbols to an old version which is soname compatible is | not allowed (can cause quite some problems). [...] cu Andreas [1] There is also an older, less fine grained way, where the development ackage simply specifies "any binary build against and linking with this version of the library gets a dependency >= x.y.z). If new symols are introduced the depinfo is bumped and the generated dependency becomes stricer even if the new symbol is not used. -- `What a good friend you are to him, Dr. Maturin. His other friends are so grateful to you.' `I sew his ears on from time to time, sure' From n.mavrogiannopoulos at gmail.com Fri Aug 11 08:59:30 2017 From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos) Date: Fri, 11 Aug 2017 08:59:30 +0200 Subject: [gnutls-devel] symbol and library versioning In-Reply-To: <20170810122129.pntvzk325aeelfuo@argenau.bebt.de> References: <20170810122129.pntvzk325aeelfuo@argenau.bebt.de> Message-ID: <1502434770.18197.1.camel@gmail.com> On Thu, 2017-08-10 at 14:21 +0200, Andreas Metzler wrote: > > The versioning that matters is via the linker script, which is then > > used by packaging software like rpm (I guess possibly dpkg as > > well), > > to detect proper dependencies. > > rpm and dpkg work differently here. > rpm looks at the library at build time and scans it for versioned > symbol > definitions. If the library exports GNUTLS_3_4_0 and GNUTLS_3_4_1 the > resulting library package provides > libgnutls.so.30 > libgnutls.so.30 (GNUTLS_3_4_0) > libgnutls.so.30 (GNUTLS_3_4_1) > > And a package linking against gnutls which uses foo at GNUTLS_3_4_1 will > get a dependency against "libgnutls.so.30 (GNUTLS_3_4_1)". > > Debian works a little bit differently. There is no benefit for dpkg > based systems in using the fine grained symbol version approach.??The > gnutls development package comes with a list of exported symbols[1] > and > the respective version that a using package needs to depend on: > ?gnutls_idna_reverse_map at GNUTLS_3_4 3.5.9 > ?gnutls_init at GNUTLS_3_4 3.5.6 > ?gnutls_key_generate at GNUTLS_3_4 3.5.0 > If a binary uses both gnutls_key_generate and gnutls_init it will > depend > on gnutls >= 3.5.6 (the maximum of 3.5.6 and 3.5.0). > There is usually a manual review process involved in updating this > list > of symbols. This is evident by the gnutls_init() dependency info. > gnutls_init was introduced ages ago, however the GNUTLS_NO_TICKETS > argument was only added in 3.5.6. Thank you for this nice summary. My target is to have that versioning suits both systems, and if I understand you correctly despite the differences in packagers, that's the case. > > Did I miss something in that? Are there are best practices I > > missed, > > or any issues in the rules I set above? > > Symbol versioning as provided by libgnutls.map have several > > advantages. > > ? 1 they allow for symbol clashing between different gnutls library > > ????versions being in the same address space. > > ? 2 they allow installers to detect the library version used for an > > ????application utilizing a specific symbol > > ? 3 the allow introducing multiple versions of a symbol a la libc, > > ????keeping the semantics of old functions while introducing new. > > 1 is what GnuTLS symbol-versioning in released versions (all symbols > versioned with a unified sonamed-specific label) currently > accomplishes. > 2 seems to be what you are aiming for, enhancing RedHat's package > dependencies. Right. > 3 Is this a goal you are envisioning of following? If not it would > clear > things up to clearly state that GnuTLS is not (currently) using > symbol > versioning for this purpose. Nice question. I was thinking to use that feature as tool to prevent breaking ABI on cases where compatibility could be kept relatively easily, or when the enhancement of a function would break all previous callers, rather than strive keep a 100% backwards compatibility on bugs. > (On a sidenote: Is there anybody except > glibc doing this?) I'm not aware of other projects using that to the level of glibc. > I think there is typo in the following text, the numbers seem to be > off. > s/GNUTLS_3_6_2/GNUTLS_3_6_3/ Thanks. It should be addressed now. > > As such for every symbol introduced on a particular version, we > > create > > an entry in libgnutls.map based on the version and containing the > > new > > symbols. For example, if in version 3.6.3 we introduce symbol > > gnutls_xyz, the entry would be: > > > > GNUTLS_3_6_2 { global: gnutls_xyz; } GNUTLS_3_6_1; > > > > where GNUTLS_3_6_1 is the last version that symbols were > > introduced, and > > indicates a dependency of 3.6.2 to symbols of 3.6.1. > > This has not been a problem with the old approach.? Do you refer to the text above or the backporting issue? regards, Nikos > > > > From ametzler at bebt.de Fri Aug 11 11:46:24 2017 From: ametzler at bebt.de (Andreas Metzler) Date: Fri, 11 Aug 2017 11:46:24 +0200 Subject: [gnutls-devel] symbol and library versioning In-Reply-To: <1502434770.18197.1.camel@gmail.com> References: <20170810122129.pntvzk325aeelfuo@argenau.bebt.de> <1502434770.18197.1.camel@gmail.com> Message-ID: <20170811094624.3kjwcuviitdvgir5@argenau.bebt.de> On 2017-08-11 Nikos Mavrogiannopoulos wrote: > On Thu, 2017-08-10 at 14:21 +0200, Andreas Metzler wrote: [...] >>> As such for every symbol introduced on a particular version, we >>> create an entry in libgnutls.map based on the version and containing >>> the new symbols. For example, if in version 3.6.3 we introduce >>> symbol gnutls_xyz, the entry would be: >>> GNUTLS_3_6_2 { global: gnutls_xyz; } GNUTLS_3_6_1; >>> where GNUTLS_3_6_1 is the last version that symbols were introduced, >>> and indicates a dependency of 3.6.2 to symbols of 3.6.1. >> This has not been a problem with the old approach.? > Do you refer to the text above or the backporting issue? Hello, I was refering to the backporting issue ("Backporting new symbols to an old version which is soname compatible is not allowed"). e.g. gnutls_pkcs7_get_embedded_data_oid() was backported to 3.4.17 from 3.5.6. Afaiui this would be forbidden/impossible now. If 3.4.17 exported gnutls_pkcs7_get_embedded_data_oid at GNUTLS3_4_17 then 3.4.17 and 3.5.6. would have been ABI incompatible, an upgrade from 3.4.x to 3.5.x would require a rebuild of all binaries using gnutls_pkcs7_get_embedded_data_oid. I am not sure about the problems caused by doing it the other way round (3.4.17 exporting gnutls_pkcs7_get_embedded_data_oid at GNUTLS3_5_6) but I guess rpm's dependency tracking might stumble. - Or would the differently chained version definitions (GNUTLS3_5_6 depending on GNUTLS3_4_8 or GNUTLS3_5_5 respectively) break the ABI? cu Andreas -- `What a good friend you are to him, Dr. Maturin. His other friends are so grateful to you.' `I sew his ears on from time to time, sure' From nmav at gnutls.org Fri Aug 11 13:24:18 2017 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Fri, 11 Aug 2017 13:24:18 +0200 Subject: [gnutls-devel] symbol and library versioning In-Reply-To: <20170811094624.3kjwcuviitdvgir5@argenau.bebt.de> References: <20170810122129.pntvzk325aeelfuo@argenau.bebt.de> <1502434770.18197.1.camel@gmail.com> <20170811094624.3kjwcuviitdvgir5@argenau.bebt.de> Message-ID: On Fri, Aug 11, 2017 at 11:46 AM, Andreas Metzler wrote: > On 2017-08-11 Nikos Mavrogiannopoulos wrote: >> On Thu, 2017-08-10 at 14:21 +0200, Andreas Metzler wrote: > [...] >>>> As such for every symbol introduced on a particular version, we >>>> create an entry in libgnutls.map based on the version and containing >>>> the new symbols. For example, if in version 3.6.3 we introduce >>>> symbol gnutls_xyz, the entry would be: > >>>> GNUTLS_3_6_2 { global: gnutls_xyz; } GNUTLS_3_6_1; > >>>> where GNUTLS_3_6_1 is the last version that symbols were introduced, >>>> and indicates a dependency of 3.6.2 to symbols of 3.6.1. > >>> This has not been a problem with the old approach. > >> Do you refer to the text above or the backporting issue? > > Hello, > I was refering to the backporting issue ("Backporting new symbols to an > old version which is soname compatible is not allowed"). e.g. > gnutls_pkcs7_get_embedded_data_oid() was backported to 3.4.17 from > 3.5.6. > > Afaiui this would be forbidden/impossible now. If 3.4.17 exported > gnutls_pkcs7_get_embedded_data_oid at GNUTLS3_4_17 then 3.4.17 and 3.5.6. > would have been ABI incompatible, an upgrade from 3.4.x to 3.5.x would > require a rebuild of all binaries using > gnutls_pkcs7_get_embedded_data_oid. Right. That's why it is now forbidden. A work around this could be for later 3.5.x versions to link the old symbol to the new one, though that is still a work around, as it still has some rough points (some 3.5.x releases would still be abi incompatible). > I am not sure about the problems caused by doing it the other way round > (3.4.17 exporting gnutls_pkcs7_get_embedded_data_oid at GNUTLS3_5_6) but I > guess rpm's dependency tracking might stumble. - Or would the > differently chained version definitions (GNUTLS3_5_6 depending on > GNUTLS3_4_8 or GNUTLS3_5_5 respectively) break the ABI? My understanding is that the chaining has no effect on rpm tracking, nor affect the ABI. However as you mention above that method will break its dependency tracking. regards, Nikos From nmav at gnutls.org Mon Aug 21 08:24:30 2017 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Mon, 21 Aug 2017 08:24:30 +0200 Subject: [gnutls-devel] gnutls 3.5.15 Message-ID: <1503296670.9015.1.camel@gnutls.org> Hello,? ?I've just released gnutls 3.5.15. This is a bug fix release on the current stable branch. * Version 3.5.15 (released 2017-08-21) ** libgnutls: Disable hardware acceleration on aarch64/ilp32 mode. There is ???no assembler code included for this CPU mode. ** certtool: Keys with provable RSA and DSA parameters are now only exported ???in PKCS#8 form, following draft-mavrogiannopoulos-pkcs8-validated-parameters-00.txt. ???This removes the need for a non-standard key format. ** API and ABI modifications: No changes since last version. Getting the Software ==================== GnuTLS may be downloaded directly from .??A list of GnuTLS mirrors can be found at . Here are the XZ compressed sources: ? ftp://ftp.gnutls.org/gcrypt/gnutls/v3.5/gnutls-3.5.15.tar.xz Here are OpenPGP detached signatures signed using key 0x96865171: ? ftp://ftp.gnutls.org/gcrypt/gnutls/v3.5/gnutls-3.5.15.tar.xz.sig Note that it has been signed with my openpgp key: pub???3104R/96865171 2008-05-04 [expires: 2028-04-29] uid??????????????????Nikos Mavrogiannopoulos gnutls.org> uid??????????????????Nikos Mavrogiannopoulos gmail.com> sub???2048R/9013B842 2008-05-04 [expires: 2018-05-02] sub???2048R/1404A91D 2008-05-04 [expires: 2018-05-02] regards, Nikos From nmav at gnutls.org Mon Aug 21 09:00:04 2017 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Mon, 21 Aug 2017 09:00:04 +0200 Subject: [gnutls-devel] GnuTLS 3.6.0 released Message-ID: <1503298804.9015.4.camel@gnutls.org> We are proud to announce a new GnuTLS release: Version 3.6.0. GnuTLS is a modern C library that implements the standard network security protocol Transport Layer Security (TLS), for use by network applications.??GnuTLS is developed for GNU/Linux, but works on many Unix-like systems as well as Windows. The GnuTLS library is distributed under the terms of the GNU Lesser General Public License version 2 (or later). The project pages of the library are available at: http://www.gnutls.org/ What's New ========== Version 3.6.0 is the first release on the 3.6.x branch and is the result of a several months of planning and work on the git master branch. The GnuTLS 3.6.x branch is marked as stable-next, meaning it is considered of stable quality but does not yet replace the?current stable releases based on 3.5.x, which will continue to be supported until the 3.6.x branch replaces it. Note that, support for the new TLS 1.3 protocol will be added on the 3.6.x branch. An extended summary of the major changes is available at: http://nmav.gnutls.org/2017/08/gnutls-3-6-0.html The NEWS entries follow. * Version 3.6.0 (released 2017-08-21) ** libgnutls: tlsfuzzer is part of the CI testsuite. This is a TLS testing and fuzzying toolkit, allowing for corner case testing, and ensuring that the behavior of the library will not change across releases. https://github.com/tomato42/tlsfuzzer ** libgnutls: Introduced a lock-free random generator which operates per-thread and eliminates random-generator related bottlenecks in multi-threaded operation. Resolves gitlab issue #141. http://nmav.gnutls.org/2017/03/improving-by-simplifying-gnutls-prng.html ** libgnutls: Replaced the Salsa20 random generator with one based on CHACHA. The goal is to reduce code needed in cache (CHACHA is also used for TLS), and the number of primitives used by the library. That does not affect the AES-DRBG random generator used in FIPS140-2 mode. ** libgnutls: Added support for RSA-PSS key type as well as signatures in certificates, and TLS key exchange. Contributed by Daiki Ueno. RSA-PSS signatures can be generated by RSA-PSS keys and normal RSA keys, but not vice-versa. The feature includes: * RSA-PSS key generation and key handling (in PKCS#8 form) * RSA-PSS key generation and key handling from PKCS#11 (with CKM_RSA_PKCS_PSS mech) * Handling of RSA-PSS subjectPublicKeyInfo parameters, when present in either the private key or certificate. * RSA-PSS signing and verification of PKIX certificates * RSA-PSS signing and verification of TLS 1.2 handshake * RSA-PSS signing and verification of PKCS#7 structures * RSA-PSS and RSA key combinations for TLS credentials. That is, when multiple keys are supplied, RSA-PSS keys are preferred over RSA for RSA-PSS TLS signatures, to contain risks of cross-protocol attacks between the algorithms. * RSA-PSS key conversion to RSA PKCS#1 form (certtool --to-rsa) Note that RSA-PSS signatures with SHA1 are (intentionally) not supported. ** libgnutls: Added support for Ed25519 signing in certificates and TLS key exchange following draft-ietf-tls-rfc4492bis-17. The feature includes: * Ed25519 key generation and key handling (in PKCS#8 form) * Ed25519 signing and verification of PKIX certificates * Ed25519 signing and verification of TLS 1.2 handshake * Ed25519 signing and verification of PKCS#7 structures ** libgnutls: Enabled X25519 key exchange by default, following draft-ietf-tls-rfc4492bis-17. ** libgnutls: Added support for Diffie-Hellman group negotiation following RFC7919. That makes the DH parameters negotiation more robust and less prone to errors due to insecure parameters. Servers are no longer required to specific explicit DH parameters, though if they do these parameters will be used. Group selection can be done via priority strings. The introduced strings are GROUP-ALL, GROUP-FFDHE2048, GROUP-FFDHE3072, GROUP-FFDHE4096 and GROUP-FFDHE8192, as well as the corresponding to curves groups. Note that the 6144 group from RFC7919 is not supported. ** libgnutls: Introduced various sanity checks on certificate import. Refuse to import certificates which have fractional seconds in Time fields, X.509v1 certificates which have the unique identifiers set, and certificates with illegal version numbers. All of these are prohibited by RFC5280. ** libgnutls: Introduced gnutls_x509_crt_set_flags(). This function can set flags in the crt structure. The only flag supported at the moment is GNUTLS_X509_CRT_FLAG_IGNORE_SANITY which skips the certificate sanity checks on import. ** libgnutls: PKIX certificates with unknown critical extensions are rejected on verification with status GNUTLS_CERT_UNKNOWN_CRIT_EXTENSIONS. This behavior can be overriden by providing the flag GNUTLS_VERIFY_IGNORE_UNKNOWN_CRIT_EXTENSIONS to verification functions. Resolves gitlab issue #177. ** libgnutls: Refuse to generate a certificate with an illegal version, or an illegal serial number. That is, gnutls_x509_crt_set_version() and gnutls_x509_crt_set_serial(), will fail on input considered to be invalid in RFC5280. ** libgnutls: Calls to gnutls_record_send() and gnutls_record_recv() prior to handshake being complete are now refused. Addresses gitlab issue #158. ** libgnutls: Added support for PKCS#12 files with no salt (zero length) in their password encoding, and PKCS#12 files using SHA384 and SHA512 as MAC. ** libgnutls: Exported functions to encode and decode DSA and ECDSA r,s values. ** libgnutls: Added new callback setting function to gnutls_privkey_t for external keys. The new function (gnutls_privkey_import_ext4), allows signing in addition to previous algorithms (RSA PKCS#1 1.5, DSA, ECDSA), with RSA-PSS and Ed25519 keys. ** libgnutls: Introduced the %VERIFY_ALLOW_BROKEN and %VERIFY_ALLOW_SIGN_WITH_SHA1 priority string options. These allows enabling all broken and SHA1-based signature algorithms in certificate verification, respectively. ** libgnutls: 3DES-CBC is no longer included in the default priorities list. It has to be explicitly enabled, e.g., with a string like "NORMAL:+3DES-CBC". ** libgnutls: SHA1 was marked as insecure for signing certificates. Verification of certificates signed with SHA1 is now considered insecure and will fail, unless flags intended to enable broken algorithms are set. Other uses of SHA1 are still allowed. This can be reverted on compile time with the configure flag --enable-sha1-support. ** libgnutls: RIPEMD160 was marked as insecure for certificate signatures. Verification of certificates signed with RIPEMD160 hash algorithm is now considered insecure and will fail, unless flags intended to enable broken algorithms are set. ** libgnutls: No longer enable SECP192R1 and SECP224R1 by default on TLS handshakes. These curves were rarely used for that purpose, provide no advantage over x25519 and were deprecated by TLS 1.3. ** libgnutls: Removed support for DEFLATE, or any other compression method. ** libgnutls: OpenPGP authentication was removed; the resulting library is ABI compatible, with the openpgp related functions being stubs that fail on invocation. ** libgnutls: Removed support for libidn (i.e., IDNA2003); gnutls can now be compiled only with libidn2 which provides IDNA2008. ** certtool: The option '--load-ca-certificate' can now accept PKCS#11 URLs in addition to files. ** certtool: The option '--load-crl' can now be used when generating PKCS#12 files (i.e., in conjunction with '--to-p12' option). ** certtool: Keys with provable RSA and DSA parameters are now only read and exported from PKCS#8 form, following draft-mavrogiannopoulos-pkcs8-validated-parameters-00.txt. This removes support for the previous a non-standard key format. ** certtool: Added support for generating, printing and handling RSA-PSS and Ed25519 keys and certificates. ** certtool: the parameters --rsa, --dsa and --ecdsa to --generate-privkey are now deprecated, replaced by the --key-type option. ** p11tool: The --generate-rsa, --generate-ecc and --generate-dsa options were replaced by the --generate-privkey option. ** psktool: Generate 256-bit keys by default. ** gnutls-server: Increase request buffer size to 16kb, and added the --alpn and --alpn-fatal options, allowing testing of ALPN negotiation. ** API and ABI modifications: gnutls_encode_rs_value: Added gnutls_decode_rs_value: Added gnutls_base64_encode2: Added gnutls_base64_decode2: Added gnutls_x509_crt_set_flags: Added gnutls_x509_crt_check_ip: Added gnutls_x509_ext_import_inhibit_anypolicy: Added gnutls_x509_ext_export_inhibit_anypolicy: Added gnutls_x509_crt_get_inhibit_anypolicy: Added gnutls_x509_crt_set_inhibit_anypolicy: Added gnutls_pubkey_export_rsa_raw2: Added gnutls_pubkey_export_dsa_raw2: Added gnutls_pubkey_export_ecc_raw2: Added gnutls_privkey_export_rsa_raw2: Added gnutls_privkey_export_dsa_raw2: Added gnutls_privkey_export_ecc_raw2: Added gnutls_x509_spki_init: Added gnutls_x509_spki_deinit: Added gnutls_x509_spki_get_pk_algorithm: Added gnutls_x509_spki_set_pk_algorithm: Added gnutls_x509_spki_get_digest_algorithm: Added gnutls_x509_spki_set_digest_algorithm: Added gnutls_x509_spki_get_salt_size: Added gnutls_x509_spki_set_salt_size: Added gnutls_x509_crt_set_spki: Added gnutls_x509_crt_get_spki: Added gnutls_x509_privkey_get_spki: Added gnutls_x509_privkey_set_spki: Added gnutls_x509_crq_get_spki: Added gnutls_x509_crq_set_spki: Added gnutls_pubkey_set_spki: Added gnutls_pubkey_get_spki: Added gnutls_privkey_set_spki: Added gnutls_privkey_get_spki: Added gnutls_privkey_import_ext4: Added GNUTLS_EXPORT_FLAG_NO_LZ: Added GNUTLS_DT_IP_ADDRESS: Added GNUTLS_X509_CRT_FLAG_IGNORE_SANITY: Added GNUTLS_CERT_UNKNOWN_CRIT_EXTENSIONS: Added GNUTLS_VERIFY_ALLOW_SIGN_WITH_SHA1: Added GNUTLS_VERIFY_DO_NOT_ALLOW_IP_MATCHES: Added GNUTLS_VERIFY_IGNORE_UNKNOWN_CRIT_EXTENSIONS: Added GNUTLS_SFLAGS_RFC7919: Added Community ========= If you need help to use GnuTLS, or want to help others, you are invited to join our help-gnutls mailing list, see: ? http://lists.gnutls.org/mailman/listinfo/gnutls-help If you wish to participate in the development of GnuTLS, you are invited to join our gnutls-dev mailing list, see: ? http://lists.gnutls.org/mailman/listinfo/gnutls-dev Internationalization ==================== The GnuTLS library messages have been translated into Czech, Dutch, French, German, Italian, Malay, Polish, Simplified Chinese, Swedish, and Vietnamese.??We welcome the addition of more translations. Getting the Software ==================== GnuTLS may be downloaded directly from .??A list of GnuTLS mirrors can be found at . Here are the XZ compressed sources: https://www.gnupg.org/ftp/gcrypt/gnutls/v3.6/gnutls-3.6.0.tar.xz ? ftp://ftp.gnutls.org/gcrypt/gnutls/v3.6/gnutls-3.6.0.tar.xz Here are OpenPGP detached signatures signed using key 0x96865171: https://www.gnupg.org/ftp/gcrypt/gnutls/v3.6/gnutls-3.6.0.tar.xz.sig ? ftp://ftp.gnutls.org/gcrypt/gnutls/v3.6/gnutls-3.6.0.tar.xz.sig Note that it has been signed with my openpgp key: pub???3104R/96865171 2008-05-04 [expires: 2028-04-29] uid??????????????????Nikos Mavrogiannopoulos gnutls.org> uid??????????????????Nikos Mavrogiannopoulos gmail.com> sub???2048R/9013B842 2008-05-04 [expires: 2018-05-02] sub???2048R/1404A91D 2008-05-04 [expires: 2018-05-02] regards, Nikos From a at juaristi.eus Sat Aug 26 10:54:01 2017 From: a at juaristi.eus (Ander Juaristi) Date: Sat, 26 Aug 2017 10:54:01 +0200 Subject: [gnutls-devel] Towards TLS 1.3 Message-ID: <1844890.DApogGho4a@mjo> Hi list, I'm a cryptography & network security MSc student at Royal Holloway, University of London. Some of you will probably know me better from wget though. It's exciting to see some known faces here ;D Starting on mid September, I'll be working on the implementation of TLS 1.3 (3.6.x branch) as part of my MSc project. I approached Nikos back at FOSDEM to propose this project idea to him, and please accept my apologies for introducing myself to the list now, rather than then (february), but I was waiting for my MSc marks to be confirmed definitely before taking the commitment. I'll start working on the new record format and on the design of a new set of APIs to accommodate 0-RTT. And then follow along with some more work after that. I'm taking as a roadmap the milestones set at [0]. I won't however, have time probably to implement all the new features so there's still plenty of room for other interested people to get involved, such as Daiki who's working on RSA- PSS or Nikos himself who recently implemented the new HKDF. Best regards, - AJ [0] https://gitlab.com/gnutls/gnutls/milestones/8 From ametzler at bebt.de Sat Aug 26 15:31:46 2017 From: ametzler at bebt.de (Andreas Metzler) Date: Sat, 26 Aug 2017 15:31:46 +0200 Subject: [gnutls-devel] GnuTLS 3.6.0 released In-Reply-To: <1503298804.9015.4.camel@gnutls.org> References: <1503298804.9015.4.camel@gnutls.org> Message-ID: <20170826133146.6hngfxsc7ms7czov@argenau.bebt.de> On 2017-08-21 Nikos Mavrogiannopoulos wrote: > We are proud to announce a new GnuTLS release: Version 3.6.0. [...] > ** libgnutls: tlsfuzzer is part of the CI testsuite. This is a TLS testing and > fuzzying toolkit, allowing for corner case testing, and ensuring that the > behavior of the library will not change across releases. > https://github.com/tomato42/tlsfuzzer Hello, I have tried building with --enable-fuzzer-target, "make check" gets stuck when running crq_key_id test and rng-fork fails. I am not sure about the proper way to run the fuzzing testsuite. I guess I should run "make -C fuzz check" *instead* of "make check"? cu Andreas -- `What a good friend you are to him, Dr. Maturin. His other friends are so grateful to you.' `I sew his ears on from time to time, sure' From ametzler at bebt.de Sat Aug 26 18:15:35 2017 From: ametzler at bebt.de (Andreas Metzler) Date: Sat, 26 Aug 2017 18:15:35 +0200 Subject: [gnutls-devel] [Patch] typoes Message-ID: <20170826161535.pcrqwgsegoryhhhn@argenau.bebt.de> Hello, find attached some typo fixes suggested by lintian. cu Andreas -- `What a good friend you are to him, Dr. Maturin. His other friends are so grateful to you.' `I sew his ears on from time to time, sure' -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Fix-some-typoes.patch Type: text/x-diff Size: 3107 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From nmav at gnutls.org Sun Aug 27 09:50:01 2017 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Sun, 27 Aug 2017 09:50:01 +0200 Subject: [gnutls-devel] GnuTLS 3.6.0 released In-Reply-To: <20170826133146.6hngfxsc7ms7czov@argenau.bebt.de> References: <1503298804.9015.4.camel@gnutls.org> <20170826133146.6hngfxsc7ms7czov@argenau.bebt.de> Message-ID: On Sat, Aug 26, 2017 at 3:31 PM, Andreas Metzler wrote: > On 2017-08-21 Nikos Mavrogiannopoulos wrote: >> We are proud to announce a new GnuTLS release: Version 3.6.0. > [...] >> ** libgnutls: tlsfuzzer is part of the CI testsuite. This is a TLS testing and >> fuzzying toolkit, allowing for corner case testing, and ensuring that the >> behavior of the library will not change across releases. >> https://github.com/tomato42/tlsfuzzer > > Hello, > > I have tried building with --enable-fuzzer-target, "make check" gets > stuck when running crq_key_id test and rng-fork fails. Hi, Maybe we need better documentation for the fuzzying options. If your intention is to run the tlsfuzzer, it doesn't need a particular configure option, it runs on the stock library. It is included in the set of tests in tests/suite/. It is only available in git repo, not on releases as it requires tlsfuzzer/tlslite which are git submodules. The AFL or libfuzzer checks (in fuzz/ subdir) automatically discover paths to search in code and modify input accordingly in order to trigger them. To reduce search space, the enable-fuzzer-mode enables a predictable random generator, removes necessary checks like the finished messages, drops iteration counts (like in PKCS#12 files), to lower levels, and so on. These are run using helper scripts in fuzz/, e.g, ./run-afl.sh, and they run indefinitely. The option make -C fuzz, during dist or CI in git, runs the most "interesting" paths discovered by fuzzers and other reproducers through the available fuzzers, i.e., something like regression testing. That itself, also doesn't need any special configure option to be run. I hope that helps. regards, Nikos From nmav at gnutls.org Sun Aug 27 09:51:56 2017 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Sun, 27 Aug 2017 09:51:56 +0200 Subject: [gnutls-devel] [Patch] typoes In-Reply-To: <20170826161535.pcrqwgsegoryhhhn@argenau.bebt.de> References: <20170826161535.pcrqwgsegoryhhhn@argenau.bebt.de> Message-ID: On Sat, Aug 26, 2017 at 6:15 PM, Andreas Metzler wrote: > Hello, > > find attached some typo fixes suggested by lintian. Thank you. They look reasonable. Could you resend with the signoff tag? regards, Nikos From nmav at gnutls.org Sun Aug 27 10:02:58 2017 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Sun, 27 Aug 2017 10:02:58 +0200 Subject: [gnutls-devel] Towards TLS 1.3 In-Reply-To: <1844890.DApogGho4a@mjo> References: <1844890.DApogGho4a@mjo> Message-ID: On Sat, Aug 26, 2017 at 10:54 AM, Ander Juaristi wrote: > Hi list, > > I'm a cryptography & network security MSc student at Royal Holloway, > University of London. Some of you will probably know me better from wget > though. It's exciting to see some known faces here ;D > > Starting on mid September, I'll be working on the implementation of TLS 1.3 > (3.6.x branch) as part of my MSc project. I approached Nikos back at FOSDEM to > propose this > project idea to him, and please accept my apologies for introducing myself to > the list now, rather than then (february), but I was waiting for my MSc marks > to be confirmed definitely before taking the commitment. > I'll start working on the new record format and on the design of a new set of > APIs to accommodate 0-RTT. And then follow along with some more work after > that. Welcome Ander, Currently I maintain a parallel branch with some patches for TLS 1.3, at tmp-tls1.3-support. This is currently a "raw" branch with unreviewed commits, and completed items from that one move with slower pace to master (at the moment only features which are not tls1.3 only). I try to add new features in parallel with their test suite, which slows things down a bit though tlsfuzzer getting tls 1.3 will help on that. The branch is still ongoing for basic 1.3 support, and only key exchange is supported. I plan to resume on this branch after holidays (I'm back on 6th of september) so let's sync prior you start working. regards, Nikos From ametzler at bebt.de Sun Aug 27 11:28:38 2017 From: ametzler at bebt.de (Andreas Metzler) Date: Sun, 27 Aug 2017 11:28:38 +0200 Subject: [gnutls-devel] [Patch] typoes In-Reply-To: References: <20170826161535.pcrqwgsegoryhhhn@argenau.bebt.de> Message-ID: <20170827092838.rvwcox3xadlrbj4o@argenau.bebt.de> On 2017-08-27 Nikos Mavrogiannopoulos wrote: > On Sat, Aug 26, 2017 at 6:15 PM, Andreas Metzler wrote: >> find attached some typo fixes suggested by lintian. > Thank you. They look reasonable. Could you resend with the signoff tag? Is it okay like this? cu Andreas -- `What a good friend you are to him, Dr. Maturin. His other friends are so grateful to you.' `I sew his ears on from time to time, sure' -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Fix-some-typoes.patch Type: text/x-diff Size: 3159 bytes Desc: not available URL: From nmav at gnutls.org Sun Aug 27 15:40:36 2017 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Sun, 27 Aug 2017 15:40:36 +0200 Subject: [gnutls-devel] [Patch] typoes In-Reply-To: <20170827092838.rvwcox3xadlrbj4o@argenau.bebt.de> References: <20170826161535.pcrqwgsegoryhhhn@argenau.bebt.de> <20170827092838.rvwcox3xadlrbj4o@argenau.bebt.de> Message-ID: <1503841236.2059.0.camel@gnutls.org> On Sun, 2017-08-27 at 11:28 +0200, Andreas Metzler wrote: > On 2017-08-27 Nikos Mavrogiannopoulos wrote: > > On Sat, Aug 26, 2017 at 6:15 PM, Andreas Metzler > > wrote: > > > find attached some typo fixes suggested by lintian. > > Thank you. They look reasonable. Could you resend with the signoff > > tag? > > Is it okay like this? > Thank you, applied. From berrange at redhat.com Tue Aug 29 17:23:35 2017 From: berrange at redhat.com (Daniel P. Berrange) Date: Tue, 29 Aug 2017 16:23:35 +0100 Subject: [gnutls-devel] GnuTLS 3.6.0 released In-Reply-To: <1503298804.9015.4.camel@gnutls.org> References: <1503298804.9015.4.camel@gnutls.org> Message-ID: <20170829152335.GF25801@redhat.com> On Mon, Aug 21, 2017 at 09:00:04AM +0200, Nikos Mavrogiannopoulos wrote: > We are proud to announce a new GnuTLS release: Version 3.6.0. > > GnuTLS is a modern C library that implements the standard network > security protocol Transport Layer Security (TLS), for use by network > applications.??GnuTLS is developed for GNU/Linux, but works on many > Unix-like systems as well as Windows. > > The GnuTLS library is distributed under the terms of the GNU Lesser > General Public License version 2 (or later). > > The project pages of the library are available at: > http://www.gnutls.org/ [snip] > ** libgnutls: SHA1 was marked as insecure for signing certificates. Verification > of certificates signed with SHA1 is now considered insecure and will > fail, unless flags intended to enable broken algorithms are set. Other uses > of SHA1 are still allowed. This can be reverted on compile time with the configure > flag --enable-sha1-support. FWIW, as a result of this change, apps that use gnutls_x509_crt_sign() will be generating certs that will never validate, since that API is hardcoded to use SHA1. This tripped up the libvirt & QEMU test suites which used that API. I'm switching libvirt/qemu to use gnutls_x509_crt_sign2() passing SHA256 explicitly, but I wondered if you'd considered updating gnutls_x509_crt_sign() to use SHA256 too, or must it stick with SHA1 for backcompat reasons ? If so, it would be worth putting a note in the API docs that any use of gnutls_x509_crt_sign() now almost certainly broken due SHA1 being untrusted. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|