From Axel.Thimm at ATrpms.net Sun Jul 2 00:41:17 2006 From: Axel.Thimm at ATrpms.net (Axel Thimm) Date: Sun Jul 2 02:26:01 2006 Subject: executable stack in libgcrypt Message-ID: <20060701224117.GP24584@neu.nirvana> Hi, libgcrypt with asm marks the object and library as requiring executable stack. On systems with enforced selinux policy this breaks w/o special rules. Therefore some distributions (for instance Fedora and Ubuntu) simply turn off asm at build time (--disable-asm). The question is whether executable stack is needed, which it probably doesn't. But only a code review of the asm code can tell for sure, or the very authors of libgcrypt :) If the asm code does not need executable stack it should contain something like .section .note.GNU-stack, "", @progbits to mark the resulting object file as not needing it. Thanks! -- Axel.Thimm at ATrpms.net -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : /pipermail/attachments/20060702/b61c8168/attachment.pgp From wk at gnupg.org Mon Jul 3 17:31:48 2006 From: wk at gnupg.org (Werner Koch) Date: Mon Jul 3 17:36:11 2006 Subject: executable stack in libgcrypt In-Reply-To: <20060701224117.GP24584@neu.nirvana> (Axel Thimm's message of "Sun, 2 Jul 2006 00:41:17 +0200") References: <20060701224117.GP24584@neu.nirvana> Message-ID: <87d5cmzouz.fsf@wheatstone.g10code.de> On Sun, 2 Jul 2006 00:41, Axel Thimm said: > libgcrypt with asm marks the object and library as requiring > executable stack. On systems with enforced selinux policy this breaks Please check the configure options as well as the mail archiove from the last few weeks. > .section .note.GNU-stack, "", @progbits This is not portable as it relies on the GNU as. Thus we require for now a configure switch. Salam-Shalom, Werner From marcus.brinkmann at ruhr-uni-bochum.de Tue Jul 4 13:15:32 2006 From: marcus.brinkmann at ruhr-uni-bochum.de (Marcus Brinkmann) Date: Tue Jul 4 13:17:34 2006 Subject: Link with -no-undefined In-Reply-To: <87hd2e34mt.fsf@latte.josefsson.org> References: <87bqsp72lz.fsf@latte.josefsson.org> <87irmxf0le.wl%marcus.brinkmann@ruhr-uni-bochum.de> <87hd2e34mt.fsf@latte.josefsson.org> Message-ID: <87sllhwrhn.wl%marcus.brinkmann@ruhr-uni-bochum.de> At Wed, 21 Jun 2006 17:33:30 +0200, Simon Josefsson wrote: > > Marcus Brinkmann writes: > > > At Mon, 19 Jun 2006 14:26:48 +0200, > > Simon Josefsson wrote: > >> > >> Adding -no-undefined flag to libtool has a side-effect of producing an > >> DLL of libgcrypt when cross-compiling to mingw32. > >> > >> The Libtool documentation for the flag is somewhat confusing, but the > >> intention is that the flag is used when the library doesn't need any > >> externals symbols, which is true for libgcrypt. > > > > In GPGME, we have something like the below. We also have a resource > > file and a symbol export definition file (eg > > libgcrypt/w32-dll/libgcrypt.def). I think that any port of libgcrypt > > to the mingw32 target should have all of these, and whatever else is > > required to make libgcrypt work on w32 targets... > > Thanks. My point was that with -no-undefined, I do get a DLL that > seems to work fine on w32 machines. So I think my patch could be > applied without your additional work. If someone wants to integrate > your additional stuff, that would be fine too, of course. Frankly, I do not understand how you get a DLL at all. It doesn't even compile for me, there is at least a msghdr issue, and there was a socklen_t issue until I fixed this right now. I committed the changes to bring the windows port framework in line with what is in GPGME, but the port remains incomplete. If you have any patches to fix the remaining issues, why not send them in? I used your socklen.m4 from gnulib, nice work. :) Thanks, Marcus From marcus.brinkmann at ruhr-uni-bochum.de Tue Jul 4 13:20:18 2006 From: marcus.brinkmann at ruhr-uni-bochum.de (Marcus Brinkmann) Date: Tue Jul 4 13:22:23 2006 Subject: libgpg-error crash on MinGW In-Reply-To: <87irmu1fwc.fsf@latte.josefsson.org> References: <87irmu1fwc.fsf@latte.josefsson.org> Message-ID: <87r711wr9p.wl%marcus.brinkmann@ruhr-uni-bochum.de> At Wed, 21 Jun 2006 21:13:07 +0200, Simon Josefsson wrote: > > I get a crash in libgpg-error: [...] > The following patch fixes this. Thanks, I put it in. 2006-07-04 Marcus Brinkmann * src/init.c (get_locale_dir): Return NULL instead of garbage. * src/gpg-error.c (get_locale_dir): Likewise. Submitted by Simon Josefsson . Marcus From marcus.brinkmann at ruhr-uni-bochum.de Tue Jul 4 13:22:01 2006 From: marcus.brinkmann at ruhr-uni-bochum.de (Marcus Brinkmann) Date: Tue Jul 4 13:22:32 2006 Subject: C'n'p error in ./cipher/sha512.c copyright statement In-Reply-To: References: Message-ID: <87psglwr6u.wl%marcus.brinkmann@ruhr-uni-bochum.de> At Sun, 25 Jun 2006 14:38:43 +0000 (UTC), Andreas Metzler wrote: > cipher/sha512.c starts with: [...] > The last paragraph should also refer to "GNU Lesser general Public License" > instead of "GNU General Public License". Thanks, I put it in. 2006-07-04 Marcus Brinkmann * sha512.c: Fix typo in copyright notice. Marcus From jas at extundo.com Tue Jul 4 13:35:26 2006 From: jas at extundo.com (Simon Josefsson) Date: Tue Jul 4 13:34:08 2006 Subject: libgpg-error crash on MinGW In-Reply-To: <87r711wr9p.wl%marcus.brinkmann@ruhr-uni-bochum.de> (Marcus Brinkmann's message of "Tue, 04 Jul 2006 13:20:18 +0200") References: <87irmu1fwc.fsf@latte.josefsson.org> <87r711wr9p.wl%marcus.brinkmann@ruhr-uni-bochum.de> Message-ID: <87bqs5vc01.fsf@latte.josefsson.org> Marcus Brinkmann writes: > At Wed, 21 Jun 2006 21:13:07 +0200, > Simon Josefsson wrote: >> >> I get a crash in libgpg-error: > > [...] > >> The following patch fixes this. > > Thanks, I put it in. Seems to work fine here, thanks! > 2006-07-04 Marcus Brinkmann > > * src/init.c (get_locale_dir): Return NULL instead of garbage. > * src/gpg-error.c (get_locale_dir): Likewise. > Submitted by Simon Josefsson . > > Marcus From jas at extundo.com Tue Jul 4 13:43:39 2006 From: jas at extundo.com (Simon Josefsson) Date: Tue Jul 4 13:42:24 2006 Subject: Link with -no-undefined In-Reply-To: <87sllhwrhn.wl%marcus.brinkmann@ruhr-uni-bochum.de> (Marcus Brinkmann's message of "Tue, 04 Jul 2006 13:15:32 +0200") References: <87bqsp72lz.fsf@latte.josefsson.org> <87irmxf0le.wl%marcus.brinkmann@ruhr-uni-bochum.de> <87hd2e34mt.fsf@latte.josefsson.org> <87sllhwrhn.wl%marcus.brinkmann@ruhr-uni-bochum.de> Message-ID: <877j2tvbmc.fsf@latte.josefsson.org> Marcus Brinkmann writes: > At Wed, 21 Jun 2006 17:33:30 +0200, > Simon Josefsson wrote: >> >> Marcus Brinkmann writes: >> >> > At Mon, 19 Jun 2006 14:26:48 +0200, >> > Simon Josefsson wrote: >> >> >> >> Adding -no-undefined flag to libtool has a side-effect of producing an >> >> DLL of libgcrypt when cross-compiling to mingw32. >> >> >> >> The Libtool documentation for the flag is somewhat confusing, but the >> >> intention is that the flag is used when the library doesn't need any >> >> externals symbols, which is true for libgcrypt. >> > >> > In GPGME, we have something like the below. We also have a resource >> > file and a symbol export definition file (eg >> > libgcrypt/w32-dll/libgcrypt.def). I think that any port of libgcrypt >> > to the mingw32 target should have all of these, and whatever else is >> > required to make libgcrypt work on w32 targets... >> >> Thanks. My point was that with -no-undefined, I do get a DLL that >> seems to work fine on w32 machines. So I think my patch could be >> applied without your additional work. If someone wants to integrate >> your additional stuff, that would be fine too, of course. > > Frankly, I do not understand how you get a DLL at all. It doesn't > even compile for me, there is at least a msghdr issue, and there was a > socklen_t issue until I fixed this right now. I'm using libgcrypt 1.2.2, which seem to work fine under Windows when build with LDFLAGS=-no-undefined. > I committed the changes to bring the windows port framework in line > with what is in GPGME, but the port remains incomplete. If you have > any patches to fix the remaining issues, why not send them in? I tried building 1.3-cvs now, and I got the same problem you did. The problems seem to be due to src/ath.h. How about reverting it to what's in 1.2.2, which seems to work? Maybe other things are fixed in 1.2.2 that is broken in 1.3-cvs too. > I used your socklen.m4 from gnulib, nice work. :) Thanks! However, is it a good idea to use #ifdef HAVE_SYS_TYPES_H checks in an installed header file? Not all programs include a config.h that have those define. How about this? Index: gcrypt.h =================================================================== --- gcrypt.h (revision 1160) +++ gcrypt.h (working copy) @@ -29,16 +29,12 @@ #include -#if HAVE_SYS_SOCKET_H -#include -#else -#if HAVE_WINSOCK2_H +#if defined _WIN32 || defined __WIN32__ # include -#endif -#if HAVE_WS2TCPIP_H # include +#else +# include #endif -#endif #include From marcus.brinkmann at ruhr-uni-bochum.de Tue Jul 4 14:34:30 2006 From: marcus.brinkmann at ruhr-uni-bochum.de (Marcus Brinkmann) Date: Tue Jul 4 14:32:37 2006 Subject: Link with -no-undefined In-Reply-To: <877j2tvbmc.fsf@latte.josefsson.org> References: <87bqsp72lz.fsf@latte.josefsson.org> <87irmxf0le.wl%marcus.brinkmann@ruhr-uni-bochum.de> <87hd2e34mt.fsf@latte.josefsson.org> <87sllhwrhn.wl%marcus.brinkmann@ruhr-uni-bochum.de> <877j2tvbmc.fsf@latte.josefsson.org> Message-ID: <87odw5wnu1.wl%marcus.brinkmann@ruhr-uni-bochum.de> At Tue, 04 Jul 2006 13:43:39 +0200, Simon Josefsson wrote: > > Marcus Brinkmann writes: > > > At Wed, 21 Jun 2006 17:33:30 +0200, > > Simon Josefsson wrote: > >> > >> Marcus Brinkmann writes: > >> > >> > At Mon, 19 Jun 2006 14:26:48 +0200, > >> > Simon Josefsson wrote: > >> >> > >> >> Adding -no-undefined flag to libtool has a side-effect of producing an > >> >> DLL of libgcrypt when cross-compiling to mingw32. > >> >> > >> >> The Libtool documentation for the flag is somewhat confusing, but the > >> >> intention is that the flag is used when the library doesn't need any > >> >> externals symbols, which is true for libgcrypt. > >> > > >> > In GPGME, we have something like the below. We also have a resource > >> > file and a symbol export definition file (eg > >> > libgcrypt/w32-dll/libgcrypt.def). I think that any port of libgcrypt > >> > to the mingw32 target should have all of these, and whatever else is > >> > required to make libgcrypt work on w32 targets... > >> > >> Thanks. My point was that with -no-undefined, I do get a DLL that > >> seems to work fine on w32 machines. So I think my patch could be > >> applied without your additional work. If someone wants to integrate > >> your additional stuff, that would be fine too, of course. > > > > Frankly, I do not understand how you get a DLL at all. It doesn't > > even compile for me, there is at least a msghdr issue, and there was a > > socklen_t issue until I fixed this right now. > > I'm using libgcrypt 1.2.2, which seem to work fine under Windows when > build with LDFLAGS=-no-undefined. I see. Werner ported libgcrypt 1.2.2 in the LIBGCRYPT-1-2-BRANCH. I just merged those changes into the trunk. This gets us closer, but still not fully there yet. > > I committed the changes to bring the windows port framework in line > > with what is in GPGME, but the port remains incomplete. If you have > > any patches to fix the remaining issues, why not send them in? > > I tried building 1.3-cvs now, and I got the same problem you did. The > problems seem to be due to src/ath.h. How about reverting it to > what's in 1.2.2, which seems to work? Maybe other things are fixed in > 1.2.2 that is broken in 1.3-cvs too. Well, 1.3 has some more development, like in the entropy module. I have to leave it here for now, as this is not what I am currently working on. If you get further, let us know, and I may get a chance to look at any patches that are sent to the mailing list. > > I used your socklen.m4 from gnulib, nice work. :) > > Thanks! > > However, is it a good idea to use #ifdef HAVE_SYS_TYPES_H checks in an > installed header file? No, it's a dumb idea. I am embarrassed! > Not all programs include a config.h that have > those define. How about this? I put it in. I don't know what header files on w32 platforms are reliable, I trust your judgement in that matter. The "right" solution to these problems is to generate the header file from a header.in file. However, we try to avoid that. 2006-07-04 Marcus Brinkmann * gcrypt.h: Revert last change, and instead: [_WIN32 || __WIN32__]: Do not include , but and . Suggested by Simon Josefsson . Thanks, Marcus From jas at extundo.com Tue Jul 4 15:18:15 2006 From: jas at extundo.com (Simon Josefsson) Date: Tue Jul 4 15:17:19 2006 Subject: Link with -no-undefined In-Reply-To: <87odw5wnu1.wl%marcus.brinkmann@ruhr-uni-bochum.de> (Marcus Brinkmann's message of "Tue, 04 Jul 2006 14:34:30 +0200") References: <87bqsp72lz.fsf@latte.josefsson.org> <87irmxf0le.wl%marcus.brinkmann@ruhr-uni-bochum.de> <87hd2e34mt.fsf@latte.josefsson.org> <87sllhwrhn.wl%marcus.brinkmann@ruhr-uni-bochum.de> <877j2tvbmc.fsf@latte.josefsson.org> <87odw5wnu1.wl%marcus.brinkmann@ruhr-uni-bochum.de> Message-ID: <87k66ttso8.fsf@latte.josefsson.org> Marcus Brinkmann writes: >> > I committed the changes to bring the windows port framework in line >> > with what is in GPGME, but the port remains incomplete. If you have >> > any patches to fix the remaining issues, why not send them in? >> >> I tried building 1.3-cvs now, and I got the same problem you did. The >> problems seem to be due to src/ath.h. How about reverting it to >> what's in 1.2.2, which seems to work? Maybe other things are fixed in >> 1.2.2 that is broken in 1.3-cvs too. > > Well, 1.3 has some more development, like in the entropy module. I > have to leave it here for now, as this is not what I am currently > working on. If you get further, let us know, and I may get a chance > to look at any patches that are sent to the mailing list. It seems 1.3-cvs require some more work to build under MinGW, and since 1.2.2 works fine for me, with the minimal patch, I think I'll stay with 1.2.2 until someone makes 1.3-cvs work under MinGW. >> Not all programs include a config.h that have >> those define. How about this? > > I put it in. I don't know what header files on w32 platforms are > reliable, I trust your judgement in that matter. I'm not exactly sure either, but I don't have any good documentation to build on. The MSDN documentation regarding which platforms have which features seems incorrect. For example, it says on http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winsock/winsock/getaddrinfo_2.asp that getaddrinfo exists on many platforms (e.g., Win95), but the only system I've actually found it on is Windows XP. I think the only sane way to deal with this is to install things that seem to work now and wait for people to report when it doesn't work. > The "right" solution to these problems is to generate the header file > from a header.in file. However, we try to avoid that. I understand, we do it that way in GnuTLS, but it complicates things, and I'm not sure it is worth the trouble. /Simon From wk at gnupg.org Tue Jul 4 15:59:44 2006 From: wk at gnupg.org (Werner Koch) Date: Tue Jul 4 16:01:14 2006 Subject: Link with -no-undefined In-Reply-To: <87k66ttso8.fsf@latte.josefsson.org> (Simon Josefsson's message of "Tue, 04 Jul 2006 15:18:15 +0200") References: <87bqsp72lz.fsf@latte.josefsson.org> <87irmxf0le.wl%marcus.brinkmann@ruhr-uni-bochum.de> <87hd2e34mt.fsf@latte.josefsson.org> <87sllhwrhn.wl%marcus.brinkmann@ruhr-uni-bochum.de> <877j2tvbmc.fsf@latte.josefsson.org> <87odw5wnu1.wl%marcus.brinkmann@ruhr-uni-bochum.de> <87k66ttso8.fsf@latte.josefsson.org> Message-ID: <871wt1xygf.fsf@wheatstone.g10code.de> On Tue, 4 Jul 2006 15:18, Simon Josefsson said: > It seems 1.3-cvs require some more work to build under MinGW, and > since 1.2.2 works fine for me, with the minimal patch, I think I'll > stay with 1.2.2 until someone makes 1.3-cvs work under MinGW. Indeed, there are some things fixed in 1.2 which have not been ported to the trunk. Salam-Shalom, Werner From Axel.Thimm at ATrpms.net Wed Jul 5 02:10:51 2006 From: Axel.Thimm at ATrpms.net (Axel Thimm) Date: Wed Jul 5 02:09:38 2006 Subject: executable stack in libgcrypt In-Reply-To: <87d5cmzouz.fsf@wheatstone.g10code.de> References: <20060701224117.GP24584@neu.nirvana> <87d5cmzouz.fsf@wheatstone.g10code.de> Message-ID: <20060705001051.GB1419@neu.nirvana> On Mon, Jul 03, 2006 at 05:31:48PM +0200, Werner Koch wrote: > On Sun, 2 Jul 2006 00:41, Axel Thimm said: > > > libgcrypt with asm marks the object and library as requiring > > executable stack. On systems with enforced selinux policy this breaks > > Please check the configure options as well as the mail archiove from > the last few weeks. Thanks, I found the configure switch (although nothing in the archives, maybe on another gnupg.org list?) > > .section .note.GNU-stack, "", @progbits > > This is not portable as it relies on the GNU as. Thus we require for > now a configure switch. It breaks with binutils 2.11.93.0.2, 2.13.90.0.2 and 2.13.90.0.18 (no warning and 2x errors out). Would these versions support the .section construct? -- Axel.Thimm at ATrpms.net -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : /pipermail/attachments/20060705/4de43ff0/attachment.pgp From haypo at inl.fr Thu Jul 6 00:19:52 2006 From: haypo at inl.fr (haypo@inl.fr) Date: Thu Jul 6 01:25:52 2006 Subject: Questions about multi-threading Message-ID: <36208.82.247.156.212.1152137992.squirrel@mail.inl.fr> Hi, I'm working on the firewall NuFW [1]. I used gprof to find functions which takes a lot of time. I'm not sure, but I think that gcrypt really uses too much mutexes lock() and unlock(). I read gcrypt source code, and used gdb on server to catch lock() calls. I didn't understood everything, but I have some questions. (a) Why do ath_mutex_lock() and ath_mutex_unlock() call mutex_init()? It looks very weird for me, since this function use another mutex! (b) Why do ath_mutex_destroy() call mutex_init()??? It's stupid to create a mutex if it doesn't exist ... to destroy it just after its creation!? (c) Can a mutex lock() or unlock() fails? (or: it is necessary to check error code?) (d) Would it be possible to make random() functions (eg. gcry_random_bytes() and gcry_create_nonce()) really thread safe? (can be called in two different threads at the same time and not block one the call until the first ends) (e) Why using REGISTER_(...); macro and not simply call them in a function like "gcrypt_global_init();"? Eg. "REGISTER_DEFAULT_DIGESTS" in cipher/md.c, it uses a mutex to check if digests are already registred or not. "REGISTER_DEFAULT_DIGESTS" is called in 7 different functions, but is it really needed? [1] Firewall NuFW http://www.nufw.org/ Victor Stinner From marcus.brinkmann at ruhr-uni-bochum.de Thu Jul 6 08:58:44 2006 From: marcus.brinkmann at ruhr-uni-bochum.de (Marcus Brinkmann) Date: Thu Jul 6 08:57:37 2006 Subject: Questions about multi-threading In-Reply-To: <36208.82.247.156.212.1152137992.squirrel@mail.inl.fr> References: <36208.82.247.156.212.1152137992.squirrel@mail.inl.fr> Message-ID: <87lkr7w76j.wl%marcus.brinkmann@ruhr-uni-bochum.de> At Thu, 6 Jul 2006 00:19:52 +0200 (CEST), haypo@inl.fr wrote: > > Hi, > > I'm working on the firewall NuFW [1]. I used gprof to find functions which > takes a lot of time. I'm not sure, but I think that gcrypt really uses too > much mutexes lock() and unlock(). I read gcrypt source code, and used gdb > on server to catch lock() calls. > > I didn't understood everything, but I have some questions. > > (a) Why do ath_mutex_lock() and ath_mutex_unlock() call mutex_init()? It > looks very weird for me, since this function use another mutex! For static initialization to work (using ATH_MUTEX_INITIALIZER). > (b) Why do ath_mutex_destroy() call mutex_init()??? It's stupid to create > a mutex if it doesn't exist ... to destroy it just after its creation!? Yeah, that seems to be superfluous. However, I don't think this is a big performance difference (or do you have data indicating otherwise?). We at least need to check if it is initialized or not. I agree that the initialization could be optimized out in the case the the mutex wasn't initialized and is going to be destroyed. However, this increases code complexity a bit for a doubtful performance gain: How often do you destroy a mutex that's not initialized? Anyway, if you think that should be changed, feel free to send in a patch. > (c) Can a mutex lock() or unlock() fails? (or: it is necessary to check > error code?) Depends on the thread implementation and configuration of the mutex. We do not check for the internal check_init_lock. Doing so would make the code a tiny bit more robust. Want to send in a patch? > (d) Would it be possible to make random() functions (eg. > gcry_random_bytes() and gcry_create_nonce()) really thread safe? (can be > called in two different threads at the same time and not block one the > call until the first ends) I think only by either using two entropy pools, thereby underutilizing a scarce resource, or by using a second process. > (e) Why using REGISTER_(...); macro and not simply call them in a function > like "gcrypt_global_init();"? Eg. "REGISTER_DEFAULT_DIGESTS" in > cipher/md.c, it uses a mutex to check if digests are already registred or > not. "REGISTER_DEFAULT_DIGESTS" is called in 7 different functions, but is > it really needed? Not my code, so I can't comment :) Thanks, Marcus From haypo at inl.fr Thu Jul 6 12:14:31 2006 From: haypo at inl.fr (haypo@inl.fr) Date: Thu Jul 6 12:13:10 2006 Subject: Patch: remove REGISTER_DEFAULT_DIGESTS in md.c Message-ID: <50602.192.168.33.162.1152180871.squirrel@mail.inl.fr> Hi, This is my first patch for gcrypt: it removes use of "REGISTER_DEFAULT_DIGESTS" macro (in cipher/md.c). It removed it because global_init() already calls _gcry_md_init(), and we can register default digests there. So I moved "REGISTER_DEFAULT_DIGESTS" code in _gcry_md_init() and removed variables default_digests_registered and digests_registered_lock. Warning: Now you have to call global_init() before any function of md.c ... but this was already true, isn't it? Victor Stinner -------------- next part -------------- A non-text attachment was scrubbed... Name: gcrypt_md_register.patch Type: text/x-diff Size: 4375 bytes Desc: not available Url : /pipermail/attachments/20060706/b6dd1f66/gcrypt_md_register-0001.bin From haypo at inl.fr Thu Jul 6 14:06:09 2006 From: haypo at inl.fr (haypo@inl.fr) Date: Thu Jul 6 14:04:42 2006 Subject: Patch: remove REGISTER_DEFAULT_PUBKEYS in pubkey.c Message-ID: <41860.192.168.33.162.1152187569.squirrel@mail.inl.fr> Hi again, A new patch which looks my first one (about md.c): remove use of "REGISTER_DEFAULT_PUBKEYS" macro (in cipher/pubkey.c). global_init() calls _gcry_pk_init(), so I do init pubkeys_registered_lock and pubkeys_registered variables there. I also removed, now useless, variable default_pubkeys_registered. I noticed that pubkeys_registered_lock wasn't right initialized: static ath_mutex_t pubkeys_registered_lock; becomes static ath_mutex_t pubkeys_registered_lock = ATH_MUTEX_INITIALIZER; I don't understand how it did work before :-) Hum, something else: this patch and my other one also add error control. Before my patch, if _gcry_module_add() fails, BUG() was called. Now error code is returned by _gcry_pk_init(), and so global_init() fails. (Same) Warning: Now you have to call global_init() before calling any function of pubkey.c ... but this was already true, isn't it? Victor Stinner -------------- next part -------------- A non-text attachment was scrubbed... Name: gcrypt_pubkey_register.patch Type: text/x-diff Size: 8009 bytes Desc: not available Url : /pipermail/attachments/20060706/5c0e3000/gcrypt_pubkey_register.bin From wk at gnupg.org Thu Jul 6 14:42:53 2006 From: wk at gnupg.org (Werner Koch) Date: Thu Jul 6 14:46:17 2006 Subject: Questions about multi-threading In-Reply-To: <36208.82.247.156.212.1152137992.squirrel@mail.inl.fr> (haypo@inl.fr's message of "Thu, 6 Jul 2006 00:19:52 +0200 (CEST)") References: <36208.82.247.156.212.1152137992.squirrel@mail.inl.fr> Message-ID: <87lkr6ucoi.fsf@wheatstone.g10code.de> On Thu, 6 Jul 2006 00:19, haypo@inl.fr said: > (d) Would it be possible to make random() functions (eg. > gcry_random_bytes() and gcry_create_nonce()) really thread safe? (can be > called in two different threads at the same time and not block one the > call until the first ends) That is not possible. Ramndom is a scare resource and trhus we serialize access. Having separate instances of the random pool makes things actually slower, because they need to be really separate and can share nothing, in particular not any entropy used as seed. The latest libgcrypt from SVN features a random daemon which might be helpful for you. There is not much documentatiion yet and definitely not well tested; you might want to try it out. You need to call gcry_control (GCRYCTL_USE_RANDOM_DAEMON, 1) tight after gcry_check_version to enable use of the daemon - there is a fallback to the internal RNG if the daemon is not running. > (e) Why using REGISTER_(...); macro and not simply call them in a function > like "gcrypt_global_init();"? Eg. "REGISTER_DEFAULT_DIGESTS" in > cipher/md.c, it uses a mutex to check if digests are already registred or > not. "REGISTER_DEFAULT_DIGESTS" is called in 7 different functions, but is > it really needed? The idea behind the module system is to allow interpreters to load new algorithms on demand and thus full fledged locking is required. I think that it is a bit over designed and we may drop this with the next version. I doubt that the module system is actually used very often. However, this will be an ABI break. Salam-Shalom, Werner From haypo at inl.fr Thu Jul 6 17:11:47 2006 From: haypo at inl.fr (haypo@inl.fr) Date: Thu Jul 6 17:10:23 2006 Subject: Questions about multi-threading Message-ID: <48218.192.168.33.162.1152198707.squirrel@mail.inl.fr> Hi, >> (d) Would it be possible to make random() functions (...) >> really thread safe? (...) > > That is not possible. (...) Ok, I understand. It was just a question :-) > The latest libgcrypt from SVN features a random daemon which might be helpful for you. Nice, I may try it since I need quickly a lot of entropy (create 100 gnutls connections or more). >> (e) Why using REGISTER_(...); macro (...) > > The idea behind the module system is to allow interpreters to load new algorithms on demand and thus full fledged locking is required. Hum, I'm not sure that you understood my question. It's a good idea to be able to add new modules. But my question was: why try to register default plugins in all functions since one call will be enough? Check my proposition (two patches) on mailing list. I replaced all "register default ..." with *one* code only called once. > However, this will be an ABI break. My patches don't break ABI nor API. Haypo From marcus.brinkmann at ruhr-uni-bochum.de Fri Jul 7 11:08:48 2006 From: marcus.brinkmann at ruhr-uni-bochum.de (Marcus Brinkmann) Date: Fri Jul 7 11:07:21 2006 Subject: Questions about multi-threading In-Reply-To: <48218.192.168.33.162.1152198707.squirrel@mail.inl.fr> References: <48218.192.168.33.162.1152198707.squirrel@mail.inl.fr> Message-ID: <87fyhdwzmn.wl%marcus.brinkmann@ruhr-uni-bochum.de> At Thu, 6 Jul 2006 17:11:47 +0200 (CEST), haypo@inl.fr wrote: > > However, this will be an ABI break. > > My patches don't break ABI nor API. Just a general note (I didn't check it against your patches): The ABI/API is changed not only if you change the call signature of functions, or memory layout of exposed data structures, but also if you change the semantic behaviour of functions. For example, if a function was guaranteed to call an initializer, and then doesn't anymore, that's a semantic change. If that is a "break" depends on what we promise about these functions. This is where one needs to check existing documentation, but also existing practice, as the documentation is probably not a 100% complete specification (some common sense applies). Thanks, Marcus From haypo at inl.fr Fri Jul 7 17:10:45 2006 From: haypo at inl.fr (haypo@inl.fr) Date: Fri Jul 7 17:09:19 2006 Subject: Patch: set value of static variables in cipher/random.c Message-ID: <54286.192.168.33.162.1152285045.squirrel@mail.inl.fr> Hi, I'm not sure, but I think that a static variable have "undefined value". So I don't understand how the code is working without value initialization :-) That's why I set some variables values. Here is the patch for random.c Most important point is the variable "is_initialized": if is_initialized is different than zero, I think that random things are never cleany initialized. Haypo -------------- next part -------------- A non-text attachment was scrubbed... Name: gcrypt_random_static.patch Type: text/x-diff Size: 1368 bytes Desc: not available Url : /pipermail/attachments/20060707/3445836b/gcrypt_random_static.bin From haypo at inl.fr Fri Jul 7 17:14:01 2006 From: haypo at inl.fr (haypo@inl.fr) Date: Fri Jul 7 17:12:32 2006 Subject: Question about md_open() and random Message-ID: <60582.192.168.33.162.1152285241.squirrel@mail.inl.fr> Hi, I don't understand why a message digest function need to call "_gcry_fast_random_poll()" !? I noticed a FIXME: /* FIXME: should we really do that? - yes [-wk] */ Does it mean that the man "-wk" answered yes? Yes, but why yes? :-) On my mind, compute a MD5 or SHA1 don't need any random bit. Haypo From wk at gnupg.org Fri Jul 7 17:25:57 2006 From: wk at gnupg.org (Werner Koch) Date: Fri Jul 7 17:31:12 2006 Subject: Questions about multi-threading In-Reply-To: <48218.192.168.33.162.1152198707.squirrel@mail.inl.fr> (haypo@inl.fr's message of "Thu, 6 Jul 2006 17:11:47 +0200 (CEST)") References: <48218.192.168.33.162.1152198707.squirrel@mail.inl.fr> Message-ID: <8764i9phbu.fsf@wheatstone.g10code.de> On Thu, 6 Jul 2006 17:11, haypo@inl.fr said: > Nice, I may try it since I need quickly a lot of entropy (create 100 > gnutls connections or more). That won't give you more entropy of course. However, the pool is persistent over invocations of applications using libgcrypt. There are still problems with it. For example applications requesting very strong random (e.g. for key generation) may stall other applications. This is much like /dev/random and /dev/urandom. > able to add new modules. But my question was: why try to register default > plugins in all functions since one call will be enough? Check my > proposition (two patches) on mailing list. I replaced all "register > default ..." with *one* code only called once. My fault, sorry. >> However, this will be an ABI break. > > My patches don't break ABI nor API. This was related to removing the module system. However, see Marcus comments. I don't think your patch would break a documented behaviour, though. BTW, we need copyright assignments for patches. Salam-Shalom, Werner From wk at gnupg.org Fri Jul 7 17:28:38 2006 From: wk at gnupg.org (Werner Koch) Date: Fri Jul 7 17:31:21 2006 Subject: Patch: set value of static variables in cipher/random.c In-Reply-To: <54286.192.168.33.162.1152285045.squirrel@mail.inl.fr> (haypo@inl.fr's message of "Fri, 7 Jul 2006 17:10:45 +0200 (CEST)") References: <54286.192.168.33.162.1152285045.squirrel@mail.inl.fr> Message-ID: <871wsxph7d.fsf@wheatstone.g10code.de> On Fri, 7 Jul 2006 17:10, haypo@inl.fr said: > I'm not sure, but I think that a static variable have "undefined value". > So I don't understand how the code is working without value initialization No they are not undefined. They are initialized to zero. Your patch actually changes the code because the variables are moved from the BSS to the DATA segment. That is in general not a problem but it increases the size of the binary. Shalom-Salam, Werner From pak21 at srcf.ucam.org Fri Jul 7 17:18:26 2006 From: pak21 at srcf.ucam.org (Philip Kendall) Date: Fri Jul 7 18:26:48 2006 Subject: Patch: set value of static variables in cipher/random.c In-Reply-To: <54286.192.168.33.162.1152285045.squirrel@mail.inl.fr> References: <54286.192.168.33.162.1152285045.squirrel@mail.inl.fr> Message-ID: <20060707151826.GA9989@srcf.ucam.org> On Fri, Jul 07, 2006 at 05:10:45PM +0200, haypo@inl.fr wrote: > > I'm not sure, but I think that a static variable have "undefined value". > So I don't understand how the code is working without value initialization > :-) "The C standard" (not sure exactly which one) specifies that static variables are initialised to zero. Cheers, Phil -- "This is the first official broadcast of the new Voice of the Resistance. [...] You can kill us. You can bomb our colonies, destroy our ships, murder innocent civilians. But you cannot kill the truth. And the truth is back in business." Ivanova: Babylon 5: Conflicts of Interest From christianbiere at gmx.de Fri Jul 7 17:36:55 2006 From: christianbiere at gmx.de (Christian Biere) Date: Fri Jul 7 19:25:38 2006 Subject: Patch: set value of static variables in cipher/random.c In-Reply-To: <54286.192.168.33.162.1152285045.squirrel@mail.inl.fr> References: <54286.192.168.33.162.1152285045.squirrel@mail.inl.fr> Message-ID: <20060707153655.GA5799@cyclonus> haypo@inl.fr wrote: > I'm not sure, but I think that a static variable have "undefined value". > So I don't understand how the code is working without value initialization This is from C99 but this hasn't changed since C89: "If an object that has static storage duration is not initialized explicitly, then: -- if it has pointer type, it is initialized to a null pointer; -- if it has arithmetic type, it is initialized to (positive or unsigned) zero; -- if it is an aggregate, every member is initialized (recursively) according to these rules; -- if it is a union, the first named member is initialized (recursively) according to these rules." -- Christian From mo at g10code.com Fri Jul 7 18:50:37 2006 From: mo at g10code.com (Moritz Schulte) Date: Fri Jul 7 20:26:08 2006 Subject: Question about md_open() and random In-Reply-To: <60582.192.168.33.162.1152285241.squirrel@mail.inl.fr> References: <60582.192.168.33.162.1152285241.squirrel@mail.inl.fr> Message-ID: <1152291037.5555.5.camel@localhost.localdomain> > I don't understand why a message digest function need to call > "_gcry_fast_random_poll()" !? I noticed a FIXME: This is to make sure that the random pool is filled with entropy on a pseudo-regular basis. Note that this function call is a no-op, in case the random pool has not been used before. Thanks, Moritz -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 205 bytes Desc: This is a digitally signed message part Url : /pipermail/attachments/20060707/8dee4be2/attachment.pgp From marcus.brinkmann at ruhr-uni-bochum.de Fri Jul 7 21:55:44 2006 From: marcus.brinkmann at ruhr-uni-bochum.de (Marcus Brinkmann) Date: Fri Jul 7 21:57:23 2006 Subject: Patch: set value of static variables in cipher/random.c In-Reply-To: <871wsxph7d.fsf@wheatstone.g10code.de> References: <54286.192.168.33.162.1152285045.squirrel@mail.inl.fr> <871wsxph7d.fsf@wheatstone.g10code.de> Message-ID: <87ejwxfav3.wl%marcus.brinkmann@ruhr-uni-bochum.de> At Fri, 07 Jul 2006 17:28:38 +0200, Werner Koch wrote: > > On Fri, 7 Jul 2006 17:10, haypo@inl.fr said: > > > I'm not sure, but I think that a static variable have "undefined value". > > So I don't understand how the code is working without value initialization > > No they are not undefined. They are initialized to zero. > > Your patch actually changes the code because the variables are moved > from the BSS to the DATA segment. That is in general not a problem > but it increases the size of the binary. Depends on the compiler, though. Recent gcc's are smart enough to put both in BSS, which means that the old rule "don't initialize static variables to 0" is somewhat obsolete. In newer code, explicitely initializing for clarity may be preferred, if you want to. Gcc's behaviour even makes sense if you don't initialize, because sometimes initialization to 0 is hidden in macro abstractions. Oh, and there is still a difference between "int foo = 0;" and "int foo;". The former allocates a variable in the BSS segment, but the latter only defines a SHN_COMMON symbol, which is only allocated in the last linking stage. This allows to put "int foo;" in a header file that is included by multiple files. Doesn't mix well with dynamic linking, though... $ cat > main.c int foo_noinit; int foo_init = 0; main () { printf ("%i %i\n", foo_noinit, foo_init); } ^D $ make main > /dev/null 2>&1 | objdump --syms main | grep foo 080495a0 g O .bss 00000004 foo_init 080495a4 g O .bss 00000004 foo_noinit Thanks, Marcus From nmav at gnutls.org Sat Jul 8 14:54:36 2006 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Sat Jul 8 16:55:53 2006 Subject: Question about md_open() and random In-Reply-To: <1152291037.5555.5.camel@localhost.localdomain> References: <60582.192.168.33.162.1152285241.squirrel@mail.inl.fr> <1152291037.5555.5.camel@localhost.localdomain> Message-ID: <200607081454.37294.nmav@gnutls.org> On Fri 07 Jul 2006 18:50, Moritz Schulte wrote: > > I don't understand why a message digest function need to call > > "_gcry_fast_random_poll()" !? I noticed a FIXME: > > This is to make sure that the random pool is filled with entropy on a > pseudo-regular basis. Note that this function call is a no-op, in > case the random pool has not been used before. Hmmm, this has the side effect of adding a non deterministic delay to other unrelated to randomness functions (because of the lock). (and might even become a bottleneck when many hashes are done in parallel). Maybe it might be more clear for this function (or a wrapper of it) to become part of the randomness api? regards, Nikos From ametzler at downhill.at.eu.org Sat Jul 8 17:52:58 2006 From: ametzler at downhill.at.eu.org (Andreas Metzler) Date: Sun Jul 9 09:32:32 2006 Subject: [patch] don't use 32bit asssembly on powerpc64 Message-ID: <20060708155258.GA26719@downhill.aus.cc> Hej, mpi/config.links looks like this: ----------- powerpc*-*-linux*) [...] ppc620-*-* | \ powerpc64*-*-*) ----------- because first match wins a 64bit linux powerpc will use the generic (32bit) powerpc linux stuff. Moving the second entry further up fixes this. (This was reported and diagnosed by Rafael ?vila de Esp?ndola on http://bugs.debian.org/284609). This applies to both 1.2.2 and svn trunk. thanks, cu andreas -- The 'Galactic Cleaning' policy undertaken by Emperor Zhark is a personal vision of the emperor's, and its inclusion in this work does not constitute tacit approval by the author or the publisher for any such projects, howsoever undertaken. (c) Jasper Ffforde -------------- next part -------------- --- libgcrypt-1.2.2-upstream/mpi/config.links 2005-07-29 15:45:42.000000000 +0200 +++ libgcrypt11-1.2.2/mpi/config.links 2006-06-25 13:30:26.876070760 +0200 @@ -221,6 +221,11 @@ path="m68k/mc68020 m68k" ;; + ppc620-*-* | \ + powerpc64*-*-*) + mpi_sflags="-Wa,-mppc" + path="powerpc64" + ;; powerpc*-*-linux*) echo '/* configured for powerpc/ELF */' >>./mpi/asm-syntax.h echo '#define ELF_SYNTAX' >>./mpi/asm-syntax.h @@ -264,11 +269,6 @@ mpi_sflags="-Wa,-mppc" path="powerpc32" ;; - ppc620-*-* | \ - powerpc64*-*-*) - mpi_sflags="-Wa,-mppc" - path="powerpc64" - ;; powerpc*-*-*) mpi_sflags="-Wa,-mppc" path="powerpc32" From ametzler at downhill.at.eu.org Sat Jul 8 18:08:45 2006 From: ametzler at downhill.at.eu.org (Andreas Metzler) Date: Sun Jul 9 09:32:36 2006 Subject: [patch] Missing casts in gcrypt.h Message-ID: <20060708160845.GA26885@downhill.aus.cc> Hej, this is http://bugs.debian.org/362636 reported by Erik Meusel. | when I try to use GCRY_THREAD_OPTION_PTHREAD_IMPL from | /usr/include/gcrypt.h with g++, there are compilation problems due to | missing casts from void * to pthread_mutex_t * and back. The following | version works for me quite well: suggested simple patch (by Erik) is attached. This applies to both 1.2.2 and svn trunk. thanks, cu andreas -- The 'Galactic Cleaning' policy undertaken by Emperor Zhark is a personal vision of the emperor's, and its inclusion in this work does not constitute tacit approval by the author or the publisher for any such projects, howsoever undertaken. (c) Jasper Ffforde -------------- next part -------------- diff -Nur libgcrypt11-1.2.2/src/gcrypt.h libgcrypt11-1.2.2.new/src/gcrypt.h --- libgcrypt11-1.2.2/src/gcrypt.h 2005-10-05 09:44:00.000000000 +0100 +++ libgcrypt11-1.2.2.new/src/gcrypt.h 2006-06-08 16:55:37.000000000 +0100 @@ -257,7 +257,7 @@ static int gcry_pthread_mutex_init (void **priv) \ { \ int err = 0; \ - pthread_mutex_t *lock = malloc (sizeof (pthread_mutex_t)); \ + pthread_mutex_t *lock = (pthread_mutex_t*) malloc (sizeof (pthread_mutex_t)); \ \ if (!lock) \ err = ENOMEM; \ @@ -272,11 +272,11 @@ return err; \ } \ static int gcry_pthread_mutex_destroy (void **lock) \ - { int err = pthread_mutex_destroy (*lock); free (*lock); return err; } \ + { int err = pthread_mutex_destroy ((pthread_mutex_t*) *lock); free (*lock); return err; } \ static int gcry_pthread_mutex_lock (void **lock) \ - { return pthread_mutex_lock (*lock); } \ + { return pthread_mutex_lock ((pthread_mutex_t*) *lock); } \ static int gcry_pthread_mutex_unlock (void **lock) \ - { return pthread_mutex_unlock (*lock); } \ + { return pthread_mutex_unlock ((pthread_mutex_t*) *lock); } \ \ static struct gcry_thread_cbs gcry_threads_pthread = \ { GCRY_THREAD_OPTION_PTHREAD, NULL, \ From haypo at inl.fr Mon Jul 10 10:16:55 2006 From: haypo at inl.fr (haypo@inl.fr) Date: Mon Jul 10 10:15:58 2006 Subject: Patch: fix ath_mutex_destroy Message-ID: <45089.192.168.33.162.1152519415.squirrel@mail.inl.fr> Hi, As Marcus Brinkmann asked, here is the patch against ath_mutex_destroy(). New function code: ======================== 8< ================================ int ath_mutex_destroy (ath_mutex_t *lock) { int err; if (!ops_set || !ops.mutex_destroy) return 0; /* if mutex isn't initialiased, just exit */ (*ops.mutex_lock) (&check_init_lock); if (*lock == ATH_MUTEX_INITIALIZER) { (*ops.mutex_unlock) (&check_init_lock); return 0; } (*ops.mutex_unlock) (&check_init_lock); err = (*ops.mutex_destroy) (lock); #ifndef NDEBUG assert (*lock == MUTEX_UNLOCKED); *lock = MUTEX_DESTROYED; #endif return err; } ======================== 8< ================================ Main change: if the lock is not initialized, just skip the function. Victor Stinner -------------- next part -------------- A non-text attachment was scrubbed... Name: gcrypt_ath_mutex_destroy.patch Type: text/x-diff Size: 839 bytes Desc: not available Url : /pipermail/attachments/20060710/3e785758/gcrypt_ath_mutex_destroy.bin From haypo at inl.fr Mon Jul 10 11:14:55 2006 From: haypo at inl.fr (haypo@inl.fr) Date: Mon Jul 10 11:13:35 2006 Subject: Patch: fix gcc warning about gcry_thread_cbs in gcrypt.h Message-ID: <55865.192.168.33.162.1152522895.squirrel@mail.inl.fr> Hi, I hate compiler warnings and attached patch fix this one: warning: missing initializer It fix "static struct gcry_thread_cbs gcry_threads_pthread" for GCRY_THREAD_OPTION_PTHREAD_IMPL and GCRY_THREAD_OPTION_PTH_IMPL. Victor Stinner -------------- next part -------------- A non-text attachment was scrubbed... Name: gcrypt_thread_cbs.patch Type: text/x-diff Size: 1153 bytes Desc: not available Url : /pipermail/attachments/20060710/f0bfa036/gcrypt_thread_cbs-0001.bin From wk at gnupg.org Mon Jul 10 12:20:06 2006 From: wk at gnupg.org (Werner Koch) Date: Mon Jul 10 12:21:18 2006 Subject: Question about md_open() and random In-Reply-To: <200607081454.37294.nmav@gnutls.org> (Nikos Mavrogiannopoulos's message of "Sat, 8 Jul 2006 14:54:36 +0200") References: <60582.192.168.33.162.1152285241.squirrel@mail.inl.fr> <1152291037.5555.5.camel@localhost.localdomain> <200607081454.37294.nmav@gnutls.org> Message-ID: <87sll9oj6x.fsf@wheatstone.g10code.de> On Sat, 8 Jul 2006 14:54, Nikos Mavrogiannopoulos said: > Hmmm, this has the side effect of adding a non deterministic delay > to other unrelated to randomness functions (because of the lock). You can't have a deterministic behaviour in a muli-threaded process. If it would be deterministic a lock wouldn't be needed. > Maybe it might be more clear for this function (or a wrapper of it) to > become part of the randomness api? It is intentional because too may crypto library users didn't care about entropy polling leading to a very weak results. BTW, other libraries do the same. Shalom-Salam, Werner From wk at gnupg.org Mon Jul 10 12:24:06 2006 From: wk at gnupg.org (Werner Koch) Date: Mon Jul 10 12:26:18 2006 Subject: Patch: set value of static variables in cipher/random.c In-Reply-To: <87ejwxfav3.wl%marcus.brinkmann@ruhr-uni-bochum.de> (Marcus Brinkmann's message of "Fri, 07 Jul 2006 21:55:44 +0200") References: <54286.192.168.33.162.1152285045.squirrel@mail.inl.fr> <871wsxph7d.fsf@wheatstone.g10code.de> <87ejwxfav3.wl%marcus.brinkmann@ruhr-uni-bochum.de> Message-ID: <87odvxoj09.fsf@wheatstone.g10code.de> On Fri, 7 Jul 2006 21:55, Marcus Brinkmann said: > Oh, and there is still a difference between "int foo = 0;" and "int > foo;". The former allocates a variable in the BSS segment, but the > latter only defines a SHN_COMMON symbol, which is only allocated in However we were talking about static allocation; there it is not the case. BTW, common symbols are not defined by the C standard but very common with most linkers. The RISC OS linker for example don't support them. Salam-Shalom, Werner From haypo at inl.fr Mon Jul 10 14:38:02 2006 From: haypo at inl.fr (haypo@inl.fr) Date: Mon Jul 10 14:36:33 2006 Subject: Question about md_open() and random Message-ID: <55363.192.168.33.162.1152535082.squirrel@mail.inl.fr> > On Fri 07 Jul 2006 18:50, Moritz Schulte wrote: >> > I don't understand why a message digest function need to call "_gcry_fast_random_poll()" !? I noticed a FIXME: >> >> This is to make sure that the random pool is filled with entropy on a pseudo-regular basis. Note that this function call is a no-op, in case the random pool has not been used before. > > Hmmm, this has the side effect of adding a non deterministic delay to other unrelated to randomness functions (because of the lock). (and might even become a bottleneck when many hashes are done in parallel). Yep, exactly! And you didn't anwer to my question: which hash algorithm may use random? Hash functions are determinitic, isn't it? Victor Stinner From haypo at inl.fr Mon Jul 10 14:38:14 2006 From: haypo at inl.fr (haypo@inl.fr) Date: Mon Jul 10 14:36:43 2006 Subject: Patch: set value of static variables in cipher/random.c Message-ID: <55365.192.168.33.162.1152535094.squirrel@mail.inl.fr> > On Fri, 7 Jul 2006 17:10, haypo@inl.fr said: > >> I'm not sure, but I think that a static variable have "undefined value". So I don't understand how the code is working without value >> initialization > > No they are not undefined. They are initialized to zero. Ah ok ;-) > Your patch actually changes the code because the variables are moved from the BSS to the DATA segment. That is in general not a problem but it increases the size of the binary. So don't apply my patch. I didn't know that. Victor Stinner PS: Ooops, I forget to set gnutls-devel@gnupg.org for "To:" email field value From haypo at inl.fr Mon Jul 10 14:57:38 2006 From: haypo at inl.fr (haypo@inl.fr) Date: Mon Jul 10 14:56:08 2006 Subject: Faster mutex lock() and unlock() Message-ID: <34777.192.168.33.162.1152536258.squirrel@mail.inl.fr> Hi, I would like to know your opinion about replacing old ath_mutex_lock() with: int ath_mutex_lock (ath_mutex_t *lock) { if (ops_set) { return (*ops.mutex_lock) (lock); } else { #ifndef NDEBUG assert (*lock == MUTEX_UNLOCKED); *lock = MUTEX_LOCKED; #endif return 0; } } ===> remove "int ret = mutex_init (lock, 1);" Since currently locking a mutex takes 3 mutex operation: 1. lock(check_init_lock) (and then maybe init mutex) 2. unlock(check_init_lock) 3. lock(mutex) That's heavy and may cost a lot in multithead environment since check_init_lock is common to all threads. I can write the patch to remove mutex_init() call in lock() and unlock(), which means check all lock() and unlock() calls (to check that mutex is already initialized). But first I would like to know if you prefer very strong function, or fast function with a big warning :-) (don't call lock/unlock before initialize them). Haypo From wk at gnupg.org Mon Jul 10 20:54:38 2006 From: wk at gnupg.org (Werner Koch) Date: Mon Jul 10 20:56:25 2006 Subject: Question about md_open() and random In-Reply-To: <55363.192.168.33.162.1152535082.squirrel@mail.inl.fr> (haypo@inl.fr's message of "Mon, 10 Jul 2006 14:38:02 +0200 (CEST)") References: <55363.192.168.33.162.1152535082.squirrel@mail.inl.fr> Message-ID: <871wsti93l.fsf@wheatstone.g10code.de> On Mon, 10 Jul 2006 14:38, haypo@inl.fr said: > Yep, exactly! And you didn't anwer to my question: which hash algorithm > may use random? Hash functions are determinitic, isn't it? It is not that the hash algorithm are using random but hashing is a very common operation and often used right before other operations require random. Thus doing a fast random collect there is a Good Thing. If you can provide profile data showing that it is really a problem for your application we can add a control call to explicitly disable these automatic polls. Shalom-Salam, Werner From marcus.brinkmann at ruhr-uni-bochum.de Tue Jul 11 11:33:14 2006 From: marcus.brinkmann at ruhr-uni-bochum.de (Marcus Brinkmann) Date: Tue Jul 11 11:32:33 2006 Subject: Faster mutex lock() and unlock() In-Reply-To: <34777.192.168.33.162.1152536258.squirrel@mail.inl.fr> References: <34777.192.168.33.162.1152536258.squirrel@mail.inl.fr> Message-ID: <87d5cch4f9.wl%marcus.brinkmann@ruhr-uni-bochum.de> At Mon, 10 Jul 2006 14:57:38 +0200 (CEST), haypo@inl.fr wrote: > > Hi, > > I would like to know your opinion about replacing old ath_mutex_lock() with: I don't think so :) We really want static initialization. Furthermore, this would constitute an API break, which is not acceptable. Look, I think you are over-engineering. Do you have the profiling data to illustrate that your proposed changes have any measurable impact in your application? Yes, there is a small overhead. The overhead is a lock instruction. That's significant, but ath was not written for performance, but for flexibility. If you really need that extra-edge, then you will have to use a customized solution. Thanks, Marcus From perky at FreeBSD.org Tue Jul 11 15:45:16 2006 From: perky at FreeBSD.org (Hye-Shik Chang) Date: Wed Jul 12 11:05:43 2006 Subject: [patch] SEED (rfc4269) support Message-ID: <20060711134516.GA19101@FreeBSD.org> Hi, This patch adds the SEED, a 128bit block cipher, support for libgcrypt trunk as of today: http://people.freebsd.org/~perky/libgcrypt-seed.diff SEED is a registered cipher on ISO/IEC and IETF and has some standard applications already (RFC4010: CMS, RFC4162: TLS, RFC4196: IPSec). More detailed information on the algorithm is available in RFC4269. Because it is widely adopted by the Korean finalcial industry and the government, it would be great for Korean open-source developers if libgcrypt includes the support. Any comments are welcome. Thanks in advance, Hye-Shik -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : /pipermail/attachments/20060711/b8406c57/attachment.pgp From perky at FreeBSD.org Wed Jul 12 16:37:38 2006 From: perky at FreeBSD.org (Hye-Shik Chang) Date: Wed Jul 12 16:42:18 2006 Subject: [patch] SEED (rfc4269) support In-Reply-To: <200607122221.03229.bradh@frogmouth.net> References: <20060711134516.GA19101@FreeBSD.org> <200607122221.03229.bradh@frogmouth.net> Message-ID: <20060712143738.GA54909@FreeBSD.org> On Wed, Jul 12, 2006 at 10:20:58PM +1000, Brad Hards wrote: > On Tuesday 11 July 2006 23:45, Hye-Shik Chang wrote: > > This patch adds the SEED, a 128bit block cipher, support for > > libgcrypt trunk as of today: > > http://people.freebsd.org/~perky/libgcrypt-seed.diff > I would like to see this incorporated, Thanks for your interest! > but I'm a big fan of unit tests. Any > chance you could add some tests for this? So do I. I looked tests/ directory but I couldn't found where I could add a unittest for the new code. Is there any conventional stub code for cipher tests in libgcrypt? > Are there standard test vectors? Yup. RFC4269 includes four set of test vectors in its appendix B: http://www.ietf.org/rfc/rfc4269.txt I used the 3rd one for selftest() routine in the patch. > I really hope you have the right documentation on file (for copyright > assignment and the like).... I'd like to provide it in public domain and let libgcrypt project relicense or do whatever it needs. I updated the patch. Is it enough for the statement? Hye-Shik -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : /pipermail/attachments/20060712/25e6c235/attachment.pgp From bradh at frogmouth.net Wed Jul 12 14:20:58 2006 From: bradh at frogmouth.net (Brad Hards) Date: Wed Jul 12 17:25:41 2006 Subject: [patch] SEED (rfc4269) support In-Reply-To: <20060711134516.GA19101@FreeBSD.org> References: <20060711134516.GA19101@FreeBSD.org> Message-ID: <200607122221.03229.bradh@frogmouth.net> On Tuesday 11 July 2006 23:45, Hye-Shik Chang wrote: > This patch adds the SEED, a 128bit block cipher, support for > libgcrypt trunk as of today: > http://people.freebsd.org/~perky/libgcrypt-seed.diff I would like to see this incorporated, but I'm a big fan of unit tests. Any chance you could add some tests for this? Are there standard test vectors? I really hope you have the right documentation on file (for copyright assignment and the like).... Brad -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : /pipermail/attachments/20060712/e17b56bf/attachment.pgp From haypo at inl.fr Wed Jul 12 18:04:09 2006 From: haypo at inl.fr (haypo@inl.fr) Date: Wed Jul 12 18:02:45 2006 Subject: My patches Message-ID: <42800.192.168.33.162.1152720249.squirrel@mail.inl.fr> Hi, I didn't get any comment about my patches. I can understand that you reject patches "remove REGISTER_DEFAULT_DIGESTS" and "REGISTER_DEFAULT_PUBKEYS", since it change function behaviour. New behaviour: _gcry_md_init()/_gcry_pk_init() have to be called (eg. call global_init()) before calling any function of md.c / pubkey.c Patch "fix gcc warning about gcry_thread_cbs in gcrypt.h" just remove a gcc warning. I need it to compile my project with "-Werror" :-) Patch "fix ath_mutex_destroy" don't really change function behaviour, it just simplify the code (don't create a mutex if it doesn't exist). -- For the copyright: Victor Stinner Is it enough? Victor Stinner From marcus.brinkmann at ruhr-uni-bochum.de Wed Jul 12 20:51:56 2006 From: marcus.brinkmann at ruhr-uni-bochum.de (Marcus Brinkmann) Date: Wed Jul 12 20:52:13 2006 Subject: Patch: fix gcc warning about gcry_thread_cbs in gcrypt.h In-Reply-To: <55865.192.168.33.162.1152522895.squirrel@mail.inl.fr> References: <55865.192.168.33.162.1152522895.squirrel@mail.inl.fr> Message-ID: <87d5cavepf.wl%marcus.brinkmann@ruhr-uni-bochum.de> At Mon, 10 Jul 2006 11:14:55 +0200 (CEST), haypo@inl.fr wrote: > > [1 ] > Hi, > > I hate compiler warnings and attached patch fix this one: > warning: missing initializer > > It fix "static struct gcry_thread_cbs gcry_threads_pthread" for > GCRY_THREAD_OPTION_PTHREAD_IMPL and GCRY_THREAD_OPTION_PTH_IMPL. I put it in, thanks. Marcus From marcus.brinkmann at ruhr-uni-bochum.de Wed Jul 12 21:04:00 2006 From: marcus.brinkmann at ruhr-uni-bochum.de (Marcus Brinkmann) Date: Wed Jul 12 21:02:08 2006 Subject: Patch: fix ath_mutex_destroy In-Reply-To: <45089.192.168.33.162.1152519415.squirrel@mail.inl.fr> References: <45089.192.168.33.162.1152519415.squirrel@mail.inl.fr> Message-ID: <87bqruve5b.wl%marcus.brinkmann@ruhr-uni-bochum.de> At Mon, 10 Jul 2006 10:16:55 +0200 (CEST), haypo@inl.fr wrote: > > As Marcus Brinkmann asked, here is the patch against ath_mutex_destroy(). I put in a slightly different change. Here are a couple of suggestions: Keep the coding style according to the surrounding code (or use the GNU coding standard, which is the same in this case). Do not make gratuitous style changes. In this case, the admittedly awkward layout of the user-specific path vs the generic path is consistent across the file. It is helpful if you submit a changelog entry as well. Thanks, Marcus 2006-07-12 Marcus Brinkmann * ath.c (ath_mutex_destroy): Microoptimize destruction of unused statitically initialized mutexes. Suggested by Victor Stinner . From marcus.brinkmann at ruhr-uni-bochum.de Wed Jul 12 21:10:05 2006 From: marcus.brinkmann at ruhr-uni-bochum.de (Marcus Brinkmann) Date: Wed Jul 12 21:07:08 2006 Subject: My patches In-Reply-To: <42800.192.168.33.162.1152720249.squirrel@mail.inl.fr> References: <42800.192.168.33.162.1152720249.squirrel@mail.inl.fr> Message-ID: <87ac7evdv6.wl%marcus.brinkmann@ruhr-uni-bochum.de> At Wed, 12 Jul 2006 18:04:09 +0200 (CEST), haypo@inl.fr wrote: > I didn't get any comment about my patches. It has only been a week. I looked over them briefly at the day they arrived, but they were all low priority so I postponed to process them as a batch. It's good to know that you are done with this batch and are stalled waiting for an answer, so here it is ;) > I can understand that you reject patches "remove REGISTER_DEFAULT_DIGESTS" > and "REGISTER_DEFAULT_PUBKEYS", since it change function behaviour. New > behaviour: _gcry_md_init()/_gcry_pk_init() have to be called (eg. call > global_init()) before calling any function of md.c / pubkey.c Silence is not rejection. In particular, Werner said that we may follow your suggestion, but as it would be an ABI break, this requires more changes beyond the ones you sent in, and needs to be coordinated with the release schedule. In particular, these two patches I want to defer to Moritz. > Patch "fix gcc warning about gcry_thread_cbs in gcrypt.h" just remove a > gcc warning. I need it to compile my project with "-Werror" :-) > > Patch "fix ath_mutex_destroy" don't really change function behaviour, it > just simplify the code (don't create a mutex if it doesn't exist). I put in those two now. Thanks, Marcus From haypo at inl.fr Thu Jul 13 10:32:39 2006 From: haypo at inl.fr (Victor Stinner) Date: Thu Jul 13 10:31:13 2006 Subject: Question about md_open() and random Message-ID: <37275.192.168.33.162.1152779559.squirrel@mail.inl.fr> > On Mon, 10 Jul 2006 14:38, haypo@inl.fr said: > >> Yep, exactly! And you didn't anwer to my question: which hash algorithm may use random? Hash functions are determinitic, isn't it? > > It is not that the hash algorithm are using random but hashing is a very common operation and often used right before other operations require random. Thus doing a fast random collect there is a Good Thing. I read _gcry_fast_random_poll() source code an finally understood :-p This function calls do_fast_random_poll() which feed random pool with current time, getrusage(), etc. So : ok, it's a very good idea :-) > If you can provide profile data showing that it is really a problem for your application we can add a control call to explicitly disable these automatic polls. It was more a question than a performance problem. Haypo From haypo at inl.fr Thu Jul 13 10:32:57 2006 From: haypo at inl.fr (Victor Stinner) Date: Thu Jul 13 10:31:28 2006 Subject: Faster mutex lock() and unlock() Message-ID: <37277.192.168.33.162.1152779577.squirrel@mail.inl.fr> Hi, > haypo@inl.fr wrote: >> >> I would like to know your opinion about replacing old ath_mutex_lock() with: > > I don't think so > (...) > Yes, there is a small overhead. The overhead is a lock instruction. That's significant, but ath was not written for performance, but for flexibility > (...) Ok, I understand. I don't want to break libgcrypt API/behaviour. I just try to understand where my program is spending a lot of time, and I have difficulties to read gprof results ... Victor Stinner aka haypo From nmav at gnutls.org Thu Jul 13 15:40:13 2006 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Thu Jul 13 20:55:47 2006 Subject: Faster mutex lock() and unlock() In-Reply-To: <37277.192.168.33.162.1152779577.squirrel@mail.inl.fr> References: <37277.192.168.33.162.1152779577.squirrel@mail.inl.fr> Message-ID: <200607131540.13924.nmav@gnutls.org> On Thu 13 Jul 2006 10:32, Victor Stinner wrote: > Ok, I understand. I don't want to break libgcrypt API/behaviour. I > just try to understand where my program is spending a lot of time, > and I have difficulties to read gprof results ... I profiled gcrypt and gnutls quite some time ago using a (multithreaded) https server (on a single cpu system). The profiler I was using was oprofile. http://oprofile.sourceforge.net/ For libgcrypt the output wasn't really helpful, but I attach it anyway since it might be usefull to somebody. regards, Nikos -------------- next part -------------- Flat profile: Each sample counts as 1 samples. % cumulative self self total time samples samples calls T1/call T1/call name 23.55 1247.00 1247.00 Loop 16.37 2114.00 867.00 Loop 14.41 2877.00 763.00 gcry_mpih_divrem 8.63 3334.00 457.00 rijndael_encrypt 5.44 3622.00 288.00 transform 3.76 3821.00 199.00 transform 2.12 3933.00 112.00 gcry_mpih_sqr_n_basecase 2.00 4039.00 106.00 Loop 1.81 4135.00 96.00 gcry_burn_stack 1.70 4225.00 90.00 Loop 1.51 4305.00 80.00 Loop 1.45 4382.00 77.00 gcry_mpi_free_limb_space 1.45 4459.00 77.00 cipher_encrypt 1.38 4532.00 73.00 gcry_mpi_copy 1.34 4603.00 71.00 gcry_mpih_addmul_1 1.27 4670.00 67.00 gcry_mpih_sqr_n 1.17 4732.00 62.00 gcry_mpih_submul_1 1.15 4793.00 61.00 md_close 0.96 4844.00 51.00 mul_n_basecase 0.68 4880.00 36.00 mul_n 0.66 4915.00 35.00 gcry_mpi_add 0.55 4944.00 29.00 gcry_mpi_invm 0.53 4972.00 28.00 gcry_mpi_set 0.38 4992.00 20.00 gcry_mpih_mul_1 0.38 5012.00 20.00 mix_pool 0.36 5031.00 19.00 frame_dummy 0.34 5049.00 18.00 L2 0.32 5066.00 17.00 gcry_mpi_rshift 0.30 5082.00 16.00 gcry_mpi_powm 0.28 5097.00 15.00 gcry_mpih_sub_n 0.26 5111.00 14.00 gcry_malloc 0.26 5125.00 14.00 gcry_mpih_add_n 0.26 5139.00 14.00 transform 0.23 5151.00 12.00 Loop2 0.23 5163.00 12.00 add_randomness 0.19 5173.00 10.00 init 0.19 5183.00 10.00 gcry_mpi_sub 0.19 5193.00 10.00 sha1_write 0.17 5202.00 9.00 gcry_free 0.13 5209.00 7.00 gcry_mpi_free 0.13 5216.00 7.00 gcry_mpih_rshift 0.13 5223.00 7.00 gcry_mpi_test_bit 0.11 5229.00 6.00 gcry_malloc 0.09 5234.00 5.00 gcry_mpi_alloc 0.09 5239.00 5.00 gcry_mpi_alloc_limb_space 0.06 5242.00 3.00 gcry_md_ctl 0.06 5245.00 3.00 gcry_md_setkey 0.06 5248.00 3.00 gcry_xmalloc 0.06 5251.00 3.00 md5_final 0.06 5254.00 3.00 md5_write 0.06 5257.00 3.00 sexp_sscan 0.04 5259.00 2.00 gcry_mpi_normalize 0.04 5261.00 2.00 gcry_mpi_set_buffer 0.04 5263.00 2.00 do_fast_random_poll 0.04 5265.00 2.00 gcry_create_nonce 0.04 5267.00 2.00 md_get_algo 0.04 5269.00 2.00 rijndael_decrypt 0.04 5271.00 2.00 rmd160_write 0.04 5273.00 2.00 sha1_final 0.02 5274.00 1.00 L0 0.02 5275.00 1.00 Lend2 0.02 5276.00 1.00 gcry_ath_mutex_lock 0.02 5277.00 1.00 gcry_ath_mutex_unlock 0.02 5278.00 1.00 gcry_fast_random_poll 0.02 5279.00 1.00 gcry_mpi_alloc_secure 0.02 5280.00 1.00 gcry_mpi_resize 0.02 5281.00 1.00 gcry_mpih_mul_karatsuba_case 0.02 5282.00 1.00 gcry_rmd160_mixblock 0.02 5283.00 1.00 gcry_cipher_close 0.02 5284.00 1.00 gcry_is_secure 0.02 5285.00 1.00 gcry_md_reset 0.02 5286.00 1.00 gcry_mpi_cmp_ui 0.02 5287.00 1.00 gcry_mpi_invm 0.02 5288.00 1.00 gcry_pk_lookup_func_name 0.02 5289.00 1.00 gcry_sexp_cdr 0.02 5290.00 1.00 gcry_sexp_find_token 0.02 5291.00 1.00 gcry_xmalloc_secure 0.02 5292.00 1.00 md_open 0.02 5293.00 1.00 md_write 0.02 5294.00 1.00 rmd160_final 0.02 5295.00 1.00 sha1_init % the percentage of the total running time of the time program used by this function. cumulative a running sum of the number of seconds accounted seconds for by this function and those listed above it. self the number of seconds accounted for by this seconds function alone. This is the major sort for this listing. calls the number of times this function was invoked, if this function is profiled, else blank. self the average number of milliseconds spent in this ms/call function per call, if this function is profiled, else blank. total the average number of milliseconds spent in this ms/call function and its descendents per call, if this function is profiled, else blank. name the name of the function. This is the minor sort for this listing. The index shows the location of the function in the gprof listing. If the index is in parenthesis it shows where it would appear in the gprof listing if it were to be printed. From wk at gnupg.org Thu Jul 13 23:36:10 2006 From: wk at gnupg.org (Werner Koch) Date: Fri Jul 14 10:16:10 2006 Subject: [patch] SEED (rfc4269) support In-Reply-To: <20060712143738.GA54909@FreeBSD.org> (Hye-Shik Chang's message of "Wed, 12 Jul 2006 23:37:38 +0900") References: <20060711134516.GA19101@FreeBSD.org> <200607122221.03229.bradh@frogmouth.net> <20060712143738.GA54909@FreeBSD.org> Message-ID: <87y7uxur05.fsf@wheatstone.g10code.de> On Wed, 12 Jul 2006 16:37, Hye-Shik Chang said: > So do I. I looked tests/ directory but I couldn't found where I > could add a unittest for the new code. Is there any conventional > stub code for cipher tests in libgcrypt? You should add support to benchmark.c as well as basic.c. > Yup. RFC4269 includes four set of test vectors in its appendix B: Given that there is an RFC out we should add it to libgcrypt. > I'd like to provide it in public domain and let libgcrypt project > relicense or do whatever it needs. I updated the patch. Is it > enough for the statement? The GNu projects requires the signing of documents. I will send you the required files by private mail. Thanks, Werner From perky at FreeBSD.org Fri Jul 14 14:41:25 2006 From: perky at FreeBSD.org (Hye-Shik Chang) Date: Fri Jul 14 14:40:06 2006 Subject: [patch] SEED (rfc4269) support In-Reply-To: <87y7uxur05.fsf@wheatstone.g10code.de> References: <20060711134516.GA19101@FreeBSD.org> <200607122221.03229.bradh@frogmouth.net> <20060712143738.GA54909@FreeBSD.org> <87y7uxur05.fsf@wheatstone.g10code.de> Message-ID: <20060714124125.GA63391@FreeBSD.org> On Thu, Jul 13, 2006 at 11:36:10PM +0200, Werner Koch wrote: > On Wed, 12 Jul 2006 16:37, Hye-Shik Chang said: > > > So do I. I looked tests/ directory but I couldn't found where I > > could add a unittest for the new code. Is there any conventional > > stub code for cipher tests in libgcrypt? > > You should add support to benchmark.c as well as basic.c. Done. I updated the patch. http://people.freebsd.org/~perky/libgcrypt-seed-060714.diff (I wonder if I could avoid adding new functions. Is there better place for SEED ECB & CBC tests in basic.c?) > > Yup. RFC4269 includes four set of test vectors in its appendix B: > > Given that there is an RFC out we should add it to libgcrypt. > > > I'd like to provide it in public domain and let libgcrypt project > > relicense or do whatever it needs. I updated the patch. Is it > > enough for the statement? > > The GNu projects requires the signing of documents. I will send you > the required files by private mail. I see. I'll sign it when your mail is delivered. Hye-Shik -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : /pipermail/attachments/20060714/236ca247/attachment-0001.pgp From wk at gnupg.org Wed Jul 19 14:39:50 2006 From: wk at gnupg.org (Werner Koch) Date: Wed Jul 19 14:41:16 2006 Subject: Profiling (was: Faster mutex lock() and unlock()) In-Reply-To: <200607131540.13924.nmav@gnutls.org> (Nikos Mavrogiannopoulos's message of "Thu, 13 Jul 2006 15:40:13 +0200") References: <37277.192.168.33.162.1152779577.squirrel@mail.inl.fr> <200607131540.13924.nmav@gnutls.org> Message-ID: <87fygxydih.fsf_-_@wheatstone.g10code.de> On Thu, 13 Jul 2006 15:40, Nikos Mavrogiannopoulos said: > time samples samples calls T1/call T1/call name > 23.55 1247.00 1247.00 Loop > 16.37 2114.00 867.00 Loop > 14.41 2877.00 763.00 gcry_mpih_divrem This shows that the big number operations are taking up most of the time. This is expected. If someone is really up to modern ia32 CPUs this can be optimized. I know that meanwhile GMP has better optimizations but due to their configuration change in the assembler functions, it is not straightforward to port them back to libgcrypt. It is of course possible and should be done. before you ask: No, libgcrypt's current configuration scheme is not subject to a change becuase we know that it works and is portable over a wide range of platforms. > 8.63 3334.00 457.00 rijndael_encrypt The current AES code is pretty standard the reference implementation. It should be possible to squeeze out more performance and maybe even make cache timing atatcks harder. Briand Gladman put quite some work in optimized implementations (http://fp.gladman.plus.com/AES/index.htm). I just noticed that the new license terms allow distribution under the GPL - so we could take the code and add it to libgcrypt using an configure option to still allow building an LGPL version of libgcrypt. If we use this code as an alternative AES implenemtation, I think it will be okay with the GNU coding standards to go without a copyright disclaimer in this case. That alternative code should be clearly separated, though. Needs a volunteer of course ;-) > 5.44 3622.00 288.00 transform Ah well, SHA-1. We should definitely look for an optimized version as SHA-1 is really often and can lead to a real performance problem. For example GnupG runs not only AES but also SHA-1 over the bulk data. The benchmark tool could give some hints on the current performance. > 3.76 3821.00 199.00 transform I guess this is another hash algorithm - probably ripe-md160 as used by the RNG. Salam-Shalom, Werner From haypo at inl.fr Tue Jul 25 18:21:44 2006 From: haypo at inl.fr (Victor Stinner) Date: Tue Jul 25 18:20:35 2006 Subject: gcrypt, MPI, GMP and powerpc64 Message-ID: <14415.195.212.14.56.1153844504.squirrel@mail.inl.fr> Hi, I'm benchmarking NuFW on PowerPC 64 bits and I see that MPI is using 80 to 90% of the CPU during gnutls_handshake(). Here are the top 5 functions: 45.94% _gcry_mpih_submul_1 21.46% _gcry_mpih_addmul_1 6.17% _gcry_mpih_add_n 4.64% _gcry_mpih_sub_n 2.88% _gcry_mpih_divrem 2.59% _gcry_mpih_mul_1 I dig into gnutls and gcrypt, and I see that there is no assembler version of MPI (for ppc64). I also learned that MPI is based on old GMP release (which release?). So questions: - Can someone write assembler code for ppc64? It should be easy since GMP prototypes look to be the same, and assembler code for ppc64 already exist in GMP. - Would it possible to use GMP in gcrypt? - Why using MPI instead of GMP? For GMP, download http://ftp.sunet.se/pub/gnu/gmp/gmp-4.2.1.tar.bz2 and read mpn/powerpc64/mode64/*.asm. I see that a 32 bits mode exists, but I don't need it. The computer is running in full 64 bits mode. Victor Stinner aka haypo From wk at gnupg.org Tue Jul 25 21:15:49 2006 From: wk at gnupg.org (Werner Koch) Date: Tue Jul 25 21:21:20 2006 Subject: gcrypt, MPI, GMP and powerpc64 In-Reply-To: <14415.195.212.14.56.1153844504.squirrel@mail.inl.fr> (Victor Stinner's message of "Tue, 25 Jul 2006 18:21:44 +0200 (CEST)") References: <14415.195.212.14.56.1153844504.squirrel@mail.inl.fr> Message-ID: <8764hlo5qy.fsf@wheatstone.g10code.de> On Tue, 25 Jul 2006 18:21, Victor Stinner said: > So questions: > - Can someone write assembler code for ppc64? It should be easy since GMP > prototypes look to be the same, and assembler code for ppc64 already exist > in GMP. Yeah, I'd really like to see that. the code from the current GMP is basically useful but it needs to be converted to the configuration scheme as used in libgcrypt. > - Would it possible to use GMP in gcrypt? No. For security reasons we don't wnat the whole GMP suff. Just the modified and meanwhile hopefuly well security audited libgcrypt code. > - Why using MPI instead of GMP? At the time I wrote GnuPG (1997) GMP development had stalled for years and was stuck at 2.something. There was no maintainer for years. Only later GMP 3 started and changed a lot of the assembler stuff - I have seen eports that GMP is far less portable than it used to be. There are quite some heavy changes in the internal working and what we actually only need are the low level fucntions - should be easy to adjust to libgcrypt but it is more work than I can spend on it. We also need better optimized code for modern ia32 CPUS as well as for ia64 or AMD. And while we are at it: An optimized SHA-1 implementaion might make an difference too. Shalom-Salam, Werner From bpgcrypt at itaparica.org Mon Jul 31 18:33:14 2006 From: bpgcrypt at itaparica.org (bpgcrypt@itaparica.org) Date: Tue Aug 1 10:38:13 2006 Subject: bug in gcrypt's bit manipulation routines Message-ID: <20060731163314.GA7608@bacardi.dt115.lan> hi, it seems to me that I found some bugs in the bit manipulation routines of libgcrypt-1.2.2: 1. The function gcry_mpi_set_bit ommits the (re-)allocation of enough memory to store an MPI of the needed size. The following code snippet demonstrates this: #include #include int main() { gcry_mpi_t a; int i; a = gcry_mpi_new(0); gcry_mpi_randomize(a, 50, GCRY_WEAK_RANDOM); // get 50 random bits gcry_mpi_set_ui(a, 0); // "reset" them to 0 for(i = 50; i >= 0; i--) // display a string of 0s printf("%d", gcry_mpi_test_bit(a, i)); printf("\n"); gcry_mpi_set_bit(a, 49); // set bit 49 for(i = 50; i >= 0; i--) // this string has unexpected printf("%d", gcry_mpi_test_bit(a, i)); // random bit content! printf("\n"); } Of course one would expect that all but one of the bits in the second output line should be cleared, but they aren't. An example output follows 000000000000000000000000000000000000000000000000000 111001001101101001100000000000000000000000000000000 The problem can be tracked down to libgcrypt-1.2.2/mpi/mpi-bit.c : void gcry_mpi_set_bit( gcry_mpi_t a, unsigned int n ) { unsigned int limbno, bitno; limbno = n / BITS_PER_MPI_LIMB; bitno = n % BITS_PER_MPI_LIMB; if( limbno >= a->nlimbs ) { /* resize */ if( a->alloced >= limbno ) /* XXXXXXXXXXXX */ mpi_resize(a, limbno+1 ); a->nlimbs = limbno+1; } a->d[limbno] |= (A_LIMB_1<=". The same is true for the function gcry_mpi_set_highbit in the same file. BTW: The code a = gcry_mpi_new(0); gcry_mpi_set_bit(a, 10000); immediately leads to a segmentation violation on my machine. 2. The routine gcry_mpi_rshift always interpretes the "shift value" N as (N % 32). This behaviour is commented (a little bit) in libgcrypt-1.2.2/mpi/generic/mpih-rshift.c * Argument constraints: * 1. 0 < CNT < BITS_PER_MP_LIMB But this is NOT documented in the gcrypt manual: -- Function: void gcry_mpi_rshift (gcry_mpi_t X, gcry_mpi_t A, unsigned int N) Shift the value of A by N bits to the right and store the result in X. I cannot decide ultimately weather the error here is in the code or in the documentation. I personally would absolutely prefer an rshift implementation that can handly arbitrary large Ns. 3. HMAC-SHA256 calculation via gcry_md_open(&mh, GCRY_MD_SHA256, GCRY_MD_FLAG_HMAC); aborts with some "no secure memory allocated" message. This can be fixed by explicitely doing a gcry_control(GCRYCTL_INIT_SECMEM, 1); but as the gcry_control function isn't documented properly, this necessity cannot be deduced from the docs. It took me hours of source reading to get the HMAC-256 to work. Again it is not me to decide weather this is a bug in the lib or a misdocumentation. Thank you for libgcrypt! bp