From wk at gnupg.org Mon Jan 2 12:06:26 2006 From: wk at gnupg.org (Werner Koch) Date: Mon, 02 Jan 2006 12:06:26 +0100 Subject: [gnutls-dev] Re: living without global variables In-Reply-To: (Simon Josefsson's message of "Wed, 28 Dec 2005 10:13:52 +0100") References: <7997.31945521689$1135626397@news.gmane.org> Message-ID: <87sls6ubwt.fsf@wheatstone.g10code.de> On Wed, 28 Dec 2005 10:13:52 +0100, Simon Josefsson said: > When we have a replacement for libgcrypt, we can reconsider this > issue, and then I think it would even be possible to get a thread safe > gnutls_global_init() -- the only remaining problem then would be the Indeed, Libgcrypt requires a call to an init fucntion and has for various reasons no deinit function. This is due to the fact that Libgcrypt deliberately keeps global state (modules, random pool, self-tests, hooks, threading model). We discussed these issues for a long time and the only way to make Libgcrypt thread safe is by delegating the initialization to the actual application - only the application knows what threading model it is going to use. Thus we use a macro to initialize the threading and then to lock things down the road. The proper way for other libraries to use libgcrypt is by wrapping this init function and to allow the caller to actually initialize Libgcrypt (hidden in a wrapper function). Unfortunately gnutls does not do this. Bryan's session group function could be used to mitigate this problem as well. However, this does not solve the problem of "unloading" libgcrypt. In fact Libgcrypt should never be unloaded but kept in memory for the lifetime of the process. Libgcrypt's init code silently ignores re-initialization and bails out only if the threading model is going to be changed. There are plans to get rid of the global state but this would involve migrating some of the code out to a daemon. Salam-Shalom, Werner From bryanh at giraffe-data.com Tue Jan 3 02:32:40 2006 From: bryanh at giraffe-data.com (Bryan Henderson) Date: 3 Jan 2006 01:32:40 +0000 Subject: [gnutls-dev] Re: living without global variables In-Reply-To: <87sls6ubwt.fsf@wheatstone.g10code.de> (wk@gnupg.org) References: <7997.31945521689$1135626397@news.gmane.org> <87sls6ubwt.fsf@wheatstone.g10code.de> Message-ID: >The proper way for other >libraries to use libgcrypt is by wrapping this init function and to >allow the caller to actually initialize Libgcrypt (hidden in a >wrapper function). > >Unfortunately gnutls does not do this. Doesn't it? gnutls' global init function does _something_ with Libgcrypt. How is that different? >In fact Libgcrypt should never be unloaded but kept in memory for the >lifetime of the process. I'm sure you have reasons for this belief, which is essentially that modularity is not appropriate where Libgcrypt is involved, but it would be nice if a user had a choice over this policy. >the only way to make >Libgcrypt thread safe is by delegating the initialization to the >actual application - only the application knows what threading model >it is going to use. Assuming you have to have global variables, of course. -- Bryan Henderson Phone 408-621-2000 San Jose, California From wk at gnupg.org Tue Jan 3 08:56:58 2006 From: wk at gnupg.org (Werner Koch) Date: Tue, 03 Jan 2006 08:56:58 +0100 Subject: [gnutls-dev] Re: living without global variables In-Reply-To: (Bryan Henderson's message of "3 Jan 2006 01:32:40 +0000") References: <7997.31945521689$1135626397@news.gmane.org> <87sls6ubwt.fsf@wheatstone.g10code.de> Message-ID: <87mzidrbg5.fsf@wheatstone.g10code.de> On 3 Jan 2006 01:32:40 +0000, Bryan Henderson said: > Doesn't it? gnutls' global init function does _something_ with > Libgcrypt. How is that different? Gnutls handles the Libgcrypt initialization internally and thus requires that the user of gnutls has knowledge about the internal use of libgcrypt. If you are using libgcrypt in a threading environment the gnutls callers needs to make sure that Libgcrypt has been initialized correctly - the last time I checked, gnutls does not know the threading system in use and thus can't initialize Libgcrypt properly. With an initialization function similar to the one of Libgcrypt it would be possible to hide the use of Libgcrypt. We do not create thread specific versions of Libgcrypt because this would require all libs using Libgcrypt to also come in different flavors (plain, pthread, pth) - not very efficient. For certain OSes we might be able to do some trickery in detecting and initializing the threading lib in use but this is not portable. > Assuming you have to have global variables, of course. The major reason for global state is the entropy pool. Collecting entropy is a time consuming task and we can't do it for each new session. Another one is libgcrypt's "secure" memory allocation functions - obviously this can't be done on a per session base or delegated to a daemon. With encrypted swap you won't need it, but this OS feature is not yet in widespread use. Thus this is all a matter of telling libgcrypt to use the correct threading system. Unloading Libgcrypt is not possible unless all users agree on shutting down all threads (but one) using Libgcrypt, restoring all hooks and releasing all secure memory. Libgcrypt can't detect such a condition and this is the reason we don't provide a deinit function. For a plugin system one could use wrapper functions to provide a managed interface to libgcrypt. The problem is that you need to do this for most libraries which are subject to "unloading" and change all those libraries to use these wrapper functions. Writing a plugin system in a portable way is not easy. Frankly, I see no way to create in-process plugins without restricting the APIs a plugin may use. Creating plugins as separate processes is far easier, more secure and really portable. Shalom-Salam, Werner From Frediano.Ziglio at vodafone.com Tue Jan 3 11:31:12 2006 From: Frediano.Ziglio at vodafone.com (ZIGLIO, Frediano, VF-IT) Date: Tue, 3 Jan 2006 11:31:12 +0100 Subject: [gnutls-dev] RE: constification patch Message-ID: .... > > > > static inline const node_asn * > > _asn1_find_up_const(const node_asn *node) > > { > > return _asn1_find_up((node_asn*) node); > > } > > > > to > > > > #if defined(__GNUC__) && __GNUC__ >= 3 > > static inline const node_asn * > > _asn1_find_up_const(const node_asn *node) > > { > > return _asn1_find_up((node_asn*) node); > > } > > #else > > #define _asn1_find_up_const(node) _asn1_find_up((node_asn*) (node)) > > #endif > > How about changing the prototype of _asn1_find_up to include the const > keyword instead? That seem more correct. > > Using typecasts like you do hide problems: what if _asn1_find_up does > not preserve const? That would lead to a disaster, and the compiler > will not warn about it. > > Further, it has not been written down, but the coding style for > minitasn1 is slightly different than GnuTLS. Minitasn1 should be > portable C89 code, and #if's like that make the code unreadable. > Unless profiling show that you'd gain a non-negligible amount of time > making that particular function inline (I highly doubt that), then it > is worth the extra CPU cycles to have readable code. Optimize only > when you need. In contrast, in GnuTLS we can use 'inline' > unconditionally because there is a gnulib M4 test that make sure > 'inline' is defined to "" if it is not available. > > If you update the patch, to change the prototype of the existing > functions to use 'const' too, I'll review it. I know there is a lot > of missing 'const' keywords in various places. > Happy New Year !! I revised a bit the patch. Attached an update. This make constification in minitasn library portable C without #define and others uglyness. Regards, Frediano Ziglio -------------- next part -------------- A non-text attachment was scrubbed... Name: gnutls.diff.gz Type: application/x-gzip Size: 4027 bytes Desc: gnutls.diff.gz URL: From Frediano.Ziglio at vodafone.com Tue Jan 3 14:36:26 2006 From: Frediano.Ziglio at vodafone.com (ZIGLIO, Frediano, VF-IT) Date: Tue, 3 Jan 2006 14:36:26 +0100 Subject: [gnutls-dev] Re: living without global variables Message-ID: > > > Doesn't it? gnutls' global init function does _something_ with > > Libgcrypt. How is that different? > > Gnutls handles the Libgcrypt initialization internally and thus > requires that the user of gnutls has knowledge about the internal use > of libgcrypt. If you are using libgcrypt in a threading environment > the gnutls callers needs to make sure that Libgcrypt has been > initialized correctly - the last time I checked, gnutls does not know > the threading system in use and thus can't initialize > Libgcrypt properly. > > With an initialization function similar to the one of Libgcrypt it > would be possible to hide the use of Libgcrypt. > > We do not create thread specific versions of Libgcrypt because this > would require all libs using Libgcrypt to also come in different > flavors (plain, pthread, pth) - not very efficient. For certain OSes > we might be able to do some trickery in detecting and initializing the > threading lib in use but this is not portable. > > > Assuming you have to have global variables, of course. > > The major reason for global state is the entropy pool. Collecting > entropy is a time consuming task and we can't do it for each new > session. Another one is libgcrypt's "secure" memory allocation > functions - obviously this can't be done on a per session base or > delegated to a daemon. With encrypted swap you won't need it, but > this OS feature is not yet in widespread use. > > Thus this is all a matter of telling libgcrypt to use the correct > threading system. > > Unloading Libgcrypt is not possible unless all users agree on shutting > down all threads (but one) using Libgcrypt, restoring all hooks and > releasing all secure memory. Libgcrypt can't detect such a condition > and this is the reason we don't provide a deinit function. > > For a plugin system one could use wrapper functions to provide a > managed interface to libgcrypt. The problem is that you need to do > this for most libraries which are subject to "unloading" and change > all those libraries to use these wrapper functions. Writing a plugin > system in a portable way is not easy. Frankly, I see no way to create > in-process plugins without restricting the APIs a plugin may use. > Creating plugins as separate processes is far easier, more secure and > really portable. > > > Shalom-Salam, > > Werner > There is also another possibility that can fix (I hope) all problems. Mainly initializers in libgcrypt are functions that manage module registrations. Module registrations just add entries to some linked list. It would be possible to initialize these lists statically. Well, let me explain with an example this solution. gcry_module is defined as struct gcry_module { struct gcry_module *next; /* List pointers. */ struct gcry_module **prevp; void *spec; /* The acctual specs. */ int flags; /* Associated flags. */ int counter; /* Use counter. */ unsigned int mod_id; /* ID of this module. */ }; we could define a statically handled list as extern gcry_module _gcry_module1; extern gcry_module _gcry_module2; static gcry_module_t ciphers_registered = &_gcry_module1; gcry_module _gcry_module1 = { &_gcry_module2, &ciphers_registered, my_cipher_spec1, /* replace with correct value*/ FLAG_MODULE_STATIC_ALLOCATED, /* or whatever you decide */ 0, /* ?? */ MODULE_ID_MIN }; gcry_module _gcry_module2 = { NULL, &_gcry_module1.next, my_cipher_spec2, FLAG_MODULE_STATIC_ALLOCATED, 0, /* ?? */ MODULE_ID_MIN+1 }; This method use a static allocation (and in this case even a static initialization but this can be changed as you like). Someone could object that if a shared library adds some module dynamically to gcrypt in this case we would still have a memory leak however is up to shared library that register the module to release it. Frediano Ziglio From wk at gnupg.org Tue Jan 3 15:37:37 2006 From: wk at gnupg.org (Werner Koch) Date: Tue, 03 Jan 2006 15:37:37 +0100 Subject: [gnutls-dev] Re: living without global variables In-Reply-To: (Frediano ZIGLIO's message of "Tue, 3 Jan 2006 14:36:26 +0100") References: Message-ID: <871wzppeby.fsf@wheatstone.g10code.de> On Tue, 3 Jan 2006 14:36:26 +0100, ZIGLIO, Frediano, VF-IT said: > There is also another possibility that can fix (I hope) all problems. > Mainly initializers in libgcrypt are functions that manage module > registrations. Module registrations just add entries to some linked Actually these are the less critical ones. For various other reasons we need a locking mechanism anyway and thus the static tablle creation does not really help. If you are only working with GNU based system, you can try to init Libgcrypt from the actual application (maybe even with a Libgcrypt loader plugin) so that the other plugins will use the already loaded and initialized Libgcrypt. To get everything right all libraries between Libgcrypt and and the application must be aware of the threading system in use. We tinkered a lot with libgcrypt to make this as easy as possible and ventually setlled for the current solution - it is by far the most portable one. All other more automagically working implementations are either non portable or lead to headaches when writing the build rules for larger applications. Shalom-Salam, Werner From bryanh at giraffe-data.com Tue Jan 3 17:42:04 2006 From: bryanh at giraffe-data.com (Bryan Henderson) Date: 3 Jan 2006 16:42:04 +0000 Subject: [gnutls-dev] Re: living without global variables In-Reply-To: <87mzidrbg5.fsf@wheatstone.g10code.de> (wk@gnupg.org) References: <7997.31945521689$1135626397@news.gmane.org> <87sls6ubwt.fsf@wheatstone.g10code.de> <87mzidrbg5.fsf@wheatstone.g10code.de> Message-ID: >The major reason for global state is the entropy pool. I suspected that, which is why it's very much a policy matter. How much entropy generation time savings is the user willing to pay for? Is it worth adding extra complexity for users of one's library (which uses Libgcrypt under the covers)? Besides, it's really rather arbitrary to throw the entropy-sharing net around the program (technically, it's the program, not the process, since it starts over with each exec). It might just as well go around one subsystem within the program or the work of one user, or the whole computer system, or more. I'm not at all comfortable having my encryption library know what a process is, and even less comfortable having it know what a thread is. All of which is to say that if Libgcrypt used private contexts (an initialization function returns a handle; other functions take that handle) instead of global variables, it would have thread-safety and whatever entropy pooling level the code above wants. Without Libgcrypt knowing what a thread, process, or program is. As a practical matter, a typical program won't make a context for each session; just one for each module in the program, which would be like 3 at the most, and usually 1. Multiple modules could share the same Libgcrypt context (and entropy pool) if they did so expressly, just by having their common caller create one and pass it in. I don't really know much about encryption, but don't some OS kernels have device drivers that provide a computer-wide entropy pool? >Another one is libgcrypt's "secure" memory allocation >functions - obviously this can't be done on a per session base or >delegated to a daemon. I don't know how that works; it isn't obvious why each session or, more relevantly, "user" couldn't do this independently. -- Bryan Henderson Phone 408-621-2000 San Jose, California From rkit at mur.at Tue Jan 3 19:49:49 2006 From: rkit at mur.at (Rupert Kittinger) Date: Tue, 03 Jan 2006 19:49:49 +0100 Subject: [gnutls-dev] Re: living without global variables In-Reply-To: <87mzidrbg5.fsf@wheatstone.g10code.de> References: <7997.31945521689$1135626397@news.gmane.org> <87sls6ubwt.fsf@wheatstone.g10code.de> <87mzidrbg5.fsf@wheatstone.g10code.de> Message-ID: <43BAC74D.3050707@mur.at> Werner Koch schrieb: ... > > Unloading Libgcrypt is not possible unless all users agree on shutting > down all threads (but one) using Libgcrypt, restoring all hooks and > releasing all secure memory. Libgcrypt can't detect such a condition > and this is the reason we don't provide a deinit function. > Probably I am missing something here, but I think "this is the reason the library does not _call_ the deinit function but only provides it so the main thread can release all resources" would also make sense. I would really appreciate this functionality (valgrind, etc...). If you do not want to merge this functionality, how about adding it to the library in a contrib directory? cheers, Rupert -- Rupert Kittinger Krenngasse 32 A-8010 Graz Austria From jas at extundo.com Mon Jan 9 14:31:04 2006 From: jas at extundo.com (Simon Josefsson) Date: Mon, 09 Jan 2006 14:31:04 +0100 Subject: [gnutls-dev] Re: living without global variables In-Reply-To: <87sls6ubwt.fsf@wheatstone.g10code.de> (Werner Koch's message of "Mon, 02 Jan 2006 12:06:26 +0100") References: <7997.31945521689$1135626397@news.gmane.org> <87sls6ubwt.fsf@wheatstone.g10code.de> Message-ID: Werner Koch writes: > On Wed, 28 Dec 2005 10:13:52 +0100, Simon Josefsson said: > >> When we have a replacement for libgcrypt, we can reconsider this >> issue, and then I think it would even be possible to get a thread safe >> gnutls_global_init() -- the only remaining problem then would be the > > Indeed, Libgcrypt requires a call to an init fucntion and has for > various reasons no deinit function. This is due to the fact that > Libgcrypt deliberately keeps global state (modules, random pool, > self-tests, hooks, threading model). > > We discussed these issues for a long time and the only way to make > Libgcrypt thread safe is by delegating the initialization to the > actual application - only the application knows what threading model > it is going to use. Thus we use a macro to initialize the threading > and then to lock things down the road. The proper way for other > libraries to use libgcrypt is by wrapping this init function and to > allow the caller to actually initialize Libgcrypt (hidden in a > wrapper function). The distinction between a library and an application is rather blurry nowadays. I'm not convinced it scale to force applications to initialize all libraries -- i.e., to force OpenOffice to initialize libgcrypt because OpenOffice may use some component that eventually may dlopen some library that may use gnutls that may use libgcrypt. I'd rather have libraries initialize the libraries they use internally, to make things modular. > Unfortunately gnutls does not do this. Improvements to the situation is welcome... I'm not sure it is possible to solve Brian's problem without changing libgcrypt though. > Bryan's session group function could be used to mitigate this > problem as well. However, this does not solve the problem of > "unloading" libgcrypt. In fact Libgcrypt should never be unloaded > but kept in memory for the lifetime of the process. Libgcrypt's > init code silently ignores re-initialization and bails out only if > the threading model is going to be changed. There are plans to get > rid of the global state but this would involve migrating some of the > code out to a daemon. I suppose the RNG stuff is the most problematic one. For the gnulib-crypto stuff, we will have to solve that aspect too, and there may be some ideas there that could be re-used. LSH's approach to this issue is interesting. LSH require a disk file with a RNG seed. I assume that, after reading it, it is confounded with some information that separate it from other processes that may access the same RNG seed file. This approach help on embedded platforms where the entropy you can gather is scarce. Then you could generate a strong random seed when you burn the flashram. /Simon From jas at extundo.com Mon Jan 9 14:42:24 2006 From: jas at extundo.com (Simon Josefsson) Date: Mon, 09 Jan 2006 14:42:24 +0100 Subject: [gnutls-dev] Re: living without global variables In-Reply-To: <17537.6049512581$1136309334@news.gmane.org> (Bryan Henderson's message of "3 Jan 2006 16:42:04 +0000") References: <7997.31945521689$1135626397@news.gmane.org> <87sls6ubwt.fsf@wheatstone.g10code.de> <87mzidrbg5.fsf@wheatstone.g10code.de> <17537.6049512581$1136309334@news.gmane.org> Message-ID: bryanh at giraffe-data.com (Bryan Henderson) writes: > I don't really know much about encryption, but don't some OS kernels have > device drivers that provide a computer-wide entropy pool? Yes, if you build GnuTLS with crypto from gnulib, it will read randomness from /dev/*random. There are at least two problems with that approach: 1) /dev/*random doesn't provide good randomness on several platforms. 2) Reading a lot from /dev/*random might deplete the system randomness pool. /Simon From jas at extundo.com Mon Jan 9 14:48:43 2006 From: jas at extundo.com (Simon Josefsson) Date: Mon, 09 Jan 2006 14:48:43 +0100 Subject: [gnutls-dev] Re: living without global variables In-Reply-To: <17537.6049512581$1136309334@news.gmane.org> (Bryan Henderson's message of "3 Jan 2006 16:42:04 +0000") References: <7997.31945521689$1135626397@news.gmane.org> <87sls6ubwt.fsf@wheatstone.g10code.de> <87mzidrbg5.fsf@wheatstone.g10code.de> <17537.6049512581$1136309334@news.gmane.org> Message-ID: bryanh at giraffe-data.com (Bryan Henderson) writes: > I'm not at all comfortable having my encryption library know what a > process is, and even less comfortable having it know what a thread is. It is difficult to avoid that: If you generate the random data in your process, it will generate the same random data in two processes after a fork(). Libgcrypt now detect this and re-seed itself when this happen. There may be similar issues with threads too. I'm beginning to think that randomness is not something an application or library should ever deal with internally. It should read it from an external device or socket. File access to a device is traditionally better synchronized, and every process that access it get "its own" device. /dev/random and /dev/urandom would work good on most desktop PCs. If you need something faster or better, write a daemon that collect entropy, and make it serve data in /dev/*random. This approach would also reduce complexity in libgcrypt. /Simon From wk at gnupg.org Mon Jan 9 15:57:52 2006 From: wk at gnupg.org (Werner Koch) Date: Mon, 09 Jan 2006 15:57:52 +0100 Subject: [gnutls-dev] Re: living without global variables In-Reply-To: (Simon Josefsson's message of "Mon, 09 Jan 2006 14:48:43 +0100") References: <7997.31945521689$1135626397@news.gmane.org> <87sls6ubwt.fsf@wheatstone.g10code.de> <87mzidrbg5.fsf@wheatstone.g10code.de> <17537.6049512581$1136309334@news.gmane.org> Message-ID: <873bjx8n4f.fsf@wheatstone.g10code.de> On Mon, 09 Jan 2006 14:48:43 +0100, Simon Josefsson said: > process, it will generate the same random data in two processes after > a fork(). Libgcrypt now detect this and re-seed itself when this > happen. There may be similar issues with threads too. Handled too. > /dev/random and /dev/urandom would work good on most desktop PCs. If The quality of these devises is not very good and thus Libgcrypt adds another layer. When unsing Linux there is even tghe problem that they both work on the saqme pool and thus any read to urandom depletest the pool. Reading just a random seed is not sufficient because the threat model we use is that random numbers available through libgcrypt's API should not be predictable if a user can read other resources than those of libgrypt's process. > you need something faster or better, write a daemon that collect > entropy, and make it serve data in /dev/*random. > This approach would also reduce complexity in libgcrypt. I'd really like to do that but there is a secre bootstrap problem: To convey the random bytes from the daemon to the consumer process (i.e. libgcrypt's code) one need to use an IPC mechanism. I have not anaylyzed how we could make use of shared memeory but I am pretty sure that this will won't be easy to get secure. Thus the remaining alternative is only a Unix domain socket. This would be pretty straightforward to implement and on some OS this would even allow a system daemon to be implemented as there are facilities for the daemon to detect the uid of the connecting client (we obviously want to separate generated random as much as possible). Okay, that will work but we would put all our eggs (in particular the random numbers used to create keys) into that mechanism and hope that there are no vulnerabilitiesin the kernel allowing to eavesdrop on the internal buffers. That violates our security principle and introduces a hard to audit code path. The obvious solution for this is to encrypt the data passing through IPC. But how? DH KEX should work nice here because we don't consider active attacks to be mountable easier than penetrating the actual libgcrypt or rng-daemon process. But well, we bneed random here again - how do we get it then? Classical problem :-( You might now say, get rid of all these paranoid principles. Well, this will obviously solve this and many other problems but weakens the system too much and libgcrypt would sone become the primart target of attack and not the application itself. We need a better and secure OS kernel and not all these Linux and BSD monsters. The RNG belong into a daemon which allows controlling how much entropy a process, may suck from the system and the kernel itself needs to provide a secure IPC mechanism. Salam-Shalom, Werner From bryanh at giraffe-data.com Mon Jan 9 17:54:07 2006 From: bryanh at giraffe-data.com (Bryan Henderson) Date: 9 Jan 2006 16:54:07 +0000 Subject: [gnutls-dev] Re: living without global variables In-Reply-To: (jas@extundo.com) References: <7997.31945521689$1135626397@news.gmane.org> <87sls6ubwt.fsf@wheatstone.g10code.de> Message-ID: >The distinction between a library and an application is rather blurry >nowadays. Yes, lets just say that a program consists of multiple modules, and some of them are related in a hierarchical way -- A uses B, B uses C. Some modules are packaged as code libraries. One module stands out, because it is at the top of the hierarchy -- the thing that gets control first and controls, at least indirectly, every other module. >I'm not convinced it scale to force applications to initialize all >libraries -- i.e., to force OpenOffice to initialize libgcrypt because >OpenOffice may use some component that eventually may dlopen some >library that may use gnutls that may use libgcrypt. > >I'd rather have libraries initialize the libraries they use >internally, to make things modular. If you take dlopen out of the picture for the sake of analysis, you can have both. OpenOffice initializes the component that it uses directly, which initializes another library, which initializes gnutls, which initializes libgcrypt. So the top level module (OpenOffice) effectively initializes libgcrypt, without ever knowing that libgcrypt exists. If you believe that libgcrypt should be participating in the program's threading, then you also believe that all those other layers should too, and therefore the "threading model" information is present in parameters on all of their initialization routines. Dlopen throws in extra complexity, but only because of the multithread problem because dlopen can happen while other threads are running. Or can it? Libraries have loader-level initialization and termination code -- maybe it's a requirement of dynamic library loading that all other threads be stopped while the load/unload takes place. The dlopen I know is part of the same component as the thread facilities (GNU libc), so that's a possibility. Anyone know? -- Bryan Henderson Phone 408-621-2000 San Jose, California From bryanh at giraffe-data.com Mon Jan 9 17:38:51 2006 From: bryanh at giraffe-data.com (Bryan Henderson) Date: 9 Jan 2006 16:38:51 +0000 Subject: [gnutls-dev] Re: living without global variables In-Reply-To: (jas@extundo.com) References: <7997.31945521689$1135626397@news.gmane.org> <87sls6ubwt.fsf@wheatstone.g10code.de> <87mzidrbg5.fsf@wheatstone.g10code.de> <17537.6049512581$1136309334@news.gmane.org> Message-ID: >2) Reading a lot from /dev/*random might deplete the system randomness > pool. If you're saying this is an issue because having one pool for everybody raises the volume of reading, then I'd say it's conceptually the same problem with the process-global pool, and identifies another reason that private contexts would be a good thing -- each independent user of randomness, whether or not in the same process with another user, can create his own pool with enough capacity to meet his randomness needs. I am, of course, beating on the point that the boundaries of a process is an arbitrary scope, and it would be better if a low-level code library were not sensitive to it. -- Bryan Henderson Phone 408-621-2000 San Jose, California From joe at manyfish.co.uk Wed Jan 11 17:17:15 2006 From: joe at manyfish.co.uk (Joe Orton) Date: Wed, 11 Jan 2006 16:17:15 +0000 Subject: [gnutls-dev] Re: [gnutls-commits] CVS gnutls/gl In-Reply-To: References: Message-ID: <20060111161715.GA8803@manyfish.co.uk> On Mon, Jan 09, 2006 at 05:40:07PM +0100, CVS User jas wrote: ... > @@ -238,8 +244,8 @@ > #define SIG_ATOMIC_MIN 0 > #define SIG_ATOMIC_MAX 127 > > -#ifndef SIZE_MAX > -# define SIZE_MAX ((size_t) -1) > +#ifndef SIZE_MAX /* SIZE_MAX may also be defined in config.h. */ > +# define SIZE_MAX (size_t)~(size_t)0) > #endif That #define needs an extra leading "("! From jas at extundo.com Wed Jan 11 18:10:43 2006 From: jas at extundo.com (Simon Josefsson) Date: Wed, 11 Jan 2006 18:10:43 +0100 Subject: [gnutls-dev] Re: [gnutls-commits] CVS gnutls/gl In-Reply-To: <20060111161715.GA8803@manyfish.co.uk> (Joe Orton's message of "Wed, 11 Jan 2006 16:17:15 +0000") References: <20060111161715.GA8803@manyfish.co.uk> Message-ID: Joe Orton writes: > On Mon, Jan 09, 2006 at 05:40:07PM +0100, CVS User jas wrote: > ... >> @@ -238,8 +244,8 @@ >> #define SIG_ATOMIC_MIN 0 >> #define SIG_ATOMIC_MAX 127 >> >> -#ifndef SIZE_MAX >> -# define SIZE_MAX ((size_t) -1) >> +#ifndef SIZE_MAX /* SIZE_MAX may also be defined in config.h. */ >> +# define SIZE_MAX (size_t)~(size_t)0) >> #endif > > That #define needs an extra leading "("! Thanks for the catch. I'll fix it upstreams in gnulib. From jas at extundo.com Thu Jan 12 22:32:03 2006 From: jas at extundo.com (Simon Josefsson) Date: Thu, 12 Jan 2006 22:32:03 +0100 Subject: [gnutls-dev] Experimental: GnuTLS 1.3.3 Message-ID: We are pleased to announce the availability of GnuTLS version 1.3.3, another release on the experimental 1.3.x branch. The goal of 1.3.x will be to add support for TLS Pre-Shared-Keys and TLS Inner Application (TLS/IA). Other planned improvements in 1.3.x are system-independent resume data structures, modularization of the bignum operations, and TLS OpenPGP improvements. So far, the TLS-PSK, TLS/IA and system-independent resume data goals have been met. GnuTLS is a modern C library that implement the standard network security protocol Transport Layer Security (TLS), for use by network applications. Improving GnuTLS is costly, but you can help! We are looking for organizations that find GnuTLS useful and wish to contribute back. You can contribute by reporting bugs, improve the software, or donate money or equipment. Commercial support contracts for GnuTLS are available, and they help finance continued maintenance. Simon Josefsson Datakonsult, a Stockholm based privately held company, is currently funding GnuTLS maintenance. We are always looking for interesting development projects. If you need help to use GnuTLS, or want to help others, you are invited to join our help-gnutls mailing list, see: . The project page of the library is available at: http://www.gnutls.org/ http://www.gnu.org/software/gnutls/ http://josefsson.org/gnutls/ (updated fastest) Here are the compressed sources: http://josefsson.org/gnutls/releases/gnutls-1.3.3.tar.gz (3.1MB) ftp://ftp.gnutls.org/pub/gnutls/gnutls-1.3.3.tar.bz2 (3.1MB) Here are GPG detached signatures signed using key 0xB565716F: http://josefsson.org/gnutls/releases/gnutls-1.3.3.tar.bz2.sig ftp://ftp.gnutls.org/pub/gnutls/gnutls-1.3.3.tar.bz2.sig The software is cryptographically signed by the author using an OpenPGP key identified by the following information: 1280R/B565716F 2002-05-05 [expires: 2006-02-28] Key fingerprint = 0424 D4EE 81A0 E3D1 19C6 F835 EDA2 1E94 B565 716F The key is available from: http://josefsson.org/key.txt dns:b565716f.josefsson.org?TYPE=CERT Here are the build reports for various platforms: http://josefsson.org/autobuild-logs/gnutls.html Here are the SHA-1 checksums: 65e255278632646afc48b284fbe03ddef996551b gnutls-1.3.3.tar.bz2 e4a401b914d4f39f13e5c714b0278b00bb11fef3 gnutls-1.3.3.tar.bz2.sig Enjoy, Nikos and Simon Noteworthy changes since version 1.3.2: ** New API to access the TLS master secret. When possible, you should use the TLS PRF functions instead. Suggested by Jouni Malinen . ** Improved handling when multiple libraries use GnuTLS at the same time. Now gnutls_global_init() can be called multiple times, and gnutls_global_deinit() will only deallocate the structure when it has been called as many times as gnutls_global_init() was called. ** Added a self test of TLS resume functionality. ** Fix crash in TLS resume code, caused by TLS/IA changes. ** Documentation fixes about thread unsafety, prompted by ** discussion with bryanh at giraffe-data.com (Bryan Henderson). In particular, gnutls_global_init() and gnutls_global_deinit() are not thread safe. Careful callers may want to protect the call using a mutex. The problem could also be ignored, which would cause a memory leak under rare conditions when two threads invoke the function roughly at the same time. ** Add 'const' keywords in various places, from Frediano ZIGLIO. ** The code was indented again, including the external header files. ** API and ABI modifications: New functions to retrieve the master secret value: gnutls_session_get_master_secret Add a 'const' keyword to existing API: gnutls_x509_crq_get_challenge_password From nmav at gnutls.org Mon Jan 16 23:00:23 2006 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Mon, 16 Jan 2006 23:00:23 +0100 Subject: [gnutls-dev] Restore gnutls session after execvp - possible? Message-ID: <200601162300.23288.nmav@gnutls.org> > Hi, > > I'm developing an IRC client called WeeChat > (http://weechat.flashtux.org). > I'm adding a new feature: /upgrade command, which does an execvp() of > weechat, without closing connections to servers (sockets are still > open after execvp). > For some servers, user may connect thru gnutls (SSL), and I need to save > session in file when upgrading, then restore it when starting new > process. > Is it possible to do that with gnutls today? Hello, It seems that I've missed this thread completely due to mail problems. However it seems that this should be possible with gnutls, though not perfectly neat. In a socket you can end the TLS connection and start a new one without closing the socket. This you can have something like: gnutls_handshake() gnutls_send/recv() gnutls_bye(RDWR) (here socket is still open) gnutls_handshake() gnutls_send/recv() gnutls_bye(RDWR) The second handshake might be a bit slow. But you can as you already thought, use the resumption feature. Thus the second handshake will be a fast one and nobody would notice a delay! regards, Nikos From nmav at gnutls.org Tue Jan 17 19:26:38 2006 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Tue, 17 Jan 2006 19:26:38 +0100 Subject: [gnutls-dev] Re: Restore gnutls session after execvp - possible? In-Reply-To: <20060117180331.GG1252@flashtux.org> References: <200601162300.23288.nmav@gnutls.org> <200601171845.39764.n.mavrogiannopoulos@gmail.com> <20060117180331.GG1252@flashtux.org> Message-ID: <200601171926.39111.nmav@gnutls.org> On Tuesday 17 January 2006 19:03, you wrote: > The peer is an IRC server, so I don't know exactly what is happening > on its side. That's the problem then. I thought you had control of both the server and the client. The server probably doesn't know that he is supposed to restart TLS after this has been terminated by the client. Probably he returns to non-TLS mode... so in that case you could order him to restart tls, or he terminates the connection (most probable). There is no easy way to solve this problem since the states that need to be saved are really more than the data needed for session resumption (state for encryption, compression algorithm, buffers and many I cannot think of). This is much work to do and doesn't seem to worth the effort (if we were in kernel side, that would have been a different story, but we are not)... regards, Nikos From bernard at brenda-arkle.demon.co.uk Wed Jan 18 00:09:17 2006 From: bernard at brenda-arkle.demon.co.uk (Bernard Leak) Date: Tue, 17 Jan 2006 23:09:17 +0000 Subject: [gnutls-dev] A tiny patch for libtasn1 (0.2.17) Message-ID: <43CD791D.4050109@brenda-arkle.demon.co.uk> Dear List, the Makefiles for libtasn (more precisely, the Makefile for the 'src' directory) don't work if building in a separate source directory: the libtasn1.h file from the 'lib' directory is added through -I ../lib which won't work (as the .h file isn't generated) unless the build tree and the source tree are the same. I offer the following patch: this is for the distributed version, so it patches both Makefile.am and Makefile.in, though from the viewpoint of this list the 'real' patch is to Makefile.am only. As can be seen, it's really very simple - just forcing a reference to $(srcdir). You might prefer to use $(top_srcdir)/lib instead, of course. As far as I have noticed, this is the only problematic place, but I can't promise I've not missed something. Anyway, it seems to work for me, and is certainly an improvement. Bernard. -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: cur.diffs URL: From bernard at brenda-arkle.demon.co.uk Wed Jan 18 00:17:32 2006 From: bernard at brenda-arkle.demon.co.uk (Bernard Leak) Date: Tue, 17 Jan 2006 23:17:32 +0000 Subject: [gnutls-dev] A tiny patch for libtasn1 (0.2.17) Message-ID: <43CD7B0C.8030507@brenda-arkle.demon.co.uk> Dear List, the Makefiles for libtasn (more precisely, the Makefile for the 'src' directory) don't work if building in a separate source directory: the libtasn1.h file from the 'lib' directory is added through -I ../lib which won't work (as the .h file isn't generated) unless the build tree and the source tree are the same. I offer the following patch: this is for the distributed version, so it patches both Makefile.am and Makefile.in, though from the viewpoint of this list the 'real' patch is to Makefile.am only. As can be seen, it's really very simple - just forcing a reference to $(srcdir). You might prefer to use $(top_srcdir)/lib instead, of course. As far as I have noticed, this is the only problematic place, but I can't promise I've not missed something. Anyway, it seems to work for me, and is certainly an improvement. Bernard Leak. -- Before they made me, they broke the mould -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: cur.diffs URL: From jas at extundo.com Wed Jan 18 10:37:05 2006 From: jas at extundo.com (Simon Josefsson) Date: Wed, 18 Jan 2006 10:37:05 +0100 Subject: [gnutls-dev] Re: A tiny patch for libtasn1 (0.2.17) In-Reply-To: <43CD791D.4050109@brenda-arkle.demon.co.uk> (Bernard Leak's message of "Tue, 17 Jan 2006 23:09:17 +0000") References: <43CD791D.4050109@brenda-arkle.demon.co.uk> Message-ID: Bernard Leak writes: > Dear List, > the Makefiles for libtasn (more precisely, the > Makefile for the > 'src' directory) don't work if building in a separate source directory: the > libtasn1.h file from the 'lib' directory is added through -I ../lib > which won't work (as the .h file isn't generated) unless the build > tree and the > source tree are the same. > > I offer the following patch: this is for the distributed version, so > it patches > both Makefile.am and Makefile.in, though from the viewpoint of this list > the 'real' patch is to Makefile.am only. As can be seen, it's really very > simple - just forcing a reference to $(srcdir). You might prefer to use > $(top_srcdir)/lib instead, of course. > > As far as I have noticed, this is the only problematic place, but I > can't promise I've not missed something. Anyway, it seems to > work for me, and is certainly an improvement. Hi Bernard. Thanks for the report, I have fixed this in libtasn1 CVS. Regards, Simon From fw at deneb.enyo.de Wed Jan 18 10:44:17 2006 From: fw at deneb.enyo.de (Florian Weimer) Date: Wed, 18 Jan 2006 10:44:17 +0100 Subject: [gnutls-dev] Feature request: not really random session keys Message-ID: <87k6cxuaz2.fsf@mid.deneb.enyo.de> Okay, the subject line might be a bit misleading. On server machines, random bits are a very scarce ressource, and you cannot really afford to throw them a way at a rate of a few kbps. Yet if you run certain network servers (or clients) with GNUTLS, this is what happens -- and these servers stall from time to time, waiting for more randomness. I would like to see an additional API which allows code to degrade session key randomness to a mere PRNG (i.e. /dev/urandom instead of /dev/random). In a theoretical sense, this sacrifices Perfect Forward Secrecy, but for some applications (MTAs, for example) this is not such a relevant issue anyway. From nmav at gnutls.org Wed Jan 18 13:40:49 2006 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Wed, 18 Jan 2006 13:40:49 +0100 Subject: [gnutls-dev] Feature request: not really random session keys In-Reply-To: <87k6cxuaz2.fsf@mid.deneb.enyo.de> References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> Message-ID: On 1/18/06, Florian Weimer wrote: > Okay, the subject line might be a bit misleading. On server machines, > random bits are a very scarce ressource, and you cannot really afford > to throw them a way at a rate of a few kbps. Yet if you run certain > network servers (or clients) with GNUTLS, this is what happens -- and > these servers stall from time to time, waiting for more randomness. > I would like to see an additional API which allows code to degrade > session key randomness to a mere PRNG (i.e. /dev/urandom instead of > /dev/random). In a theoretical sense, this sacrifices Perfect Forward > Secrecy, but for some applications (MTAs, for example) this is not > such a relevant issue anyway. Well, gnutls shouldn't use /dev/random on normal server use. For example if you use only TLS /dev/random shouldn't be accessed. Only if you generate private keys (or RSA parameters) /dev/random will be used. regards, Nikos From jas at extundo.com Wed Jan 18 13:51:57 2006 From: jas at extundo.com (Simon Josefsson) Date: Wed, 18 Jan 2006 13:51:57 +0100 Subject: [gnutls-dev] Re: Feature request: not really random session keys In-Reply-To: <87k6cxuaz2.fsf@mid.deneb.enyo.de> (Florian Weimer's message of "Wed, 18 Jan 2006 10:44:17 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> Message-ID: Florian Weimer writes: > Okay, the subject line might be a bit misleading. On server machines, > random bits are a very scarce ressource, and you cannot really afford > to throw them a way at a rate of a few kbps. Yet if you run certain > network servers (or clients) with GNUTLS, this is what happens -- and > these servers stall from time to time, waiting for more randomness. > > I would like to see an additional API which allows code to degrade > session key randomness to a mere PRNG (i.e. /dev/urandom instead of > /dev/random). In a theoretical sense, this sacrifices Perfect Forward > Secrecy, but for some applications (MTAs, for example) this is not > such a relevant issue anyway. We are working on this issue, but we don't really know how to solve it currently. See Werner's last post on the 'living without global variables' thread for some problems. I have added the following to doc/TODO to make sure we don't forget it. * RNG improvements, to avoid /dev/random blocking on busy servers. You can specify --without-libgcrypt and then specify --enable-random-devices=/dev/urandom, but the non-libgcrypt mode is not well tested yet. The goal is probably to provide some new APIs for applications that want to modify this themselves, and some environment variable so users can override this per-application. The environment variable could contain a path to a device/socket that contain the random data. I'm beginning to think we should read from /dev/urandom by default. People with stronger RNG needs could change the path through the environment variable, and start some RNG daemon that collect entropy and serve it through that device or socket. I'd also like to support a strong PRNG seeded from a file, but I find it very complicated to do this. The problem is how to avoid having multiple GnuTLS instances read the same PRNG seed and thus generate the same random data. Doing that in a portable fashion is a challenge. Thanks. From jas at extundo.com Wed Jan 18 14:03:44 2006 From: jas at extundo.com (Simon Josefsson) Date: Wed, 18 Jan 2006 14:03:44 +0100 Subject: [gnutls-dev] Re: Feature request: not really random session keys In-Reply-To: (Nikos Mavrogiannopoulos's message of "Wed, 18 Jan 2006 13:40:49 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> Message-ID: Nikos Mavrogiannopoulos writes: > On 1/18/06, Florian Weimer wrote: >> Okay, the subject line might be a bit misleading. On server machines, >> random bits are a very scarce ressource, and you cannot really afford >> to throw them a way at a rate of a few kbps. Yet if you run certain >> network servers (or clients) with GNUTLS, this is what happens -- and >> these servers stall from time to time, waiting for more randomness. >> I would like to see an additional API which allows code to degrade >> session key randomness to a mere PRNG (i.e. /dev/urandom instead of >> /dev/random). In a theoretical sense, this sacrifices Perfect Forward >> Secrecy, but for some applications (MTAs, for example) this is not >> such a relevant issue anyway. > > Well, gnutls shouldn't use /dev/random on normal server use. For example if > you use only TLS /dev/random shouldn't be accessed. Only if you generate > private keys (or RSA parameters) /dev/random will be used. Perhaps someone with this problem could debug exactly what is accessing /dev/random? Perhaps libgcrypt does it internally for some reason. From fw at deneb.enyo.de Wed Jan 18 14:13:39 2006 From: fw at deneb.enyo.de (Florian Weimer) Date: Wed, 18 Jan 2006 14:13:39 +0100 Subject: [gnutls-dev] Feature request: not really random session keys In-Reply-To: (Nikos Mavrogiannopoulos's message of "Wed, 18 Jan 2006 13:40:49 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> Message-ID: <877j8xptks.fsf@mid.deneb.enyo.de> * Nikos Mavrogiannopoulos: > Well, gnutls shouldn't use /dev/random on normal server use. For > example if you use only TLS /dev/random shouldn't be accessed. Only > if you generate private keys (or RSA parameters) /dev/random will be > used. TLS needs session keys, and they seem to use cryptographically strong random numbers. From n.mavrogiannopoulos at gmail.com Wed Jan 18 14:19:59 2006 From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos) Date: Wed, 18 Jan 2006 14:19:59 +0100 Subject: [gnutls-dev] Feature request: not really random session keys In-Reply-To: <877j8xptks.fsf@mid.deneb.enyo.de> References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <877j8xptks.fsf@mid.deneb.enyo.de> Message-ID: On 1/18/06, Florian Weimer wrote: > > Well, gnutls shouldn't use /dev/random on normal server use. For > > example if you use only TLS /dev/random shouldn't be accessed. Only > > if you generate private keys (or RSA parameters) /dev/random will be > > used. > TLS needs session keys, and they seem to use cryptographically strong > random numbers. Indeed and /dev/urandom is adequate for this kind of keys. Only for long-lived keys such as the private keys /dev/random is used. From n.mavrogiannopoulos at gmail.com Wed Jan 18 14:26:15 2006 From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos) Date: Wed, 18 Jan 2006 14:26:15 +0100 Subject: [gnutls-dev] Feature request: not really random session keys In-Reply-To: <87k6cxuaz2.fsf@mid.deneb.enyo.de> References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> Message-ID: > I would like to see an additional API which allows code to degrade > session key randomness to a mere PRNG (i.e. /dev/urandom instead of > /dev/random). In a theoretical sense, this sacrifices Perfect Forward > Secrecy This is not really true. Only if you consider the /dev/urandom algorithms and the libgcrypt PRNG broken. regards, Nikos From fw at deneb.enyo.de Wed Jan 18 14:30:39 2006 From: fw at deneb.enyo.de (Florian Weimer) Date: Wed, 18 Jan 2006 14:30:39 +0100 Subject: [gnutls-dev] Re: Feature request: not really random session keys In-Reply-To: (Simon Josefsson's message of "Wed, 18 Jan 2006 14:03:44 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> Message-ID: <8764ohpssg.fsf@mid.deneb.enyo.de> * Simon Josefsson: > Perhaps someone with this problem could debug exactly what is > accessing /dev/random? Okay, I'll try to get a backtrace. In the meantime, here's what I found by looking at the source code. The RSA-based authentication function generates a session key only on the client side (lib/auth_rsa.c, _gnutls_gen_rsa_client_kx): if (gc_pseudo_random(session->key->key.data, session->key->key.size) != GC_OK) { gnutls_assert(); return GNUTLS_E_RANDOM_FAILED; } gc_pseudo_random (in gl/gc-libgcrypt.c) looks like this: gc_pseudo_random (char *data, size_t datalen) { gcry_randomize ((unsigned char *) data, datalen, GCRY_STRONG_RANDOM); return GC_OK; } The various DH implementations seem to end up calling gnutls_calc_dh_secret in lib/gnutls_dh.c: do { _gnutls_mpi_randomize(x, (x_size / 8) * 8, GCRY_STRONG_RANDOM); /* Check whether x is zero. */ } while( _gnutls_mpi_cmp_ui( x, 0)==0); _gnutls_mpi_randomize is actually gcry_mpi_randomize. If I read the libgcrypt source correctly GCRY_STRONG_RANDOM maps to level 2, and this means that a corresponding number of bits has to be read from /dev/random. From fw at deneb.enyo.de Wed Jan 18 14:39:28 2006 From: fw at deneb.enyo.de (Florian Weimer) Date: Wed, 18 Jan 2006 14:39:28 +0100 Subject: [gnutls-dev] Feature request: not really random session keys In-Reply-To: (Nikos Mavrogiannopoulos's message of "Wed, 18 Jan 2006 14:26:15 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> Message-ID: <87r775odtb.fsf@mid.deneb.enyo.de> * Nikos Mavrogiannopoulos: >> I would like to see an additional API which allows code to degrade >> session key randomness to a mere PRNG (i.e. /dev/urandom instead of >> /dev/random). In a theoretical sense, this sacrifices Perfect Forward >> Secrecy > > This is not really true. Only if you consider the /dev/urandom algorithms > and the libgcrypt PRNG broken. Yes, that's why I wrote "theoretical". 8-) I think it's completely acceptable to use a PRNG (instead of a real RNG) for those session keys. What I don't understand is that you say you are already using the PRNG source, and I find this hard to match with the source code and some reports from the trenches. From n.mavrogiannopoulos at gmail.com Wed Jan 18 14:43:41 2006 From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos) Date: Wed, 18 Jan 2006 14:43:41 +0100 Subject: [gnutls-dev] Feature request: not really random session keys In-Reply-To: <87r775odtb.fsf@mid.deneb.enyo.de> References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <87r775odtb.fsf@mid.deneb.enyo.de> Message-ID: On 1/18/06, Florian Weimer wrote: > Yes, that's why I wrote "theoretical". 8-) I think it's completely > acceptable to use a PRNG (instead of a real RNG) for those session > keys. > What I don't understand is that you say you are already using the PRNG > source, and I find this hard to match with the source code and some > reports from the trenches. Hmmm, I cannot verify it right now, but everything up to STRONG_RANDOM should have been using /dev/urandom. Only the VERY_STRONG_RANDOM in libgcrypt should use /dev/random, but this is not used for normal TLS sessions. If this is not the case then it's probably a bug either in libgcrypt or in gnutls. From jas at extundo.com Wed Jan 18 15:01:16 2006 From: jas at extundo.com (Simon Josefsson) Date: Wed, 18 Jan 2006 15:01:16 +0100 Subject: [gnutls-dev] Re: Feature request: not really random session keys In-Reply-To: <8764ohpssg.fsf@mid.deneb.enyo.de> (Florian Weimer's message of "Wed, 18 Jan 2006 14:30:39 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <8764ohpssg.fsf@mid.deneb.enyo.de> Message-ID: Florian Weimer writes: > The various DH implementations seem to end up calling > gnutls_calc_dh_secret in lib/gnutls_dh.c: > > do { > _gnutls_mpi_randomize(x, (x_size / 8) * 8, GCRY_STRONG_RANDOM); > /* Check whether x is zero. > */ > } while( _gnutls_mpi_cmp_ui( x, 0)==0); > > _gnutls_mpi_randomize is actually gcry_mpi_randomize. If I read the > libgcrypt source correctly GCRY_STRONG_RANDOM maps to level 2, and > this means that a corresponding number of bits has to be read from > /dev/random. STRONG_RANDOM is 1: typedef enum gcry_random_level { GCRY_WEAK_RANDOM = 0, GCRY_STRONG_RANDOM = 1, GCRY_VERY_STRONG_RANDOM = 2 } I believe only >= 2 should ever block. From wk at gnupg.org Thu Jan 19 09:48:48 2006 From: wk at gnupg.org (Werner Koch) Date: Thu, 19 Jan 2006 09:48:48 +0100 Subject: [gnutls-dev] Feature request: not really random session keys In-Reply-To: (Nikos Mavrogiannopoulos's message of "Wed, 18 Jan 2006 14:19:59 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <877j8xptks.fsf@mid.deneb.enyo.de> Message-ID: <87slrkwqkv.fsf@wheatstone.g10code.de> On Wed, 18 Jan 2006 14:19:59 +0100, Nikos Mavrogiannopoulos said: > Indeed and /dev/urandom is adequate for this kind of keys. Only > for long-lived keys such as the private keys /dev/random is used. Said hundreds of times over the years: /dev/urandom drains the same entropy people as /dev/random does. It just does not block but falls back into a PRNG. Other applications in need for some bytes out of /dev/randomj might then stall forever. Salam-Shalom, Werner From wk at gnupg.org Thu Jan 19 10:05:05 2006 From: wk at gnupg.org (Werner Koch) Date: Thu, 19 Jan 2006 10:05:05 +0100 Subject: [gnutls-dev] Feature request: not really random session keys In-Reply-To: (Nikos Mavrogiannopoulos's message of "Wed, 18 Jan 2006 14:43:41 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <87r775odtb.fsf@mid.deneb.enyo.de> Message-ID: <87irsgwptq.fsf@wheatstone.g10code.de> On Wed, 18 Jan 2006 14:43:41 +0100, Nikos Mavrogiannopoulos said: > Hmmm, I cannot verify it right now, but everything up to STRONG_RANDOM > should have been using /dev/urandom. Only the VERY_STRONG_RANDOM > in libgcrypt should use /dev/random, but this is not used for normal We have a similar problem in gpg: Saving libcgrypt's random seed to a file is not protected by a file locking. So when several gppg processes run it may happen that one sees an empty seed file and now starts to fill it up agains - this time from /dev/random! The solution is to lock the seed file. The same may happen with libgcrypt applications if several short living processes are running (Exim?). I am not sure whether GnuTLS sets a random seed file at all. Does it? The quick solution would be to uses fcntl locks which are available on all modern OSes. In the long term there will be no other way than to have a Libgcrypt specific daemon to maintain the entropy pool. I already wrote about the problems we are facing with such a solution (<873bjx8n4f.fsf at wheatstone.g10code.de>). Shalom-Salam, Werner From fw at deneb.enyo.de Fri Jan 20 12:17:36 2006 From: fw at deneb.enyo.de (Florian Weimer) Date: Fri, 20 Jan 2006 12:17:36 +0100 Subject: [gnutls-dev] Re: Feature request: not really random session keys In-Reply-To: (Simon Josefsson's message of "Wed, 18 Jan 2006 15:01:16 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <8764ohpssg.fsf@mid.deneb.enyo.de> Message-ID: <87ek33rvvz.fsf@mid.deneb.enyo.de> * Simon Josefsson: > STRONG_RANDOM is 1: > > typedef enum gcry_random_level > { > GCRY_WEAK_RANDOM = 0, > GCRY_STRONG_RANDOM = 1, > GCRY_VERY_STRONG_RANDOM = 2 > } > > I believe only >= 2 should ever block. I stand corrected. Hopefully, I'll receive more debugging information, so we can finally figure out what's going on. From Frediano.Ziglio at vodafone.com Wed Jan 25 13:11:10 2006 From: Frediano.Ziglio at vodafone.com (ZIGLIO, Frediano, VF-IT) Date: Wed, 25 Jan 2006 13:11:10 +0100 Subject: [gnutls-dev] constification patch Message-ID: I don't remember if I posted my updated patch for constification. Here you are. bye freddy77 -------------- next part -------------- A non-text attachment was scrubbed... Name: vedi.diff.gz Type: application/x-gzip Size: 4043 bytes Desc: vedi.diff.gz URL: From bernard at brenda-arkle.demon.co.uk Wed Jan 25 17:25:23 2006 From: bernard at brenda-arkle.demon.co.uk (Bernard Leak) Date: Wed, 25 Jan 2006 16:25:23 +0000 Subject: [gnutls-dev] A tiny patch for libtasn1 (0.2.17) In-Reply-To: <43CD7B0C.8030507@brenda-arkle.demon.co.uk> References: <43CD7B0C.8030507@brenda-arkle.demon.co.uk> Message-ID: <43D7A673.7050608@brenda-arkle.demon.co.uk> Dear List, I sent a supplementary patch for the testsuite, which became stuck in the "submission too large" queue. It's been in there (or deleted) since the 20th of January. The patch includes a renamed file, so the diff output includes the entire text of the old and new versions, surrounding a quite small set of substantive changes. Is a list moderator looking into it? Suggestions for reducing the bulk of the patch without breaking everyone's autopilot use of 'patch' are welcome... Bernard Leak -- Before they made me, they broke the mould (Bother, it appears that Groucho Marx said this about Sid Perelman, before I thought of it) From jas at extundo.com Wed Jan 25 18:04:19 2006 From: jas at extundo.com (Simon Josefsson) Date: Wed, 25 Jan 2006 18:04:19 +0100 Subject: [gnutls-dev] Re: A tiny patch for libtasn1 (0.2.17) In-Reply-To: <43D7A673.7050608@brenda-arkle.demon.co.uk> (Bernard Leak's message of "Wed, 25 Jan 2006 16:25:23 +0000") References: <43CD7B0C.8030507@brenda-arkle.demon.co.uk> <43D7A673.7050608@brenda-arkle.demon.co.uk> Message-ID: Bernard Leak writes: > Dear List, > I sent a supplementary patch for the testsuite, > which became stuck in > the "submission too large" queue. It's been in there (or deleted) > since the 20th of > January. The patch includes a renamed file, so the diff output > includes the entire > text of the old and new versions, surrounding a quite small set of > substantive > changes. Is a list moderator looking into it? Suggestions for > reducing the bulk > of the patch without breaking everyone's autopilot use of 'patch' are > welcome... Hi. Sorry about your problem. Try publishing the patch on a web site, and post the URL. Thanks! From bernard at brenda-arkle.demon.co.uk Fri Jan 20 17:27:04 2006 From: bernard at brenda-arkle.demon.co.uk (Bernard Leak) Date: Fri, 20 Jan 2006 16:27:04 +0000 Subject: [gnutls-dev] oops - slightly more patch for libtasn (0.2.17) Message-ID: <43D10F58.70709@brenda-arkle.demon.co.uk> Dear List, I now find that I never ran the testsuite for libtasn before submitting the previous patch. Here are the associated changes to the testsuite. Unfortunately, I can't find an elegant way of prepending a path in the test sources, so the top-level Test_parser.c and Test_tree.c files are now generated from .in files by configure, using AC_SUBST(srcdir) . The "obvious" alternative - with a -D define - is actually no better, because I'd still have to modify the sources, and I can't find a clean way to adjoin anything to the DEFS variable in a local fashion. Rather than play hob with the build rules, I make configure do the work. Note that the .c files disappear, and reappear as .c.in files, with slight changes. That's because there I don't know a graceful way of renaming files in CVS. Apologies for all the horrors: these very small and simple changes were surprisingly burdensome. Explanations of how to do this sort of thing elegantly would be very welcome! The attached patch is my "working" patch, so it includes the previous patch changes. It now includes patches (not really wanted for the CVS tree) for both configure and Makefile.in . Bernard Leak -- Before they made me, they broke the mould -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: libtasn1-0.2.17-sepbuild-2.patch URL: From yoann at prelude-ids.org Wed Jan 25 21:52:25 2006 From: yoann at prelude-ids.org (Yoann Vandoorselaere) Date: Wed, 25 Jan 2006 21:52:25 +0100 Subject: [gnutls-dev] [PATCH]: broken gnutls_bye() state machine Message-ID: <1138222345.32113.49.camel@arwen.prelude-ids.org> Hi folks, Attached is a patch that fix several issues in GnuTLS 1.2.9: # When gnutls_bye() reach STATE62, if _gnutls_recv_int() return EAGAIN, then upcoming gnutls_bye() call will resume at STATE61. This mean that the GNUTLS_A_CLOSE_NOTIFY alert will be sent several time, even through the peer might have stopped reading the socket. This lead to garbage remaining to be read, even through gnutls_bye() returned 0 on both end of the connection (thus, unusable connection). # gnutls_bye() not returning an alert sent by the peer, in the specific case where: 1. the peer send an alert, followed by a call to gnutls_bye() then immediate connection closure. 2. Before noticing the hang-up, we try to send something to the peer, and get GNUTLS_E_PUSH_ERROR, since the peer has already closed the socket. The problem happen here: the internal GnuTLS code will call the session_invalidate() function as a result of the send() error. Unfortunately, this mean that we won't be able to read the remaining data (the alert sent earlier, in #1), since _gnutls_recv_int() check whether the session has been invalidated before reading the socket. This is why the attached patch remove the session_invalidate() call, and set session->internals.may_not_write to 1. 3. We thus call gnutls_bye(), but never get the alert sent in #1, since _gnutls_recv_int() will always return an error prior trying to read the socket, due to the session being already invalidated. With the attached patch, I can not reproduce both problem anymore. I'm not sure whether the patch is fully correct through. Hope it help, -- Yoann Vandoorselaere | Responsable R&D / CTO | PreludeIDS Technologies Tel: +33 (0)8 70 70 21 58 Fax: +33(0)4 78 42 21 58 http://www.prelude-ids.com -- Yoann Vandoorselaere -------------- next part -------------- A non-text attachment was scrubbed... Name: gnutls-1.2.9-bye.diff Type: text/x-patch Size: 596 bytes Desc: not available URL: From jas at extundo.com Fri Jan 27 13:44:21 2006 From: jas at extundo.com (Simon Josefsson) Date: Fri, 27 Jan 2006 13:44:21 +0100 Subject: [gnutls-dev] Re: oops - slightly more patch for libtasn (0.2.17) In-Reply-To: <43D10F58.70709@brenda-arkle.demon.co.uk> (Bernard Leak's message of "Fri, 20 Jan 2006 16:27:04 +0000") References: <43D10F58.70709@brenda-arkle.demon.co.uk> Message-ID: Bernard Leak writes: > Dear List, > I now find that I never ran the testsuite for libtasn > before submitting > the previous patch. Here are the associated changes to the testsuite. > Unfortunately, I can't find an elegant way of prepending a path in the test > sources, so the top-level Test_parser.c and Test_tree.c files are now > generated from .in files by configure, using AC_SUBST(srcdir) . The > "obvious" alternative - with a -D define - is actually no better, because > I'd still have to modify the sources, and I can't find a clean way to adjoin > anything to the DEFS variable in a local fashion. Rather than play hob > with the build rules, I make configure do the work. > > Note that the .c files disappear, and reappear as .c.in files, with slight > changes. That's because there I don't know a graceful way of renaming files > in CVS. > > Apologies for all the horrors: these very small and simple changes were > surprisingly burdensome. Explanations of how to do this sort of thing > elegantly would be very welcome! > > The attached patch is my "working" patch, so it includes the previous patch > changes. It now includes patches (not really wanted for the CVS tree) > for both configure and Makefile.in . Hi Bernard! I fixed this in a different way: I made automake define two environment variables for the filename, and then used 'getenv' inside the program to find out the filename to read. GnuTLS uses a similar approach. I'd appreciate if you would test the latest CVS. It builds fine here with objdir != srcdir. I have made a release candidate of 0.2.18 available: http://josefsson.org/libtasn1/rc/libtasn1-0.2.18.tar.gz Please test it! If there is no reports, I'll release it in a few days. Thanks, Simon From jas at extundo.com Fri Jan 27 14:45:24 2006 From: jas at extundo.com (Simon Josefsson) Date: Fri, 27 Jan 2006 14:45:24 +0100 Subject: [gnutls-dev] Re: living without global variables In-Reply-To: <200601091753.k09HrEUL024566@mail-relay.eunet.no> (Bryan Henderson's message of "9 Jan 2006 16:54:07 +0000") References: <7997.31945521689$1135626397@news.gmane.org> <87sls6ubwt.fsf@wheatstone.g10code.de> <200601091753.k09HrEUL024566@mail-relay.eunet.no> Message-ID: bryanh at giraffe-data.com (Bryan Henderson) writes: >>The distinction between a library and an application is rather blurry >>nowadays. > > Yes, lets just say that a program consists of multiple modules, and some > of them are related in a hierarchical way -- A uses B, B uses C. Some > modules are packaged as code libraries. One module stands out, because > it is at the top of the hierarchy -- the thing that gets control first > and controls, at least indirectly, every other module. > >>I'm not convinced it scale to force applications to initialize all >>libraries -- i.e., to force OpenOffice to initialize libgcrypt because >>OpenOffice may use some component that eventually may dlopen some >>library that may use gnutls that may use libgcrypt. >> >>I'd rather have libraries initialize the libraries they use >>internally, to make things modular. > > If you take dlopen out of the picture for the sake of analysis, you > can have both. OpenOffice initializes the component that it uses > directly, which initializes another library, which initializes gnutls, > which initializes libgcrypt. So the top level module (OpenOffice) > effectively initializes libgcrypt, without ever knowing that libgcrypt > exists. If you believe that libgcrypt should be participating in the > program's threading, then you also believe that all those other layers > should too, and therefore the "threading model" information is present > in parameters on all of their initialization routines. I think we are saying the same thing in different ways. The problem with libgcrypt is that gnutls cannot initialize it correctly, because libgcrypt need to know about threads, of which gnutls know nothing. The "solution" is to require that OpenOffice initialize libgcrypt if it uses some library that uses gnutls. I don't think that approach scale. I believe the approach you outline is better. I do not think all libraries need to participate in the threading, most libraries do not use threads so it is irrelevant to them. Adding thread initialization code just because the library uses libgcrypt seem wrong to me. We will need a cleaner solution to this. The remaining problem with the gnulib approach is to add APIs for MPI, and to solve the RNG situation. Help welcome. Thanks, Simon From nmav at gnutls.org Mon Jan 30 03:19:49 2006 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Mon, 30 Jan 2006 03:19:49 +0100 Subject: [gnutls-dev] Re: [PATCH]: broken gnutls_bye() state machine In-Reply-To: <1138222345.32113.49.camel@arwen.prelude-ids.org> References: <1138222345.32113.49.camel@arwen.prelude-ids.org> Message-ID: <200601300319.49880.nmav@gnutls.org> On Wednesday 25 January 2006 21:52, Yoann Vandoorselaere wrote: > Hi folks, > Attached is a patch that fix several issues in GnuTLS 1.2.9: > > # When gnutls_bye() reach STATE62, if _gnutls_recv_int() return > EAGAIN, then upcoming gnutls_bye() call will resume at STATE61. [...] > # gnutls_bye() not returning an alert sent by the peer, in the > specific case where: Hello, I've applied the patch to fix the above issues. -- Nikos Mavrogiannopoulos From fw at deneb.enyo.de Mon Jan 30 14:18:43 2006 From: fw at deneb.enyo.de (Florian Weimer) Date: Mon, 30 Jan 2006 14:18:43 +0100 Subject: [gnutls-dev] Feature request: not really random session keys In-Reply-To: <87irsgwptq.fsf@wheatstone.g10code.de> (Werner Koch's message of "Thu, 19 Jan 2006 10:05:05 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <87r775odtb.fsf@mid.deneb.enyo.de> <87irsgwptq.fsf@wheatstone.g10code.de> Message-ID: <87irs1c0r0.fsf@mid.deneb.enyo.de> * Werner Koch: > The same may happen with libgcrypt applications if several short > living processes are running (Exim?). I am not sure whether GnuTLS > sets a random seed file at all. Does it? In case of Exim, it's regeneration of the RSA_EXPORT key. It is not serialized, either, so multiple Exim processes try to regenerate it and consume increasing amounts of entropy. > In the long term there will be no other way than to have a Libgcrypt > specific daemon to maintain the entropy pool. Why not fix /dev/random instead, and add the functionality which is missing there? With all the trouble with threading, forking, and so on, it might make sense to put this into the kernel. From n.mavrogiannopoulos at gmail.com Mon Jan 30 14:32:45 2006 From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos) Date: Mon, 30 Jan 2006 14:32:45 +0100 Subject: [gnutls-dev] Feature request: not really random session keys In-Reply-To: <87irs1c0r0.fsf@mid.deneb.enyo.de> References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <87r775odtb.fsf@mid.deneb.enyo.de> <87irsgwptq.fsf@wheatstone.g10code.de> <87irs1c0r0.fsf@mid.deneb.enyo.de> Message-ID: On 1/30/06, Florian Weimer wrote: > > The same may happen with libgcrypt applications if several short > > living processes are running (Exim?). I am not sure whether GnuTLS > > sets a random seed file at all. Does it? > In case of Exim, it's regeneration of the RSA_EXPORT key. It is not > serialized, either, so multiple Exim processes try to regenerate it > and consume increasing amounts of entropy. As far as I remember it was saving it to a file to eliminate the need for regeneration every time. Isn't this the case any more? From jas at extundo.com Mon Jan 30 15:13:48 2006 From: jas at extundo.com (Simon Josefsson) Date: Mon, 30 Jan 2006 15:13:48 +0100 Subject: [gnutls-dev] Re: Feature request: not really random session keys In-Reply-To: <87irs1c0r0.fsf@mid.deneb.enyo.de> (Florian Weimer's message of "Mon, 30 Jan 2006 14:18:43 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <87r775odtb.fsf@mid.deneb.enyo.de> <87irsgwptq.fsf@wheatstone.g10code.de> <87irs1c0r0.fsf@mid.deneb.enyo.de> Message-ID: Florian Weimer writes: > * Werner Koch: > >> The same may happen with libgcrypt applications if several short >> living processes are running (Exim?). I am not sure whether GnuTLS >> sets a random seed file at all. Does it? > > In case of Exim, it's regeneration of the RSA_EXPORT key. It is not > serialized, either, so multiple Exim processes try to regenerate it > and consume increasing amounts of entropy. I recall the same problem in some other application. The solution was to have a separate process devoted to regenerate the keys, store it to a file, and have the other processes use it. This circumvent the synchronization problem, which can be quite complicated, and also guarantee that the Exim process will never block on /dev/random. The process that regenerate the keys can be invoked through cron. From fw at deneb.enyo.de Mon Jan 30 15:14:22 2006 From: fw at deneb.enyo.de (Florian Weimer) Date: Mon, 30 Jan 2006 15:14:22 +0100 Subject: [gnutls-dev] Feature request: not really random session keys In-Reply-To: (Nikos Mavrogiannopoulos's message of "Mon, 30 Jan 2006 14:32:45 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <87r775odtb.fsf@mid.deneb.enyo.de> <87irsgwptq.fsf@wheatstone.g10code.de> <87irs1c0r0.fsf@mid.deneb.enyo.de> Message-ID: <87zmldajlt.fsf@mid.deneb.enyo.de> * Nikos Mavrogiannopoulos: > On 1/30/06, Florian Weimer wrote: > >> > The same may happen with libgcrypt applications if several short >> > living processes are running (Exim?). I am not sure whether GnuTLS >> > sets a random seed file at all. Does it? >> In case of Exim, it's regeneration of the RSA_EXPORT key. It is not >> serialized, either, so multiple Exim processes try to regenerate it >> and consume increasing amounts of entropy. > > As far as I remember it was saving it to a file to eliminate the need > for regeneration every time. Isn't this the case any more? It does, but when it's not there (or outdated, apparently), every delivery process which needs it tries to regenerate it in parallel. If you have a busy mail server, this is quite noticeable. (It doesn't matter if you only process a few thousand messages per day.) From wk at gnupg.org Mon Jan 30 15:21:34 2006 From: wk at gnupg.org (Werner Koch) Date: Mon, 30 Jan 2006 15:21:34 +0100 Subject: [gnutls-dev] Feature request: not really random session keys In-Reply-To: <87irs1c0r0.fsf@mid.deneb.enyo.de> (Florian Weimer's message of "Mon, 30 Jan 2006 14:18:43 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <87r775odtb.fsf@mid.deneb.enyo.de> <87irsgwptq.fsf@wheatstone.g10code.de> <87irs1c0r0.fsf@mid.deneb.enyo.de> Message-ID: <87wtgh4x01.fsf@wheatstone.g10code.de> On Mon, 30 Jan 2006 14:18:43 +0100, Florian Weimer said: > Why not fix /dev/random instead, and add the functionality which is > missing there? With all the trouble with threading, forking, and so > on, it might make sense to put this into the kernel. Sure. That was orginally Ted Tso's plan but he could not get a solid RNG into the kernel because the kernel hackers required to amke /dev/random optional and Ted's plan was to have a solid RNG in the kernel as a standard service. With all the changes to the RNG (or better the so-called entropy sources) I still feel safer to add some extra processing to /dev/random. Some OSes don't have a /dev/random or worse a predictable one (some OS X). Thus we need to do it on our own to be portable. Salam-Shalom, Werner From fw at deneb.enyo.de Mon Jan 30 15:34:50 2006 From: fw at deneb.enyo.de (Florian Weimer) Date: Mon, 30 Jan 2006 15:34:50 +0100 Subject: [gnutls-dev] Feature request: not really random session keys In-Reply-To: <87k6cxuaz2.fsf@mid.deneb.enyo.de> (Florian Weimer's message of "Wed, 18 Jan 2006 10:44:17 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> Message-ID: <87bqxtainp.fsf@mid.deneb.enyo.de> I tracked this down to the generation of the RSA_EXPORT key. In this case, bits from /dev/random are used, even though the generated key is horribly insecure anyway. Wouldn't it make sense to use only STRONG_RANDOM in this case, and not VERY_STRONG_RANDOM? (I have a distinct dej?-vu feeling about the whole matter. Odd.) From n.mavrogiannopoulos at gmail.com Mon Jan 30 15:40:09 2006 From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos) Date: Mon, 30 Jan 2006 15:40:09 +0100 Subject: [gnutls-dev] Feature request: not really random session keys In-Reply-To: <87zmldajlt.fsf@mid.deneb.enyo.de> References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <87r775odtb.fsf@mid.deneb.enyo.de> <87irsgwptq.fsf@wheatstone.g10code.de> <87irs1c0r0.fsf@mid.deneb.enyo.de> <87zmldajlt.fsf@mid.deneb.enyo.de> Message-ID: On 1/30/06, Florian Weimer wrote: > > As far as I remember it was saving it to a file to eliminate the need > > for regeneration every time. Isn't this the case any more? > It does, but when it's not there (or outdated, apparently), every > delivery process which needs it tries to regenerate it in parallel. > If you have a busy mail server, this is quite noticeable. (It doesn't > matter if you only process a few thousand messages per day.) Hmmm then it's a problem... the process shouldn't check if it is outdated or not (or could check but in that case disable the corresponding ciphersuites, instead of generating the key). The easier way to fix that is to generate the RSA key and the DH parameters by other means --say certtool running on the bg once per day or something like that. From jas at extundo.com Mon Jan 30 16:30:25 2006 From: jas at extundo.com (Simon Josefsson) Date: Mon, 30 Jan 2006 16:30:25 +0100 Subject: [gnutls-dev] Re: Feature request: not really random session keys In-Reply-To: <87bqxtainp.fsf@mid.deneb.enyo.de> (Florian Weimer's message of "Mon, 30 Jan 2006 15:34:50 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <87bqxtainp.fsf@mid.deneb.enyo.de> Message-ID: Florian Weimer writes: > I tracked this down to the generation of the RSA_EXPORT key. In this > case, bits from /dev/random are used, even though the generated key is > horribly insecure anyway. > > Wouldn't it make sense to use only STRONG_RANDOM in this case, and not > VERY_STRONG_RANDOM? Perhaps. But doesn't this happen for non-RSA_EXPORT keys too? We wouldn't want to make that change there. It seems better to fix Exim here. From fw at deneb.enyo.de Mon Jan 30 17:00:13 2006 From: fw at deneb.enyo.de (Florian Weimer) Date: Mon, 30 Jan 2006 17:00:13 +0100 Subject: [gnutls-dev] Feature request: not really random session keys In-Reply-To: (Nikos Mavrogiannopoulos's message of "Mon, 30 Jan 2006 15:40:09 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <87r775odtb.fsf@mid.deneb.enyo.de> <87irsgwptq.fsf@wheatstone.g10code.de> <87irs1c0r0.fsf@mid.deneb.enyo.de> <87zmldajlt.fsf@mid.deneb.enyo.de> Message-ID: <87oe1t904y.fsf@mid.deneb.enyo.de> * Nikos Mavrogiannopoulos: > Hmmm then it's a problem... the process shouldn't check if it is > outdated or not (or could check but in that case disable the > corresponding ciphersuites, instead of generating the key). I don't think RSA_EXPORT is terribly important anyway. 8-> > The easier way to fix that is to generate the RSA key and the DH > parameters by other means --say certtool running on the bg once per > day or something like that. The params file seems to be in some kind of proprietary file format, so this is not as easy as it sounds. But we will likely do something like this when it's been decided that we cannot scrap RSA_EXPORT support. From fw at deneb.enyo.de Mon Jan 30 17:11:30 2006 From: fw at deneb.enyo.de (Florian Weimer) Date: Mon, 30 Jan 2006 17:11:30 +0100 Subject: [gnutls-dev] Re: Feature request: not really random session keys In-Reply-To: (Simon Josefsson's message of "Mon, 30 Jan 2006 16:30:25 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <87bqxtainp.fsf@mid.deneb.enyo.de> Message-ID: <87k6ch8zm5.fsf@mid.deneb.enyo.de> * Simon Josefsson: > Florian Weimer writes: > >> I tracked this down to the generation of the RSA_EXPORT key. In this >> case, bits from /dev/random are used, even though the generated key is >> horribly insecure anyway. >> >> Wouldn't it make sense to use only STRONG_RANDOM in this case, and not >> VERY_STRONG_RANDOM? > > Perhaps. But doesn't this happen for non-RSA_EXPORT keys too? We > wouldn't want to make that change there. I think you'd need to pass an additional flag, yes. It's certainly not a two-line change, and I can understand if you don't want to make it for such a silly feature. > It seems better to fix Exim here. Even if we follow the advice in your other message, a busy mail server will deplete the pool at an alarming rate (each TLS-enabled SMTP connection consumes 600 bytes from the kernel pool -- which can only store 4096 bits). This means that gathering the required true randomness may take a long time. We'll see if it's still acceptable, or if the randomness is distributed so unfairly that it won't work. From n.mavrogiannopoulos at gmail.com Mon Jan 30 17:17:56 2006 From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos) Date: Mon, 30 Jan 2006 17:17:56 +0100 Subject: [gnutls-dev] Feature request: not really random session keys In-Reply-To: <87oe1t904y.fsf@mid.deneb.enyo.de> References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <87oe1t904y.fsf@mid.deneb.enyo.de> Message-ID: <200601301717.56600.n.mavrogiannopoulos@gmail.com> On Monday 30 January 2006 17:00, Florian Weimer wrote: > > The easier way to fix that is to generate the RSA key and the DH > > parameters by other means --say certtool running on the bg once per > > day or something like that. > The params file seems to be in some kind of proprietary file format, > so this is not as easy as it sounds. But we will likely do something > like this when it's been decided that we cannot scrap RSA_EXPORT > support. I had written a patch for that so the format was a standard pkcs1 pem encoded key. I thought it had been applied in the latest version of exim. regards, Nikos From fw at deneb.enyo.de Mon Jan 30 17:20:03 2006 From: fw at deneb.enyo.de (Florian Weimer) Date: Mon, 30 Jan 2006 17:20:03 +0100 Subject: [gnutls-dev] Re: living without global variables In-Reply-To: (Simon Josefsson's message of "Fri, 27 Jan 2006 14:45:24 +0100") References: <7997.31945521689$1135626397@news.gmane.org> <87sls6ubwt.fsf@wheatstone.g10code.de> <200601091753.k09HrEUL024566@mail-relay.eunet.no> Message-ID: <87d5i98z7w.fsf@mid.deneb.enyo.de> * Simon Josefsson: > The problem with libgcrypt is that gnutls cannot initialize it > correctly, because libgcrypt need to know about threads, of which > gnutls know nothing. This means that you probably need to do away with the need for initialization in libgcrypt. Use .init sections (aka C++ constructors)? Nowadays, they are available on most systems. Or what about pthread_once? You only need a static initialize to use it. From fw at deneb.enyo.de Mon Jan 30 17:26:00 2006 From: fw at deneb.enyo.de (Florian Weimer) Date: Mon, 30 Jan 2006 17:26:00 +0100 Subject: [gnutls-dev] Feature request: not really random session keys In-Reply-To: <87wtgh4x01.fsf@wheatstone.g10code.de> (Werner Koch's message of "Mon, 30 Jan 2006 15:21:34 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <87r775odtb.fsf@mid.deneb.enyo.de> <87irsgwptq.fsf@wheatstone.g10code.de> <87irs1c0r0.fsf@mid.deneb.enyo.de> <87wtgh4x01.fsf@wheatstone.g10code.de> Message-ID: <87bqxt8yxz.fsf@mid.deneb.enyo.de> * Werner Koch: > On Mon, 30 Jan 2006 14:18:43 +0100, Florian Weimer said: > >> Why not fix /dev/random instead, and add the functionality which is >> missing there? With all the trouble with threading, forking, and so >> on, it might make sense to put this into the kernel. > > Sure. That was orginally Ted Tso's plan but he could not get a solid > RNG into the kernel because the kernel hackers required to amke > /dev/random optional and Ted's plan was to have a solid RNG in the > kernel as a standard service. /dev/random is no longer optional, it's needed by the network stack (to generate random keys for internal hash functions, for instance, or for the secret which is used to compute SYN cookies). Some changes are still desirable, though. Someone needs to review the entropy sources and verify that the estimates are correct. It would be a good idea to incorporate a few of Peter Gutmann's suggestions, I think. (I'll see what I can do about this, but this particular issue has been on my to-do list for four or five years. 8-/) > Some OSes don't have a /dev/random or worse a predictable one (some OS X). > Thus we need to do it on our own to be portable. Then you need a special daemon. However, I would like to avoid the additional administrative overhead on systems where the kernel can be fixed. From jas at extundo.com Mon Jan 30 17:27:42 2006 From: jas at extundo.com (Simon Josefsson) Date: Mon, 30 Jan 2006 17:27:42 +0100 Subject: [gnutls-dev] Re: Feature request: not really random session keys In-Reply-To: <87k6ch8zm5.fsf@mid.deneb.enyo.de> (Florian Weimer's message of "Mon, 30 Jan 2006 17:11:30 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <87bqxtainp.fsf@mid.deneb.enyo.de> <87k6ch8zm5.fsf@mid.deneb.enyo.de> Message-ID: Florian Weimer writes: > * Simon Josefsson: > >> Florian Weimer writes: >> >>> I tracked this down to the generation of the RSA_EXPORT key. In this >>> case, bits from /dev/random are used, even though the generated key is >>> horribly insecure anyway. >>> >>> Wouldn't it make sense to use only STRONG_RANDOM in this case, and not >>> VERY_STRONG_RANDOM? >> >> Perhaps. But doesn't this happen for non-RSA_EXPORT keys too? We >> wouldn't want to make that change there. > > I think you'd need to pass an additional flag, yes. It's certainly > not a two-line change, and I can understand if you don't want to make > it for such a silly feature. I dunno. Let's see what options we have. There isn't a patch to consider at this point anyway. >> It seems better to fix Exim here. > > Even if we follow the advice in your other message, a busy mail server > will deplete the pool at an alarming rate (each TLS-enabled SMTP > connection consumes 600 bytes from the kernel pool -- which can only > store 4096 bits). This means that gathering the required true > randomness may take a long time. We'll see if it's still acceptable, > or if the randomness is distributed so unfairly that it won't work. Does /dev/urandom deplete the entropy pool that quickly? That seem like bad design. It seems to me that /dev/urandom should be a PRNG seeded with, say, 256 bytes of good randomness. It would be quick after that, and not require more "real" randomness. For improved security, it could be re-seeded sometimes but that shouldn't be done so often that it destroy the /dev/random pool. It is probably important to re-seed it, because the /dev/random datat used to seed the PRNG may contain little entropy if the machine was just re-started, or if it is an embedded system with few entropy sources. It may be wise for systems to save the /dev/random pool on shutdown and restore it on startup. Thanks. From jas at extundo.com Mon Jan 30 17:30:46 2006 From: jas at extundo.com (Simon Josefsson) Date: Mon, 30 Jan 2006 17:30:46 +0100 Subject: [gnutls-dev] Re: living without global variables In-Reply-To: <87d5i98z7w.fsf@mid.deneb.enyo.de> (Florian Weimer's message of "Mon, 30 Jan 2006 17:20:03 +0100") References: <7997.31945521689$1135626397@news.gmane.org> <87sls6ubwt.fsf@wheatstone.g10code.de> <200601091753.k09HrEUL024566@mail-relay.eunet.no> <87d5i98z7w.fsf@mid.deneb.enyo.de> Message-ID: Florian Weimer writes: > * Simon Josefsson: > >> The problem with libgcrypt is that gnutls cannot initialize it >> correctly, because libgcrypt need to know about threads, of which >> gnutls know nothing. > > This means that you probably need to do away with the need for > initialization in libgcrypt. > > Use .init sections (aka C++ constructors)? Nowadays, they are > available on most systems. Or what about pthread_once? You only need > a static initialize to use it. That seems like a libgcrypt improvement... perhaps the gcrypt-dev list is more appropriate. It sounds somewhat unportable to me, though. I don't know, and doubt, that even fairly popular targets such as mingw32 has it. Thanks! From fw at deneb.enyo.de Mon Jan 30 17:44:41 2006 From: fw at deneb.enyo.de (Florian Weimer) Date: Mon, 30 Jan 2006 17:44:41 +0100 Subject: [gnutls-dev] Re: Feature request: not really random session keys In-Reply-To: (Simon Josefsson's message of "Mon, 30 Jan 2006 17:27:42 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <87bqxtainp.fsf@mid.deneb.enyo.de> <87k6ch8zm5.fsf@mid.deneb.enyo.de> Message-ID: <87vew17jie.fsf@mid.deneb.enyo.de> * Simon Josefsson: >> Even if we follow the advice in your other message, a busy mail server >> will deplete the pool at an alarming rate (each TLS-enabled SMTP >> connection consumes 600 bytes from the kernel pool -- which can only >> store 4096 bits). This means that gathering the required true >> randomness may take a long time. We'll see if it's still acceptable, >> or if the randomness is distributed so unfairly that it won't work. > > Does /dev/urandom deplete the entropy pool that quickly? Yes, there is only a single pool, shared by /dev/random and /dev/urandom. Each bit you read from the pool reduces the entropy estimate by one. There is no way to avoid that, short of artificially thinning the output (i.e. read 32 bits and turn them into 64 or something like that). > It seems to me that /dev/urandom should be a PRNG seeded with, say, > 256 bytes of good randomness. It would be quick after that, and not > require more "real" randomness. For improved security, it could be > re-seeded sometimes but that shouldn't be done so often that it > destroy the /dev/random pool. This makes a lot of sense, but I could imagine that this is problematic to get past the kernel developers (because it intentionally decreases the quality of /dev/urandom output). > It is probably important to re-seed it, because the /dev/random > datat used to seed the PRNG may contain little entropy if the > machine was just re-started, After a reboot, there is a lot of disk activity, and according to the current estimates, this creates a lot of entropy. So it's not a real issue, unless you need random bits very early in the boot process. In that case, you could temporarily switch to a hardware RNG (which is included in most current systems). > It may be wise for systems to save the /dev/random pool on shutdown > and restore it on startup. Is this really a good idea? I mean, exposing the pool state like this? From jas at extundo.com Mon Jan 30 17:51:01 2006 From: jas at extundo.com (Simon Josefsson) Date: Mon, 30 Jan 2006 17:51:01 +0100 Subject: [gnutls-dev] Re: Feature request: not really random session keys In-Reply-To: <87bqxt8yxz.fsf@mid.deneb.enyo.de> (Florian Weimer's message of "Mon, 30 Jan 2006 17:26:00 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <87r775odtb.fsf@mid.deneb.enyo.de> <87irsgwptq.fsf@wheatstone.g10code.de> <87irs1c0r0.fsf@mid.deneb.enyo.de> <87wtgh4x01.fsf@wheatstone.g10code.de> <87bqxt8yxz.fsf@mid.deneb.enyo.de> Message-ID: Florian Weimer writes: >> Some OSes don't have a /dev/random or worse a predictable one (some OS X). >> Thus we need to do it on our own to be portable. > > Then you need a special daemon. However, I would like to avoid the > additional administrative overhead on systems where the kernel can be > fixed. Hear, hear. Moving this complexity away from applications (GnuTLS, GNU SASL, Shishi, ...) seem like something very useful. Simply moving it to an external daemon is good enough, improving /dev/random on Linux would be an optimization. Should we write a simple daemon 'grngd', based on libgcrypt, and start to use it? That should be simple. It should likely register two sockets, one suitable for short-term session keys and one for long-term keys, matching /dev/urandom and /dev/random. Is there any point for us to look at EGD? I think I'll take up on this exercise soon. Thanks. From jas at extundo.com Mon Jan 30 18:02:45 2006 From: jas at extundo.com (Simon Josefsson) Date: Mon, 30 Jan 2006 18:02:45 +0100 Subject: [gnutls-dev] Re: Feature request: not really random session keys In-Reply-To: <87vew17jie.fsf@mid.deneb.enyo.de> (Florian Weimer's message of "Mon, 30 Jan 2006 17:44:41 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <87bqxtainp.fsf@mid.deneb.enyo.de> <87k6ch8zm5.fsf@mid.deneb.enyo.de> <87vew17jie.fsf@mid.deneb.enyo.de> Message-ID: Florian Weimer writes: > * Simon Josefsson: > >>> Even if we follow the advice in your other message, a busy mail server >>> will deplete the pool at an alarming rate (each TLS-enabled SMTP >>> connection consumes 600 bytes from the kernel pool -- which can only >>> store 4096 bits). This means that gathering the required true >>> randomness may take a long time. We'll see if it's still acceptable, >>> or if the randomness is distributed so unfairly that it won't work. >> >> Does /dev/urandom deplete the entropy pool that quickly? > > Yes, there is only a single pool, shared by /dev/random and > /dev/urandom. Each bit you read from the pool reduces the entropy > estimate by one. There is no way to avoid that, short of artificially > thinning the output (i.e. read 32 bits and turn them into 64 or > something like that). Ouch. This seem sub-optimal to me. >> It seems to me that /dev/urandom should be a PRNG seeded with, say, >> 256 bytes of good randomness. It would be quick after that, and not >> require more "real" randomness. For improved security, it could be >> re-seeded sometimes but that shouldn't be done so often that it >> destroy the /dev/random pool. > > This makes a lot of sense, but I could imagine that this is > problematic to get past the kernel developers (because it > intentionally decreases the quality of /dev/urandom output). I don't see how it decreases the quality. /dev/urandom is documented to potentially be vulnerable to a cryptographic attack. Pick a PRNG and you fulfil the documented security problem, as far as I can see. Do you know if anyone has done any work on /dev/*random in the kernel lately? Perhaps we could talk to them. >> It is probably important to re-seed it, because the /dev/random >> datat used to seed the PRNG may contain little entropy if the >> machine was just re-started, > > After a reboot, there is a lot of disk activity, and according to the > current estimates, this creates a lot of entropy. If it is using delays from the disk as entropy sources, I think it may not be impossible to predict them, or at least predict a set of possible delays. Especially if you are booting from flash disks or similar. > So it's not a real issue, unless you need random bits very early in > the boot process. In that case, you could temporarily switch to a > hardware RNG (which is included in most current systems). That's probably a good idea anyway. >> It may be wise for systems to save the /dev/random pool on shutdown >> and restore it on startup. > > Is this really a good idea? I mean, exposing the pool state like > this? Perhaps not on desktops, but perhaps on embedded systems. Thanks! From lunz at falooley.org Mon Jan 30 17:58:19 2006 From: lunz at falooley.org (Jason Lunz) Date: Mon, 30 Jan 2006 16:58:19 +0000 (UTC) Subject: [gnutls-dev] ongoing entropy problems Message-ID: Are the gnutls developers aware of the ongoing entropy-pool-draining problems with gnutls in exim4? For example: http://article.gmane.org/gmane.linux.debian.devel.exim4.user/477 Is this a known problem? Can something be done about it? Jason From ametzler at downhill.at.eu.org Mon Jan 30 19:20:10 2006 From: ametzler at downhill.at.eu.org (Andreas Metzler) Date: Mon, 30 Jan 2006 19:20:10 +0100 Subject: [gnutls-dev] Feature request: not really random session keys In-Reply-To: <87oe1t904y.fsf@mid.deneb.enyo.de> References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <87r775odtb.fsf@mid.deneb.enyo.de> <87irsgwptq.fsf@wheatstone.g10code.de> <87irs1c0r0.fsf@mid.deneb.enyo.de> <87zmldajlt.fsf@mid.deneb.enyo.de> <87oe1t904y.fsf@mid.deneb.enyo.de> Message-ID: <20060130182010.GB2861@downhill.aus.cc> On 2006-01-30 Florian Weimer wrote: > * Nikos Mavrogiannopoulos: [...] > > The easier way to fix that is to generate the RSA key and the DH > > parameters by other means --say certtool running on the bg once per > > day or something like that. > The params file seems to be in some kind of proprietary file format, > so this is not as easy as it sounds. But we will likely do something > like this when it's been decided that we cannot scrap RSA_EXPORT > support. It is not anymore. Since 4.54? Quoting /usr/share/doc/exim4-base/changelog.Debian.gz exim4 (4.52-2) unstable; urgency=low [...] * Use certtool from gnutls-bin in cron.daily to re-generate gnutls-params instead of only removing the file and letting exim4 re-generate it at SMTP time after receiving STARTTLS. The maximum runtime of certtool is limited to 2*1800 seconds per default by using timeout.pl by Piotr Roszatycki (checked and beautified by Marc 'HE' Brockschmidt). Add gnutls-bin to exim4-base' Suggests. (am) Closes: #285371 [...] cu andreas -- The 'Galactic Cleaning' policy undertaken by Emperor Zhark is a personal vision of the emperor's, and its inclusion in this work does not constitute tacit approval by the author or the publisher for any such projects, howsoever undertaken. (c) Jasper Ffforde From wk at gnupg.org Tue Jan 31 10:17:08 2006 From: wk at gnupg.org (Werner Koch) Date: Tue, 31 Jan 2006 10:17:08 +0100 Subject: [gnutls-dev] Re: Feature request: not really random session keys In-Reply-To: <87vew17jie.fsf@mid.deneb.enyo.de> (Florian Weimer's message of "Mon, 30 Jan 2006 17:44:41 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <87bqxtainp.fsf@mid.deneb.enyo.de> <87k6ch8zm5.fsf@mid.deneb.enyo.de> <87vew17jie.fsf@mid.deneb.enyo.de> Message-ID: <87y80wzrhn.fsf@wheatstone.g10code.de> On Mon, 30 Jan 2006 17:44:41 +0100, Florian Weimer said: > After a reboot, there is a lot of disk activity, and according to the > current estimates, this creates a lot of entropy. So it's not a real I have seen reports that this is really predictable and allows for real world attacks. >> It may be wise for systems to save the /dev/random pool on shutdown >> and restore it on startup. > Is this really a good idea? I mean, exposing the pool state like > this? All systems I know are doing just this (e.g. /etc/init.d/urandom). This mitigates the problem described above. Shalom-Salam, Werner From wk at gnupg.org Tue Jan 31 10:23:18 2006 From: wk at gnupg.org (Werner Koch) Date: Tue, 31 Jan 2006 10:23:18 +0100 Subject: [gnutls-dev] Feature request: not really random session keys In-Reply-To: <87bqxt8yxz.fsf@mid.deneb.enyo.de> (Florian Weimer's message of "Mon, 30 Jan 2006 17:26:00 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <87r775odtb.fsf@mid.deneb.enyo.de> <87irsgwptq.fsf@wheatstone.g10code.de> <87irs1c0r0.fsf@mid.deneb.enyo.de> <87wtgh4x01.fsf@wheatstone.g10code.de> <87bqxt8yxz.fsf@mid.deneb.enyo.de> Message-ID: <87u0bkzr7d.fsf@wheatstone.g10code.de> On Mon, 30 Jan 2006 17:26:00 +0100, Florian Weimer said: >> Sure. That was orginally Ted Tso's plan but he could not get a solid >> RNG into the kernel because the kernel hackers required to amke >> /dev/random optional and Ted's plan was to have a solid RNG in the >> kernel as a standard service. > /dev/random is no longer optional, it's needed by the network stack It has never been optional. This was a top priority design criteria and the one-pool-only design is due to this. Thanks to the stubbornness of some Linux hackers. Salam-Shalom, Werner From wk at gnupg.org Tue Jan 31 10:29:12 2006 From: wk at gnupg.org (Werner Koch) Date: Tue, 31 Jan 2006 10:29:12 +0100 Subject: [gnutls-dev] Re: Feature request: not really random session keys In-Reply-To: (Simon Josefsson's message of "Mon, 30 Jan 2006 17:51:01 +0100") References: <87k6cxuaz2.fsf@mid.deneb.enyo.de> <87r775odtb.fsf@mid.deneb.enyo.de> <87irsgwptq.fsf@wheatstone.g10code.de> <87irs1c0r0.fsf@mid.deneb.enyo.de> <87wtgh4x01.fsf@wheatstone.g10code.de> <87bqxt8yxz.fsf@mid.deneb.enyo.de> Message-ID: <87psm8zqxj.fsf@wheatstone.g10code.de> On Mon, 30 Jan 2006 17:51:01 +0100, Simon Josefsson said: > Should we write a simple daemon 'grngd', based on libgcrypt, and start > to use it? That should be simple. It should likely register two I already talked about this. The problem is that we can't be sure that no traces of the random bytes are left in internal kernel buffers. That won't be a problem for me if it is about session keys but for long term keys I'd hesitate to use an IPC mechanism to get the key material to the application. All what is actually needed is to make sure that Libgcrypts saves and restores its own random pool realiable without producing random zero length files (which is easy to fix). And not using GCRY_VERY_STRONG_RANDOM. Shalom-Salam, Werner From wk at gnupg.org Tue Jan 31 10:30:39 2006 From: wk at gnupg.org (Werner Koch) Date: Tue, 31 Jan 2006 10:30:39 +0100 Subject: [OT] Re: [gnutls-dev] ongoing entropy problems In-Reply-To: (Jason Lunz's message of "Mon, 30 Jan 2006 16:58:19 +0000 (UTC)") References: Message-ID: <87lkwwzqv4.fsf@wheatstone.g10code.de> On Mon, 30 Jan 2006 16:58:19 +0000 (UTC), Jason Lunz said: > http://article.gmane.org/gmane.linux.debian.devel.exim4.user/477 Please don't just send a link. Not everyone has online access while processing mails. Thanks, Werner From jas at extundo.com Tue Jan 31 16:27:05 2006 From: jas at extundo.com (Simon Josefsson) Date: Tue, 31 Jan 2006 16:27:05 +0100 Subject: [gnutls-dev] Re: ongoing entropy problems In-Reply-To: (Jason Lunz's message of "Mon, 30 Jan 2006 16:58:19 +0000 (UTC)") References: Message-ID: Jason Lunz writes: > Are the gnutls developers aware of the ongoing entropy-pool-draining > problems with gnutls in exim4? For example: > > http://article.gmane.org/gmane.linux.debian.devel.exim4.user/477 > > Is this a known problem? Can something be done about it? We are working on it, please see the other current threads on the mailing list. I believe we have identified the problem, and proposed a solution, so a exim developer could probably implement it. Any exim developers following this? Thanks, Simon From ametzler at downhill.at.eu.org Tue Jan 31 19:30:29 2006 From: ametzler at downhill.at.eu.org (Andreas Metzler) Date: Tue, 31 Jan 2006 19:30:29 +0100 Subject: [gnutls-dev] Re: ongoing entropy problems In-Reply-To: References: Message-ID: <20060131183029.GA3568@downhill.aus.cc> On 2006-01-31 Simon Josefsson wrote: > Jason Lunz writes: > > Are the gnutls developers aware of the ongoing entropy-pool-draining > > problems with gnutls in exim4? For example: > > http://article.gmane.org/gmane.linux.debian.devel.exim4.user/477 > > Is this a known problem? Can something be done about it? > We are working on it, please see the other current threads on the > mailing list. > I believe we have identified the problem, and proposed a solution, so > a exim developer could probably implement it. Any exim developers > following this? Hello, As far as I gather from reading http://bugs.debian.org/343085 and the thread "Feature request: not really random session keys" on gnutls-dev there are two problems. #1: Online, blocking generation of RSA-params and DH-params in exim. This is already fixed for quite some time in exim (thanks Nikos Mavrogiannopoulos), it switched to using a certtool-compatible fileformat for these parameters and certtool can be used for generating them offline. #2: Florian Weimer wrote in http://bugs.debian.org/343085 | As a side note: With GNU TLS, every _single_ encrypted mail | transmission _totally_ depletes my entropy pool (going from ~3500 to | ~150), but after recompiling Exim4 with OpenSSL, only about 200 bits | (the number is difficult to measure, but it is way less than with GNU | TLS) are used. For bug #2 /dev/urandom is used, therefore there is no blocking in exim, just the fact that anything using /dev/random will block, as there is no entropy left. The issue Jason is refereing to is #2, which is not fixed /yet/ but known. cu andreas -- The 'Galactic Cleaning' policy undertaken by Emperor Zhark is a personal vision of the emperor's, and its inclusion in this work does not constitute tacit approval by the author or the publisher for any such projects, howsoever undertaken. (c) Jasper Ffforde From lunz at falooley.org Tue Jan 31 21:30:44 2006 From: lunz at falooley.org (Jason Lunz) Date: Tue, 31 Jan 2006 20:30:44 +0000 (UTC) Subject: [gnutls-dev] Re: ongoing entropy problems References: <87lkwwzqv4.fsf@wheatstone.g10code.de> Message-ID: wk at gnupg.org said: > Please don't just send a link. Not everyone has online access while > processing mails. Apologies. I see now that this thread: > From: Florian Weimer > Subject: Feature request: not really random session keys > > Okay, the subject line might be a bit misleading. On server machines, > random bits are a very scarce ressource, and you cannot really afford > to throw them a way at a rate of a few kbps. is exactly what I was asking about. For the record, the specific problem I was referring to is this: > From: Sven Hartge svenhartge.de> > Subject: Re: Downgrading or removing TLS due to lack of entropy > > Um 15:43 Uhr am 19.01.06 schrieb Marc Haber: > > > Additionally, the latest exim4 packages (starting with 4.60-3) allow > > optionally build with openssl instead of GnuTLS. If you have the > > possiblity to re-build exim4 locally, this may be an option. I would > > also be interested in learning whether this actually works better > > than GnuTLS. > > I have been hit by the entropy problem as well, but it was really bad, > since only some encrypted mails caused a major DoS on my server, since the > entropie pool was depleted so fast (in fact, just _one_ mail was needed > for the pool to go from 3500 to about 120), the kernel was not able to > refill it fast enough. > > After recompiling exim with OpenSSL, this problem went away. > > So in my opinion, the is definitely something wrong with gnutls as it uses > _way_ to much entropie from the pool as compared to openssl. What exacerbates the situation is that the default MTA for debian stable is exim4, which easily experiences this entropy problem if TLS is enabled. (It isn't, by default). Maybe an example will illustrate just how bad the problem is: I run a small mail server on an old AMD-K6 475MHz system. It runs only ssh, smtp, and imaps, serving a total of TWO (2) email accounts. The smtp server is exim4, using gnutls. The imaps server is dovecot, linked with openssl. That small of a load was enough to cause unusable smtp STARTTLS service as a result of blocked reads on /dev/random. Jason