From ametzler at downhill.at.eu.org Sat Mar 8 09:48:18 2008 From: ametzler at downhill.at.eu.org (Andreas Metzler) Date: Sat, 8 Mar 2008 09:48:18 +0100 Subject: Bug#448775: Uses too much entropy (Debian Bug #343085) In-Reply-To: <87fxwe4d0g.fsf@wheatstone.g10code.de> References: <87tzlt6ddp.fsf@wheatstone.g10code.de> <82wsqpg492.fsf@mid.bfk.de> <87fxxdbw59.fsf@mocca.josefsson.org> <87r6gx4sdf.fsf@wheatstone.g10code.de> <87tzltr7za.fsf@mocca.josefsson.org> <87k5mp385s.fsf@wheatstone.g10code.de> <8763y9r35b.fsf@mocca.josefsson.org> <87wsqk4pf6.fsf@wheatstone.g10code.de> <20080130182010.GA5173@downhill.g.la> <87fxwe4d0g.fsf@wheatstone.g10code.de> Message-ID: <20080308084818.GC3091@downhill.g.la> On 2008-01-31 Werner Koch wrote: > On Wed, 30 Jan 2008 19:20, ametzler at downhill.at.eu.org said: >> Any obvious breakage? Exim does not use any threading. I have not >> included an gcry_check_version(NULL) since I thought gcry_control() >> would fail as reliably as gcry_check_version() would, if gcrypt was > Better insert a gcry_check_version because this is the safe way to > initialize gcrypt. The initialization is also done with most other > gcry_control calls but that is a failsafe feature. Explicitly > initialization is better (you don't need to check the return code, just > call it.) Hello, we still seem have not been able to find a really working solution, this patch causes crashes in exim. Quoting Marc Haber in http://bugs.exim.org/show_bug.cgi?id=654 | However, the secondary MX (which delivers some spam to the primary MX) noted | that the primary box had become unreliable in TLS: | 2008-01-30 14:51:21 1JKDKT-0003ME-AG Remote host mailgate.zugschlus.de | [85.214.68.41] closed connection in response to STARTTLS | | When this happened (a couple of times per hour), I didn't get any | atypical log entries on mailgate. | | This was a repeating, but intermittent failure since mailgate | continued to work normally, and STARTTLS was successful most of the | time. [...] | Going back to the same exim sans the random-seed patch, entropy | average went back to 200, but the STARTTLS failures vanished. We have tried to isolate what actually breaks, by only applying parts of the patch (e.g setting up dthe seed file, but not updating it.), but it looks like the mere presence of gcry_control (GCRYCTL_SET_RANDOM_SEED_FILE,filename) causes the crashes. I am really wondering whether using the experimental daemon might be worthwile, there does not seem to be much cost involved, and /dev/(u)random has not got any policy controls either. cu andreas -- `What a good friend you are to him, Dr. Maturin. His other friends are so grateful to you.' `I sew his ears on from time to time, sure' From simon at josefsson.org Sat Mar 8 10:44:13 2008 From: simon at josefsson.org (Simon Josefsson) Date: Sat, 08 Mar 2008 10:44:13 +0100 Subject: Bug#448775: Uses too much entropy (Debian Bug #343085) In-Reply-To: <20080308084818.GC3091@downhill.g.la> (Andreas Metzler's message of "Sat, 8 Mar 2008 09:48:18 +0100") References: <87tzlt6ddp.fsf@wheatstone.g10code.de> <82wsqpg492.fsf@mid.bfk.de> <87fxxdbw59.fsf@mocca.josefsson.org> <87r6gx4sdf.fsf@wheatstone.g10code.de> <87tzltr7za.fsf@mocca.josefsson.org> <87k5mp385s.fsf@wheatstone.g10code.de> <8763y9r35b.fsf@mocca.josefsson.org> <87wsqk4pf6.fsf@wheatstone.g10code.de> <20080130182010.GA5173@downhill.g.la> <87fxwe4d0g.fsf@wheatstone.g10code.de> <20080308084818.GC3091@downhill.g.la> Message-ID: <87k5kdblea.fsf@mocca.josefsson.org> Andreas Metzler writes: > On 2008-01-31 Werner Koch wrote: >> On Wed, 30 Jan 2008 19:20, ametzler at downhill.at.eu.org said: > >>> Any obvious breakage? Exim does not use any threading. I have not >>> included an gcry_check_version(NULL) since I thought gcry_control() >>> would fail as reliably as gcry_check_version() would, if gcrypt was > >> Better insert a gcry_check_version because this is the safe way to >> initialize gcrypt. The initialization is also done with most other >> gcry_control calls but that is a failsafe feature. Explicitly >> initialization is better (you don't need to check the return code, just >> call it.) > > Hello, > > we still seem have not been able to find a really working solution, > this patch > causes crashes in exim. > > Quoting Marc Haber in http://bugs.exim.org/show_bug.cgi?id=654 > | However, the secondary MX (which delivers some spam to the primary MX) noted > | that the primary box had become unreliable in TLS: > | 2008-01-30 14:51:21 1JKDKT-0003ME-AG Remote host mailgate.zugschlus.de > | [85.214.68.41] closed connection in response to STARTTLS > | > | When this happened (a couple of times per hour), I didn't get any > | atypical log entries on mailgate. > | > | This was a repeating, but intermittent failure since mailgate > | continued to work normally, and STARTTLS was successful most of the > | time. > [...] > | Going back to the same exim sans the random-seed patch, entropy > | average went back to 200, but the STARTTLS failures vanished. > > We have tried to isolate what actually breaks, by only applying parts > of the patch (e.g setting up dthe seed file, but not updating it.), > but it looks like the mere presence of > gcry_control (GCRYCTL_SET_RANDOM_SEED_FILE,filename) > causes the crashes. Did you follow Werner's suggestion to the patch? I.e., to call gcry_check_version to initialize libgcrypt properly. > I am really wondering whether using the experimental daemon might be > worthwile, there does not seem to be much cost involved, and > /dev/(u)random has not got any policy controls either. I think doing whatever it takes to make libgcrypt return useful amounts of entropy would be a good thing. Forcing every application to set a seed file is quite costly maintenance and documentation wise, so running one libgcrypt-specific daemon may be simpler. /Simon From ametzler at downhill.at.eu.org Sat Mar 8 11:11:51 2008 From: ametzler at downhill.at.eu.org (Andreas Metzler) Date: Sat, 8 Mar 2008 11:11:51 +0100 Subject: Bug#448775: Uses too much entropy (Debian Bug #343085) In-Reply-To: <87k5kdblea.fsf@mocca.josefsson.org> References: <87fxxdbw59.fsf@mocca.josefsson.org> <87r6gx4sdf.fsf@wheatstone.g10code.de> <87tzltr7za.fsf@mocca.josefsson.org> <87k5mp385s.fsf@wheatstone.g10code.de> <8763y9r35b.fsf@mocca.josefsson.org> <87wsqk4pf6.fsf@wheatstone.g10code.de> <20080130182010.GA5173@downhill.g.la> <87fxwe4d0g.fsf@wheatstone.g10code.de> <20080308084818.GC3091@downhill.g.la> <87k5kdblea.fsf@mocca.josefsson.org> Message-ID: <20080308101151.GD3091@downhill.g.la> On 2008-03-08 Simon Josefsson wrote: > Andreas Metzler writes: [...] > > We have tried to isolate what actually breaks, by only applying parts > > of the patch (e.g setting up dthe seed file, but not updating it.), > > but it looks like the mere presence of > > gcry_control (GCRYCTL_SET_RANDOM_SEED_FILE,filename) > > causes the crashes. > Did you follow Werner's suggestion to the patch? I.e., to call > gcry_check_version to initialize libgcrypt properly. Yes, we did. cu andreas -- `What a good friend you are to him, Dr. Maturin. His other friends are so grateful to you.' `I sew his ears on from time to time, sure' From ametzler at downhill.at.eu.org Mon Mar 10 19:24:52 2008 From: ametzler at downhill.at.eu.org (Andreas Metzler) Date: Mon, 10 Mar 2008 19:24:52 +0100 Subject: How experimental is --enable-random-daemon? Message-ID: Hello, since using a random seed file for exim does not work reliably with the (perhaps broken) patch I am considering the other easy way, using --enable-random-daemon. However I am unsure on whether --enable-random-daemon will eat small children or whether it simply is not built by default but should work alright. Would you outright recommed against using it in production environments? Other stuff I have been wondering about: - A pid file would be nice. - Why isn't the daemon used by default if specified at compile time? Needing to patch every gcrypt using application (or at least libgnutls) to get best benefits seems to be suboptimal. gcry_control (GCRYCTL_USE_RANDOM_DAEMON, 1) thanks, cu andreas -- `What a good friend you are to him, Dr. Maturin. His other friends are so grateful to you.' `I sew his ears on from time to time, sure' From wk at gnupg.org Tue Mar 11 17:01:03 2008 From: wk at gnupg.org (Werner Koch) Date: Tue, 11 Mar 2008 17:01:03 +0100 Subject: How experimental is --enable-random-daemon? In-Reply-To: (Andreas Metzler's message of "Mon, 10 Mar 2008 19:24:52 +0100") References: Message-ID: <87skyxi728.fsf@wheatstone.g10code.de> On Mon, 10 Mar 2008 19:24, ametzler at downhill.at.eu.org said: > alright. Would you outright recommed against using it in production > environments? I wrote it once but did not gave it proper testing. In case it helps with exim it should be better than nothing. > - A pid file would be nice. Noted. > - Why isn't the daemon used by default if specified at compile time? > Needing to patch every gcrypt using application (or at least > libgnutls) to get best benefits seems to be suboptimal. > gcry_control (GCRYCTL_USE_RANDOM_DAEMON, 1) It is not matured enough and it gives a new option to snoop on the random numbers, namely the socket used for the connection. I would not use it for key generation or other critical appications. For Exim use it should be fine and you should enable this only within Exim. However, I would prefer to see why the patch crashes Exim. I have not yet looked at it, though. Salam-Shalom, Werner -- Die Gedanken sind frei. Auschnahme regelt ein Bundeschgesetz. From wk at gnupg.org Tue Mar 11 17:09:19 2008 From: wk at gnupg.org (Werner Koch) Date: Tue, 11 Mar 2008 17:09:19 +0100 Subject: Bug#448775: Uses too much entropy (Debian Bug #343085) In-Reply-To: <20080308084818.GC3091@downhill.g.la> (Andreas Metzler's message of "Sat, 8 Mar 2008 09:48:18 +0100") References: <87tzlt6ddp.fsf@wheatstone.g10code.de> <82wsqpg492.fsf@mid.bfk.de> <87fxxdbw59.fsf@mocca.josefsson.org> <87r6gx4sdf.fsf@wheatstone.g10code.de> <87tzltr7za.fsf@mocca.josefsson.org> <87k5mp385s.fsf@wheatstone.g10code.de> <8763y9r35b.fsf@mocca.josefsson.org> <87wsqk4pf6.fsf@wheatstone.g10code.de> <20080130182010.GA5173@downhill.g.la> <87fxwe4d0g.fsf@wheatstone.g10code.de> <20080308084818.GC3091@downhill.g.la> Message-ID: <87od9li6og.fsf@wheatstone.g10code.de> On Sat, 8 Mar 2008 09:48, ametzler at downhill.at.eu.org said: > but it looks like the mere presence of > gcry_control (GCRYCTL_SET_RANDOM_SEED_FILE,filename) > causes the crashes. That should be easy to debug: void _gcry_set_random_seed_file( const char *name ) { if (seed_file_name) BUG (); seed_file_name = gcry_xstrdup (name); } I guess you are running the init code twice. Isn't there any output to stderr? Shalom-Salam, Werner -- Die Gedanken sind frei. Auschnahme regelt ein Bundeschgesetz. From ametzler at downhill.at.eu.org Wed Mar 12 20:20:44 2008 From: ametzler at downhill.at.eu.org (Andreas Metzler) Date: Wed, 12 Mar 2008 20:20:44 +0100 Subject: Bug#448775: Uses too much entropy (Debian Bug #343085) In-Reply-To: <87od9li6og.fsf@wheatstone.g10code.de> References: <87fxxdbw59.fsf@mocca.josefsson.org> <87r6gx4sdf.fsf@wheatstone.g10code.de> <87tzltr7za.fsf@mocca.josefsson.org> <87k5mp385s.fsf@wheatstone.g10code.de> <8763y9r35b.fsf@mocca.josefsson.org> <87wsqk4pf6.fsf@wheatstone.g10code.de> <20080130182010.GA5173@downhill.g.la> <87fxwe4d0g.fsf@wheatstone.g10code.de> <20080308084818.GC3091@downhill.g.la> <87od9li6og.fsf@wheatstone.g10code.de> Message-ID: <20080312192044.GA3513@downhill.g.la> On 2008-03-11 Werner Koch wrote: > On Sat, 8 Mar 2008 09:48, ametzler at downhill.at.eu.org said: > > but it looks like the mere presence of > > gcry_control (GCRYCTL_SET_RANDOM_SEED_FILE,filename) > > causes the crashes. > That should be easy to debug: > void > _gcry_set_random_seed_file( const char *name ) > { > if (seed_file_name) > BUG (); > seed_file_name = gcry_xstrdup (name); > } > I guess you are running the init code twice. Isn't there any output to > stderr? Hello, The exim daemon forks and closes stderr/out. Marc, can you try running exim in the foreground and check whether gcrypt throws an error (case GCRY_LOG_BUG: fputs("Ohhhh jeeee: ", stderr); break;)? thanks, cu andreas -- `What a good friend you are to him, Dr. Maturin. His other friends are so grateful to you.' `I sew his ears on from time to time, sure' From mh+debian-bugs at zugschlus.de Wed Mar 12 21:16:30 2008 From: mh+debian-bugs at zugschlus.de (Marc Haber) Date: Wed, 12 Mar 2008 21:16:30 +0100 Subject: [Pkg-gnutls-maint] Bug#448775: Uses too much entropy (Debian Bug #343085) In-Reply-To: <20080312192044.GA3513@downhill.g.la> References: <87r6gx4sdf.fsf@wheatstone.g10code.de> <87tzltr7za.fsf@mocca.josefsson.org> <87k5mp385s.fsf@wheatstone.g10code.de> <8763y9r35b.fsf@mocca.josefsson.org> <87wsqk4pf6.fsf@wheatstone.g10code.de> <20080130182010.GA5173@downhill.g.la> <87fxwe4d0g.fsf@wheatstone.g10code.de> <20080308084818.GC3091@downhill.g.la> <87od9li6og.fsf@wheatstone.g10code.de> <20080312192044.GA3513@downhill.g.la> Message-ID: <20080312201630.GE20186@torres.zugschlus.de> On Wed, Mar 12, 2008 at 08:20:44PM +0100, Andreas Metzler wrote: > The exim daemon forks and closes stderr/out. Marc, can you try running > exim in the foreground and check whether gcrypt throws an error > (case GCRY_LOG_BUG: fputs("Ohhhh jeeee: ", stderr); break;)? Yes, it does. $ sudo exim -bdf -q30m -oX 587:465:25 -oP /var/run/exim4/exim.pid Ohhhh jeeee: ... this is a bug (random.c:528:_gcry_set_random_seed_file) Ohhhh jeeee: ... this is a bug (random.c:528:_gcry_set_random_seed_file) Ohhhh jeeee: ... this is a bug (random.c:528:_gcry_set_random_seed_file) Ohhhh jeeee: ... this is a bug (random.c:528:_gcry_set_random_seed_file) Ohhhh jeeee: ... this is a bug (random.c:528:_gcry_set_random_seed_file) Ohhhh jeeee: ... this is a bug (random.c:528:_gcry_set_random_seed_file) Ohhhh jeeee: ... this is a bug (random.c:528:_gcry_set_random_seed_file) Ohhhh jeeee: ... this is a bug (random.c:528:_gcry_set_random_seed_file) Ohhhh jeeee: ... this is a bug (random.c:528:_gcry_set_random_seed_file) Ohhhh jeeee: ... this is a bug (random.c:528:_gcry_set_random_seed_file) Ohhhh jeeee: ... this is a bug (random.c:528:_gcry_set_random_seed_file) Ohhhh jeeee: ... this is a bug (random.c:528:_gcry_set_random_seed_file) Greetings Marc -- ----------------------------------------------------------------------------- Marc Haber | "I don't trust Computers. They | Mailadresse im Header Mannheim, Germany | lose things." Winona Ryder | Fon: *49 621 72739834 Nordisch by Nature | How to make an American Quilt | Fax: *49 3221 2323190 From fweimer at bfk.de Thu Mar 13 09:32:17 2008 From: fweimer at bfk.de (Florian Weimer) Date: Thu, 13 Mar 2008 09:32:17 +0100 Subject: Bug#448775: Uses too much entropy (Debian Bug #343085) In-Reply-To: <20080308084818.GC3091@downhill.g.la> (Andreas Metzler's message of "Sat, 8 Mar 2008 09:48:18 +0100") References: <87tzlt6ddp.fsf@wheatstone.g10code.de> <82wsqpg492.fsf@mid.bfk.de> <87fxxdbw59.fsf@mocca.josefsson.org> <87r6gx4sdf.fsf@wheatstone.g10code.de> <87tzltr7za.fsf@mocca.josefsson.org> <87k5mp385s.fsf@wheatstone.g10code.de> <8763y9r35b.fsf@mocca.josefsson.org> <87wsqk4pf6.fsf@wheatstone.g10code.de> <20080130182010.GA5173@downhill.g.la> <87fxwe4d0g.fsf@wheatstone.g10code.de> <20080308084818.GC3091@downhill.g.la> Message-ID: <82fxuvyqge.fsf@mid.bfk.de> * Andreas Metzler: > we still seem have not been able to find a really working solution, > this patch > causes crashes in exim. IIRC, I have already posted this, but perhaps my wording was a bit unclear. I don't think the seed file approach works for a forking daemon like Exim because you cannot guaranteed an undisturbed read/modify/write cycle on the seed file. Locking is out of the question, too, because it would bring the mail system to a standstill. And it's arguably not a good idea to reuse the same seed file in different forked children. You need a separate daemon, or trust the kernel and read fewer bytes from /dev/urandom. -- Florian Weimer BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstra?e 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99 From wk at gnupg.org Thu Mar 13 13:55:47 2008 From: wk at gnupg.org (Werner Koch) Date: Thu, 13 Mar 2008 13:55:47 +0100 Subject: Bug#448775: Uses too much entropy (Debian Bug #343085) In-Reply-To: <82fxuvyqge.fsf@mid.bfk.de> (Florian Weimer's message of "Thu, 13 Mar 2008 09:32:17 +0100") References: <87tzlt6ddp.fsf@wheatstone.g10code.de> <82wsqpg492.fsf@mid.bfk.de> <87fxxdbw59.fsf@mocca.josefsson.org> <87r6gx4sdf.fsf@wheatstone.g10code.de> <87tzltr7za.fsf@mocca.josefsson.org> <87k5mp385s.fsf@wheatstone.g10code.de> <8763y9r35b.fsf@mocca.josefsson.org> <87wsqk4pf6.fsf@wheatstone.g10code.de> <20080130182010.GA5173@downhill.g.la> <87fxwe4d0g.fsf@wheatstone.g10code.de> <20080308084818.GC3091@downhill.g.la> <82fxuvyqge.fsf@mid.bfk.de> Message-ID: <871w6eg4vg.fsf@wheatstone.g10code.de> On Thu, 13 Mar 2008 09:32, fweimer at bfk.de said: > I don't think the seed file approach works for a forking daemon like > Exim because you cannot guaranteed an undisturbed read/modify/write > cycle on the seed file. Locking is out of the question, too, because It depends on how much entropy you want to have in the pool. Even with all processes using the same random seed, libgcrypt will add extra random to each process (e.g. 16 bytes from /dev/urandom). This turns it into a PRNG which is not worse than using /dev/urandom directly. The advantage reading a the seed file is that libgcrypt does not need to initialize its own random pool. You may view this initialization as assuring that the random pool is not plain zeroed out. If you don't like that approach you cat limit the reuse of the seed file by using a seed file name depending on the pid, like asprintf ("/foo/bar/andom_seed.%d", ((int)getpid() % 5)) Salam-Shalom, Werner -- Die Gedanken sind frei. Auschnahme regelt ein Bundeschgesetz. From wk at gnupg.org Mon Mar 17 19:23:28 2008 From: wk at gnupg.org (Werner Koch) Date: Mon, 17 Mar 2008 19:23:28 +0100 Subject: Improved AES performance Message-ID: <878x0h5hwf.fsf@wheatstone.g10code.de> Hi! I hacked libgcrypt a bit to see how to increase the symmetrical cipher performance. This should benefit all architecture except for Padlock, which I have not yet changed. The current SVN has the changes for AES in CFB and CBC mode. On my box it gives about 25% speedup with the benchmark tool. That tool has two new options: --large-buffers passes larger blocks of data to the encryption function and reduces the loop count instead. --cipher-repetitions N may be used to increase the loop count by a factor N. This is gives more accurate timings on fas machines. Example usage: tests/benchmark --cipher-repetition 10 cipher aes aes192 aes256 I did a real work test with gpg2 on a 2.9GB MPEG file. First a plain store operation to see the I/O overhead. $ time gpg2 --store -z0 -v --batch --passphrase 'test'\