High performance GnuPG - Assuan interface to g10?

[Robin H. Johnson]

Hi

I'm trying to get a high performance GnuPG going - using pgme just isn't
cutting it - the repeated execv overhead is killing me.

First, an explanation of my problem/objectives - somebody might be able
to offer an alternative to my plan.

I have a dataset, consisting of N chunks of data, of which some are GPG
signed (not all of them, although we are encouraging everybody to move
towards signing). 

The existing format of the data is at least one chunk of data per file,
although in some cases a file has multiple chunks, with no separators -
if two adjacent chunks are unsigned, they are just considered as one
chunk.

The present value of N is 11k, but it will shortly increase to at
least 4x.

For each chunk, I want to be able to verify it using an existing keyring
(gpg --verify), and be told "Good Signature"/"Bad Signature"/"No valid
OpenPGP data found".

Using gpgme leads to N execv's of gpg, which hurts badly in performance.
I clock it at ~120 seconds for the present 11k items.

Using gpg --multifile, I get it down to 40 seconds for 11k items, but I
have to first put each chunk into a unique file, and then pass that long
commandline into GnuPG. But file writing is a not acceptable for the
final application.

I don't mind separating the chunks myself - I need to do that for
another part of the application anyway, but I'm still stuck with the
problem of verifying lots of them efficiently.

What I'd like considered (I may implement the majority of it myself if
the idea is acceptable) is the following:
- An Assuan interface to the g10 code, much like the Assuan interface to
  GPGSM, so a user could call 'gpg --server', and then use that
  interface as needed - feeding off the status output for verification
  results.

-- 
Robin Hugh Johnson
E-Mail     : robbat2 at gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 241 bytes
Desc: not available
Url : /pipermail/attachments/20060519/49d6070b/attachment.pgp