[Sks-devel] pool.sks-keyservers.net issues

[Phil Pennock]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

Short version: bad interaction of GnuPG, cURL and Apache.  Can probably
be worked around in Apache config, can definitely be worked around in
GnuPG code, should aim to get both done.

On 2013-02-28 at 10:01 -0800, Doug Barton wrote:
> 2001:470:1f09:5e7::2	worked
> 2001:470:1f0a:5d7::2	failed
> 
> In any case thanks for clarifying the cause of the error ... it's just
> hard to debug without putting those options in first obviously, and
> with those options the output is very noisy.

417 is not load-related, Kristian's goofed.

417 _only_ happens when the client sends "Expect: 100-continue", in an
HTTP/1.1 request, and a reverse proxy (or forward-proxy) knows that the
backend is HTTP/1.0 only.

GnuPG will use such a header with a _POST_ request, ie when _sending_
keys; that expectation is a POST optimisation, and it's coming from
cURL.

In this case, the failing server is keys.wuschelpuschel.org, which is
running GnuKS behind Apache.  Since SKS/GnuKS only support HTTP/1.0,
Apache has been "correctly" configured, per standards.  nginx doesn't
support returning 417, so this is only seen with Apache-fronted
keyservers.

I see nothing in RFC2616 which explains how 1.1 clients speaking to 1.1
proxies with 1.0 origin servers should handle this.  Presumably, they
should retry.

Can someone running SKS/GnuKS behind Apache please use:
  gpg -v --keyserver-options verbose,debug --keyserver THEIRS --send-key WHATEVER
and confirm that they see this (as I do, reliably and reproducibly) and
then try:

  <IfModule mod_headers.c>
  RequestHeader unset Expect early
  </IfModule>

I suspect that just ripping the header out of the request and pretending
it's not there will let things "work".


GnuPG developers: with GnuPG 2.0.19, in keyserver/gpgkeys_hkp.c main(),
around line 783, I added an empty "Expect:" header to force cURL to not
send that header, and it worked.  I could then send a key to
keys.wuschelpuschel.org.

- ----------------------------8< cut here >8------------------------------
  headers=curl_slist_append(headers,"Pragma: no-cache");
  if(headers)
    headers=curl_slist_append(headers,"Cache-Control: no-cache");
/* Added this next bit: */
  if(headers)
    headers=curl_slist_append(headers,"Expect:");
- ----------------------------8< cut here >8------------------------------


So: I know GnuPG can work around it, that will take a while to work
around.  If we can get someone using Apache to confirm that the above
fix works for them, we can push people to include it.

Kristian: I suggest that a pool check might try an HTTP/1.1 POST with an
Expect: header and exclude from the main pools any server where this
fails.

(Similarly, need to figure out what to do about nginx/FreeBSD/KQUEUE for
 GnuPG curl-shim ... harder to deterministically detect.  For myself, I
 might suck it up and rebuild without KQUEUE support.)

Regards,
- -Phil
-----BEGIN PGP SIGNATURE-----

iEYEAREDAAYFAlEwYPQACgkQQDBDFTkDY38a2gCfUmlbJqHy+CcNNRtyWRicvhoJ
cu8AnAgfFYYnjP5I0yi7jdBiSI14Q6Hn
=o4bt
-----END PGP SIGNATURE-----