[Help-gnutls] NIST X.509 self tests

[Simon Josefsson]

I spent today to run the GnuTLS X.509 certificate chain validator on
NIST's self tests, and thought I'd share some of the findings.

First, it should be noted that all of these tests where done using
'certtool --verify-chain' which is not the same verifier that is used
by GnuTLS when you verify server certificates in TLS.  We should
probably merge these verifiers eventually.  I expect that large parts
of the verifiers are similar.

I started with the old tests from
<http://csrc.nist.gov/pki/testing/x509paths_old.html>.  They are
installed in CVS into tests/x509paths.  Running './chain' in that
directory should test all chains.  We do fail some of the self tests,
here are my notes:

Chain 13-14,65: We probably should not fail fatally, although this is
not a real problem.

Chain 15-18: We should succeed, the reason we don't is that we use
memcmp for DN comparisons.

Chain 19: I don't understand why this test should fail?  The chain
seems fine to me.

Chain 28-29: We fail to check keyCertSign (non-)critical key usage in
intermediate certificates.  XXX

Chain 31-32: The CRL is issued by a issuer without CRLSign
(non-)critical keyCertSign.  We don't check the CRL, so this is not a
real problem.

Chain 54-63: We don't check path length constraints properly. XXX

I then started with NIST's current self tests,
<http://csrc.nist.gov/pki/testing/x509paths.html>.  They are installed
in CVS into tests/nist-pkits/.  You can run ./pkits in that directory
to run the simple tests, which just check the parser for all files.

The script "pkits_test" will build NIST's tool to generate HTML for
tests.  It will start a Glade interface, and you should type
e.g. 'foo.html' and then 'Generate tables'.  Since GnuTLS supports
DSA, you should typically click on 'DSA signature verification' too.
The interface invokes the script "gnutls_test_entry" which verifies
certificates chains, after building them using the script
"build-chain".

Since building NIST's tool require some non-standard stuff, I made one
run and stored the output in CVS too.  You can access it from:

http://cvs.gnupg.org/cgi-bin/viewcvs.cgi/*checkout*/gnutls/tests/nist-pkits/gnutls-nist-tests.html?root=GNU+TLS+Library

We again fail some tests, notably are the same as in the NIST's old
test suite, i.e. the keyCertSign and pathLenConstraint related ones.
One new set of failures are due to lack of support for policies.  Some
failures are date-related, and I'm not sure they are important.

I don't have resources to make GnuTLS pass these self tests, so this
is a request for volunteers that want to work on improving the X.509
validator.

If anyone knows of other X.509 self tests, that would be useful.

/Simon