RSA sign/verify and hash generation functions

[Alessandro Vesely]

On 08/Dec/10 23:56, Murray S. Kucherawy wrote:
>> -----Original Message-----
>> From: Nikos Mavrogiannopoulos [mailto:n.mavrogiannopoulos at gmail.com] On Behalf Of Nikos Mavrogiannopoulos
>> Sent: Wednesday, December 08, 2010 2:25 PM
>> To: Murray S. Kucherawy
>> Cc: help-gnutls at gnu.org
>> Subject: Re: RSA sign/verify and hash generation functions
>> 
>> Which signing method do you use with openssl? In gnutls we support only
>> PKCS #1 1.5 signatures (that one required by TLS).
> 
> Ah, maybe that's the problem.  The RSA_sign() man page from OpenSSL says:
> 
>        RSA_sign() signs the message digest m of size m_len using the private
>        key rsa as specified in PKCS #1 v2.0.

I'd be surprised if PKCS#1 v2.0 introduced incompatibilities with the
previous version.  At any rate, RFC 4871 says: "

3.3.1. The rsa-sha1 Signing Algorithm


   The rsa-sha1 Signing Algorithm computes a message hash as described
   in Section 3.7 below using SHA-1 [FIPS.180-2.2002] as the hash-alg.
   That hash is then signed by the signer using the RSA algorithm
   (defined in PKCS#1 version 1.5 [RFC3447]) as the crypt-alg and the
   signer's private key. [...]"