[PATCH] Fix encryption/decryption return type for GOST28147

[Dmitry Eremin-Solenikov]

On Wed, Sep 18, 2013 at 6:26 PM, Jussi Kivilinna <jussi.kivilinna at iki.fi> wrote:
> * cipher/gost.h (_gcry_gost_enc_one): Change return type to
> 'unsigned int'.
> * cipher/gost28147.c (max): New macro.
> (gost_encrypt_block, gost_decrypt_block): Return burn stack depth.
> (_gcry_gost_enc_one): Return burn stack depth from gost_encrypt_block.
> --
>
> Return type for block cipher functions was lately changed from 'void' to
> 'unsigned int' to pass burn stack depth to cipher mode code. Patch fixes
> gost28147 to return stack burn value.
>
> Signed-off-by: Jussi Kivilinna <jussi.kivilinna at iki.fi>
> ---
>  cipher/gost.h      |    2 +-
>  cipher/gost28147.c |   23 +++++++++++++++++++----
>  2 files changed, 20 insertions(+), 5 deletions(-)
>
> diff --git a/cipher/gost.h b/cipher/gost.h
> index e1cf033..42021aa 100644
> --- a/cipher/gost.h
> +++ b/cipher/gost.h
> @@ -27,7 +27,7 @@ typedef struct {
>  } GOST28147_context;
>
>  /* This is a simple interface that will be used by GOST R 34.11-94 */
> -extern void _gcry_gost_enc_one (GOST28147_context *c, const byte *key,
> +extern unsigned int _gcry_gost_enc_one (GOST28147_context *c, const byte *key,
>      byte *out, byte *in);
>
>  #endif
> diff --git a/cipher/gost28147.c b/cipher/gost28147.c
> index 5d6d1e7..862e7d6 100644
> --- a/cipher/gost28147.c
> +++ b/cipher/gost28147.c
> @@ -35,6 +35,9 @@
>  #include "cipher.h"
>
>
> +#define max(a, b) (((a) > (b)) ? (a) : (b))
> +
> +
>  /* This is an s-box from RFC4357, named GostR3411-94-TestParamSet
>   * For now it is the only s-box supported, as libgcrypt lacks mechanism
>   * for passing parameters to cipher in a usefull way. */
> @@ -107,7 +110,7 @@ gost_val (GOST28147_context *ctx, u32 cm1, int subkey)
>    return (cm1 << 11) | (cm1 >> 21);
>  }
>
> -static void
> +static unsigned int
>  gost_encrypt_block (void *c, byte *outbuf, const byte *inbuf)
>  {
>    GOST28147_context *ctx = c;
> @@ -153,16 +156,22 @@ gost_encrypt_block (void *c, byte *outbuf, const byte *inbuf)
>    outbuf[1 + 4] = (n1 >> (1 * 8)) & 0xff;
>    outbuf[2 + 4] = (n1 >> (2 * 8)) & 0xff;
>    outbuf[3 + 4] = (n1 >> (3 * 8)) & 0xff;
> +
> +  return /* burn_stack */ 4*sizeof(void*) /* func call */ +
> +                          3*sizeof(void*) /* stack */ +
> +                          max( 4*sizeof(void*) /* gost_val call */,
> +                               3*sizeof(void*) /* gost_set_subst call */ +
> +                               2*sizeof(void*) /* gost_set subst stack*/ );

I think you can be pretty much sure here that 4 is less than 3 + 2, can't you?
And also it looks like you have forgot those n1 and n2 variables...
I would suggest to just select 384 or 512 and don't get into details of stack.
What do you think?


-- 
With best wishes
Dmitry