Access violation using gpgme

[Daniel Lo Nigro]

I ended up working this out.

.NET Core is cross-platform and runs on Linux too, so I thought I'd test it
on Linux too. One thing I ended up noticing was that the same code worked
fine on 64-bit Linux, but failed on 32-bit Windows. I can't find a 64-bit
build of Gpg4Win so I was unable to test 64-bit Windows, but I think it
would have worked too. This led me to look at the potential differences
between 32-bit and 64-bit.

I found that all the calls from C# into GPGME were correctly marked as
using the "cdecl" calling convention, however the callback was not,
Microsoft compilers default to using "stdcall" calling convention. The main
difference between them is that with cdecl the caller is responsible for
cleaning up the stack, whereas with stdcall the callee is responsible for
cleaning up the stack. Using the wrong one corrupts your stack.  The
callback actually seemed to execute fine, but the function that called the
callback ended up crashing when it tried to return. 64-bit systems only use
one calling convention (Windows differs from "everything else", but it's
consistent within the same platform), which is why it worked fine on 64-bit.

So in the end, I spent around a week debugging it, but it ended up just
being a single-line change to fix it:
https://github.com/Daniel15/gpgme-sharp/commit/108b6af6abaadd52e31798e0aea930fc18a14498
:D

Thanks,
--
Regards,
Daniel Lo Nigro
https://d.sb/ | Twitter <http://twitter.com/Daniel15> | Facebook
<http://www.facebook.com/daaniel>


On Thu, Feb 7, 2019 at 10:53 PM Daniel Lo Nigro <d at d.sb> wrote:

> Hey Andre, I was just wondering if you had any other ideas about this? I
> tried to use WinDbg to debug it, which showed this error message:
>
> > Attempt to execute non-executable address 004fef7c
>
> Full output:
> https://gist.github.com/Daniel15/c8c31b9e46d8c2eea6385d7dd1ba6c40
>
> I've been stepping through the code and haven't been able to work out why
> this happens, particularly since it happens after the passphrase callback
> has returned (so gpgme is calling the callback fine). However, I've never
> written any C code, so I'm not familiar with debugging it. The thing I'm
> not sure how to determine is where that pointer 0x004fef7c is coming from,
> and whether it's inside the gpgme library or in my app.
>
> Thanks!
>
> --
> Regards,
> Daniel Lo Nigro
> https://d.sb/ | Twitter <http://twitter.com/Daniel15> | Facebook
> <http://www.facebook.com/daaniel>
>
>
> On Tue, Feb 5, 2019 at 10:59 AM Daniel Lo Nigro <d at d.sb> wrote:
>
>> The code was using gpgme_io_write to write the passphrase followed by a
>> null byte:
>> https://github.com/wget/gpgme-sharp/blob/master/gpgme-sharp/Context.cs#L353-L354.
>> I changed it to gpgme_io_writen and newline. That code was written by
>> someone else way back in 2013 so I wonder if it's been broken this entire
>> time. Here's the changes I've made so far:
>> https://github.com/wget/gpgme-sharp/compare/master...Daniel15:pinentry
>>
>> Now I'm getting closer, but there's still some odd behaviour. The first
>> time I run my test app, I get an access violation. However, it does spawn
>> gpg-agent correctly, and the second time I run it, it works fine! I guess
>> gpg-agent is properly receiving/caching the passphrase.
>>
>> --
>> Regards,
>> Daniel Lo Nigro
>> https://d.sb/ | Twitter <http://twitter.com/Daniel15> | Facebook
>> <http://www.facebook.com/daaniel>
>>
>>
>> On Tue, Feb 5, 2019 at 10:32 AM Andre Heinecke <aheinecke at gnupg.org>
>> wrote:
>>
>>> Hi,
>>>
>>> On Tuesday, February 5, 2019 6:21:45 PM GMT Daniel Lo Nigro wrote:
>>> > I'm using gpgme_set_pinentry_mode to set the pinentry mode to
>>> "loopback",
>>> > and the access violation is being encountered after my pinentry
>>> callback
>>> > returns. It works fine if I don't use loopback, and instead use the
>>> > system's pinentry mechanism.
>>>
>>> This sounds like your passphrase callback returns invalid data. I would
>>> have
>>> to look how the PassphraseResult is mapped in the C# bindings but from
>>> the C
>>> API:
>>>
>>> The user must write the passphrase, followed by a newline character,
>>> to the file descriptor @var{fd}.  The function @code{gpgme_io_writen}
>>> should be used for the write operation.  Note that if the user returns
>>> 0 to indicate success, the user must at least write a newline
>>> character before returning from the callback.
>>>
>>>
>>> I do not think that you need to post your full code, but I would be
>>> interested
>>> in seeing your Passphrase callback.
>>> (Sorry If I'm  the only one answering here but in the GnuPG Team I'm the
>>> Guy
>>> for "all things windows" ;-) )
>>>
>>>
>>> Regards,
>>> Andre
>>>
>>> --
>>> GnuPG e.V., Rochusstr. 44, D-40479 Düsseldorf.  VR 11482 Düsseldorf
>>> Vorstand: W.Koch, M.Gollowitzer, A.Heinecke.    Mail: board at gnupg.org
>>> Finanzamt D-Altstadt, St-Nr: 103/5923/1779.   Tel: +49-2104-4938799
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.gnupg.org/pipermail/gnupg-devel/attachments/20190210/5d45313f/attachment.html>