Python bindings HOWTO proof reader request

[Ben McGinnes]

On Fri, Mar 23, 2018 at 11:44:53PM -0400, Daniel Kahn Gillmor wrote:
> On Sat 2018-03-24 09:01:51 +1100, Ben McGinnes wrote:
>> Also, I do view this type of binary release as essentially an
>> additional amount of work which is only made necessary by the
>> deliberate actions of third parties to break functionality.  That is,
>> it is only necessary because distribution managers have manually
>> removed the bindings from the library in the first place or simply not
>> updated the versions of the libraries they bundle with their
>> distribution in the first place.  In some cases both and in some cases
>> the latter is caused by the lack of resources at their end to conduct
>> the same manual intervention of the former on the more recent
>> releases.
> 
> speaking as a "distribution manager" (if i understand your terms
> correctly), i am not "manually removing the bindings from the
> library".
> 
> debian administrators want modular systems, and some people run
> systems with some dependencies installed; others do not.  Not
> everyone who wants libgpgme11 wants the python bindings.

Which I have no problem with, but if they were to subsequently change
their minds, surely the solution would be to reinstall the relevant
package(s) with the bits left out?

Not return to the upstream project and request a custom binary package
specifically for the their system architecture and OS and/or
distribution (and/or package management format) to be distributed via
a third party (regardless of whether or not that third party is
related to one or more of the languages the project is implemented
in).

> Furthermore, some gpgme python bindings are incompatible with some
> versions of python, but this is not "sabotage" on the part of the
> distribution manager"

I also said, "for lack of a better term" for a reason.  I don't think
it's actually sabotage, but I do see deliberate actions, often made
for fairly logical reasons, which remove a particular function.  Now
that removal is resulting in requests for other means of circumventing
those decisions to restore those functions via a means which adds work
unnecessarily.

> -- python3-gpg doesn't work with python3.7 in debian because i
> *haven't* deviated from upstream practice, so the package was built
> against python3.6 only.

Python 3.7 does not yet have a stable production release and, IIRC,
has only just recently entered beta (assuming the release timetable I
glanced at recently is on track).  That said, it is scheduled for a
stable release in the middle of the year and if there are build
failures then that should be raised as a bug.

> distribution managers are *trying* to do the work needed to make the
> GnuPG project easier to adopt on their distro.

Which we greatly appreciate.  Particularly your work and the team
involved in the MacPorts project in particular.  I expect to have
greater discussions with FreeBSD people in the not too distant future
as well (stemming from having a friend in FreeBSD Core).

> As you've noticed from thinking about the python bindings, it's
> quite difficult to do, because:
> 
>  a) merely shipping gpgme binary shared objects on a specific platform
>     is a non-trivial task.

Yes and though with some platforms, like Windows, it might be
necessary; it's not something I'm eagerly rushing in to doing unless
there really is no other means of getting the thing to work.

With Windows it's quite likely that I may need to do something like
utilise the CFFI ABI calling methods to get it to play nicely.  If
that happens then that might turn out to be the means by which Tobias'
request might be realised rather than building a Python binary from
the SWIG bindings.  That sort of thing would still be best served by a
platform specific compilation of GPGME, but the ABI calls through CFFI
would then reference version specific dynamic libraries in order to
Just Work™.

>  b) getting the python bindings built against the current version of
>     python is challenging, particularly if the user upgrades their
>     own python installation.

Which I do all the time, though not originally just to test this.

>  c) even if both of the above are sorted out, the GnuPG binary
>     software itself (and all associated daemons) needs to be
>     correctly installed for gpgme to work.

Yes, but even the other common GPG wrappers still need most of the
same things available.  Both python-gnupg (Vinay Sajip's project) and
the other one with the conflicting module name and annoying numbering
(deliberately inflated to a higher version number in order to override
python-gnupg installations) require everything short of GPGME
installed to just work.

The other major Python cryptography packages are cryptography.py
(which depends on recent versions of OpenSSL), PyNaCl (which depends
on libsodium and only implements DJB's curves) and PyCryptodome (which
implements cryptographic primitives directly in Python instead).  None
of them actually do exactly what the GnuPG Project does and so there's
definitely still value for us continuing to pursue this.

> distro managers are just about the only people capable of sorting
> out these complex dependency trees (binaries, shared-objects, python
> bindings) and wrapping it all up into code that Just Works™.

It's not an easy job to package an entire operating system, no matter
which approach is taken and that's in both the F[L]OSS and commercial
worlds.  Having seen the inside of doing the same for commercial UNIX
systems, specifically Solaris on all architectures Sun ever sold
(including the re-badged Cray, aka the Sun Fire E10K, and the rest of
the big iron).

> If we're failing, it's not due to some sort of sabotage, it's due to
> the complexity of the GnuPG architecture.

Yes, which is why I preceded that comment with "for lack of a better
term" and "sabotage" isn't really very accurate.  Unfortunately I
can't think of another word to indicate a deliberate action which
produces and unwanted (from one point of view) effect in English (and
not for lack of vocabulary ... I know mine's fairly large).

> Not many projects require both shared objects and system-level
> binaries in order for the python bindings to work :/

I can't actually think of any other projects quite like this.  I can
think of plenty of other very complex ones, of course, but they're all
significantly larger and often either complete operating systems or
environments, but there's very few projects which require this level
of complexity in order to do one basic thing.  In this case I'm
referring to all these cryptographic operations as "one basic thing"
(so more like one basic type of thing).

> If you see a way to simplify this further, i'd be happy to hear
> about it.

If I think of something, this list will be the first to know (probably
shortly after double-checking with a few people off-list that whatever
it is isn't me being stupid).  ;)


Regards,
Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 228 bytes
Desc: not available
URL: <https://lists.gnupg.org/pipermail/gnupg-devel/attachments/20180325/4f247cd8/attachment.sig>