GNUPG and Google Cloud

Wed Aug 17 00:22:11 CEST 2022

C.J.,

Pub/Sub & Data Flow would require us to have a larger "footprint" on-prem that we wanted. We want to keep our on-prem environment as small as possible. Additionally, most of our data will be transferred using flat files that do not lend themselves to being sent via pub-sub.

For the on-prem encrypting, could I have the GKE container(s), shell out to GnuPG to do the encrypting and then "send" the encrypted file to Cloud Storage? Would I then use Secret Manager or Cloud KMS to store my keys or would there be a way to use Kleopatra?

We need to encrypt the data all stages because it contains PII information and we don't want it un-encrypted at any stage especially when it is being sent to GCP.

Thanks,
David

From: C.J. Collier <cjac at colliertech.org>
Sent: Tuesday, August 16, 2022 4:29 PM
To: David Gordon <DavidWGordon1011 at outlook.com>
Cc: gnupg-users at gnupg.org
Subject: Re: GNUPG and Google Cloud

Hi David,

I would take a look at Secret Manager[1] as a way to store your private key material confidentially.  Perhaps consider Cloud Run[2] as a mechanism for execution of arbitrary code, in this case for instance with the encryption/decryption pipeline using the python runtime and python-gnupg[3] library.

You might instead find Cloud Pub/Sub[4] and Dataflow[5] to be useful for streaming the data from your on-prem environment to GCS, and from GCS to BigQuery.

In short, yes, there are a variety of ways to perform the steps that you're talking about on GCP.  You should be able to develop a proof of concept on a small scale while staying within the limits of the free tier[6].  I'm not quite clear on why you would want to encrypt the data when you will eventually decrypt it for storage into BigQuery, but yes, it is feasible.

C.J.

[1] https://cloud.google.com/secret-manager<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcloud.google.com%2Fsecret-manager&data=05%7C01%7C%7C5c9de3dd8dd740ffbe1408da7fc6064a%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637962785758109746%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qlEFZ5Eo49fcqhJnDeZN90nPWiyEeX6RMr7Ia7Cngtc%3D&reserved=0>
[2] https://cloud.google.com/run<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcloud.google.com%2Frun&data=05%7C01%7C%7C5c9de3dd8dd740ffbe1408da7fc6064a%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637962785758265995%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=TTIQBdbbyrjjALfzBxARfFBLA1XoKXzd85uFhz%2FzQvg%3D&reserved=0>
[3] https://pypi.org/project/python-gnupg/<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpypi.org%2Fproject%2Fpython-gnupg%2F&data=05%7C01%7C%7C5c9de3dd8dd740ffbe1408da7fc6064a%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637962785758265995%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=7KAaWdaGK8x2RC9BTfdNoTbmfVcJbWFhQpM7PYxaKgs%3D&reserved=0>
[4] https://cloud.google.com/pubsub<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcloud.google.com%2Fpubsub&data=05%7C01%7C%7C5c9de3dd8dd740ffbe1408da7fc6064a%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637962785758265995%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=TzOZ%2B7P46bfYd12qK4%2Fs5r7ZJiZqUWkK4UJii9nFBss%3D&reserved=0>
[5] https://cloud.google.com/dataflow<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcloud.google.com%2Fdataflow&data=05%7C01%7C%7C5c9de3dd8dd740ffbe1408da7fc6064a%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637962785758265995%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=tUjsrNg%2FN2CKLA6%2BGy45I02Hx2kfeRVF27zIwPi0NWE%3D&reserved=0>
[6] https://cloud.google.com/free<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcloud.google.com%2Ffree&data=05%7C01%7C%7C5c9de3dd8dd740ffbe1408da7fc6064a%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637962785758265995%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=HlLFA9mwijkPHNbJf38gG3e1lI3YtCP9skCfnWxxrYw%3D&reserved=0>

On Tue, Aug 16, 2022 at 11:33 AM David Gordon <DavidWGordon1011 at outlook.com<mailto:DavidWGordon1011 at outlook.com>> wrote:
CJ,

We were looking for a server-less solution. What we want to do is take data from a legacy mainframe system, encrypt it via PGP, and then via GKE transfer it to Cloud Storage. From there we want to decrypt it via GnuPG, save it in Cloud Storage and then load it into Big Query.

Thanks,
David

From: C.J. Collier <cjac at colliertech.org<mailto:cjac at colliertech.org>>
Sent: Tuesday, August 16, 2022 10:23 AM
To: David Gordon <DavidWGordon1011 at outlook.com<mailto:DavidWGordon1011 at outlook.com>>
Cc: gnupg-users at gnupg.org<mailto:gnupg-users at gnupg.org>
Subject: Re: GNUPG and Google Cloud

Hi there!

Are you looking for a server-less solution or will a Debian instance on GCE or GKE suffice?

You can "deploy" GNUPG with apt-get.  Decrypting content would require getting a private key or an agent onto the system.

Can you give more details about what you're looking for?

C.J. in Cloud Support, Seattle
GCP Technical Solutions Engineer

On Tue, Aug 16, 2022, 05:49 David Gordon via Gnupg-users <gnupg-users at gnupg.org<mailto:gnupg-users at gnupg.org>> wrote:
Can GnuPG be deployed to GCP to decrypt files? If so, is there a recommended approach?

Thanks,
David

Sent from Mail<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgo.microsoft.com%2Ffwlink%2F%3FLinkId%3D550986&data=05%7C01%7C%7C5c9de3dd8dd740ffbe1408da7fc6064a%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637962785758265995%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=WeUmiEJmJohnRrW%2FJ6hhmwYVpM%2FrBGc0ZT94bA2iwOI%3D&reserved=0> for Windows

_______________________________________________
Gnupg-users mailing list
Gnupg-users at gnupg.org<mailto:Gnupg-users at gnupg.org>
https://lists.gnupg.org/mailman/listinfo/gnupg-users<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.gnupg.org%2Fmailman%2Flistinfo%2Fgnupg-users&data=05%7C01%7C%7C5c9de3dd8dd740ffbe1408da7fc6064a%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637962785758265995%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ZjDyXV5qlDMlZEMT9Z9XR6C%2Br17h3JbWjU5RmY6grJs%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.gnupg.org/pipermail/gnupg-users/attachments/20220816/00764790/attachment-0001.html>