[Haskell-cafe] Encrypting streamed data

[Ivan Lazar Miljenovic]

On 6 July 2017 at 17:23, David Turner <dct25-561bs at mythic-beasts.com> wrote:
> Apologies, just seen the bit about wanting to pass the key in directly
> rather than using the GPG keyring because there are no email addresses
> attached to your various keys.
>
> Maybe a silly question, but can you give them email addresses to identify
> them in a GPG-compatible manner? They don't have to have associated
> mailboxes so the addresses can be totally made-up.

Yes, that's my fallback, since it's possible to tell gpg to use a
different directory so I can provide a key externally with my
transmission request, load it into a temporary store, grab the
identity out and use that.  It would just be more convenient to have a
"here's a ByteString with the public key" option (which I can always
implement as a wrapper function).

>
> If not, I'd probably start to look to something like openssl to do the
> symmetric encryption and manage the keys for that separately. It looks
> possible to build a streaming AES implementation using the nonstreaming
> functions in `cryptonite`, but the usual recommendation is that its far too
> risky to do any low-level crypto yourself so this seems like a bad idea.
>
>
> On 6 Jul 2017 07:40, "David Turner" <dct25-561bs at mythic-beasts.com> wrote:
>>
>> Hi,
>>
>> I do not know of a library to do this, sorry. Note that the way public-key
>> crypto works in a streaming fashion is typically to use the public-key bit
>> only to encrypt a key for a symmetric cipher and then use the (much-faster)
>> symmetric encryption for the actual data. The symmetric bit could well be
>> something like AES256-CBC or AES256-CTR.
>>
>> This means that the format of the resulting data is a bit
>> implementation-defined as it has to include the asymetrically-encrypted data
>> first, followed by the stream of symmetrically-encrypted data. GnuPG
>> includes quite a bit of metadata in its files that describes the algorithms
>> used and delimits the pieces, so if you want the resulting files to be
>> GnuPG-compatible you'll need to take this into account.
>>
>> If it were me, I'd probably just shell out to `gpg`. It's fast and
>> low-risk.
>>
>> Hope that helps,
>>
>> David
>>
>>
>> On 6 Jul 2017 05:59, "Ivan Lazar Miljenovic" <ivan.miljenovic at gmail.com>
>> wrote:
>>
>> I have a use case for needing to use public key cryptography to
>> encrypt a large amount of data in a streaming fashion (get it out of a
>> DB, encrypt, put into an AWS S3 bucket).
>>
>> The command-line gpg tool seems to be able to encrypt/decrypt data
>> from stdin to stdout in a streaming fashion, but in my attempts to use
>> it it seems very file-based for things like the keys to use (whereas I
>> would prefer to be able to pass the public key as an actual value
>> rather than a file; if nothing else because this is for tools that
>> don't have email addresses to use and base their keys on for
>> addressing).
>>
>> Is there an existing library that can achieve this using
>> conduit/pipes/whatever? cryptonite-conduit only covers hashing,
>> hOpenPGP is poorly documented and I can't work out how to use it
>> ("just follow the types" is difficult when Haddock docs don't link to
>> the required types (seems to be because it uses the "import Module as
>> X" trick for re-exporting everything but then everything from those
>> modules isn't available).
>>
>> Can anyone recommend a solution?
>>
>> --
>> Ivan Lazar Miljenovic
>> Ivan.Miljenovic at gmail.com
>> http://IvanMiljenovic.wordpress.com
>> _______________________________________________
>> Haskell-Cafe mailing list
>> To (un)subscribe, modify options or view archives go to:
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>> Only members subscribed via the mailman list are allowed to post.
>>
>>
>



-- 
Ivan Lazar Miljenovic
Ivan.Miljenovic at gmail.com
http://IvanMiljenovic.wordpress.com