[Haskell-cafe] Encrypting streamed data

[Ivan Lazar Miljenovic]

On 6 July 2017 at 19:17, Patrick Chilton <chpatrick at gmail.com> wrote:
> If your data fits in RAM maybe it's best to forget about the streaming and
> use the saltine library's Box module. Then you can't really go wrong on the
> crypto front.

If that was the case, I wouldn't be asking ;-)

>
> On Jul 6, 2017 09:41, "Ivan Lazar Miljenovic" <ivan.miljenovic at gmail.com>
> wrote:
>>
>> On 6 July 2017 at 17:23, David Turner <dct25-561bs at mythic-beasts.com>
>> wrote:
>> > Apologies, just seen the bit about wanting to pass the key in directly
>> > rather than using the GPG keyring because there are no email addresses
>> > attached to your various keys.
>> >
>> > Maybe a silly question, but can you give them email addresses to
>> > identify
>> > them in a GPG-compatible manner? They don't have to have associated
>> > mailboxes so the addresses can be totally made-up.
>>
>> Yes, that's my fallback, since it's possible to tell gpg to use a
>> different directory so I can provide a key externally with my
>> transmission request, load it into a temporary store, grab the
>> identity out and use that.  It would just be more convenient to have a
>> "here's a ByteString with the public key" option (which I can always
>> implement as a wrapper function).
>>
>> >
>> > If not, I'd probably start to look to something like openssl to do the
>> > symmetric encryption and manage the keys for that separately. It looks
>> > possible to build a streaming AES implementation using the nonstreaming
>> > functions in `cryptonite`, but the usual recommendation is that its far
>> > too
>> > risky to do any low-level crypto yourself so this seems like a bad idea.
>> >
>> >
>> > On 6 Jul 2017 07:40, "David Turner" <dct25-561bs at mythic-beasts.com>
>> > wrote:
>> >>
>> >> Hi,
>> >>
>> >> I do not know of a library to do this, sorry. Note that the way
>> >> public-key
>> >> crypto works in a streaming fashion is typically to use the public-key
>> >> bit
>> >> only to encrypt a key for a symmetric cipher and then use the
>> >> (much-faster)
>> >> symmetric encryption for the actual data. The symmetric bit could well
>> >> be
>> >> something like AES256-CBC or AES256-CTR.
>> >>
>> >> This means that the format of the resulting data is a bit
>> >> implementation-defined as it has to include the asymetrically-encrypted
>> >> data
>> >> first, followed by the stream of symmetrically-encrypted data. GnuPG
>> >> includes quite a bit of metadata in its files that describes the
>> >> algorithms
>> >> used and delimits the pieces, so if you want the resulting files to be
>> >> GnuPG-compatible you'll need to take this into account.
>> >>
>> >> If it were me, I'd probably just shell out to `gpg`. It's fast and
>> >> low-risk.
>> >>
>> >> Hope that helps,
>> >>
>> >> David
>> >>
>> >>
>> >> On 6 Jul 2017 05:59, "Ivan Lazar Miljenovic"
>> >> <ivan.miljenovic at gmail.com>
>> >> wrote:
>> >>
>> >> I have a use case for needing to use public key cryptography to
>> >> encrypt a large amount of data in a streaming fashion (get it out of a
>> >> DB, encrypt, put into an AWS S3 bucket).
>> >>
>> >> The command-line gpg tool seems to be able to encrypt/decrypt data
>> >> from stdin to stdout in a streaming fashion, but in my attempts to use
>> >> it it seems very file-based for things like the keys to use (whereas I
>> >> would prefer to be able to pass the public key as an actual value
>> >> rather than a file; if nothing else because this is for tools that
>> >> don't have email addresses to use and base their keys on for
>> >> addressing).
>> >>
>> >> Is there an existing library that can achieve this using
>> >> conduit/pipes/whatever? cryptonite-conduit only covers hashing,
>> >> hOpenPGP is poorly documented and I can't work out how to use it
>> >> ("just follow the types" is difficult when Haddock docs don't link to
>> >> the required types (seems to be because it uses the "import Module as
>> >> X" trick for re-exporting everything but then everything from those
>> >> modules isn't available).
>> >>
>> >> Can anyone recommend a solution?
>> >>
>> >> --
>> >> Ivan Lazar Miljenovic
>> >> Ivan.Miljenovic at gmail.com
>> >> http://IvanMiljenovic.wordpress.com
>> >> _______________________________________________
>> >> Haskell-Cafe mailing list
>> >> To (un)subscribe, modify options or view archives go to:
>> >> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>> >> Only members subscribed via the mailman list are allowed to post.
>> >>
>> >>
>> >
>>
>>
>>
>> --
>> Ivan Lazar Miljenovic
>> Ivan.Miljenovic at gmail.com
>> http://IvanMiljenovic.wordpress.com
>> _______________________________________________
>> Haskell-Cafe mailing list
>> To (un)subscribe, modify options or view archives go to:
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>> Only members subscribed via the mailman list are allowed to post.



-- 
Ivan Lazar Miljenovic
Ivan.Miljenovic at gmail.com
http://IvanMiljenovic.wordpress.com