[Haskell-cafe] Encrypting streamed data

[Rein Henrichs]

I'm also not sure what benefit you would get from rolling your own versus
using AWS's server-side S3 bucket encryption[1], now that I think about it.

[1]
http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html

On Thu, Jul 6, 2017 at 3:56 PM Rein Henrichs <rein.henrichs at gmail.com>
wrote:

> If you are not a security expert, I would strongly recommend against
> rolling your own encryption scheme. If you are a security expert, you would
> probably not be asking us for advice on how to roll your own encryption
> scheme. I would suggest that you find something off the rack that might
> meet your needs (whether it's written in Haskell or not) and make sure you
> understand how it works and what your threat model is well enough to decide
> whether it does, in fact, meet your needs.
>
> On Thu, Jul 6, 2017 at 3:29 PM Ivan Lazar Miljenovic <
> ivan.miljenovic at gmail.com> wrote:
>
>> On 7 July 2017 at 01:44, Viktor Dukhovni <ietf-dane at dukhovni.org> wrote:
>> >
>> >> On Jul 6, 2017, at 12:58 AM, Ivan Lazar Miljenovic <
>> ivan.miljenovic at gmail.com> wrote:
>> >>
>> >> I have a use case for needing to use public key cryptography to
>> >> encrypt a large amount of data in a streaming fashion (get it out of a
>> >> DB, encrypt, put into an AWS S3 bucket).
>> >
>> > What are the data-format requirements?  Do you need (binary) CMS output?
>> > GPG-compatible output?  Or just roll your own?
>>
>> The intent is to be able to transfer data between two parties such
>> that only the recipient is able to view it (hence the usage of public
>> key cryptography).  GPG/PGP compatability is preferable as it's
>> common, but anything that is sufficiently standardised (as this will
>> potentially be used by others that aren't me doing so with Haskell and
>> thus can't just use a library to do so) will suffice.
>>
>> (The other advantage of GPG/PGP is that the security testing team is
>> more familiar with it and thus likely to sign off on it.)
>>
>> >
>> > Integrity protection can be tricky with large data streams.  Most data
>> > formats for enveloped data have a single MAC at the end, which means
>> > that the decoder has to consume all the data before it is known to be
>> > valid!
>> >
>> > So if you're in a position to avoid a standard all-in-one format, it
>> > makes sense to "packetize" the stream, with integrity protection for
>> > each "packet", and packet sequence numbers to preserve overall stream
>> > integrity.  With vast amounts of data, you'll want to be careful with
>> > the symmetric cipher modes, AEAD (AES-GCM, for example) protects only
>> > a limited amount of data before you need to rekey.  It may be simplest
>> > to just generate a new symmetric key for every N megabytes of data.
>> >
>> > With a careful design of the "packet" format, you can use in-memory
>> > crypto for each packet.  Don't forget to include an "end-of-stream"
>> > packet to defeat truncation attacks.
>>
>> This sounds good in theory, but in practice I'm not versed enough in
>> security to want to try and roll my own if I could avoid it, and
>> trying to document such a format for others to use could be
>> problematic.allowed to post.
>>
>> --
>> Ivan Lazar Miljenovic
>> Ivan.Miljenovic at gmail.com
>> http://IvanMiljenovic.wordpress.com
>> _______________________________________________
>> Haskell-Cafe mailing list
>> To (un)subscribe, modify options or view archives go to:
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>> Only members subscribed via the mailman list are allowed to post.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20170706/358881f0/attachment.html>