[Haskell-cafe] Encrypting streamed data

Rein Henrichs rein.henrichs at gmail.com
Thu Jul 6 23:27:17 UTC 2017


Ah, yes. That would do it.

On Thu, Jul 6, 2017 at 4:25 PM Ivan Lazar Miljenovic <
ivan.miljenovic at gmail.com> wrote:

> On 7 July 2017 at 09:04, Rein Henrichs <rein.henrichs at gmail.com> wrote:
> > I'm also not sure what benefit you would get from rolling your own versus
> > using AWS's server-side S3 bucket encryption[1], now that I think about
> it.
> >
> > [1]
> >
> http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html
>
> Due to regulatory requirements, we can't rely upon that and need to
> ensure we never put any private data in AWS that even someone with
> login access to AWS can read.
>
> >
> > On Thu, Jul 6, 2017 at 3:56 PM Rein Henrichs <rein.henrichs at gmail.com>
> > wrote:
> >>
> >> If you are not a security expert, I would strongly recommend against
> >> rolling your own encryption scheme. If you are a security expert, you
> would
> >> probably not be asking us for advice on how to roll your own encryption
> >> scheme. I would suggest that you find something off the rack that might
> meet
> >> your needs (whether it's written in Haskell or not) and make sure you
> >> understand how it works and what your threat model is well enough to
> decide
> >> whether it does, in fact, meet your needs.
> >>
> >> On Thu, Jul 6, 2017 at 3:29 PM Ivan Lazar Miljenovic
> >> <ivan.miljenovic at gmail.com> wrote:
> >>>
> >>> On 7 July 2017 at 01:44, Viktor Dukhovni <ietf-dane at dukhovni.org>
> wrote:
> >>> >
> >>> >> On Jul 6, 2017, at 12:58 AM, Ivan Lazar Miljenovic
> >>> >> <ivan.miljenovic at gmail.com> wrote:
> >>> >>
> >>> >> I have a use case for needing to use public key cryptography to
> >>> >> encrypt a large amount of data in a streaming fashion (get it out
> of a
> >>> >> DB, encrypt, put into an AWS S3 bucket).
> >>> >
> >>> > What are the data-format requirements?  Do you need (binary) CMS
> >>> > output?
> >>> > GPG-compatible output?  Or just roll your own?
> >>>
> >>> The intent is to be able to transfer data between two parties such
> >>> that only the recipient is able to view it (hence the usage of public
> >>> key cryptography).  GPG/PGP compatability is preferable as it's
> >>> common, but anything that is sufficiently standardised (as this will
> >>> potentially be used by others that aren't me doing so with Haskell and
> >>> thus can't just use a library to do so) will suffice.
> >>>
> >>> (The other advantage of GPG/PGP is that the security testing team is
> >>> more familiar with it and thus likely to sign off on it.)
> >>>
> >>> >
> >>> > Integrity protection can be tricky with large data streams.  Most
> data
> >>> > formats for enveloped data have a single MAC at the end, which means
> >>> > that the decoder has to consume all the data before it is known to be
> >>> > valid!
> >>> >
> >>> > So if you're in a position to avoid a standard all-in-one format, it
> >>> > makes sense to "packetize" the stream, with integrity protection for
> >>> > each "packet", and packet sequence numbers to preserve overall stream
> >>> > integrity.  With vast amounts of data, you'll want to be careful with
> >>> > the symmetric cipher modes, AEAD (AES-GCM, for example) protects only
> >>> > a limited amount of data before you need to rekey.  It may be
> simplest
> >>> > to just generate a new symmetric key for every N megabytes of data.
> >>> >
> >>> > With a careful design of the "packet" format, you can use in-memory
> >>> > crypto for each packet.  Don't forget to include an "end-of-stream"
> >>> > packet to defeat truncation attacks.
> >>>
> >>> This sounds good in theory, but in practice I'm not versed enough in
> >>> security to want to try and roll my own if I could avoid it, and
> >>> trying to document such a format for others to use could be
> >>> problematic.allowed to post.
> >>>
> >>> --
> >>> Ivan Lazar Miljenovic
> >>> Ivan.Miljenovic at gmail.com
> >>> http://IvanMiljenovic.wordpress.com
> >>> _______________________________________________
> >>> Haskell-Cafe mailing list
> >>> To (un)subscribe, modify options or view archives go to:
> >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
> >>> Only members subscribed via the mailman list are allowed to post.
>
>
>
> --
> Ivan Lazar Miljenovic
> Ivan.Miljenovic at gmail.com
> http://IvanMiljenovic.wordpress.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20170706/567d34d6/attachment.html>


More information about the Haskell-Cafe mailing list