[Haskell-cafe] Streaming to JuicyPixels

[Vincent Berthoux]

Hi,

  I can understand your performance problems, I bumped into them before the
first release of Juicy Pixels and took a long time to get 'correct'
performance out of the box, and the IDCT is not the only 'hot point', I got
problems with intermediate data structure as well. Any list has proven a
bit problematic performance wise, so I rewrote some things which would have
been easily implemented with forM_ or mapM_ with manual recursion.

I didn't knew STT existed, so it open a new area of reflection for the
streaming Jpeg decoder, instead of using the (remaining >> getBytes) combo,
staying in the Get monad might help. The reason to expose the ST monad is
that I use it internally to mutate the final image directly, and I'd prefer
to avoid freezing/unfreezing the underlying array. So in order to give
access to the intermediate representation, a type could be (STT s (StateT
JpegDecodingState Get) (MutableImage s PixelYCbCr8)) (the Jpeg decoder only
produce (Image PixelYCbCr8 internally)). This should allow a freeze then a
color conversion.

  As it would induce performance loss, this version should exist alongside
current implementation. This is not trivial, but it's far from impossible.
For the IDCT implementation, I don't think a package make much sense, if
you want to use it, just grab it and customize the interface to your needs
:).

  Regards

     Vincent Berthoux

Le 21 février 2012 06:16, Myles C. Maxfield <myles.maxfield at gmail.com> a
écrit :

Hello, and thanks for the quick reply.
>
> You're right that using (remaining >> getBytes) won't work for streaming,
> as it would pull the rest of the stream into a buffer, thereby delaying
> further processing until the image is completely downloaded. :-(
>
> I'm a little unclear about what you mean about the use of the ST monad.
> There is an STT<http://hackage.haskell.org/packages/archive/STMonadTrans/0.2/doc/html/Control-Monad-ST-Trans.html> monad
> transformer, so you could wrap that around Get. Is that what you're
> referring to?
>
> As an aside: I didn't realize that JuicyPixels existed, so I wrote a JPEG
> decoder specifically designed for streaming - it doesn't request a byte of
> input unless it has to, uses a StateT (wrapped around Attoparsec) to keep
> track of which bit in the current byte is "next", and does the Huffman
> decoding in-line. However, I didn't use ST for the IDCT, so my own decoder
> has serious performance problems. This prompted me to start searching
> around for a solution, and I came across JuicyPixels, which already exists
> and is much faster than my own implementation. I'm hoping to get rid of my
> own decoder and just use JuicyPixels. If you're curious, my own code is
> here: https://github.com/litherum/jpeg.
>
> Is it reasonable to extend JuicyPixels to fit my use case? It sounds like
> JuicyPixels wouldn't work so well as it stands. I'd be happy to do whatever
> work is necessary to help out and get JuicyPixels usable for me. However,
> if that would require a full (or near-full) rewrite, it might make more
> sense for me to use my own implementation with your IDCT. Is there a way we
> can share just the IDCT between our two repositories? Perhaps making a new
> IDCT8 library that we can both depend on?
>
> As for what API I'd like to be able to use, just a "Get DynamicImage"
> should suffice (assuming it has streaming semantics as described above). It
> would be really nice if it was possible to get at the incomplete image
> before the stream is completed (so the image could slowly update as more
> data arrives from the network), but I'm not quite sure how to elegantly
> express that. Do you have any ideas?
>
> I think that having 2 native jpeg decoders (Actually 3, because of this
> package <http://hackage.haskell.org/package/jpeg>) is detrimental to the
> Haskell community, and I would really like to use JuicyPixels :D
>
> Thanks,
> Myles C. Maxfield
>
>
> On Mon, Feb 20, 2012 at 3:01 PM, Vincent Berthoux <
> vincent.berthoux at gmail.com> wrote:
>
>> Hi,
>>
>>  I can expose the low level parsing, but you would only get the
>> chunks/frames/sections of the image, Cereal is mainly used to parse the
>> structure of the image, not to do the raw processing. For the raw
>> processing, I rely on `remaining >> getBytes` to be able to manipulate data
>> at bit level or to feed it to zlib, and the documentation clearly state
>> that remaining doesn't work well with runGetPartial, so no read ahead, but
>> even worse for streaming :).
>>
>> To be fair, I never thought of this use case, and exposing a partially
>> decoded image would impose the use of the ST Monad somehow, and Serialize
>> is not a monad transformer, making it a bit hard to implement.
>>
>> By curiosity what kind of API would you hope for this kind of
>> functionality?
>>
>> Regards
>>
>> Vincent Berthoux
>>
>> Le 20 février 2012 22:08, Myles C. Maxfield <myles.maxfield at gmail.com> a
>> écrit :
>>
>> Hello,
>>> I am interested in the possibility of using JuicyPixels for streaming
>>> images from the web. It doesn't appear to expose any of its internally-used
>>> Serialize.Get functionality, which is problematic for streaming - I would
>>> not like to have to stream the whole image into a buffer before the decoder
>>> can start decoding. Are there any plans on exposing this API, so I can use
>>> the runGetPartial function to facilitate streaming?
>>>
>>> In addition, does the library do much readahead? There's no point in
>>> exposing a Get interface if it's just going to wait until the stream is
>>> done to start decoding anyway.
>>>
>>>  Thanks,
>>> Myles C. Maxfield
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20120222/8e7c9251/attachment.htm>