[Haskell-cafe] Streaming to JuicyPixels

Wed Feb 22 20:20:54 CET 2012

Let's put aside the issue of getting access to the pixels before the stream
is complete.

How would you feel if I implemented use of the STT monad transformer on top
of Get in JuicyPixels, in order to get rid of the (remaining >> getBytes)
call, and then expose the underlying Get interface to callers? This would
allow use for streaming.

Is this something that you feel that I should pursue? I can send you a
GitHub Pull Request when I'm done.

Thanks,
Myles

On Wed, Feb 22, 2012 at 5:01 AM, Vincent Berthoux <
vincent.berthoux at gmail.com> wrote:

> Hi,
>
>   I can understand your performance problems, I bumped into them before
> the first release of Juicy Pixels and took a long time to get 'correct'
> performance out of the box, and the IDCT is not the only 'hot point', I got
> problems with intermediate data structure as well. Any list has proven a
> bit problematic performance wise, so I rewrote some things which would have
> been easily implemented with forM_ or mapM_ with manual recursion.
>
> I didn't knew STT existed, so it open a new area of reflection for the
> streaming Jpeg decoder, instead of using the (remaining >> getBytes) combo,
> staying in the Get monad might help. The reason to expose the ST monad is
> that I use it internally to mutate the final image directly, and I'd prefer
> to avoid freezing/unfreezing the underlying array. So in order to give
> access to the intermediate representation, a type could be (STT s (StateT
> JpegDecodingState Get) (MutableImage s PixelYCbCr8)) (the Jpeg decoder only
> produce (Image PixelYCbCr8 internally)). This should allow a freeze then a
> color conversion.
>
>   As it would induce performance loss, this version should exist alongside
> current implementation. This is not trivial, but it's far from impossible.
> For the IDCT implementation, I don't think a package make much sense, if
> you want to use it, just grab it and customize the interface to your needs
> :).
>
>   Regards
>
>      Vincent Berthoux
>
> Le 21 février 2012 06:16, Myles C. Maxfield <myles.maxfield at gmail.com> a
> écrit :
>
> Hello, and thanks for the quick reply.
>>
>> You're right that using (remaining >> getBytes) won't work for streaming,
>> as it would pull the rest of the stream into a buffer, thereby delaying
>> further processing until the image is completely downloaded. :-(
>>
>> I'm a little unclear about what you mean about the use of the ST monad.
>> There is an STT<http://hackage.haskell.org/packages/archive/STMonadTrans/0.2/doc/html/Control-Monad-ST-Trans.html> monad
>> transformer, so you could wrap that around Get. Is that what you're
>> referring to?
>>
>> As an aside: I didn't realize that JuicyPixels existed, so I wrote a JPEG
>> decoder specifically designed for streaming - it doesn't request a byte of
>> input unless it has to, uses a StateT (wrapped around Attoparsec) to keep
>> track of which bit in the current byte is "next", and does the Huffman
>> decoding in-line. However, I didn't use ST for the IDCT, so my own decoder
>> has serious performance problems. This prompted me to start searching
>> around for a solution, and I came across JuicyPixels, which already exists
>> and is much faster than my own implementation. I'm hoping to get rid of my
>> own decoder and just use JuicyPixels. If you're curious, my own code is
>> here: https://github.com/litherum/jpeg.
>>
>> Is it reasonable to extend JuicyPixels to fit my use case? It sounds like
>> JuicyPixels wouldn't work so well as it stands. I'd be happy to do whatever
>> work is necessary to help out and get JuicyPixels usable for me. However,
>> if that would require a full (or near-full) rewrite, it might make more
>> sense for me to use my own implementation with your IDCT. Is there a way we
>> can share just the IDCT between our two repositories? Perhaps making a new
>> IDCT8 library that we can both depend on?
>>
>> As for what API I'd like to be able to use, just a "Get DynamicImage"
>> should suffice (assuming it has streaming semantics as described above). It
>> would be really nice if it was possible to get at the incomplete image
>> before the stream is completed (so the image could slowly update as more
>> data arrives from the network), but I'm not quite sure how to elegantly
>> express that. Do you have any ideas?
>>
>> I think that having 2 native jpeg decoders (Actually 3, because of this
>> package <http://hackage.haskell.org/package/jpeg>) is detrimental to the
>> Haskell community, and I would really like to use JuicyPixels :D
>>
>> Thanks,
>> Myles C. Maxfield
>>
>>
>> On Mon, Feb 20, 2012 at 3:01 PM, Vincent Berthoux <
>> vincent.berthoux at gmail.com> wrote:
>>
>>> Hi,
>>>
>>>  I can expose the low level parsing, but you would only get the
>>> chunks/frames/sections of the image, Cereal is mainly used to parse the
>>> structure of the image, not to do the raw processing. For the raw
>>> processing, I rely on `remaining >> getBytes` to be able to manipulate data
>>> at bit level or to feed it to zlib, and the documentation clearly state
>>> that remaining doesn't work well with runGetPartial, so no read ahead, but
>>> even worse for streaming :).
>>>
>>> To be fair, I never thought of this use case, and exposing a partially
>>> decoded image would impose the use of the ST Monad somehow, and Serialize
>>> is not a monad transformer, making it a bit hard to implement.
>>>
>>> By curiosity what kind of API would you hope for this kind of
>>> functionality?
>>>
>>> Regards
>>>
>>> Vincent Berthoux
>>>
>>> Le 20 février 2012 22:08, Myles C. Maxfield <myles.maxfield at gmail.com>a écrit :
>>>
>>> Hello,
>>>> I am interested in the possibility of using JuicyPixels for streaming
>>>> images from the web. It doesn't appear to expose any of its internally-used
>>>> Serialize.Get functionality, which is problematic for streaming - I would
>>>> not like to have to stream the whole image into a buffer before the decoder
>>>> can start decoding. Are there any plans on exposing this API, so I can use
>>>> the runGetPartial function to facilitate streaming?
>>>>
>>>> In addition, does the library do much readahead? There's no point in
>>>> exposing a Get interface if it's just going to wait until the stream is
>>>> done to start decoding anyway.
>>>>
>>>>  Thanks,
>>>> Myles C. Maxfield
>>>>
>>>
>>>
>>
>
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20120222/2c166cfd/attachment-0001.htm>