[Haskell-cafe] Streaming to JuicyPixels

Wed Feb 22 21:30:52 CET 2012

Hi,
  Please go ahead, and github is the perfect medium for code sharing :)

  Regards

   Vincent Berthoux

Le 22 février 2012 20:20, Myles C. Maxfield <myles.maxfield at gmail.com> a
écrit :

Let's put aside the issue of getting access to the pixels before the stream
> is complete.
>
> How would you feel if I implemented use of the STT monad transformer on
> top of Get in JuicyPixels, in order to get rid of the (remaining >>
> getBytes) call, and then expose the underlying Get interface to callers?
> This would allow use for streaming.
>
> Is this something that you feel that I should pursue? I can send you a
> GitHub Pull Request when I'm done.
>
> Thanks,
> Myles
>
> On Wed, Feb 22, 2012 at 5:01 AM, Vincent Berthoux <
> vincent.berthoux at gmail.com> wrote:
>
>> Hi,
>>
>>   I can understand your performance problems, I bumped into them before
>> the first release of Juicy Pixels and took a long time to get 'correct'
>> performance out of the box, and the IDCT is not the only 'hot point', I got
>> problems with intermediate data structure as well. Any list has proven a
>> bit problematic performance wise, so I rewrote some things which would have
>> been easily implemented with forM_ or mapM_ with manual recursion.
>>
>> I didn't knew STT existed, so it open a new area of reflection for the
>> streaming Jpeg decoder, instead of using the (remaining >> getBytes) combo,
>> staying in the Get monad might help. The reason to expose the ST monad is
>> that I use it internally to mutate the final image directly, and I'd prefer
>> to avoid freezing/unfreezing the underlying array. So in order to give
>> access to the intermediate representation, a type could be (STT s (StateT
>> JpegDecodingState Get) (MutableImage s PixelYCbCr8)) (the Jpeg decoder only
>> produce (Image PixelYCbCr8 internally)). This should allow a freeze then a
>> color conversion.
>>
>>   As it would induce performance loss, this version should exist
>> alongside current implementation. This is not trivial, but it's far from
>> impossible. For the IDCT implementation, I don't think a package make much
>> sense, if you want to use it, just grab it and customize the interface to
>> your needs :).
>>
>>   Regards
>>
>>      Vincent Berthoux
>>
>> Le 21 février 2012 06:16, Myles C. Maxfield <myles.maxfield at gmail.com> a
>> écrit :
>>
>> Hello, and thanks for the quick reply.
>>>
>>> You're right that using (remaining >> getBytes) won't work for
>>> streaming, as it would pull the rest of the stream into a buffer, thereby
>>> delaying further processing until the image is completely downloaded. :-(
>>>
>>> I'm a little unclear about what you mean about the use of the ST monad.
>>> There is an STT<http://hackage.haskell.org/packages/archive/STMonadTrans/0.2/doc/html/Control-Monad-ST-Trans.html> monad
>>> transformer, so you could wrap that around Get. Is that what you're
>>> referring to?
>>>
>>> As an aside: I didn't realize that JuicyPixels existed, so I wrote a
>>> JPEG decoder specifically designed for streaming - it doesn't request a
>>> byte of input unless it has to, uses a StateT (wrapped around Attoparsec)
>>> to keep track of which bit in the current byte is "next", and does the
>>> Huffman decoding in-line. However, I didn't use ST for the IDCT, so my own
>>> decoder has serious performance problems. This prompted me to start
>>> searching around for a solution, and I came across JuicyPixels, which
>>> already exists and is much faster than my own implementation. I'm hoping to
>>> get rid of my own decoder and just use JuicyPixels. If you're curious, my
>>> own code is here: https://github.com/litherum/jpeg.
>>>
>>> Is it reasonable to extend JuicyPixels to fit my use case? It sounds
>>> like JuicyPixels wouldn't work so well as it stands. I'd be happy to do
>>> whatever work is necessary to help out and get JuicyPixels usable for me.
>>> However, if that would require a full (or near-full) rewrite, it might make
>>> more sense for me to use my own implementation with your IDCT. Is there a
>>> way we can share just the IDCT between our two repositories? Perhaps making
>>> a new IDCT8 library that we can both depend on?
>>>
>>> As for what API I'd like to be able to use, just a "Get DynamicImage"
>>> should suffice (assuming it has streaming semantics as described above). It
>>> would be really nice if it was possible to get at the incomplete image
>>> before the stream is completed (so the image could slowly update as more
>>> data arrives from the network), but I'm not quite sure how to elegantly
>>> express that. Do you have any ideas?
>>>
>>> I think that having 2 native jpeg decoders (Actually 3, because of this
>>> package <http://hackage.haskell.org/package/jpeg>) is detrimental to
>>> the Haskell community, and I would really like to use JuicyPixels :D
>>>
>>> Thanks,
>>> Myles C. Maxfield
>>>
>>>
>>> On Mon, Feb 20, 2012 at 3:01 PM, Vincent Berthoux <
>>> vincent.berthoux at gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>>  I can expose the low level parsing, but you would only get the
>>>> chunks/frames/sections of the image, Cereal is mainly used to parse the
>>>> structure of the image, not to do the raw processing. For the raw
>>>> processing, I rely on `remaining >> getBytes` to be able to manipulate data
>>>> at bit level or to feed it to zlib, and the documentation clearly state
>>>> that remaining doesn't work well with runGetPartial, so no read ahead, but
>>>> even worse for streaming :).
>>>>
>>>> To be fair, I never thought of this use case, and exposing a partially
>>>> decoded image would impose the use of the ST Monad somehow, and Serialize
>>>> is not a monad transformer, making it a bit hard to implement.
>>>>
>>>> By curiosity what kind of API would you hope for this kind of
>>>> functionality?
>>>>
>>>> Regards
>>>>
>>>> Vincent Berthoux
>>>>
>>>> Le 20 février 2012 22:08, Myles C. Maxfield <myles.maxfield at gmail.com>a écrit :
>>>>
>>>> Hello,
>>>>> I am interested in the possibility of using JuicyPixels for streaming
>>>>> images from the web. It doesn't appear to expose any of its internally-used
>>>>> Serialize.Get functionality, which is problematic for streaming - I would
>>>>> not like to have to stream the whole image into a buffer before the decoder
>>>>> can start decoding. Are there any plans on exposing this API, so I can use
>>>>> the runGetPartial function to facilitate streaming?
>>>>>
>>>>> In addition, does the library do much readahead? There's no point in
>>>>> exposing a Get interface if it's just going to wait until the stream is
>>>>> done to start decoding anyway.
>>>>>
>>>>>  Thanks,
>>>>> Myles C. Maxfield
>>>>>
>>>>
>>>>
>>>
>>
>>
>> _______________________________________________
>> Haskell-Cafe mailing list
>> Haskell-Cafe at haskell.org
>> http://www.haskell.org/mailman/listinfo/haskell-cafe
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20120222/e7db3290/attachment.htm>