[Haskell-cafe] Streaming to JuicyPixels

Fri Feb 24 06:42:00 CET 2012

So I started working on moving JuicyPixels to a streaming interface, and
have some observations. This is going to be a pretty major change, touching
pretty much every function, and the end result will end up looking very
close to the code that I have already written. I'm not nearly as close to
the code as the author is, and I've already made some mistakes due to not
understanding how the original code is structured and layed out.

Because this is essentially turning out to be a rewrite, I think it makes
more sense for me to just work on my own library, and release it as a
streaming alternative to JuicyPixels. How do you feel about this, Vincent?

Thanks,
Myles

On Wed, Feb 22, 2012 at 12:30 PM, Vincent Berthoux <
vincent.berthoux at gmail.com> wrote:

> Hi,
>   Please go ahead, and github is the perfect medium for code sharing :)
>
>   Regards
>
>    Vincent Berthoux
>
> Le 22 février 2012 20:20, Myles C. Maxfield <myles.maxfield at gmail.com> a
> écrit :
>
> Let's put aside the issue of getting access to the pixels before the
>> stream is complete.
>>
>> How would you feel if I implemented use of the STT monad transformer on
>> top of Get in JuicyPixels, in order to get rid of the (remaining >>
>> getBytes) call, and then expose the underlying Get interface to callers?
>> This would allow use for streaming.
>>
>> Is this something that you feel that I should pursue? I can send you a
>> GitHub Pull Request when I'm done.
>>
>> Thanks,
>> Myles
>>
>> On Wed, Feb 22, 2012 at 5:01 AM, Vincent Berthoux <
>> vincent.berthoux at gmail.com> wrote:
>>
>>> Hi,
>>>
>>>   I can understand your performance problems, I bumped into them before
>>> the first release of Juicy Pixels and took a long time to get 'correct'
>>> performance out of the box, and the IDCT is not the only 'hot point', I got
>>> problems with intermediate data structure as well. Any list has proven a
>>> bit problematic performance wise, so I rewrote some things which would have
>>> been easily implemented with forM_ or mapM_ with manual recursion.
>>>
>>> I didn't knew STT existed, so it open a new area of reflection for the
>>> streaming Jpeg decoder, instead of using the (remaining >> getBytes) combo,
>>> staying in the Get monad might help. The reason to expose the ST monad is
>>> that I use it internally to mutate the final image directly, and I'd prefer
>>> to avoid freezing/unfreezing the underlying array. So in order to give
>>> access to the intermediate representation, a type could be (STT s (StateT
>>> JpegDecodingState Get) (MutableImage s PixelYCbCr8)) (the Jpeg decoder only
>>> produce (Image PixelYCbCr8 internally)). This should allow a freeze then a
>>> color conversion.
>>>
>>>   As it would induce performance loss, this version should exist
>>> alongside current implementation. This is not trivial, but it's far from
>>> impossible. For the IDCT implementation, I don't think a package make much
>>> sense, if you want to use it, just grab it and customize the interface to
>>> your needs :).
>>>
>>>   Regards
>>>
>>>      Vincent Berthoux
>>>
>>> Le 21 février 2012 06:16, Myles C. Maxfield <myles.maxfield at gmail.com>a écrit :
>>>
>>> Hello, and thanks for the quick reply.
>>>>
>>>> You're right that using (remaining >> getBytes) won't work for
>>>> streaming, as it would pull the rest of the stream into a buffer, thereby
>>>> delaying further processing until the image is completely downloaded. :-(
>>>>
>>>> I'm a little unclear about what you mean about the use of the ST monad.
>>>> There is an STT<http://hackage.haskell.org/packages/archive/STMonadTrans/0.2/doc/html/Control-Monad-ST-Trans.html> monad
>>>> transformer, so you could wrap that around Get. Is that what you're
>>>> referring to?
>>>>
>>>> As an aside: I didn't realize that JuicyPixels existed, so I wrote a
>>>> JPEG decoder specifically designed for streaming - it doesn't request a
>>>> byte of input unless it has to, uses a StateT (wrapped around Attoparsec)
>>>> to keep track of which bit in the current byte is "next", and does the
>>>> Huffman decoding in-line. However, I didn't use ST for the IDCT, so my own
>>>> decoder has serious performance problems. This prompted me to start
>>>> searching around for a solution, and I came across JuicyPixels, which
>>>> already exists and is much faster than my own implementation. I'm hoping to
>>>> get rid of my own decoder and just use JuicyPixels. If you're curious, my
>>>> own code is here: https://github.com/litherum/jpeg.
>>>>
>>>> Is it reasonable to extend JuicyPixels to fit my use case? It sounds
>>>> like JuicyPixels wouldn't work so well as it stands. I'd be happy to do
>>>> whatever work is necessary to help out and get JuicyPixels usable for me.
>>>> However, if that would require a full (or near-full) rewrite, it might make
>>>> more sense for me to use my own implementation with your IDCT. Is there a
>>>> way we can share just the IDCT between our two repositories? Perhaps making
>>>> a new IDCT8 library that we can both depend on?
>>>>
>>>> As for what API I'd like to be able to use, just a "Get DynamicImage"
>>>> should suffice (assuming it has streaming semantics as described above). It
>>>> would be really nice if it was possible to get at the incomplete image
>>>> before the stream is completed (so the image could slowly update as more
>>>> data arrives from the network), but I'm not quite sure how to elegantly
>>>> express that. Do you have any ideas?
>>>>
>>>> I think that having 2 native jpeg decoders (Actually 3, because of this
>>>> package <http://hackage.haskell.org/package/jpeg>) is detrimental to
>>>> the Haskell community, and I would really like to use JuicyPixels :D
>>>>
>>>> Thanks,
>>>> Myles C. Maxfield
>>>>
>>>>
>>>> On Mon, Feb 20, 2012 at 3:01 PM, Vincent Berthoux <
>>>> vincent.berthoux at gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>>  I can expose the low level parsing, but you would only get the
>>>>> chunks/frames/sections of the image, Cereal is mainly used to parse the
>>>>> structure of the image, not to do the raw processing. For the raw
>>>>> processing, I rely on `remaining >> getBytes` to be able to manipulate data
>>>>> at bit level or to feed it to zlib, and the documentation clearly state
>>>>> that remaining doesn't work well with runGetPartial, so no read ahead, but
>>>>> even worse for streaming :).
>>>>>
>>>>> To be fair, I never thought of this use case, and exposing a partially
>>>>> decoded image would impose the use of the ST Monad somehow, and Serialize
>>>>> is not a monad transformer, making it a bit hard to implement.
>>>>>
>>>>> By curiosity what kind of API would you hope for this kind of
>>>>> functionality?
>>>>>
>>>>> Regards
>>>>>
>>>>> Vincent Berthoux
>>>>>
>>>>> Le 20 février 2012 22:08, Myles C. Maxfield <myles.maxfield at gmail.com>a écrit :
>>>>>
>>>>> Hello,
>>>>>> I am interested in the possibility of using JuicyPixels for streaming
>>>>>> images from the web. It doesn't appear to expose any of its internally-used
>>>>>> Serialize.Get functionality, which is problematic for streaming - I would
>>>>>> not like to have to stream the whole image into a buffer before the decoder
>>>>>> can start decoding. Are there any plans on exposing this API, so I can use
>>>>>> the runGetPartial function to facilitate streaming?
>>>>>>
>>>>>> In addition, does the library do much readahead? There's no point in
>>>>>> exposing a Get interface if it's just going to wait until the stream is
>>>>>> done to start decoding anyway.
>>>>>>
>>>>>>  Thanks,
>>>>>> Myles C. Maxfield
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>> _______________________________________________
>>> Haskell-Cafe mailing list
>>> Haskell-Cafe at haskell.org
>>> http://www.haskell.org/mailman/listinfo/haskell-cafe
>>>
>>>
>>
>
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20120223/b7661f3f/attachment.htm>