[Haskell-cafe] safe lazy IO or Iteratee?

[John Lato]

I've put my benchmarking code online at:

http://inmachina.net/~jwlato/haskell/research-iteratee.tar.bz2

unpack it so you have this directory structure:

./iteratee
./research-iteratee/

Also download my criterionProcessor programs.  The darcs repo is at

http://inmachina.net/~jwlato/haskell/criterionProcessor/

to use it, go into the criterionProcessor directory, edit the
testrunner.hs script for your environment, and run it.  This runs all
the benchmarks.  Then you can use the CritProc program (build with
cabal) to generate pictures.  I'm pretty sure you need Chart HEAD in
order to build CritProc (I hacked my Chart install, but I think the
only important change has been applied to HEAD).

I make no guarantees that these will all build properly, it's
basically a work-in-progress dump.

John


On Fri, Feb 5, 2010 at 10:25 PM, John Millikin <jmillikin at gmail.com> wrote:
> Benchmark attached. It just enumerates a list until EOF is reached.
>
> An interesting thing I've noticed is that IterateeMCPS performs better
> with no optimization, but -O2 gives IterateeM the advantage. Their
> relative performance depends heavily on the chunk size -- for example,
> CPS is much faster at chunk size 1, but slower with 100-element
> chunks.
>
> On Fri, Feb 5, 2010 at 08:56, John Lato <jwlato at gmail.com> wrote:
>> On Fri, Feb 5, 2010 at 4:31 PM, Valery V. Vorotyntsev
>> <valery.vv at gmail.com> wrote:
>>>> John Lato <jwlato at gmail.com> wrote:
>>>>
>>>>> Both designs appear to offer similar performance in aggregate,
>>>>> although there are differences for particular functions.  I haven't
>>>>> yet had a chance to test the performance of the CPS variant, although
>>>>> Oleg has indicated he expects it will be higher.
>>>
>>> @jwlato:
>>> Do you mind creating `IterateeCPS' tree in
>>> <http://inmachina.net/~jwlato/haskell/iteratee/src/Data/>, so we can
>>> start writing CPS performance testing code?
>>
>> I'm working on the CPS version and will make it public when it's done.
>>  It may take a week or so; this term started at 90 and has picked up.
>> I have several benchmark sources that aren't public yet, but I can put
>> them online for your perusal.
>>
>>>
>>> AFAICS, you have benchmarks for IterateeM-driven code already:
>>> http://inmachina.net/~jwlato/haskell/iteratee/tests/benchmarks.hs
>>
>> Those will make more sense when I've added the context of the
>> codebases in use.  There are several more sets of output that I simply
>> haven't published yet, including bytestring-based variants.
>>
>>>
>>> John Millikin <jmillikin at gmail.com> wrote:
>>>
>>>> I wrote some criterion benchmarks for IterateeM vs IterateeCPS, and
>>>> the CPS version was notably slower. I don't understand enough about
>>>> CPS to diagnose why, but the additional runtime was present in even
>>>> simple cases (reading from a file, writing back out).
>>
>> That's very interesting.  I wonder if I'll see the same, and if I'd be
>> able to figure it out myself...
>>
>> Did you benchmark any cases without doing IO?  Sometimes the cost of
>> the IO can overwhelm any other measurable differences, and also disk
>> caching can affect results.  Criterion should highlight any major
>> outliers, but I still like to avoid IO when benchmarking unless
>> strictly necessary.
>>
>>>
>>> @jmillikin:
>>> Could you please publish those benchmarks?
>>
>> +1
>>
>> John
>>
>