[Haskell-cafe] Conduit+GHC high memory use for simple Sink
Bryan Vicknair
bryanvick at gmail.com
Thu Aug 28 18:27:52 UTC 2014
Michael,
When I was first digging into this bug, I also ran into the case where doing an
action twice would trigger a large increase in memory usage. Also strange was
that doing 'sequence_ [action]' was causing the problem for me, but 'do action'
was not.
Strangely enough though, on my machine there is no memory difference between
running 'action' once or twice in your 1st example [1] in comment 4 of bug
#9520. Whether action is done once or twice, the maximum resident memory as
reported by /usr/bin/time -v is about 1.1Mb. I get roughly the same memory
usage from the second code example in that same comment.
Attached are the .prof and .hp files from running the 'mem' binary using sink2
on my machine.
Here is the output from the +RTS -s switch:
2,191,403,328 bytes allocated in the heap
4,269,946,560 bytes copied during GC
528,829,096 bytes maximum residency (21 sample(s))
21,830,752 bytes maximum slop
1070 MB total memory in use (0 MB lost due to fragmentation)
Tot time (elapsed) Avg pause Max pause
Gen 0 3826 colls, 0 par 0.37s 0.42s 0.0001s 0.0032s
Gen 1 21 colls, 0 par 4.02s 14.98s 0.7131s 7.6060s
INIT time 0.00s ( 0.00s elapsed)
MUT time 0.90s ( 6.22s elapsed)
GC time 2.74s ( 11.04s elapsed)
RP time 0.00s ( 0.00s elapsed)
PROF time 1.65s ( 4.35s elapsed)
EXIT time 0.04s ( 0.05s elapsed)
Total time 5.33s ( 17.31s elapsed)
%GC time 51.4% (63.8% elapsed)
Alloc rate 2,432,664,887 bytes per MUT second
Productivity 17.6% of total user, 5.4% of total elapsed
Bryan Vicknair
[1] https://ghc.haskell.org/trac/ghc/ticket/9520#comment:4
On Thu, Aug 28, 2014 at 09:37:45AM +0300, Michael Snoyman wrote:
> Can you provide the output from +RTS -s, as well as the heap profile for
> -hy?
>
> I believe I'm now also reproducing the memory leak on conduit 1.2, so it
> must have been a mistake in my testing last night when I thought 1.2 fixed
> it.
>
>
> On Thu, Aug 28, 2014 at 8:58 AM, Bryan Vicknair <bryanvick at gmail.com> wrote:
>
> > Thanks for the interesting blog posts Michael. I updated the example
> > project
> > [1] to use conduit 1.2. Unfortunately, on my machine [2], my original
> > sink2
> > still uses about 500Mb of memory when processing 4 gzip files of about 5Mb
> > each, while sink1 only uses about 8Mb. I added sink3, which does the same
> > as
> > sink2 but uses fold from Conduit.List as you recommended, and that seems to
> > work, using about 8Mb.
> >
> > Looking at the code for sink2 vs sink3, I don't understand what would be
> > occupying so much memory in sink2 even in the case of expensive monadic
> > binding, or exclusion from stream fusion. I'm curious if sink2 adds
> > thunks to
> > the heap that sink3 doesn't, or if the GC is failing to clean up heap
> > objects
> > in sink2 that is cleans up in sink3. I'm new at memory profiling, but the
> > chart I get with '+RTS -h' or '+RTS -hr' basically just tells me that the
> > action function is expensive.
> >
> > In the real project that inspired this example I'm going to do some
> > cleanup,
> > replacing manual recursion with higher-level functions from Conduit.List,
> > as
> > that seems like an all around good idea.
> >
> >
> > Bryan Vicknair
> >
> > [1] https://bitbucket.org/bryanvick/conduit-mem
> > [2] GHC 7.8.3, Arch Linux 3.16.1 kernel x86-64
> >
> >
> > On Thu, Aug 28, 2014 at 07:00:41AM +0300, Michael Snoyman wrote:
> > <snip>
> > > But looking at the code again with fresher eyes than last night: I really
> > > don't understand why it had such abysmal performance. I'll look into
> > this a
> > > bit more, looks like it should be interesting.
> > >
> > >
> > > On Thu, Aug 28, 2014 at 1:39 AM, Dan Burton <danburton.email at gmail.com>
> > > wrote:
> > >
> > > > Michael, I don't see how your code sample for (3) is any different to
> > the
> > > > compiler than Roman's original sink2.
> > > >
> > > > I also don't see how the original sink2 creates a bad bind tree. I
> > presume
> > > > that the reason "fold" works is due to the streaming optimization
> > rule, and
> > > > not due to its implementation, which looks almost identical to (3).
> > > >
> > > > I worry about using fold in this case, which is only strict up to WHNF,
> > > > and therefore wouldn't necessarily force the integers in the tuples;
> > > > instead it would create tons of integer thunks, wouldn't it? Roman's
> > > > hand-coded sink2 avoids this issue so I presume that's not what is
> > causing
> > > > his memory woes.
> > > >
> > > > -- Dan Burton
> > > >
> > > >
> > > > On Wed, Aug 27, 2014 at 2:55 PM, Roman Cheplyaka <roma at ro-che.info>
> > wrote:
> > > >
> > > >> * Michael Snoyman <michael at snoyman.com> [2014-08-27 23:48:06+0300]
> > > >> > > The problem is the following Sink, which counts how many even/odd
> > > >> Tokens
> > > >> > > are
> > > >> > > seen:
> > > >> > >
> > > >> > > type SinkState = (Integer, Integer)
> > > >> > >
> > > >> > > sink2 :: (Monad m) => SinkState -> Sink Token m SinkState
> > > >> > > sink2 state@(!evenCount, !oddCount) = do
> > > >> > > maybeToken <- await
> > > >> > > case maybeToken of
> > > >> > > Nothing -> return state
> > > >> > > (Just Even) -> sink2 (evenCount + 1, oddCount )
> > > >> > > (Just Odd ) -> sink2 (evenCount , oddCount + 1)
> > > >> >
> > > >> > Wow, talk about timing! What you've run into here is expensive
> > monadic
> > > >> > bindings. As it turns out, this is exactly what my blog post from
> > last
> > > >> > week[1] covered. You have three options to fix this:
> > > >> >
> > > >> > 1. Just upgrade to conduit 1.2.0, which I released a few hours ago,
> > and
> > > >> > uses the codensity transform to avoid the problem. (I just tested
> > your
> > > >> > code; you get constant memory usage under conduit 1.2.0, seemingly
> > > >> without
> > > >> > any code change necessary.)
> >
-------------- next part --------------
JOB "mem /tmp/tcmsdb/src/471ecbb8d3e679fe0d2caf945edcd3f1.gz /tmp/tcmsdb/src/4bf2525ad7483b1938228ae25835beee.gz /tmp/tcmsdb/src/952afd64c15d57903dfa6958c382641d.gz /tmp/tcmsdb/src/980dfea7c997644e5bd256f7d4fea1b5.gz +RTS -p -s -hy"
DATE "Thu Aug 28 11:02 2014"
SAMPLE_UNIT "seconds"
VALUE_UNIT "bytes"
BEGIN_SAMPLE 0.00
END_SAMPLE 0.00
BEGIN_SAMPLE 0.08
IO 80
ThreadId 16
MonadIO 24
MonadBase 48
Pipe 16
NewlineMode 24
Bool 24
IO 32
BufferMode 24
FD 24
->IO 16
->(,) 24
->ResourceT 24
MonadIO 24
stg_sel_upd 48
stg_ap_2_upd_info 32
->ConduitM 40
->>ResourceT 16
* 32
* 24
stg_sel_upd 48
->* 24
->>* 24
->>ResourceT 16
* 24
->ResourceT 16
ConduitM 32
Monad 40
->* 16
->->Pipe 16
->(#,#) 16
->>IO 24
->IO 24
->Pipe 72
Pipe 32
* 24
->>* 24
* 24
* 24
->>>Pipe 16
Pipe 40
->IO 32
->>(#,#) 16
STRef 16
Pipe 32
* 40
->>>* 32
->>>* 32
->>>Pipe 24
->ConduitM 24
->>>>>ConduitM 24
* 24
Handle__ 272
STArray 40
IO 16
->IO 16
Int32 16
IO 16
MonadResource 48
* 24
Finalizers 16
->>* 112
->* 24
->* 64
ForeignPtrContents 16
* 32
->IO 16
Dynamic 24
->[] 96
->>IO 16
BufferState 24
[] 192
BufferCodec 48
BufferList 24
Buffer 280
Pipe 32
->Pipe 32
->Pipe 16
->Pipe 32
->Pipe 32
->ResourceT 16
->(#,#) 16
->* 16
->* 40
->Pipe 24
Pipe 24
IO 32
IO 32
->Pipe 16
->Pipe 16
TextEncoding 64
->Pipe 16
->ResourceT 16
->* 24
->* 32
PAP 168
->ConduitM 16
->IO 16
ForeignPtr 96
->Pipe 16
->>>Pipe 16
->* 80
Popper 40
Monad 24
->Pipe 16
->>>Pipe 16
Obj 16
ForeignPtrContents 144
Maybe 64
MUT_VAR_CLEAN 288
MUT_ARR_PTRS_CLEAN 552
MVAR 96
C_FINALIZER_LIST 48
[] 4512
Handle 48
MVar 16
WEAK 192
ConduitM 32
Pipe 4431120
Integer 2954032
* 32
(,) 4431216
Sink 4504352
Sink 1403776
Pipe 64
Pipe 5908064
Pipe 4504352
Pipe 1403776
->Pipe 32
->Pipe 2954048
->Pipe 4431072
Pipe 24
* 24
->Pipe 80
->Pipe 80
BLACKHOLE 48
ForeignPtrContents 32
ByteString 120
ARR_WORDS 163712
END_SAMPLE 0.08
BEGIN_SAMPLE 0.17
IO 80
ThreadId 16
MonadIO 24
MonadBase 48
NewlineMode 24
Bool 24
IO 32
BufferMode 24
FD 24
->IO 16
->(,) 24
->ResourceT 24
MonadIO 24
stg_sel_upd 48
stg_ap_2_upd_info 32
->ConduitM 40
->>ResourceT 16
* 32
* 24
stg_sel_upd 48
->* 24
->>* 24
->>ResourceT 16
* 24
->ResourceT 16
ConduitM 32
Monad 40
->* 16
->->Pipe 16
->(#,#) 16
->>IO 24
->IO 24
->Pipe 72
Pipe 32
* 24
->>* 24
* 24
* 24
->>>Pipe 16
->>>Pipe 16
Pipe 40
->IO 32
->>(#,#) 16
STRef 16
Pipe 32
* 40
->>>* 32
->>>* 32
->>>Pipe 24
* 24
->>>Pipe 16
Handle__ 272
STArray 40
IO 16
->IO 16
Int32 16
IO 16
MonadResource 48
* 24
Finalizers 16
->>* 112
->* 24
->* 64
->Pipe 16
ByteString 40
ForeignPtrContents 16
* 32
->IO 16
Dynamic 24
->[] 96
->>IO 16
BufferState 24
[] 192
BufferCodec 48
BufferList 24
Buffer 280
ConduitM 32
->* 40
Pipe 32
->Pipe 24
Pipe 24
->Pipe 32
->Pipe 16
->Pipe 40
->Pipe 40
->Pipe 16
->Pipe 16
->Pipe 32
->Pipe 32
->ResourceT 16
IO 32
IO 32
TextEncoding 64
->Pipe 16
->ResourceT 16
->* 24
->* 32
PAP 168
->ConduitM 16
->IO 16
ForeignPtr 96
->ConduitM 24
->* 80
Popper 40
->>>>>ConduitM 24
Monad 24
->Pipe 16
Obj 16
ForeignPtrContents 144
Maybe 64
MUT_VAR_CLEAN 288
MUT_ARR_PTRS_CLEAN 552
MVAR 96
C_FINALIZER_LIST 48
[] 4512
Handle 48
MVar 16
WEAK 192
->(#,#) 16
->* 16
Pipe 16
Pipe 2444704
->Pipe 8413152
->Pipe 5608768
Pipe 8772832
Pipe 8413200
Pipe 11217504
* 32
Pipe 64
ForeignPtrContents 32
Integer 5608768
->Pipe 32
ConduitM 56
(,) 8413320
Sink 8772864
Sink 2444736
Pipe 24
* 24
Pipe 32
Pipe 24
ResourceT 16
BLACKHOLE 32
->Pipe 32
ARR_WORDS 163712
END_SAMPLE 0.17
BEGIN_SAMPLE 0.26
ThreadId 16
IO 80
MonadIO 24
MonadBase 48
NewlineMode 24
Bool 24
BufferMode 24
FD 24
IO 32
->IO 16
->(,) 24
stg_ap_2_upd_info 32
* 24
stg_sel_upd 48
->->Pipe 16
->(#,#) 16
->>IO 24
->IO 24
* 24
->>>Pipe 16
->>>Pipe 16
Pipe 40
->IO 32
->>(#,#) 16
STRef 16
->>>* 32
->>>* 32
->>>Pipe 24
* 24
->>>Pipe 16
Handle__ 272
STArray 40
IO 16
->IO 16
Pipe 24
->Pipe 32
->Pipe 16
->Pipe 40
->Pipe 40
IO 32
IO 32
->Pipe 16
->Pipe 16
->Pipe 32
->Pipe 32
->ResourceT 16
TextEncoding 64
Int32 16
IO 16
MonadResource 48
Finalizers 16
* 24
->>* 112
->* 24
ForeignPtrContents 16
->* 64
MonadIO 24
stg_sel_upd 48
->Pipe 16
->>ResourceT 16
* 32
->Pipe 16
->ResourceT 16
->* 24
->* 32
->ConduitM 16
->ResourceT 24
->ConduitM 24
->[] 96
->>IO 16
BufferState 24
[] 192
BufferCodec 48
BufferList 24
Buffer 280
->(#,#) 16
->* 16
PAP 168
ConduitM 32
->* 40
Pipe 32
->Pipe 24
->* 24
->>* 24
->>ResourceT 16
* 24
->ResourceT 16
Monad 40
->* 16
->* 80
->ConduitM 40
->>>>>ConduitM 24
->Pipe 72
* 24
->>* 24
* 24
Monad 24
->Pipe 16
Obj 16
ForeignPtrContents 144
->IO 16
Dynamic 24
MUT_VAR_CLEAN 288
MUT_ARR_PTRS_CLEAN 552
MVAR 96
C_FINALIZER_LIST 48
[] 4512
Handle 48
MVar 16
WEAK 192
Pipe 16
Pipe 3417024
->Pipe 12359376
->Pipe 8239584
Pipe 13062144
Pipe 12359424
Pipe 16479136
* 32
ConduitM 32
Pipe 64
Integer 8239584
Pipe 32
->Pipe 32
ConduitM 56
(,) 12359544
Sink 13062176
Sink 3417056
Pipe 32
* 40
Pipe 24
* 24
Pipe 32
Pipe 24
ResourceT 16
BLACKHOLE 32
->Pipe 32
ForeignPtrContents 32
ByteString 40
->IO 16
Maybe 64
ForeignPtr 96
* 32
Popper 40
ARR_WORDS 163712
END_SAMPLE 0.26
BEGIN_SAMPLE 0.35
IO 80
ThreadId 16
MonadIO 24
MonadBase 48
NewlineMode 24
Bool 24
IO 32
BufferMode 24
FD 24
->IO 16
->(,) 24
->ResourceT 24
MonadIO 24
stg_sel_upd 48
stg_ap_2_upd_info 32
->ConduitM 40
->>ResourceT 16
* 32
* 24
stg_sel_upd 48
->* 24
->>* 24
->>ResourceT 16
* 24
->ResourceT 16
ConduitM 32
Monad 40
->* 16
->->Pipe 16
->(#,#) 16
->>IO 24
->IO 24
->Pipe 72
Pipe 32
* 24
->>* 24
* 24
* 24
->>>Pipe 16
->>>Pipe 16
Pipe 40
->IO 32
->>(#,#) 16
STRef 16
Pipe 32
* 40
->>>* 32
->>>* 32
->>>Pipe 24
* 24
->>>Pipe 16
Handle__ 272
STArray 40
IO 16
->IO 16
Int32 16
IO 16
MonadResource 48
* 24
Finalizers 16
->>* 112
->* 24
->* 64
->Pipe 16
ByteString 40
ForeignPtrContents 16
* 32
->IO 16
Dynamic 24
->[] 96
->>IO 16
BufferState 24
[] 192
BufferCodec 48
BufferList 24
Buffer 280
ConduitM 32
->* 40
Pipe 32
->Pipe 24
Pipe 24
->Pipe 32
->Pipe 16
->Pipe 40
->Pipe 40
->Pipe 16
->Pipe 16
->Pipe 32
->Pipe 32
->ResourceT 16
IO 32
IO 32
TextEncoding 64
->Pipe 16
->ResourceT 16
->* 24
->* 32
PAP 168
->ConduitM 16
->IO 16
ForeignPtr 96
->ConduitM 24
->* 80
Popper 40
->>>>>ConduitM 24
Monad 24
->Pipe 16
Obj 16
ForeignPtrContents 144
Maybe 64
MUT_VAR_CLEAN 288
MUT_ARR_PTRS_CLEAN 552
MVAR 96
C_FINALIZER_LIST 48
[] 4512
Handle 48
MVar 16
WEAK 192
->(#,#) 16
->* 16
Pipe 16
Pipe 4371968
->Pipe 16011120
->Pipe 10674080
Pipe 16976192
Pipe 16011168
Pipe 21348128
* 32
Pipe 64
ForeignPtrContents 32
Integer 10674080
->Pipe 32
ConduitM 56
(,) 16011288
Sink 16976224
Sink 4372000
Pipe 24
* 24
Pipe 32
Pipe 24
ResourceT 16
BLACKHOLE 32
->Pipe 32
ARR_WORDS 163712
END_SAMPLE 0.35
BEGIN_SAMPLE 0.42
ThreadId 16
IO 80
MonadIO 24
MonadBase 48
NewlineMode 24
Bool 24
IO 32
BufferMode 24
FD 24
->IO 16
->(,) 24
->ResourceT 24
MonadIO 24
stg_sel_upd 48
stg_ap_2_upd_info 32
->ConduitM 40
->>ResourceT 16
* 32
* 24
stg_sel_upd 48
->* 24
->>* 24
->>ResourceT 16
* 24
->ResourceT 16
ConduitM 32
Monad 40
->* 16
->->Pipe 16
->(#,#) 16
->>IO 24
->IO 24
->Pipe 72
Pipe 32
* 24
->>* 24
* 24
* 24
->>>Pipe 16
->>>Pipe 16
Pipe 40
->IO 32
->>(#,#) 16
STRef 16
Pipe 32
* 40
->>>* 32
->>>* 32
->>>Pipe 24
* 24
->>>Pipe 16
Handle__ 272
STArray 40
IO 16
->IO 16
Int32 16
IO 16
MonadResource 48
* 24
Finalizers 16
->>* 112
->* 24
->* 64
->Pipe 16
ByteString 40
ForeignPtrContents 16
* 32
->IO 16
Dynamic 24
->[] 96
->>IO 16
BufferState 24
[] 192
BufferCodec 48
BufferList 24
Buffer 280
ConduitM 32
->* 40
Pipe 32
->Pipe 24
Pipe 24
->Pipe 32
->Pipe 16
->Pipe 40
->Pipe 40
->Pipe 16
->Pipe 16
->Pipe 32
->Pipe 32
->ResourceT 16
IO 32
IO 32
TextEncoding 64
->Pipe 16
->ResourceT 16
->* 24
->* 32
PAP 168
->ConduitM 16
->IO 16
ForeignPtr 96
->ConduitM 24
->* 80
Popper 40
->>>>>ConduitM 24
Monad 24
->Pipe 16
Obj 16
ForeignPtrContents 144
Maybe 64
MUT_VAR_CLEAN 288
MUT_ARR_PTRS_CLEAN 552
MVAR 96
C_FINALIZER_LIST 48
[] 4512
Handle 48
MVar 16
WEAK 192
->(#,#) 16
->* 16
Pipe 16
Pipe 5284064
->Pipe 19532640
->Pipe 13021760
Pipe 20759456
Pipe 19532688
Pipe 26043488
* 32
Pipe 64
ForeignPtrContents 32
Integer 13021760
->Pipe 32
ConduitM 56
(,) 19532808
Sink 20759488
Sink 5284096
Pipe 24
* 24
Pipe 32
Pipe 24
ResourceT 16
BLACKHOLE 32
->Pipe 32
ARR_WORDS 163712
END_SAMPLE 0.42
BEGIN_SAMPLE 0.50
IO 80
ThreadId 16
MonadIO 24
MonadBase 48
NewlineMode 24
Bool 24
IO 32
BufferMode 24
FD 24
->IO 16
->(,) 24
->ResourceT 24
MonadIO 24
stg_sel_upd 48
stg_ap_2_upd_info 32
->ConduitM 40
->>ResourceT 16
* 32
* 24
stg_sel_upd 48
->* 24
->>* 24
->>ResourceT 16
* 24
->ResourceT 16
ConduitM 32
Monad 40
->* 16
->->Pipe 16
->(#,#) 16
->>IO 24
->IO 24
->Pipe 72
Pipe 32
* 24
->>* 24
* 24
* 24
->>>Pipe 16
Pipe 40
->IO 32
->>(#,#) 16
STRef 16
Pipe 32
* 40
->>>* 32
->>>* 32
->>>Pipe 24
->ConduitM 24
->>>>>ConduitM 24
* 24
Handle__ 272
STArray 40
IO 16
->IO 16
Int32 16
IO 16
MonadResource 48
* 24
Finalizers 16
->>* 112
->* 24
->* 64
ForeignPtrContents 16
* 32
->IO 16
Dynamic 24
->[] 96
->>IO 16
BufferState 24
[] 192
BufferCodec 48
BufferList 24
Buffer 280
Pipe 32
->Pipe 32
->Pipe 16
->Pipe 32
->Pipe 32
->ResourceT 16
->(#,#) 16
->* 16
->* 40
->Pipe 24
Pipe 24
IO 32
IO 32
->Pipe 16
->Pipe 16
TextEncoding 64
->Pipe 16
->ResourceT 16
->* 24
->* 32
PAP 168
->ConduitM 16
->IO 16
ForeignPtr 96
->Pipe 16
->>>Pipe 16
->* 80
Popper 40
Monad 24
->Pipe 16
->>>Pipe 16
Obj 16
ForeignPtrContents 144
Maybe 64
MUT_VAR_CLEAN 288
MUT_ARR_PTRS_CLEAN 552
MVAR 96
C_FINALIZER_LIST 48
[] 4512
Handle 48
MVar 16
WEAK 192
ConduitM 32
Pipe 16
Pipe 22922448
Integer 15281584
* 32
(,) 22922544
Sink 24390560
Sink 6172672
Pipe 64
Pipe 30563168
Pipe 24390560
Pipe 6172672
->Pipe 32
->Pipe 15281600
->Pipe 22922400
Pipe 24
* 24
->Pipe 80
->Pipe 80
BLACKHOLE 48
ForeignPtrContents 32
ByteString 120
ARR_WORDS 163712
END_SAMPLE 0.50
BEGIN_SAMPLE 0.57
ThreadId 16
IO 80
MonadIO 24
MonadBase 48
NewlineMode 24
Bool 24
IO 32
BufferMode 24
FD 24
->IO 16
->(,) 24
->ResourceT 24
MonadIO 24
stg_sel_upd 48
stg_ap_2_upd_info 32
->ConduitM 40
->>ResourceT 16
* 32
* 24
stg_sel_upd 48
->* 24
->>* 24
->>ResourceT 16
* 24
->ResourceT 16
ConduitM 32
Monad 40
->* 16
->->Pipe 16
->(#,#) 16
->>IO 24
->IO 24
->Pipe 72
Pipe 32
* 24
->>* 24
* 24
* 24
->>>Pipe 16
Pipe 40
->IO 32
->>(#,#) 16
STRef 16
Pipe 32
* 40
->>>* 32
->>>* 32
->>>Pipe 24
->ConduitM 24
->>>>>ConduitM 24
* 24
Handle__ 272
STArray 40
IO 16
->IO 16
Int32 16
IO 16
MonadResource 48
* 24
Finalizers 16
->>* 112
->* 24
->* 64
ForeignPtrContents 16
* 32
->IO 16
Dynamic 24
->[] 96
->>IO 16
BufferState 24
[] 192
BufferCodec 48
BufferList 24
Buffer 280
Pipe 32
->Pipe 32
->Pipe 16
->Pipe 32
->Pipe 32
->ResourceT 16
->(#,#) 16
->* 16
->* 40
->Pipe 24
Pipe 24
IO 32
IO 32
->Pipe 16
->Pipe 16
TextEncoding 64
->Pipe 16
->ResourceT 16
->* 24
->* 32
PAP 168
->ConduitM 16
->IO 16
ForeignPtr 96
->Pipe 16
->>>Pipe 16
->* 80
Popper 40
Monad 24
->Pipe 16
->>>Pipe 16
Obj 16
ForeignPtrContents 144
Maybe 64
MUT_VAR_CLEAN 288
MUT_ARR_PTRS_CLEAN 552
MVAR 96
C_FINALIZER_LIST 48
[] 4512
Handle 48
MVar 16
WEAK 192
ConduitM 32
Pipe 16
Pipe 26276616
Integer 17517696
* 32
(,) 26276712
Sink 28008064
Sink 7027392
Pipe 64
Pipe 35035392
Pipe 28008064
Pipe 7027392
->Pipe 32
->Pipe 17517712
->Pipe 26276568
Pipe 24
* 24
->Pipe 80
->Pipe 80
BLACKHOLE 48
ForeignPtrContents 32
ByteString 120
ARR_WORDS 163712
END_SAMPLE 0.57
BEGIN_SAMPLE 0.65
IO 80
ThreadId 16
MonadIO 24
MonadBase 48
NewlineMode 24
Bool 24
IO 32
BufferMode 24
FD 24
->IO 16
->(,) 24
->ResourceT 24
MonadIO 24
stg_sel_upd 48
stg_ap_2_upd_info 32
->ConduitM 40
->>ResourceT 16
* 32
* 24
stg_sel_upd 48
->* 24
->>* 24
->>ResourceT 16
* 24
->ResourceT 16
ConduitM 32
Monad 40
->* 16
->->Pipe 16
->(#,#) 16
->>IO 24
->IO 24
->Pipe 72
Pipe 32
* 24
->>* 24
* 24
* 24
->>>Pipe 16
->>>Pipe 16
Pipe 40
->IO 32
->>(#,#) 16
STRef 16
Pipe 32
* 40
->>>* 32
->>>* 32
->>>Pipe 24
* 24
->>>Pipe 16
Handle__ 272
STArray 40
IO 16
->IO 16
Int32 16
IO 16
MonadResource 48
* 24
Finalizers 16
->>* 112
->* 24
->* 64
->Pipe 16
ByteString 40
ForeignPtrContents 16
* 32
->IO 16
Dynamic 24
->[] 96
->>IO 16
BufferState 24
[] 192
BufferCodec 48
BufferList 24
Buffer 280
ConduitM 32
->* 40
Pipe 32
->Pipe 24
Pipe 24
->Pipe 32
->Pipe 16
->Pipe 40
->Pipe 40
->Pipe 16
->Pipe 16
->Pipe 32
->Pipe 32
->ResourceT 16
IO 32
IO 32
TextEncoding 64
->Pipe 16
->ResourceT 16
->* 24
->* 32
PAP 168
->ConduitM 16
->IO 16
ForeignPtr 96
->ConduitM 24
->* 80
Popper 40
->>>>>ConduitM 24
Monad 24
->Pipe 16
Obj 16
ForeignPtrContents 144
Maybe 64
MUT_VAR_CLEAN 288
MUT_ARR_PTRS_CLEAN 552
MVAR 96
C_FINALIZER_LIST 48
[] 4512
Handle 48
MVar 16
WEAK 192
->(#,#) 16
->* 16
Pipe 16
Pipe 7879360
->Pipe 29587296
->Pipe 19724864
Pipe 31570368
Pipe 29587344
Pipe 39449696
* 32
Pipe 64
ForeignPtrContents 32
Integer 19724864
->Pipe 32
ConduitM 56
(,) 29587464
Sink 31570400
Sink 7879392
Pipe 24
* 24
Pipe 32
Pipe 24
ResourceT 16
BLACKHOLE 32
->Pipe 32
ARR_WORDS 163712
END_SAMPLE 0.65
BEGIN_SAMPLE 0.74
IO 32
->IO 16
->(,) 24
stg_ap_2_upd_info 32
->Pipe 16
* 24
->Pipe 16
->->Pipe 16
->(#,#) 16
->>IO 24
->IO 24
* 24
->>>Pipe 16
->Pipe 16
Pipe 40
->IO 32
->>(#,#) 16
STRef 16
->>>* 32
->>>* 32
->>>Pipe 24
* 24
->>>Pipe 16
->>>Pipe 16
BufferState 24
[] 192
BufferCodec 48
BufferList 24
STArray 40
IO 16
->IO 16
->ResourceT 16
->* 24
->* 32
->ConduitM 16
ForeignPtrContents 16
->* 64
->* 80
MonadIO 24
->>ResourceT 16
* 32
->ResourceT 24
->ConduitM 24
->* 24
->>* 24
->>ResourceT 16
* 24
->ResourceT 16
Monad 40
->* 16
->ConduitM 40
->>>>>ConduitM 24
->Pipe 72
* 24
->>* 24
* 24
Monad 24
Obj 16
->IO 16
Dynamic 24
->[] 96
->>IO 16
MUT_ARR_PTRS_CLEAN 552
C_FINALIZER_LIST 96
Handle 48
MVar 16
DEAD_WEAK 96
WEAK 192
->(#,#) 16
->Pipe 40
->Pipe 16
->Pipe 16
->Pipe 32
->Pipe 32
->ResourceT 16
Int32 16
IO 16
MonadResource 48
Finalizers 16
* 24
->>* 112
->* 24
->* 16
PAP 168
ConduitM 32
->* 40
Pipe 32
->Pipe 24
Pipe 24
->Pipe 32
->Pipe 16
->Pipe 40
ForeignPtrContents 168
Buffer 336
NewlineMode 48
Bool 48
stg_sel_upd 72
stg_sel_upd 72
BufferMode 48
MUT_VAR_CLEAN 368
FD 48
Handle__ 408
MVAR 128
[] 4488
ThreadId 16
IO 120
IO 32
IO 32
MonadIO 24
MonadBase 48
TextEncoding 64
Pipe 32
Pipe 33667488
Integer 22444944
ConduitM 32
(,) 33667608
Sink 35482464
Sink 9407488
Pipe 32
->Pipe 32
ConduitM 56
Pipe 44889856
Pipe 35482464
Pipe 9407488
Pipe 32
* 40
BLACKHOLE 16
Pipe 24
* 24
Pipe 32
->Pipe 22444960
->Pipe 33667440
ForeignPtrContents 32
ByteString 40
->IO 16
Maybe 64
ForeignPtr 96
* 32
Popper 40
Pipe 64
* 32
ARR_WORDS 171888
END_SAMPLE 0.74
BEGIN_SAMPLE 0.81
IO 80
ThreadId 16
MonadIO 24
MonadBase 48
NewlineMode 24
Bool 24
IO 32
BufferMode 24
FD 24
->IO 16
->(,) 24
->ResourceT 24
MonadIO 24
stg_sel_upd 48
stg_ap_2_upd_info 32
->ConduitM 40
->>ResourceT 16
* 32
* 24
stg_sel_upd 48
->* 24
->>* 24
->>ResourceT 16
* 24
->ResourceT 16
ConduitM 32
Monad 40
->* 16
->->Pipe 16
->(#,#) 16
->>IO 24
->IO 24
->Pipe 72
Pipe 32
* 24
->>* 24
* 24
* 24
->>>Pipe 16
Pipe 40
->IO 32
->>(#,#) 16
STRef 16
Pipe 32
* 40
->>>* 32
->>>* 32
->>>Pipe 24
BufferCodec 48
BufferList 24
Handle__ 272
STArray 40
IO 16
->IO 16
ConduitM 32
->* 40
Pipe 32
->Pipe 24
Pipe 24
->Pipe 32
->Pipe 16
IO 32
IO 32
->Pipe 16
->Pipe 16
->Pipe 32
->Pipe 32
->ResourceT 16
TextEncoding 64
Int32 16
IO 16
MonadResource 48
* 24
Finalizers 16
->>* 112
->* 24
->* 64
ForeignPtrContents 16
MUT_ARR_PTRS_CLEAN 552
MVAR 96
C_FINALIZER_LIST 48
[] 3264
Handle 48
MVar 16
WEAK 192
->(#,#) 16
->* 16
* 32
->Pipe 16
->ResourceT 16
->* 24
->* 32
PAP 168
->ConduitM 16
->IO 16
Maybe 64
->ConduitM 24
->Pipe 16
->>>Pipe 16
->* 80
Popper 40
->>>>>ConduitM 24
Monad 24
->Pipe 16
->>>Pipe 16
* 24
Obj 16
MUT_VAR_CLEAN 288
->IO 16
Dynamic 24
->>IO 16
ForeignPtr 96
Buffer 280
BufferState 24
ForeignPtrContents 144
->[] 96
[] 192
Pipe 32
Pipe 36667848
* 32
(,) 36667944
Sink 37403072
Sink 11487360
Pipe 64
Pipe 48890336
Pipe 37403072
Pipe 11487360
ByteString 80
ForeignPtrContents 32
->Pipe 32
->Pipe 24445200
->Pipe 36667800
ByteString 32
ConduitM 56
Pipe 24
* 24
->Pipe 80
->Pipe 80
BLACKHOLE 32
->Pipe 32
Integer 24445184
ARR_WORDS 163712
END_SAMPLE 0.81
BEGIN_SAMPLE 0.87
ThreadId 16
IO 80
NewlineMode 24
Bool 24
IO 32
BufferMode 24
FD 24
->IO 16
->(,) 24
->ResourceT 24
MonadIO 24
stg_sel_upd 48
stg_ap_2_upd_info 32
->ConduitM 40
->>ResourceT 16
* 32
* 24
stg_sel_upd 48
->* 24
->>* 24
->>ResourceT 16
* 24
->ResourceT 16
ConduitM 32
Monad 40
->* 16
->->Pipe 16
->(#,#) 16
->>IO 24
->IO 24
->Pipe 72
Pipe 32
* 24
->>* 24
* 24
* 24
->>>Pipe 16
Pipe 40
->IO 32
->>(#,#) 16
STRef 16
Pipe 32
* 40
->>>* 32
->>>* 32
->>>Pipe 24
BufferCodec 48
BufferList 24
Handle__ 272
STArray 40
IO 16
->IO 16
ConduitM 32
->* 40
Pipe 32
->Pipe 24
Pipe 24
->Pipe 32
->Pipe 16
IO 32
->Pipe 16
->Pipe 16
->Pipe 32
->Pipe 32
->ResourceT 16
Int32 16
IO 16
MonadResource 48
* 24
Finalizers 16
->>* 112
->* 24
->* 64
ForeignPtrContents 16
MUT_ARR_PTRS_CLEAN 552
MVAR 96
C_FINALIZER_LIST 48
[] 3264
Handle 48
MVar 16
WEAK 192
->(#,#) 16
->* 16
* 32
->Pipe 16
->ResourceT 16
->* 24
->* 32
PAP 168
->ConduitM 16
->IO 16
Maybe 64
->ConduitM 24
->Pipe 16
->>>Pipe 16
->* 80
Popper 40
->>>>>ConduitM 24
Monad 24
->Pipe 16
->>>Pipe 16
* 24
Obj 16
MUT_VAR_CLEAN 288
->IO 16
Dynamic 24
->>IO 16
ForeignPtr 96
Buffer 280
BufferState 24
ForeignPtrContents 144
->[] 96
[] 192
IO 32
MonadIO 24
MonadBase 48
TextEncoding 64
Pipe 32
Pipe 38681160
* 32
(,) 38681256
Sink 38597856
Sink 12976992
Pipe 64
Pipe 51574752
Pipe 38597856
Pipe 12976992
ByteString 80
ForeignPtrContents 32
->Pipe 32
->Pipe 25787408
->Pipe 38681112
ByteString 32
ConduitM 56
Pipe 24
* 24
->Pipe 80
->Pipe 80
BLACKHOLE 32
->Pipe 32
Integer 25787392
ARR_WORDS 163712
END_SAMPLE 0.87
BEGIN_SAMPLE 0.94
END_SAMPLE 0.94
-------------- next part --------------
Thu Aug 28 11:02 2014 Time and Allocation Profiling Report (Final)
mem +RTS -p -s -hy -RTS /tmp/tcmsdb/src/471ecbb8d3e679fe0d2caf945edcd3f1.gz /tmp/tcmsdb/src/4bf2525ad7483b1938228ae25835beee.gz /tmp/tcmsdb/src/952afd64c15d57903dfa6958c382641d.gz /tmp/tcmsdb/src/980dfea7c997644e5bd256f7d4fea1b5.gz
total time = 1.10 secs (1096 ticks @ 1000 us, 1 processor)
total alloc = 1,510,912,856 bytes (excludes profiling overheads)
COST CENTRE MODULE %time %alloc
action Main 88.6 57.3
sink2 Main 6.7 34.4
token Main 2.9 8.4
token.tokenize Main 1.8 0.0
individual inherited
COST CENTRE MODULE no. entries %time %alloc %time %alloc
MAIN MAIN 140 0 0.0 0.0 100.0 100.0
main Main 281 0 0.0 0.0 93.5 65.6
action Main 291 4 88.6 57.3 93.5 65.6
action.sink Main 311 0 0.0 0.0 0.2 0.0
sink2 Main 312 0 0.2 0.0 0.2 0.0
token Main 309 0 2.9 8.4 4.7 8.4
token.tokenize Main 310 1753770 1.8 0.0 1.8 0.0
main.tryArgs Main 282 1 0.0 0.0 0.0 0.0
parseArgs Main 283 1 0.0 0.0 0.0 0.0
parseArgs.parse.ext Main 288 4 0.0 0.0 0.0 0.0
parseArgs.parse Main 286 4 0.0 0.0 0.0 0.0
parseArgs.parse.(...) Main 287 4 0.0 0.0 0.0 0.0
CAF:main1 Main 278 0 0.0 0.0 0.0 0.0
main Main 280 1 0.0 0.0 0.0 0.0
CAF:parseArgs1 Main 276 0 0.0 0.0 0.0 0.0
parseArgs Main 284 0 0.0 0.0 0.0 0.0
CAF:parseArgs2 Main 275 0 0.0 0.0 0.0 0.0
parseArgs Main 285 0 0.0 0.0 0.0 0.0
CAF:lvl11_r8nQ Main 274 0 0.0 0.0 0.0 0.0
parseArgs Main 289 0 0.0 0.0 0.0 0.0
parseArgs.parse Main 290 0 0.0 0.0 0.0 0.0
CAF:lvl9_r8nO Main 273 0 0.0 0.0 0.0 0.0
action Main 292 0 0.0 0.0 0.0 0.0
CAF:lvl8_r8nN Main 272 0 0.0 0.0 0.0 0.0
action Main 293 0 0.0 0.0 0.0 0.0
CAF:lvl7_r8nM Main 271 0 0.0 0.0 0.0 0.0
action Main 305 0 0.0 0.0 0.0 0.0
CAF:lvl6_r8nJ Main 270 0 0.0 0.0 0.0 0.0
action Main 307 0 0.0 0.0 0.0 0.0
CAF:lvl5_r8nI Main 269 0 0.0 0.0 3.3 18.6
action Main 294 0 0.0 0.0 3.3 18.6
action.sink Main 300 0 0.0 0.0 3.3 18.6
sink2 Main 301 0 3.3 18.6 3.3 18.6
CAF:lvl4_r8nH Main 268 0 0.0 0.0 0.0 0.0
action Main 302 0 0.0 0.0 0.0 0.0
token Main 304 0 0.0 0.0 0.0 0.0
CAF:ds1_r8nG Main 267 0 0.0 0.0 0.0 0.0
action Main 295 0 0.0 0.0 0.0 0.0
action.sink Main 296 1 0.0 0.0 0.0 0.0
CAF:lvl3_r8nF Main 266 0 0.0 0.0 3.2 15.8
action Main 297 0 0.0 0.0 3.2 15.8
action.sink Main 298 0 0.0 0.0 3.2 15.8
sink2 Main 299 1753746 3.2 15.8 3.2 15.8
CAF:lvl2_r8nE Main 265 0 0.0 0.0 0.0 0.0
action Main 306 0 0.0 0.0 0.0 0.0
CAF:lvl1_r8nD Main 264 0 0.0 0.0 0.0 0.0
action Main 308 0 0.0 0.0 0.0 0.0
CAF:action6 Main 263 0 0.0 0.0 0.0 0.0
CAF:action11 Main 262 0 0.0 0.0 0.0 0.0
CAF:action7 Main 260 0 0.0 0.0 0.0 0.0
CAF:token1 Main 251 0 0.0 0.0 0.0 0.0
token Main 303 1 0.0 0.0 0.0 0.0
CAF:action8 Main 249 0 0.0 0.0 0.0 0.0
CAF GHC.Conc.Signal 183 0 0.0 0.0 0.0 0.0
CAF GHC.IO.Encoding 178 0 0.0 0.0 0.0 0.0
CAF GHC.IO.Encoding.Iconv 177 0 0.0 0.0 0.0 0.0
CAF GHC.IO.Handle.FD 175 0 0.0 0.0 0.0 0.0
CAF GHC.IO.FD 160 0 0.0 0.0 0.0 0.0
More information about the Haskell-Cafe
mailing list