[Haskell-cafe] Help with profiling

[Carter Schonwald]

you may want to try out using the Strict variant of IntMap
http://hackage.haskell.org/package/containers-0.5.3.1/docs/Data-IntMap-Strict.html

I've definitely had some scary space usage explosions with the lazy one on
occasions! (one embarrassing instance nearly two years ago was a 40gb space
blowup that turned into 100mb usage once i switched to a strict map)


On Thu, Dec 5, 2013 at 9:00 PM, Carter Schonwald <carter.schonwald at gmail.com
> wrote:

> Hey Uri,
> could you explain what those numbers represent? many of use the profiling
> tools, but we may not remember what columns / numbers mean what. What
> number seems wrong and what does the number mean? Also its very hard to
> help debug if you don't link to the code
>
> If you're hitting space leak problems, one culprit could be your data
> structure or the operations acting on it is too lazy! Have you tried making
> it stricter?
>
> cheers
>  -Carter
>
>
> On Thu, Dec 5, 2013 at 8:13 PM, Uri Braun <uribraun at eecs.harvard.edu>wrote:
>
>>  Hi Café,
>>
>> I'm attempting to profile my rather large (~18K LOC) Haskell program that
>> primarily manipulates graph structures based on Data.IntMap. The program
>> is chewing up _lots_ of memory (approx. 4G for 2.5K nodes & 2.5K edges)and I have profiled it in an attempt to figure out why. It
>> appears that the .prof and .hp (which I visualize with hp2ps -c) files
>> do not agree on the source of the problem. The hp results are located at:
>> *http://www.eecs.harvard.edu/~uribraun/sybil.ps*<http://www.eecs.harvard.edu/~uribraun/sybil.ps>and the top of the .prof file looks like this:
>>
>> COST CENTRE                   MODULE                  %time %alloc
>>
>> compare                       Policy.Edge              21.7   50.0
>>
>> eqPartition.edges             Policy.EdgeQuad           6.5    3.7
>>
>> toConstructedSet              Types.TaggedIntMultimap   6.1    3.6
>>
>> elems                         Types.TaggedIntMap        6.0    6.8
>>
>> elems                         Types.TaggedIntSet        5.7    3.8
>>
>> toPairs                       Types.TaggedIntMultimap   3.9    5.2
>>
>> assocs                        Types.TaggedIntMap        3.2    4.1
>>
>> eqIsValid                     Policy.EdgeQuad           2.9    0.5
>>
>> rIsValid                      Policy.Relations          2.2    0.2
>>
>> assocs.\                      Types.TaggedIntMap        1.8    0.0
>>
>> keysSet                       Types.TaggedIntMap        1.5    1.7
>>
>> mergeIdenticalNodesT.exactMap Minimize.MergeIdentical   1.4    1.2
>>
>> fromList                      Types.TaggedIntSet        1.3    1.6
>>
>> tePartitionSrc                Policy.TaggedEdge         1.2    0.9
>>
>> null                          Types.TaggedIntSet        1.2    0.0
>>
>> scc.comps                     Policy.Scc                1.2    0.9
>>
>> tePartitionTgt                Policy.TaggedEdge         1.1    0.8
>>
>> member                        Types.TaggedIntSet        1.1    0.8
>>
>> alter                         Types.TaggedIntMap        1.0    1.1
>>
>>  Also, the line in the .prof that corresponds to the anormUnion.name
>> (what the hprof image claims is taking up space) looks like (some spaces
>> removed):
>>
>> anormUnion.name Anormalize.Anormalize                2914          60
>> 0.0    0.0     0.0    0.0
>>
>> My main questions at this point are:
>>
>> 1.      What are the chances the hprof image is just wrong?
>>
>> 2.      What would you suggest I try next in debugging my space issue?
>>
>> This is my first big Haskell program, please do inform me of any big war
>> tools I don’t know about.
>>
>> Many thanks in advance!
>>
>> +Uri
>>
>> _______________________________________________
>> Haskell-Cafe mailing list
>> Haskell-Cafe at haskell.org
>> http://www.haskell.org/mailman/listinfo/haskell-cafe
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20131205/9275932d/attachment.html>