[Haskell-cafe] [13/16] SBM: Graphs that show the infidelity of
-sstderr
Peter Firefly Brodersen Lund
firefly at vax64.dk
Sat Dec 22 04:17:24 EST 2007
Everybody seems to use +RTS -sstderr to measure the memory performance of their
Haskell (ghc) code. That's what I started out doing, too.
However, top seemed to occasionally disagree with the run-time system.
I decided to dig deeper and ask the operating system at process exit time.
I use LD_PRELOAD to get my code injected into each benchmarked process where
it gets activated when the process is about to exit and copies /proc/self/maps
and /proc/self/status to stderr where the test harness captures them.
Unfortunately, -sstderr really *IS* unreliable!
tools/regensstderr.sh and tools/sstderr.pl are two scripts that together
generate bar graphs for an existing measurement tarball. I used the following
shell code to generate graphs for 7 different ghc/bytestring combinations:
(for F in ghc-thorough-6.6.1.tgz \
ghc-thorough-6.8.2.tgz ghc-thorough-6.8.2-bs0902.tgz \
ghc-thorough-6.9.tgz ghc-thorough-6.9-bs0902.tgz \
ghc-thorough-head.tgz ghc-thorough-head-bs0902.tgz; \
do \
printf "%s\n-------\n" $F; \
tools/regensstderr.sh $F; \
printf "\n\n"; \
done \
) > xx
The graphs clearly show that -sstderr sometimes severely underestimates the
memory use and sometimes somewhat overestimates it. Moreover, the errors are
not always for the same programs.
This holds for all ghc versions I've tested: 6.6.1, 6.8.2, 6.9.20071119, and
head as of noon 2007-12-19.
-Peter
ghc-thorough-6.6.1.tgz
-------
hs/byte-bs----acc: 1 147448 â |âââââââââââââââââââ |
hs/byte-bs----foldlx: 145 147468 âââââââââââ |âââââââââââââââââââ |
hs/byte-bs----foldrx: 1 147452 â |âââââââââââââââââââ |
hs/byte-bsl---acc: 2 2872 â |â |
hs/byte-xxxxx-acc-1: 1 1572 â |â |
hs/byte-xxxxx-acc-2: 1 1568 â |â |
hs/byte-xxxxx-foldl: 1 1572 â |â |
hs/space-bs-c8-acc-1: 1 147448 â |âââââââââââââââââââ |
hs/space-bs-c8-count: 1 147448 â |âââââââââââââââââââ |
hs/space-bs-c8-foldlx-1: 145 147468 âââââââââââ |âââââââââââââââââââ |
hs/space-bs-c8-foldlx-2: 145 147468 âââââââââââ |âââââââââââââââââââ |
hs/space-bs-c8-foldrx: 1 147448 â |âââââââââââââââââââ |
hs/space-bs-c8-lenfil: 145 147464 âââââââââââ |âââââââââââââââââââ |
hs/space-bslc8-acc-1: 2 2872 â |â |
hs/space-bslc8-acc-2: 2 2872 â |â |
hs/space-bslc8-acc-3: 2 2868 â |â |
hs/space-bslc8-count: 2 1476 â |â |
hs/space-bslc8-foldl: 2 1552 â |â |
hs/space-bslc8-foldlx-1: 2 1556 â |â |
hs/space-bslc8-foldlx-2: 2 1552 â |â |
hs/space-bslc8-foldr-1: 297 169480 âââââââââââââââââââââ|âââââââââââââââââââââ|
hs/space-bslc8-foldr-2: 297 169480 âââââââââââââââââââââ|âââââââââââââââââââââ|
hs/space-bslc8-lenfil-1: 38 38632 âââ |âââââ |
hs/space-bslc8-lenfil-2: 38 38628 âââ |âââââ |
hs/space-bsl---foldlx: 2 1560 â |â |
hs/space-xxxxx-acc-1: 1 1568 â |â |
hs/space-xxxxx-acc-2: 1 1572 â |â |
hs/space-xxxxx-foldl: 1 1572 â |â |
hs/space-xxxxx-lenfil: 1 1556 â |â |
c/byte-getchar: - -
c/byte-getchar-u: - -
c/byte-4k: - -
c/space-getchar: - -
c/space-getchar-u: - -
c/space-4k: - -
c/space-megabuf: - -
c/space-getwchar: - -
c/space-getwchar-u: - -
c/space-32k: - -
c/space-32k-8: - -
ghc-thorough-6.8.2.tgz
-------
hs/byte-bs----acc: 1 147500 â |âââââââââââââââââââ |
hs/byte-bs----foldlx: 145 147528 âââââââââââ |âââââââââââââââââââ |
hs/byte-bs----foldrx: 1 147500 â |âââââââââââââââââââ |
hs/byte-bsl---acc: 2 2920 â |â |
hs/byte-xxxxx-acc-1: 1 1632 â |â |
hs/byte-xxxxx-acc-2: 1 1636 â |â |
hs/byte-xxxxx-foldl: 1 1632 â |â |
hs/space-bs-c8-acc-1: 1 147492 â |âââââââââââââââââââ |
hs/space-bs-c8-count: 1 147500 â |âââââââââââââââââââ |
hs/space-bs-c8-foldlx-1: 145 147524 âââââââââââ |âââââââââââââââââââ |
hs/space-bs-c8-foldlx-2: 145 147520 âââââââââââ |âââââââââââââââââââ |
hs/space-bs-c8-foldrx: 1 147496 â |âââââââââââââââââââ |
hs/space-bs-c8-lenfil: 145 147524 âââââââââââ |âââââââââââââââââââ |
hs/space-bslc8-acc-1: 2 2920 â |â |
hs/space-bslc8-acc-2: 2 2924 â |â |
hs/space-bslc8-acc-3: 2 2924 â |â |
hs/space-bslc8-chunk-1: 2 1512 â |â |
hs/space-bslc8-chunk-2: 2 1512 â |â |
hs/space-bslc8-chunk-3: 2 1548 â |â |
hs/space-bslc8-chunk-4: 2 1512 â |â |
hs/space-bslc8-count: 2 1520 â |â |
hs/space-bslc8-foldl: 2 1620 â |â |
hs/space-bslc8-foldlx-1: 2 1620 â |â |
hs/space-bslc8-foldlx-2: 2 1616 â |â |
hs/space-bslc8-foldr-1: 297 169544 âââââââââââââââââââââ|âââââââââââââââââââââ|
hs/space-bslc8-foldr-2: 297 169544 âââââââââââââââââââââ|âââââââââââââââââââââ|
hs/space-bslc8-lenfil-1: 38 38644 âââ |âââââ |
hs/space-bslc8-lenfil-2: 38 38648 âââ |âââââ |
hs/space-bsl---foldlx: 2 1616 â |â |
hs/space-xxxxx-acc-1: 1 1636 â |â |
hs/space-xxxxx-acc-2: 1 1636 â |â |
hs/space-xxxxx-foldl: 1 1636 â |â |
hs/space-xxxxx-lenfil: 1 1608 â |â |
c/byte-getchar: - -
c/byte-getchar-u: - -
c/byte-4k: - -
c/space-getchar: - -
c/space-getchar-u: - -
c/space-4k: - -
c/space-megabuf: - -
c/space-getwchar: - -
c/space-getwchar-u: - -
c/space-32k: - -
c/space-32k-8: - -
ghc-thorough-6.8.2-bs0902.tgz
-------
hs/byte-bs----acc: 1 147616 â |âââââââââââââââââââ |
hs/byte-bs----foldlx: 1 147624 â |âââââââââââââââââââ |
hs/byte-bs----foldrx: 1 147624 â |âââââââââââââââââââ |
hs/byte-bsl---acc: 2 3040 â |â |
hs/byte-xxxxx-acc-1: 1 1636 â |â |
hs/byte-xxxxx-acc-2: 1 1636 â |â |
hs/byte-xxxxx-foldl: 1 1636 â |â |
hs/space-bs-c8-acc-1: 1 147608 â |âââââââââââââââââââ |
hs/space-bs-c8-count: 1 147612 â |âââââââââââââââââââ |
hs/space-bs-c8-foldlx-1: 1 147608 â |âââââââââââââââââââ |
hs/space-bs-c8-foldlx-2: 1 147608 â |âââââââââââââââââââ |
hs/space-bs-c8-foldrx: 1 147608 â |âââââââââââââââââââ |
hs/space-bs-c8-lenfil: 21 167176 âââ |âââââââââââââââââââââ|
hs/space-bslc8-acc-1: 2 3048 â |â |
hs/space-bslc8-acc-2: 2 3048 â |â |
hs/space-bslc8-acc-3: 2 3052 â |â |
hs/space-bslc8-chunk-1: 2 1636 â |â |
hs/space-bslc8-chunk-2: 2 1640 â |â |
hs/space-bslc8-chunk-3: 2 1644 â |â |
hs/space-bslc8-chunk-4: 2 1636 â |â |
hs/space-bslc8-count: 2 1656 â |â |
hs/space-bslc8-foldl: 2 1632 â |â |
hs/space-bslc8-foldlx-1: 2 1636 â |â |
hs/space-bslc8-foldlx-2: 2 1636 â |â |
hs/space-bslc8-foldr-1: 149 149076 âââââââââââââââââââââ|âââââââââââââââââââ |
hs/space-bslc8-foldr-2: 149 149076 âââââââââââââââââââââ|âââââââââââââââââââ |
hs/space-bslc8-lenfil-1: 2 1940 â |â |
hs/space-bslc8-lenfil-2: 2 1940 â |â |
hs/space-bsl---foldlx: 2 1620 â |â |
hs/space-xxxxx-acc-1: 1 1636 â |â |
hs/space-xxxxx-acc-2: 1 1636 â |â |
hs/space-xxxxx-foldl: 1 1636 â |â |
hs/space-xxxxx-lenfil: 1 1604 â |â |
c/byte-getchar: - -
c/byte-getchar-u: - -
c/byte-4k: - -
c/space-getchar: - -
c/space-getchar-u: - -
c/space-4k: - -
c/space-megabuf: - -
c/space-getwchar: - -
c/space-getwchar-u: - -
c/space-32k: - -
c/space-32k-8: - -
ghc-thorough-6.9.tgz
-------
hs/byte-bs----acc: 1 147752 â |ââââââââââââââââââ |
hs/byte-bs----foldlx: 145 147748 ââââââââââ |ââââââââââââââââââ |
hs/byte-bs----foldrx: 1 147744 â |ââââââââââââââââââ |
hs/byte-bsl---acc: 2 3172 â |â |
hs/byte-xxxxx-acc-1: 1 1904 â |â |
hs/byte-xxxxx-acc-2: 1 1904 â |â |
hs/byte-xxxxx-foldl: 1 1900 â |â |
hs/space-bs-c8-acc-1: 1 147736 â |ââââââââââââââââââ |
hs/space-bs-c8-count: 1 147744 â |ââââââââââââââââââ |
hs/space-bs-c8-foldlx-1: 145 147756 ââââââââââ |ââââââââââââââââââ |
hs/space-bs-c8-foldlx-2: 145 147752 ââââââââââ |ââââââââââââââââââ |
hs/space-bs-c8-foldrx: 1 147732 â |ââââââââââââââââââ |
hs/space-bs-c8-lenfil: 145 147756 ââââââââââ |ââââââââââââââââââ |
hs/space-bslc8-acc-1: 2 3192 â |â |
hs/space-bslc8-acc-2: 2 3192 â |â |
hs/space-bslc8-acc-3: 2 3196 â |â |
hs/space-bslc8-chunk-1: 64 65192 âââââ |ââââââââ |
hs/space-bslc8-chunk-2: 64 65192 âââââ |ââââââââ |
hs/space-bslc8-chunk-3: 74 75240 âââââ |âââââââââ |
hs/space-bslc8-chunk-4: 75 76188 âââââ |âââââââââ |
hs/space-bslc8-count: 78 79204 ââââââ |ââââââââââ |
hs/space-bslc8-foldl: 85 85868 ââââââ |âââââââââââ |
hs/space-bslc8-foldlx-1: 85 85864 ââââââ |âââââââââââ |
hs/space-bslc8-foldlx-2: 85 85868 ââââââ |âââââââââââ |
hs/space-bslc8-foldr-1: 297 169700 ââââââââââââââââââââ|ââââââââââââââââââââ|
hs/space-bslc8-foldr-2: 297 169700 ââââââââââââââââââââ|ââââââââââââââââââââ|
hs/space-bslc8-lenfil-1: 115 109404 ââââââââ |âââââââââââââ |
hs/space-bslc8-lenfil-2: 115 109404 ââââââââ |âââââââââââââ |
hs/space-bsl---foldlx: 85 85836 ââââââ |âââââââââââ |
hs/space-xxxxx-acc-1: 1 1904 â |â |
hs/space-xxxxx-acc-2: 1 1904 â |â |
hs/space-xxxxx-foldl: 1 1900 â |â |
hs/space-xxxxx-lenfil: 1 1892 â |â |
hand/byte-bs----acc-a: 1 147764 â |ââââââââââââââââââ |
hand/byte-bs----acc-b: 1 147764 â |ââââââââââââââââââ |
hand/byte-bs----acc-c: 1 147760 â |ââââââââââââââââââ |
hand/byte-bs----acc-d: 1 147760 â |ââââââââââââââââââ |
hand/space-bs-c8-acc-1-a: 1 147752 â |ââââââââââââââââââ |
hand/space-bs-c8-acc-1-b: 1 147752 â |ââââââââââââââââââ |
hand/space-bs-c8-acc-1-c: 1 147752 â |ââââââââââââââââââ |
hand/space-bs-c8-acc-1-d: 1 147756 â |ââââââââââââââââââ |
hand/space-bs-c8-acc-1-e: 1 147752 â |ââââââââââââââââââ |
hand/space-bs-c8-acc-1-f: 1 147748 â |ââââââââââââââââââ |
hand/space-bs-c8-acc-1-g: 1 147752 â |ââââââââââââââââââ |
hand/space-bs-c8-acc-1-h: 1 147748 â |ââââââââââââââââââ |
hand/space-bs-c8-acc-1-i: 1 147748 â |ââââââââââââââââââ |
hand/space-bs-c8-acc-1-j: 1 147748 â |ââââââââââââââââââ |
hand/space-bs-c8-acc-1-k: 1 147752 â |ââââââââââââââââââ |
hand/space-bs-c8-acc-1-l: 1 147752 â |ââââââââââââââââââ |
hand/space-bs-c8-acc-1-m: 1 147748 â |ââââââââââââââââââ |
hand/space-bs-c8-acc-1-n: 1 147752 â |ââââââââââââââââââ |
hand/space-bs-c8-acc-1-o: 1 147752 â |ââââââââââââââââââ |
hand/space-bs-c8-acc-1-p: 1 147752 â |ââââââââââââââââââ |
hand/space-bs-c8-acc-1-q: 1 147748 â |ââââââââââââââââââ |
hand/space-bs-c8-acc-1-r: 1 147748 â |ââââââââââââââââââ |
hand/space-bs-c8-acc-1-s: 1 147744 â |ââââââââââââââââââ |
c/byte-getchar: - -
c/byte-getchar-u: - -
c/byte-4k: - -
c/space-getchar: - -
c/space-getchar-u: - -
c/space-4k: - -
c/space-megabuf: - -
c/space-getwchar: - -
c/space-getwchar-u: - -
c/space-32k: - -
c/space-32k-8: - -
ghc-thorough-6.9-bs0902.tgz
-------
hs/byte-bs----acc: 1 147732 â |âââââââââââââââââââ |
hs/byte-bs----foldlx: 1 147736 â |âââââââââââââââââââ |
hs/byte-bs----foldrx: 1 147740 â |âââââââââââââââââââ |
hs/byte-bsl---acc: 2 3184 â |â |
hs/byte-xxxxx-acc-1: 1 1904 â |â |
hs/byte-xxxxx-acc-2: 1 1900 â |â |
hs/byte-xxxxx-foldl: 1 1904 â |â |
hs/space-bs-c8-acc-1: 1 147740 â |âââââââââââââââââââ |
hs/space-bs-c8-count: 1 147756 â |âââââââââââââââââââ |
hs/space-bs-c8-foldlx-1: 1 147744 â |âââââââââââââââââââ |
hs/space-bs-c8-foldlx-2: 1 147748 â |âââââââââââââââââââ |
hs/space-bs-c8-foldrx: 1 147748 â |âââââââââââââââââââ |
hs/space-bs-c8-lenfil: 21 167300 âââ |âââââââââââââââââââââ|
hs/space-bslc8-acc-1: 2 3228 â |â |
hs/space-bslc8-acc-2: 2 3228 â |â |
hs/space-bslc8-acc-3: 2 3232 â |â |
hs/space-bslc8-chunk-1: 65 66216 ââââââââââ |âââââââââ |
hs/space-bslc8-chunk-2: 65 66216 ââââââââââ |âââââââââ |
hs/space-bslc8-chunk-3: 65 66208 ââââââââââ |âââââââââ |
hs/space-bslc8-chunk-4: 78 79232 âââââââââââ |ââââââââââ |
hs/space-bslc8-count: 81 82248 ââââââââââââ |âââââââââââ |
hs/space-bslc8-foldl: 78 79224 âââââââââââ |ââââââââââ |
hs/space-bslc8-foldlx-1: 78 79224 âââââââââââ |ââââââââââ |
hs/space-bslc8-foldlx-2: 78 79224 âââââââââââ |ââââââââââ |
hs/space-bslc8-foldr-1: 149 149704 âââââââââââââââââââââ|âââââââââââââââââââ |
hs/space-bslc8-foldr-2: 149 149708 âââââââââââââââââââââ|âââââââââââââââââââ |
hs/space-bslc8-lenfil-1: 81 82324 ââââââââââââ |âââââââââââ |
hs/space-bslc8-lenfil-2: 81 82328 ââââââââââââ |âââââââââââ |
hs/space-bsl---foldlx: 78 79184 âââââââââââ |ââââââââââ |
hs/space-xxxxx-acc-1: 1 1904 â |â |
hs/space-xxxxx-acc-2: 1 1900 â |â |
hs/space-xxxxx-foldl: 1 1904 â |â |
hs/space-xxxxx-lenfil: 1 1892 â |â |
c/byte-getchar: - -
c/byte-getchar-u: - -
c/byte-4k: - -
c/space-getchar: - -
c/space-getchar-u: - -
c/space-4k: - -
c/space-megabuf: - -
c/space-getwchar: - -
c/space-getwchar-u: - -
c/space-32k: - -
c/space-32k-8: - -
ghc-thorough-head.tgz
-------
hs/byte-bs----acc: 1 147772 â |âââââââââââââââââââ |
hs/byte-bs----foldlx: 145 147796 âââââââââââ |âââââââââââââââââââ |
hs/byte-bs----foldrx: 1 147772 â |âââââââââââââââââââ |
hs/byte-bsl---acc: 2 3212 â |â |
hs/byte-xxxxx-acc-1: 1 1900 â |â |
hs/byte-xxxxx-acc-2: 1 1900 â |â |
hs/byte-xxxxx-foldl: 1 1900 â |â |
hs/space-bs-c8-acc-1: 1 147752 â |âââââââââââââââââââ |
hs/space-bs-c8-count: 1 147764 â |âââââââââââââââââââ |
hs/space-bs-c8-foldlx-1: 145 147796 âââââââââââ |âââââââââââââââââââ |
hs/space-bs-c8-foldlx-2: 145 147796 âââââââââââ |âââââââââââââââââââ |
hs/space-bs-c8-foldrx: 1 147760 â |âââââââââââââââââââ |
hs/space-bs-c8-lenfil: 145 147788 âââââââââââ |âââââââââââââââââââ |
hs/space-bslc8-acc-1: 2 3180 â |â |
hs/space-bslc8-acc-2: 2 3180 â |â |
hs/space-bslc8-acc-3: 2 3172 â |â |
hs/space-bslc8-chunk-1: 64 65204 âââââ |âââââââââ |
hs/space-bslc8-chunk-2: 64 65204 âââââ |âââââââââ |
hs/space-bslc8-chunk-3: 74 75216 ââââââ |ââââââââââ |
hs/space-bslc8-chunk-4: 75 76176 ââââââ |ââââââââââ |
hs/space-bslc8-count: 78 79220 ââââââ |ââââââââââ |
hs/space-bslc8-foldl: 85 85852 ââââââ |âââââââââââ |
hs/space-bslc8-foldlx-1: 85 85852 ââââââ |âââââââââââ |
hs/space-bslc8-foldlx-2: 85 85852 ââââââ |âââââââââââ |
hs/space-bslc8-foldr-1: 297 169680 âââââââââââââââââââââ|âââââââââââââââââââââ|
hs/space-bslc8-foldr-2: 297 169676 âââââââââââââââââââââ|âââââââââââââââââââââ|
hs/space-bslc8-lenfil-1: 115 109388 âââââââââ |ââââââââââââââ |
hs/space-bslc8-lenfil-2: 115 109392 âââââââââ |ââââââââââââââ |
hs/space-bsl---foldlx: 85 85864 ââââââ |âââââââââââ |
hs/space-xxxxx-acc-1: 1 1900 â |â |
hs/space-xxxxx-acc-2: 1 1904 â |â |
hs/space-xxxxx-foldl: 1 1904 â |â |
hs/space-xxxxx-lenfil: 1 1888 â |â |
c/byte-getchar: - -
c/byte-getchar-u: - -
c/byte-4k: - -
c/space-getchar: - -
c/space-getchar-u: - -
c/space-4k: - -
c/space-megabuf: - -
c/space-getwchar: - -
c/space-getwchar-u: - -
c/space-32k: - -
c/space-32k-8: - -
ghc-thorough-head-bs0902.tgz
-------
hs/byte-bs----acc: 1 147780 â |âââââââââââââââââââ |
hs/byte-bs----foldlx: 1 147780 â |âââââââââââââââââââ |
hs/byte-bs----foldrx: 1 147784 â |âââââââââââââââââââ |
hs/byte-bsl---acc: 2 3216 â |â |
hs/byte-xxxxx-acc-1: 1 1900 â |â |
hs/byte-xxxxx-acc-2: 1 1896 â |â |
hs/byte-xxxxx-foldl: 1 1896 â |â |
hs/space-bs-c8-acc-1: 1 147752 â |âââââââââââââââââââ |
hs/space-bs-c8-count: 1 147776 â |âââââââââââââââââââ |
hs/space-bs-c8-foldlx-1: 1 147752 â |âââââââââââââââââââ |
hs/space-bs-c8-foldlx-2: 1 147756 â |âââââââââââââââââââ |
hs/space-bs-c8-foldrx: 1 147756 â |âââââââââââââââââââ |
hs/space-bs-c8-lenfil: 21 167320 âââ |âââââââââââââââââââââ|
hs/space-bslc8-acc-1: 2 3196 â |â |
hs/space-bslc8-acc-2: 2 3200 â |â |
hs/space-bslc8-acc-3: 2 3208 â |â |
hs/space-bslc8-chunk-1: 65 66224 ââââââââââ |âââââââââ |
hs/space-bslc8-chunk-2: 65 66228 ââââââââââ |âââââââââ |
hs/space-bslc8-chunk-3: 65 66224 ââââââââââ |âââââââââ |
hs/space-bslc8-chunk-4: 78 79204 âââââââââââ |ââââââââââ |
hs/space-bslc8-count: 81 82224 ââââââââââââ |âââââââââââ |
hs/space-bslc8-foldl: 78 79196 âââââââââââ |ââââââââââ |
hs/space-bslc8-foldlx-1: 78 79196 âââââââââââ |ââââââââââ |
hs/space-bslc8-foldlx-2: 78 79196 âââââââââââ |ââââââââââ |
hs/space-bslc8-foldr-1: 149 149680 âââââââââââââââââââââ|âââââââââââââââââââ |
hs/space-bslc8-foldr-2: 149 149676 âââââââââââââââââââââ|âââââââââââââââââââ |
hs/space-bslc8-lenfil-1: 81 82304 ââââââââââââ |âââââââââââ |
hs/space-bslc8-lenfil-2: 81 82300 ââââââââââââ |âââââââââââ |
hs/space-bsl---foldlx: 78 79208 âââââââââââ |ââââââââââ |
hs/space-xxxxx-acc-1: 1 1904 â |â |
hs/space-xxxxx-acc-2: 1 1900 â |â |
hs/space-xxxxx-foldl: 1 1904 â |â |
hs/space-xxxxx-lenfil: 1 1888 â |â |
c/byte-getchar: - -
c/byte-getchar-u: - -
c/byte-4k: - -
c/space-getchar: - -
c/space-getchar-u: - -
c/space-4k: - -
c/space-megabuf: - -
c/space-getwchar: - -
c/space-getwchar-u: - -
c/space-32k: - -
c/space-32k-8: - -
More information about the Haskell-Cafe
mailing list