[Haskell-cafe] [10/16] SBM: Graphs for 6.9.x across four cpus
Peter Firefly Brodersen Lund
firefly at vax64.dk
Sat Dec 22 04:17:17 EST 2007
This is what you get if you merge the previous four reports (and filter out the
hand-tweaked assembly benchmarks).
I generated the report with the following command:
EXCLUDE='^hand/' \
tools/merge.pl \
ghc-armada-thorough-6.9.tgz \
ghc-fischer-thorough-6.9.tgz \
ghc-albatross-thorough-6.9.tgz \
ghc-thorough-6.9.tgz \
> xx
This graph shows the memory usage to be (almost) exactly the same. The
difference can easily be explained with slightly different versions of the
C library (there was also a security update or two from Ubuntu in the last
few days - the kernel was definitely updated and I think the C library was
too). We are also talking about two versions of Ubuntu and one (old) version
of SuSE. And we are talking about three not quite identical versions of ghc.
The fischer and albatross runs were made before the inclusion of the three
benchmarks from Don Stewart (hs/space-bs-c8-count, hs/space-bslc8-count, and
hs/space-bslc8-chunk-4) so they are a couple of holes in the graphs.
The speed pattern is more fun. It makes no sense to compare absolute times
here so the graphs were not rescaled. One would naïvely expect the bars to
be about the same when taken in groups of four but that really turns out to be
far from the case!
The explanation is most likely that we are looking at four very different
microarchitectures.
**********
This should hammer home the point that benchmarking on any single machine
isn't enough!
**********
Two probably aren't enough either...
-Peter
ls-search
ghc 6.9.20071119
Pentium III (Coppermine)
596.932 MHz
TESTKIND=THOROUGH
SUFFIX=
Fischer's machine
ghc 6.9.20071124
AMD Duron(tm) processor
1200.089 MHz
TESTKIND=THOROUGH
SUFFIX=
albatross
ghc 6.9.20071208 (or thereabouts)
Genuine Intel(R) CPU T2300 @ 1.66GHz (Core Duo)
1667.000 MHz
TESTKIND=THOROUGH
SUFFIX=
charybdis
ghc 6.9.20071119
AMD Athlon(tm) 64 Processor 3000+
2009.160 MHz
TESTKIND=THOROUGH
SUFFIX=
Time (byte counting) std
-------------------- avg dev slack
hs/byte-bs----acc: 3.274 1â° 0.1 ââââ |
-- 1.892 21â° 0.4 âââ |
-- 0.918 4â° 0.1 ââââââ |
-- 0.705 7â° 0.1 ââââ |
hs/byte-bs----foldlx: 4.027 0â° 0.0 ââââ |
-- 2.258 3â° 0.1 ââââ |
-- 1.014 2â° 0.2 ââââââ |
-- 1.002 1â° 0.5 ââââââ |
hs/byte-bs----foldrx: 4.184 1â° 0.0 ââââ |
-- 2.933 0â° 0.1 âââââ |
-- 0.999 2â° 0.1 ââââââ |
-- 1.112 2â° 0.1 ââââââ |
hs/byte-bsl---acc: 28.005 10â° 0.0 âââââââââââââââââââââââââââ |
-- 14.319 45â° 0.1 âââââââââââââââââââââ |
-- 1.957 1â° 0.1 ââââââââââââ |
-- 2.595 14â° 0.1 ââââââââââââââ |
hs/byte-xxxxx-acc-1: 25.852 4â° 0.0 âââââââââââââââââââââââââ |
-- 20.915 17â° 4.0 ââââââââââââââââââââââââââââââ |
-- 3.591 2â° 0.1 ââââââââââââââââââââââ |
-- 4.436 5â° 0.0 ââââââââââââââââââââââââ |
hs/byte-xxxxx-acc-2: 25.622 5â° 0.0 âââââââââââââââââââââââââ |
-- 20.691 8â° 0.1 ââââââââââââââââââââââââââââââ |
-- 3.598 4â° 0.1 ââââââââââââââââââââââ |
-- 4.473 8â° 0.0 ââââââââââââââââââââââââ |
hs/byte-xxxxx-foldl: 25.803 5â° 0.0 âââââââââââââââââââââââââ |
-- 20.610 5â° 1.4 ââââââââââââââââââââââââââââââ |
-- 3.594 1â° 0.1 ââââââââââââââââââââââ |
-- 4.412 10â° 0.0 âââââââââââââââââââââââ |
c/byte-getchar: 13.016 0â° 0.0 âââââââââââââ |
-- 9.042 0â° 0.1 âââââââââââââ |
-- 3.761 2â° 0.1 âââââââââââââââââââââââ |
-- 2.422 27â° 0.1 âââââââââââââ |
c/byte-getchar-u: 1.662 1â° 0.1 ââ |
-- 1.314 3â° 0.2 ââ |
-- 0.719 2â° 0.3 âââââ |
-- 0.632 3â° 0.3 ââââ |
c/byte-4k: 0.543 2â° 0.2 â |
-- 0.419 5â° 0.5 â |
-- 0.101 10â° 3.1 â |
-- 0.094 26â° 2.2 â |
Memory: Peak
------- KB
hs/byte-bs----acc: 147752 ââââââââââââââââââââââââââââââââââââââââ |
-- 147492 ââââââââââââââââââââââââââââââââââââââââ |
-- 147784 âââââââââââââââââââââââââââââââââââââââââ |
-- 147752 ââââââââââââââââââââââââââââââââââââââââ |
hs/byte-bs----foldlx: 147756 ââââââââââââââââââââââââââââââââââââââââ |
-- 147492 ââââââââââââââââââââââââââââââââââââââââ |
-- 147812 âââââââââââââââââââââââââââââââââââââââââ |
-- 147748 ââââââââââââââââââââââââââââââââââââââââ |
hs/byte-bs----foldrx: 147760 ââââââââââââââââââââââââââââââââââââââââ |
-- 147488 ââââââââââââââââââââââââââââââââââââââââ |
-- 147788 âââââââââââââââââââââââââââââââââââââââââ |
-- 147744 ââââââââââââââââââââââââââââââââââââââââ |
hs/byte-bsl---acc: 3180 â |
-- 2896 â |
-- 3228 â |
-- 3172 â |
hs/byte-xxxxx-acc-1: 1916 â |
-- 1612 â |
-- 1904 â |
-- 1904 â |
hs/byte-xxxxx-acc-2: 1912 â |
-- 1612 â |
-- 1908 â |
-- 1904 â |
hs/byte-xxxxx-foldl: 1912 â |
-- 1612 â |
-- 1908 â |
-- 1900 â |
c/byte-getchar: 436 â |
-- 384 â |
-- 432 â |
-- 440 â |
c/byte-getchar-u: 432 â |
-- 384 â |
-- 440 â |
-- 440 â |
c/byte-4k: 436 â |
-- 380 â |
-- 436 â |
-- 440 â |
Time (space counting) std
--------------------- avg dev slack
hs/space-bs-c8-acc-1: 4.318 1â° 0.0 ââââ |
-- 2.467 1â° 0.3 ââââ |
-- 1.313 0â° 0.2 ââââââââ |
-- 1.145 1â° 0.2 ââââââ |
hs/space-bs-c8-count: 3.118 1â° 0.1 âââ |
-- n/a
-- n/a
-- 0.521 1â° 0.2 âââ |
hs/space-bs-c8-foldlx-1: 4.631 1â° 0.0 âââââ |
-- 2.585 2â° 0.1 ââââ |
-- 1.326 1â° 0.1 ââââââââ |
-- 1.221 1â° 0.1 âââââââ |
hs/space-bs-c8-foldlx-2: 4.632 1â° 0.0 âââââ |
-- 2.576 2â° 0.3 ââââ |
-- 1.328 1â° 0.1 ââââââââ |
-- 1.219 2â° 0.2 âââââââ |
hs/space-bs-c8-foldrx: 4.678 0â° 0.0 âââââ |
-- 2.982 8â° 2.3 âââââ |
-- 1.128 1â° 0.2 âââââââ |
-- 1.172 2â° 0.3 âââââââ |
hs/space-bs-c8-lenfil: 4.634 1â° 0.1 âââââ |
-- 2.599 1â° 0.2 ââââ |
-- 1.327 2â° 0.2 ââââââââ |
-- 1.223 1â° 0.2 âââââââ |
hs/space-bslc8-acc-1: 32.733 7â° 0.0 âââââââââââââââââââââââââââââââ |
-- 15.228 8â° 0.1 ââââââââââââââââââââââ |
-- 3.002 1â° 0.1 ââââââââââââââââââ |
-- 3.388 10â° 0.1 ââââââââââââââââââ |
hs/space-bslc8-acc-2: 33.112 11â° 0.0 ââââââââââââââââââââââââââââââââ |
-- 15.855 38â° 0.0 âââââââââââââââââââââââ |
-- 3.010 2â° 0.0 ââââââââââââââââââ |
-- 3.386 5â° 0.1 ââââââââââââââââââ |
hs/space-bslc8-acc-3: 30.294 12â° 0.0 âââââââââââââââââââââââââââââ |
-- 14.980 14â° 0.0 ââââââââââââââââââââââ |
-- 2.586 1â° 0.1 ââââââââââââââââ |
-- 2.832 4â° 0.1 âââââââââââââââ |
hs/space-bslc8-chunk-1: 3.888 0â° 0.1 ââââ |
-- 2.443 2â° 0.2 ââââ |
-- 1.192 0â° 0.0 âââââââ |
-- 1.094 0â° 0.3 ââââââ |
hs/space-bslc8-chunk-2: 3.887 1â° 0.0 ââââ |
-- 2.449 1â° 0.3 ââââ |
-- 1.189 0â° 0.1 âââââââ |
-- 1.094 1â° 0.3 ââââââ |
hs/space-bslc8-chunk-3: 3.881 0â° 0.1 ââââ |
-- 2.534 3â° 0.3 ââââ |
-- 1.272 1â° 0.2 ââââââââ |
-- 1.087 1â° 0.3 ââââââ |
hs/space-bslc8-chunk-4: 3.910 1â° 0.0 ââââ |
-- n/a
-- n/a
-- 1.099 0â° 0.3 ââââââ |
hs/space-bslc8-count: 2.355 1â° 0.1 âââ |
-- n/a
-- n/a
-- 0.477 2â° 0.4 âââ |
hs/space-bslc8-foldl: 4.674 0â° 0.0 âââââ |
-- 2.938 1â° 0.2 âââââ |
-- 1.517 0â° 0.1 âââââââââ |
-- 1.458 1â° 0.3 ââââââââ |
hs/space-bslc8-foldlx-1: 4.676 0â° 0.0 âââââ |
-- 2.928 1â° 0.0 âââââ |
-- 1.516 0â° 0.0 âââââââââ |
-- 1.458 0â° 0.1 ââââââââ |
hs/space-bslc8-foldlx-2: 4.676 0â° 0.0 âââââ |
-- 2.937 2â° 0.2 âââââ |
-- 1.511 0â° 0.0 âââââââââ |
-- 1.457 1â° 0.1 ââââââââ |
hs/space-bslc8-foldr-1: 5.222 0â° 0.0 âââââ |
-- 4.043 6â° 0.1 ââââââ |
-- 1.205 1â° 0.2 ââââââââ |
-- 1.312 1â° 0.2 âââââââ |
hs/space-bslc8-foldr-2: 5.224 1â° 0.1 âââââ |
-- 4.007 4â° 0.1 ââââââ |
-- 1.202 3â° 0.1 ââââââââ |
-- 1.315 1â° 0.2 âââââââ |
hs/space-bslc8-lenfil-1: 4.712 1â° 0.0 âââââ |
-- 3.240 1â° 0.2 âââââ |
-- 1.400 0â° 0.0 âââââââââ |
-- 1.323 1â° 0.1 âââââââ |
hs/space-bslc8-lenfil-2: 4.714 1â° 0.0 âââââ |
-- 3.236 1â° 0.2 âââââ |
-- 1.396 0â° 0.0 âââââââââ |
-- 1.325 1â° 0.1 âââââââ |
hs/space-bsl---foldlx: 4.522 0â° 0.1 âââââ |
-- 2.821 1â° 0.1 ââââ |
-- 1.476 1â° 0.1 âââââââââ |
-- 1.391 1â° 0.1 ââââââââ |
hs/space-xxxxx-acc-1: 27.529 6â° 0.0 ââââââââââââââââââââââââââ |
-- 21.002 4â° 0.1 ââââââââââââââââââââââââââââââ |
-- 4.245 1â° 0.0 âââââââââââââââââââââââââ |
-- 4.991 1â° 0.1 ââââââââââââââââââââââââââ |
hs/space-xxxxx-acc-2: 27.200 3â° 0.0 ââââââââââââââââââââââââââ |
-- 21.270 22â° 4.7 âââââââââââââââââââââââââââââââ |
-- 4.241 2â° 0.1 âââââââââââââââââââââââââ |
-- 4.985 1â° 0.0 ââââââââââââââââââââââââââ |
hs/space-xxxxx-foldl: 27.436 8â° 0.0 ââââââââââââââââââââââââââ |
-- 20.934 1â° 0.1 ââââââââââââââââââââââââââââââ |
-- 4.239 2â° 0.1 âââââââââââââââââââââââââ |
-- 5.035 2â° 0.1 âââââââââââââââââââââââââââ |
hs/space-xxxxx-lenfil: 39.343 3â° 0.0 âââââââââââââââââââââââââââââââââââââ|
-- 25.915 3â° 0.0 âââââââââââââââââââââââââââââââââââââ|
-- 6.287 2â° 0.0 âââââââââââââââââââââââââââââââââââââ|
-- 7.106 8â° 0.1 âââââââââââââââââââââââââââââââââââââ|
c/space-getchar: 13.268 0â° 0.0 âââââââââââââ |
-- 9.354 0â° 0.0 ââââââââââââââ |
-- 4.031 18â° 0.0 ââââââââââââââââââââââââ |
-- 2.465 15â° 0.1 âââââââââââââ |
c/space-getchar-u: 2.170 0â° 0.1 ââ |
-- 1.676 2â° 0.2 âââ |
-- 0.609 2â° 0.7 ââââ |
-- 0.631 1â° 0.5 ââââ |
c/space-4k: 1.815 1â° 0.1 ââ |
-- 1.293 2â° 0.5 ââ |
-- 0.465 2â° 0.6 âââ |
-- 0.319 3â° 0.6 ââ |
c/space-megabuf: 2.763 2â° 0.1 âââ |
-- 1.830 3â° 0.5 âââ |
-- 0.613 1â° 0.3 ââââ |
-- 0.515 3â° 0.4 âââ |
c/space-getwchar: 15.105 29â° 0.0 âââââââââââââââ |
-- 14.721 1â° 0.1 âââââââââââââââââââââ |
-- 4.584 1â° 0.1 âââââââââââââââââââââââââââ |
-- 3.956 6â° 0.1 âââââââââââââââââââââ |
c/space-getwchar-u: 8.979 1â° 0.0 âââââââââ |
-- 4.814 0â° 0.0 âââââââ |
-- 2.437 0â° 0.1 âââââââââââââââ |
-- 2.217 1â° 0.1 ââââââââââââ |
c/space-32k: 1.786 9â° 0.1 ââ |
-- 1.276 2â° 0.6 ââ |
-- 0.445 1â° 0.5 âââ |
-- 0.303 3â° 1.0 ââ |
c/space-32k-8: 1.777 1â° 0.1 ââ |
-- 1.275 2â° 0.2 ââ |
-- 0.448 0â° 0.0 âââ |
-- 0.304 2â° 0.3 ââ |
Memory: Peak
------- KB
hs/space-bs-c8-acc-1: 147748 ââââââââââââââââââââââââââââââââââââââââ |
-- 147488 ââââââââââââââââââââââââââââââââââââââââ |
-- 147772 ââââââââââââââââââââââââââââââââââââââââ |
-- 147736 ââââââââââââââââââââââââââââââââââââââââ |
hs/space-bs-c8-count: 147760 ââââââââââââââââââââââââââââââââââââââââ |
-- n/a
-- n/a
-- 147744 ââââââââââââââââââââââââââââââââââââââââ |
hs/space-bs-c8-foldlx-1: 147760 ââââââââââââââââââââââââââââââââââââââââ |
-- 147492 ââââââââââââââââââââââââââââââââââââââââ |
-- 147800 âââââââââââââââââââââââââââââââââââââââââ |
-- 147756 ââââââââââââââââââââââââââââââââââââââââ |
hs/space-bs-c8-foldlx-2: 147764 ââââââââââââââââââââââââââââââââââââââââ |
-- 147492 ââââââââââââââââââââââââââââââââââââââââ |
-- 147804 âââââââââââââââââââââââââââââââââââââââââ |
-- 147752 ââââââââââââââââââââââââââââââââââââââââ |
hs/space-bs-c8-foldrx: 147748 ââââââââââââââââââââââââââââââââââââââââ |
-- 147488 ââââââââââââââââââââââââââââââââââââââââ |
-- 147776 ââââââââââââââââââââââââââââââââââââââââ |
-- 147732 ââââââââââââââââââââââââââââââââââââââââ |
hs/space-bs-c8-lenfil: 147764 ââââââââââââââââââââââââââââââââââââââââ |
-- 147492 ââââââââââââââââââââââââââââââââââââââââ |
-- 147796 âââââââââââââââââââââââââââââââââââââââââ |
-- 147756 ââââââââââââââââââââââââââââââââââââââââ |
hs/space-bslc8-acc-1: 3196 â |
-- 2896 â |
-- 3192 â |
-- 3192 â |
hs/space-bslc8-acc-2: 3200 â |
-- 2896 â |
-- 3192 â |
-- 3192 â |
hs/space-bslc8-acc-3: 3204 â |
-- 2896 â |
-- 3188 â |
-- 3196 â |
hs/space-bslc8-chunk-1: 65204 ââââââââââââââââââ |
-- 65892 ââââââââââââââââââ |
-- 65208 ââââââââââââââââââ |
-- 65192 ââââââââââââââââââ |
hs/space-bslc8-chunk-2: 65200 ââââââââââââââââââ |
-- 65892 ââââââââââââââââââ |
-- 65212 ââââââââââââââââââ |
-- 65192 ââââââââââââââââââ |
hs/space-bslc8-chunk-3: 75248 âââââââââââââââââââââ |
-- 76472 âââââââââââââââââââââ |
-- 75232 âââââââââââââââââââââ |
-- 75240 âââââââââââââââââââââ |
hs/space-bslc8-chunk-4: 76204 âââââââââââââââââââââ |
-- n/a
-- n/a
-- 76188 âââââââââââââââââââââ |
hs/space-bslc8-count: 79224 ââââââââââââââââââââââ |
-- n/a
-- n/a
-- 79204 ââââââââââââââââââââââ |
hs/space-bslc8-foldl: 85880 ââââââââââââââââââââââââ |
-- 86772 ââââââââââââââââââââââââ |
-- 85856 ââââââââââââââââââââââââ |
-- 85868 ââââââââââââââââââââââââ |
hs/space-bslc8-foldlx-1: 85880 ââââââââââââââââââââââââ |
-- 86772 ââââââââââââââââââââââââ |
-- 85852 ââââââââââââââââââââââââ |
-- 85864 ââââââââââââââââââââââââ |
hs/space-bslc8-foldlx-2: 85880 ââââââââââââââââââââââââ |
-- 86772 ââââââââââââââââââââââââ |
-- 85856 ââââââââââââââââââââââââ |
-- 85868 ââââââââââââââââââââââââ |
hs/space-bslc8-foldr-1: 169704 ââââââââââââââââââââââââââââââââââââââââââââââ|
-- 169360 ââââââââââââââââââââââââââââââââââââââââââââââ|
-- 169684 ââââââââââââââââââââââââââââââââââââââââââââââ|
-- 169700 ââââââââââââââââââââââââââââââââââââââââââââââ|
hs/space-bslc8-foldr-2: 169700 ââââââââââââââââââââââââââââââââââââââââââââââ|
-- 169360 ââââââââââââââââââââââââââââââââââââââââââââââ|
-- 169684 ââââââââââââââââââââââââââââââââââââââââââââââ|
-- 169700 ââââââââââââââââââââââââââââââââââââââââââââââ|
hs/space-bslc8-lenfil-1: 109408 ââââââââââââââââââââââââââââââ |
-- 110704 âââââââââââââââââââââââââââââââ |
-- 109404 ââââââââââââââââââââââââââââââ |
-- 109404 ââââââââââââââââââââââââââââââ |
hs/space-bslc8-lenfil-2: 109404 ââââââââââââââââââââââââââââââ |
-- 110704 âââââââââââââââââââââââââââââââ |
-- 109404 ââââââââââââââââââââââââââââââ |
-- 109404 ââââââââââââââââââââââââââââââ |
hs/space-bsl---foldlx: 85844 ââââââââââââââââââââââââ |
-- 86776 ââââââââââââââââââââââââ |
-- 85880 ââââââââââââââââââââââââ |
-- 85836 ââââââââââââââââââââââââ |
hs/space-xxxxx-acc-1: 1912 â |
-- 1612 â |
-- 1916 â |
-- 1904 â |
hs/space-xxxxx-acc-2: 1912 â |
-- 1612 â |
-- 1916 â |
-- 1904 â |
hs/space-xxxxx-foldl: 1912 â |
-- 1612 â |
-- 1916 â |
-- 1900 â |
hs/space-xxxxx-lenfil: 1904 â |
-- 1588 â |
-- 1892 â |
-- 1892 â |
c/space-getchar: 436 â |
-- 384 â |
-- 436 â |
-- 444 â |
c/space-getchar-u: 436 â |
-- 384 â |
-- 436 â |
-- 444 â |
c/space-4k: 472 â |
-- 412 â |
-- 468 â |
-- 468 â |
c/space-megabuf: 146968 ââââââââââââââââââââââââââââââââââââââââ |
-- 146904 ââââââââââââââââââââââââââââââââââââââââ |
-- 146968 ââââââââââââââââââââââââââââââââââââââââ |
-- 146968 ââââââââââââââââââââââââââââââââââââââââ |
c/space-getwchar: 484 â |
-- 440 â |
-- 484 â |
-- 496 â |
c/space-getwchar-u: 484 â |
-- 440 â |
-- 480 â |
-- 500 â |
c/space-32k: 500 â |
-- 436 â |
-- 496 â |
-- 496 â |
c/space-32k-8: 496 â |
-- 436 â |
-- 500 â |
-- 500 â |
More information about the Haskell-Cafe
mailing list