<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.Code, li.Code, div.Code
{mso-style-name:Code;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
font-size:9.0pt;
font-family:"Courier New";}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;
font-weight:normal;
font-style:normal;
text-decoration:none none;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
.MsoPapDefault
{mso-style-type:export-only;
margin-top:6.0pt;
margin-right:0cm;
margin-bottom:6.0pt;
margin-left:0cm;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:12.0pt">Evan<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">That’s truly great information, thank you. Could you start a GHC ticket for it? Stuff gets lots in email, less so in tickets.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">The big take-away I got from your message is this:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">score max mb total mb prd derive lily perform ghc<br>
6 72.26 3279.22 0.88 0.79~0.84 0.70~0.74 0.31~0.33 8.0.2<br>
6 76.63 3419.20 0.58 1.45~1.59 1.05~1.07 0.33~0.36 8.4.1<br>
<br>
bloom 70.69 2456.14 <b><span style="color:red">0.89</span></b><span style="color:red">
</span>1.32~1.36 0.15~0.16 8.0.2<br>
bloom 67.86 2589.97 <b><span style="color:red">0.62</span></b><span style="color:red">
</span>1.94~1.99 0.20~0.22 8.4.1<br>
<br>
</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">The bytes-allocated number has gone up a bit. (Not too surprising… the compiler is doing more.) But the productivity number is down sharply, and consistently so, which translates directly into longer compile
times. Somehow, although residency is not increasing, GC time is greatly increased.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">We have some compiler/perf regression tests that track bytes-allocated, but not that track productivity, which is why we have noticed.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">It’d be good to figure out what’s gone wrong here. Maybe a change in nursery size or something stupid like that?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">It’d be great if someone felt up to investigating a bit<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">Thanks for gathering this data.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">Simon<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Glasgow-haskell-users <glasgow-haskell-users-bounces@haskell.org>
<b>On Behalf Of </b>Evan Laforge<br>
<b>Sent:</b> 21 March 2018 22:05<br>
<b>To:</b> GHC users <glasgow-haskell-users@haskell.org><br>
<b>Subject:</b> ghc 8.0.2 vs 8.4.1 compilation time and performance<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:6.0pt;margin-right:0cm;margin-bottom:6.0pt;margin-left:0cm">
I just upgraded from 8.0.2 to 8.4.1, and I took the opportunity to do a few<br>
informal compile time and run time tests. There's been a lot of talk about<br>
compile time regressions, so maybe these will be of interest, informal as<br>
they are.<br>
<br>
I wound up skipping 8.2.1 due to <a href="https://ghc.haskell.org/trac/ghc/ticket/13604">
https://ghc.haskell.org/trac/ghc/ticket/13604</a>,<br>
but I could still test with 8.2 perfectly well. Just haven't done it yet.<br>
<br>
In this context, RunTests is more code with no optimization (and -fhpc, if it<br>
matters). debug/seq and opt/seq are the same code but with no optimization and<br>
-O respectively. I found that -O2 hurt compile time but didn't improve run<br>
time, but it's been a long time so I should run that experiment again.<br>
<br>
compile times:<br>
<br>
OS X, macbook pro:<br>
<br>
RunTests 549.10s user 118.45s system 343% cpu 3:14.53 total 8.0.2<br>
RunTests 548.71s user 117.10s system 347% cpu 3:11.78 total 8.0.2<br>
RunTests 450.92s user 109.63s system 343% cpu 2:43.13 total 8.4.1<br>
RunTests 445.48s user 107.99s system 341% cpu 2:42.19 total 8.4.1<br>
<br>
debug/seq 284.47s user 55.95s system 345% cpu 1:38.58 total 8.0.2<br>
debug/seq 283.33s user 55.27s system 343% cpu 1:38.53 total 8.0.2<br>
debug/seq 220.92s user 50.21s system 337% cpu 1:20.32 total 8.4.1<br>
debug/seq 218.39s user 49.20s system 345% cpu 1:17.47 total 8.4.1<br>
<br>
opt/seq 732.63s user 70.86s system 338% cpu 3:57.30 total 8.0.2<br>
opt/seq 735.21s user 71.48s system 327% cpu 4:06.31 total 8.0.2<br>
opt/seq 785.12s user 65.42s system 327% cpu 4:19.84 total 8.4.1<br>
opt/seq 765.52s user 64.01s system 321% cpu 4:18.29 total 8.4.1<br>
<br>
Linux, PC:<br>
<br>
RunTests 781.31s user 58.21s system 363% cpu 3:50.70 total 8.0.2<br>
RunTests 613.11s user 49.84s system 357% cpu 3:05.52 total 8.4.1<br>
<br>
debug/seq 429.44s user 31.34s system 362% cpu 2:07.03 total 8.0.2<br>
debug/seq 329.67s user 23.86s system 352% cpu 1:40.38 total 8.4.1<br>
<br>
opt/seq 1277.20s user 45.85s system 358% cpu 6:08.68 total 8.0.2<br>
opt/seq 1339.73s user 39.87s system 341% cpu 6:43.50 total 8.4.1<br>
<br>
So it looks like non-optimized compile times have gotten significantly better<br>
since 8.0.2. However, optimized has gotten a little worse, but not much.<br>
<br>
The performance numbers are a bit more disappointing. At first it appeared<br>
that allocation went down in 8.4.1 while overall time is up significantly.<br>
However, the 8.4.1 used newer dependencies, so to try to control for those, I<br>
tested again after using a cabal freeze from the 8.4.1 test. Of course I had<br>
to remove the ghc distributed packages, like container and 'ghc' itself, but<br>
the rest of the deps should be the same. Those have the 'libs' suffix on<br>
Linux.<br>
<br>
From that, it looks like the improved memory in 8.4.1 was due to external<br>
dependencies, and in fact 8.4.1 bumped memory usage up again. Ow.<br>
<br>
In the graphs, 'score' is just the input file. 'max mb' and 'total mb' and<br>
'prd' come from the post-run GC report, specifically '* bytes maximum<br>
residency', '* bytes allocated in the heap', and productivity fields.<br>
'derive', 'lily' and 'perform' are just different kinds of processes. They are<br>
CPU time bracketing the specific action, after initialization, and the range is<br>
min and max over 6 runs, so no fancy criterion-like analysis. Each run is a<br>
separate process, so they should be independent.<br>
<br>
I was hoping for some gains due to the join points stuff, but it kind of looks<br>
like things get worse across the board. I don't know why productivity goes<br>
down so much, and I don't know why the effect seems so much worse on OS X.<br>
<br>
Of course the obvious next step is to see where 8.2.1 lies, but I thought I'd<br>
see if there's interest before going to the trouble. Of course, I should track<br>
down the regressions for my own purposes, but it's a bit of a daunting task.<br>
The step of reducing to a minimal example seems a lot harder for performance<br>
than for a bug! Probably some old fashioned SCC annotations await me, but that<br>
can be a long and confusing process.<br>
<br>
<span style="font-family:"Courier New"">OS X, macbook pro:<br>
score max mb total mb prd derive lily perform ghc<br>
6 72.26 3279.22 0.88 0.79~0.84 0.70~0.74 0.31~0.33 8.0.2<br>
6 76.63 3419.20 0.58 1.45~1.59 1.05~1.07 0.33~0.36 8.4.1<br>
<br>
bloom 70.69 2456.14 0.89 1.32~1.36 0.15~0.16 8.0.2<br>
bloom 67.86 2589.97 0.62 1.94~1.99 0.20~0.22 8.4.1<br>
<br>
cerucuk-punyah 138.01 10080.55 0.93 6.98~7.16 1.24~1.30 8.0.2<br>
cerucuk-punyah 130.78 9617.35 0.68 8.91~9.22 1.57~1.68 8.4.1<br>
<br>
hex 32.86 2120.95 0.91 0.76~0.88 0.16~0.19 8.0.2<br>
hex 32.67 2194.82 0.66 1.09~1.16 0.28~0.30 8.4.1<br>
<br>
p1 67.01 4039.82 0.92 2.63~2.73 0.47~0.50 8.0.2<br>
p1 64.80 3899.85 0.68 3.35~3.43 0.58~0.59 8.4.1<br>
<br>
viola-sonata 69.32 6083.65 0.92 2.48~2.56 2.07~2.13 0.25~0.26 8.0.2<br>
viola-sonata 66.76 6120.26 0.68 3.32~3.43 2.90~2.93 0.32~0.34 8.4.1<br>
<br>
Linux, PC:<br>
<br>
score max mb total mb prd derive lily perform ghc<br>
<br>
6 79.76 3310.69 0.89 0.88~0.89 0.73~0.75 0.27~0.27 8.0.2<br>
6 72.21 3421.45 0.90 0.87~0.87 0.72~0.79 0.28~0.28 8.0.2 libs<br>
6 76.56 3419.05 0.77 1.16~1.17 0.87~0.93 0.33~0.33 8.4.1<br>
<br>
bloom 69.82 2461.95 0.89 1.35~1.36 0.17~0.17 8.0.2<br>
bloom 63.45 2554.89 0.90 1.33~1.35 0.18~0.18 8.0.2 libs<br>
bloom 67.79 2589.85 0.79 1.64~1.65 0.20~0.20 8.4.1<br>
<br>
cerucuk-punyah 137.05 10113.41 0.94 7.44~7.50 1.31~1.33 8.0.2<br>
cerucuk-punyah 128.09 10278.03 0.94 7.50~7.55 1.37~1.38 8.0.2 libs<br>
cerucuk-punyah 131.20 9617.22 0.84 7.35~7.40 1.49~1.50 8.4.1<br>
<br>
hex 32.02 2096.87 0.92 0.73~0.74 0.18~0.18 8.0.2<br>
hex 32.05 2200.30 0.91 0.73~0.80 0.19~0.19 8.0.2 libs<br>
hex 32.46 2144.22 0.83 0.89~0.90 0.20~0.20 8.4.1<br>
<br>
p1 65.88 4054.66 0.93 2.84~2.87 0.49~0.50 8.0.2<br>
p1 62.60 4127.68 0.94 2.83~2.92 0.51~0.51 8.0.2 libs<br>
p1 64.72 3899.72 0.81 2.80~2.81 0.54~0.55 8.4.1<br>
<br>
viola-sonata 68.68 6086.49 0.93 2.55~2.56 2.10~2.12 0.27~0.27 8.0.2<br>
viola-sonata 65.05 6212.57 0.93 2.52~2.55 2.07~2.16 0.28~0.28 8.0.2 libs<br>
viola-sonata 66.85 6120.15 0.83 2.91~2.92 2.48~2.51 0.30~0.31 8.4.1</span><o:p></o:p></p>
</div>
</div>
</div>
</body>
</html>