From syd.kerckhove at gmail.com Sat Apr 1 21:59:12 2017 From: syd.kerckhove at gmail.com (Tom Sydney Kerckhove) Date: Sat, 1 Apr 2017 23:59:12 +0200 Subject: Translation of GHC typechecker output to haskell-src-exts's 'Type' Message-ID: <20170401215912.GA5496@quintus.localdomain> Dear GHC Devs, Because of the unwieldy nature of the data that the GHC type checker outputs, I am trying to convert a GHC 'Type' [1] to a haskell-src-ext 'Type' [2]. The translation does not need to be perfect for now, but I would at least like to be able to translate function types and types that involve type-class constraints. (See my initial attempt in attachment) Has this ever been done before? Could you point me to some documentation on GHC's 'Type' [1] that could help me with writing this function? (The comments in code aren't nearly enough for me.) In particular, I am having trouble finding type class constraints in the 'Type'. Thank you for your time. [1]: https://downloads.haskell.org/~ghc/8.0.2/docs/html/libraries/ghc-8.0.2/src/TyCoRep.html#Type [2]: https://hackage.haskell.org/package/haskell-src-exts-1.18.2/docs/Language-Haskell-Exts-Syntax.html#t:Type -- Tom Sydney Kerckhove -------------- next part -------------- A non-text attachment was scrubbed... Name: attempt.hs Type: text/x-haskell Size: 1751 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From ben at smart-cactus.org Sat Apr 1 23:47:15 2017 From: ben at smart-cactus.org (Ben Gamari) Date: Sat, 01 Apr 2017 19:47:15 -0400 Subject: Translation of GHC typechecker output to haskell-src-exts's 'Type' In-Reply-To: <20170401215912.GA5496@quintus.localdomain> References: <20170401215912.GA5496@quintus.localdomain> Message-ID: <87h927667w.fsf@ben-laptop.smart-cactus.org> Tom Sydney Kerckhove writes: > Dear GHC Devs, Hi Tom, > Because of the unwieldy nature of the data that the GHC type checker > outputs, I am trying to convert a GHC 'Type' [1] to a haskell-src-ext > 'Type' [2]. > > The translation does not need to be perfect for now, but I would at > least like to be able to translate function types and types that involve > type-class constraints. (See my initial attempt in attachment) > > Has this ever been done before? I'm not aware of any implementations, but I can't claim to have seen every usage of haskell-src-exts. > Could you point me to some documentation on GHC's 'Type' [1] that could > help me with writing this function? (The comments in code aren't nearly > enough for me.) > I'm afraid the notes in TyCoRep are really all we have. Note that you should also familiarize yourself with the TyCon type (for reasons you'll see below). > In particular, I am having trouble finding type class constraints in the > 'Type'. > During typechecking class constraints are turned into dictionary arguments. If I'm not mistaken you can pick these out by looking for AlgTyCons (a constructor of the TyCon type, see TyCon.hs) with a algTcParent matching (ClassTyCon cls _). cls will be the name of the associated class. I hope this helps. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From syd.kerckhove at gmail.com Sun Apr 2 12:40:05 2017 From: syd.kerckhove at gmail.com (Tom Sydney Kerckhove) Date: Sun, 2 Apr 2017 14:40:05 +0200 Subject: Translation of GHC typechecker output to haskell-src-exts's 'Type' In-Reply-To: <87h927667w.fsf@ben-laptop.smart-cactus.org> References: <20170401215912.GA5496@quintus.localdomain> <87h927667w.fsf@ben-laptop.smart-cactus.org> Message-ID: <20170402124005.GC5496@quintus.localdomain> On 01-04-17 19:47:15, Ben Gamari wrote: > Tom Sydney Kerckhove writes: > > > Dear GHC Devs, > > Hi Tom, Hi Mr Gamari > I'm afraid the notes in TyCoRep are really all we have. Note that you > should also familiarize yourself with the TyCon type (for reasons you'll > see below). > > > In particular, I am having trouble finding type class constraints in the > > 'Type'. > > > During typechecking class constraints are turned into dictionary > arguments. Oh, that's already done by then? It makes a lot more sense now why I get this: ghcs translation: Ord a -> [a] -> [a] haskell-src-exts: Ord a => [a] -> [a] I'm guessing that this 'Ord a' that I get, is the type of the dictionary for 'Ord' in the case of 'a'. Is there a way to access the types before this translation happens? It's okay if I have to assume that type-checking succeeds... > If I'm not mistaken you can pick these out by looking for > AlgTyCons (a constructor of the TyCon type, see TyCon.hs) with a > algTcParent matching (ClassTyCon cls _). cls will be the name of the > associated class. > > I hope this helps. This does help, thank you! > Cheers, Thank you for your time. > - Ben > -- Tom Sydney Kerckhove -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From syd.kerckhove at gmail.com Sun Apr 2 13:01:43 2017 From: syd.kerckhove at gmail.com (Tom Sydney Kerckhove) Date: Sun, 2 Apr 2017 15:01:43 +0200 Subject: Translation of GHC typechecker output to haskell-src-exts's 'Type' In-Reply-To: <20170402124005.GC5496@quintus.localdomain> References: <20170401215912.GA5496@quintus.localdomain> <87h927667w.fsf@ben-laptop.smart-cactus.org> <20170402124005.GC5496@quintus.localdomain> Message-ID: <20170402130143.GD5496@quintus.localdomain> On 02-04-17 14:40:05, Tom Sydney Kerckhove wrote: > Is there a way to access the types before this translation happens? > It's okay if I have to assume that type-checking succeeds... I have found a way to do what I want, even after this translation. It relies on the fact that type class constraints always occur on the left side of the result of splitFunTy. Thank you for your help! -- Tom Sydney Kerckhove -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From ben at smart-cactus.org Sun Apr 2 16:18:52 2017 From: ben at smart-cactus.org (Ben Gamari) Date: Sun, 02 Apr 2017 12:18:52 -0400 Subject: Translation of GHC typechecker output to haskell-src-exts's 'Type' In-Reply-To: <20170402130143.GD5496@quintus.localdomain> References: <20170401215912.GA5496@quintus.localdomain> <87h927667w.fsf@ben-laptop.smart-cactus.org> <20170402124005.GC5496@quintus.localdomain> <20170402130143.GD5496@quintus.localdomain> Message-ID: <878tni6avn.fsf@ben-laptop.smart-cactus.org> Tom Sydney Kerckhove writes: > On 02-04-17 14:40:05, Tom Sydney Kerckhove wrote: >> Is there a way to access the types before this translation happens? >> It's okay if I have to assume that type-checking succeeds... > > I have found a way to do what I want, even after this translation. > It relies on the fact that type class constraints always occur on the > left side of the result of splitFunTy. > Yep, that is what I would do as well. > Thank you for your help! > No worries and good luck. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From simonpj at microsoft.com Mon Apr 3 10:54:21 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Mon, 3 Apr 2017 10:54:21 +0000 Subject: FW: [Haskell-cafe] [Call for Contributions] Haskell Communities and Activities Report, May 2017 edition (32nd edition) In-Reply-To: References: Message-ID: Ben Will you coordinate the GHC status report for HCAR? Everyone: please don’t leave Ben doing all the heavy lifting. He'll sketch a structure but it's up to the rest of us to fill it in. These six monthly reports are widely read, I think. Simon -----Original Message----- From: Haskell-Cafe [mailto:haskell-cafe-bounces at haskell.org] On Behalf Of Mihai Maruseac Sent: 03 April 2017 01:42 To: Haskell ; haskell ; Haskell Beginners Subject: [Haskell-cafe] [Call for Contributions] Haskell Communities and Activities Report, May 2017 edition (32nd edition) Dear all, We would like to collect contributions for the 32nd edition (10000th edition) of the ============================================================ Haskell Communities & Activities Report http://www.haskell.org/haskellwiki/Haskell_Communities_and_Activities_Report Submission deadline: 30 April 2017 (please send your contributions to hcar at haskell.org, in plain text or LaTeX format, both are equally accepted) ============================================================ This is the short story: * If you are working on any project that is in some way related to Haskell, please write a short entry and submit it. Even if the project is very small or unfinished or you think it is not important enough --- please reconsider and submit an entry anyway! * If you are interested in an existing project related to Haskell that has not previously been mentioned in the HCAR, please tell us, so that we can contact the project leaders and ask them to submit an entry. * If you are working on a project that is looking for contributors, please write a short entry and submit it, mentioning that your are looking for contributors. * Feel free to pass on this call for contributions to others that might be interested. More detailed information: The Haskell Communities & Activities Report is a bi-annual overview of the state of Haskell as well as Haskell-related projects over the last, and possibly the upcoming six months. If you have only recently been exposed to Haskell, it might be a good idea to browse the previous edition --- you will find interesting projects described as well as several starting points and links that may provide answers to many questions. Contributions will be collected until the submission deadline. They will then be compiled into a coherent report that is published online as soon as it is ready. As always, this is a great opportunity to update your webpages, make new releases, announce or even start new projects, or to talk about developments you want every Haskeller to know about! Looking forward to your contributions, Mihai Maruseac FAQ: Q: What format should I write in? A: The usual format is a LaTeX source file, adhering to the template that is available at: http://haskell.org/communities/05-2017/template.tex There is also a LaTeX style file at http://haskell.org/communities/05-2017/hcar.sty that you can use to preview your entry. If you do not know LaTeX or don't want to use it or don't have time to translate your entry into it, then please use plain text, it is better to have an entry in plain-text which we will translate than not have it at all. If you modify an old entry that you have written for an earlier edition of the report, you should soon receive your old entry as a template (provided we have your valid email address). Please modify that template, rather than using your own version of the old entry as a template. Q: Can I include Haskell code? A: Yes. Please use lhs2tex syntax (http://www.andres-loeh.de/lhs2tex/). The report is compiled in mode polycode.fmt. Q: Can I include images? A: Yes, you are even encouraged to do so. Please use .jpg or .png format, then. PNG is preferred for simplicity. Q: Should I send files in .zip archives or similar? A: No, plain file attachments are the way. Q: How much should I write? A: Authors are asked to limit entries to about one column of text. A general introduction is helpful. Apart from that, you should focus on recent or upcoming developments. Pointers to online content can be given for more comprehensive or "historic" overviews of a project. Images do not count towards the length limit, so you may want to use this opportunity to pep up entries. There is no minimum length of an entry! The report aims for being as complete as possible, so please consider writing an entry, even if it is only a few lines long. Q: Which topics are relevant? A: All topics which are related to Haskell in some way are relevant. We usually had reports from users of Haskell (private, academic, or commercial), from authors or contributors to projects related to Haskell, from people working on the Haskell language, libraries, on language extensions or variants. We also like reports about distributions of Haskell software, Haskell infrastructure, books and tutorials on Haskell. Reports on past and upcoming events related to Haskell are also relevant. Finally, there might be new topics we do not even think about. As a rule of thumb: if in doubt, then it probably is relevant and has a place in the HCAR. You can also simply ask us. Q: Is unfinished work relevant? Are ideas for projects relevant? A: Yes! You can use the HCAR to talk about projects you are currently working on. You can use it to look for other developers that might help you. You can use HCAR to ask for more contributors to your project, it is a good way to gain visibility and traction. Q: If I do not update my entry, but want to keep it in the report, what should I do? A: Tell us that there are no changes. The old entry will typically be reused in this case, but it might be dropped if it is older than a year, to give more room and more attention to projects that change a lot. Do not resend complete entries if you have not changed them. Q: Will I get confirmation if I send an entry? How do I know whether my email has even reached its destination, and not ended up in a spam folder? A: Prior to publication of the final report, we will send a draft to all contributors, for possible corrections. So if you do not hear from us within two weeks after the deadline, it is safer to send another mail and check whether your first one was received. -- Mihai Maruseac (MM) "If you can't solve a problem, then there's an easier problem you can solve: find it." -- George Polya _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post. From simonpj at microsoft.com Mon Apr 3 10:59:39 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Mon, 3 Apr 2017 10:59:39 +0000 Subject: Translation of GHC typechecker output to haskell-src-exts's 'Type' In-Reply-To: <20170402130143.GD5496@quintus.localdomain> References: <20170401215912.GA5496@quintus.localdomain> <87h927667w.fsf@ben-laptop.smart-cactus.org> <20170402124005.GC5496@quintus.localdomain> <20170402130143.GD5496@quintus.localdomain> Message-ID: Tom Please start a wiki page, somewhere under https://ghc.haskell.org/trac/ghc/wiki/Commentary that gives the information you wish was more clearly described. And/or add a Note in TyCoRep to explain. My answer to your question would be: The type that Haskell programmers write Ord a => a -> a is represented in Type using FunTy, thus Ord a -> a -> a There is no "=>" data contructor. Instead you can tell the difference between -> and => by looking at the kind of the bit before the ->. Thus ty1 -> ty2 if ty1 :: Constraint, then it it's a "=>" otherwise a "->". Use (isPredTy ty1) to tell the difference. Does that help Simon | -----Original Message----- | From: ghc-devs [mailto:ghc-devs-bounces at haskell.org] On Behalf Of Tom | Sydney Kerckhove | Sent: 02 April 2017 14:02 | To: Ben Gamari | Cc: ghc-devs at haskell.org | Subject: Re: Translation of GHC typechecker output to haskell-src- | exts's 'Type' | | On 02-04-17 14:40:05, Tom Sydney Kerckhove wrote: | > Is there a way to access the types before this translation happens? | > It's okay if I have to assume that type-checking succeeds... | | I have found a way to do what I want, even after this translation. | It relies on the fact that type class constraints always occur on the | left side of the result of splitFunTy. | | Thank you for your help! | | -- | Tom Sydney Kerckhove From ben at well-typed.com Mon Apr 3 13:56:50 2017 From: ben at well-typed.com (Ben Gamari) Date: Mon, 03 Apr 2017 09:56:50 -0400 Subject: FW: [Haskell-cafe] [Call for Contributions] Haskell Communities and Activities Report, May 2017 edition (32nd edition) In-Reply-To: References: Message-ID: <87y3vh4msd.fsf@ben-laptop.smart-cactus.org> Simon Peyton Jones via ghc-devs writes: > Ben > > Will you coordinate the GHC status report for HCAR? > Of course. > Everyone: please don’t leave Ben doing all the heavy lifting. He'll > sketch a structure but it's up to the rest of us to fill it in. These > six monthly reports are widely read, I think. > Thanks! Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 483 bytes Desc: not available URL: From ben at well-typed.com Tue Apr 4 04:21:17 2017 From: ben at well-typed.com (Ben Gamari) Date: Tue, 04 Apr 2017 00:21:17 -0400 Subject: GHC 8.2.1-rc1 source tarball availability References: <877ffpu3ax.fsf@smart-cactus.org> Message-ID: <87r3184xc2.fsf@ben-laptop.smart-cactus.org> tl;dr: If you would like to produce a binary distribution for GHC 8.2.1-rc1 then let me know, grab the source distribution and start building. The binary distributions will be announced one week from today. Hello GHC packagers, I am happy to announce the release of the 8.2.1-rc1 source distribution to binary packagers. You will find the usual source artifacts at http://downloads.haskell.org/~ghc/8.2.1-rc1/ As usual, the sooner we can get the binary distributions together the better, but I will hold off on announcing the distributions until next Sunday to ensure we're all on the same page. It would be appreciated if you could reply to this message confirming that you intend to offer a binary distribution this release. Otherwise, let me know if you have any trouble building your distribution. I have yet to push the ghc-8.2.1-rc1 tag in case we encounter unexpected issues but all of my builds with this tarball thusfar have gone swimmingly save a few known issues (namely #13426, #13233, and #13509). Good luck and thanks for all of your work! Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From alan.zimm at gmail.com Tue Apr 4 07:01:07 2017 From: alan.zimm at gmail.com (Alan & Kim Zimmerman) Date: Tue, 4 Apr 2017 09:01:07 +0200 Subject: GHC 8.2.1-rc1 source tarball availability In-Reply-To: <87r3184xc2.fsf@ben-laptop.smart-cactus.org> References: <877ffpu3ax.fsf@smart-cactus.org> <87r3184xc2.fsf@ben-laptop.smart-cactus.org> Message-ID: There is no tag in the source tree, which commit has been used? Alan On 4 April 2017 at 06:21, Ben Gamari wrote: > > tl;dr: If you would like to produce a binary distribution for GHC > 8.2.1-rc1 then let me know, grab the source distribution and > start building. The binary distributions will be announced one > week from today. > > Hello GHC packagers, > > I am happy to announce the release of the 8.2.1-rc1 source distribution > to binary packagers. You will find the usual source artifacts at > > http://downloads.haskell.org/~ghc/8.2.1-rc1/ > > As usual, the sooner we can get the binary distributions together the > better, but I will hold off on announcing the distributions until next > Sunday to ensure we're all on the same page. It would be appreciated if > you could reply to this message confirming that you intend to offer a > binary distribution this release. > > Otherwise, let me know if you have any trouble building your > distribution. I have yet to push the ghc-8.2.1-rc1 tag in case we > encounter unexpected issues but all of my builds with this tarball > thusfar have gone swimmingly save a few known issues (namely #13426, > #13233, and #13509). > > Good luck and thanks for all of your work! > > Cheers, > > - Ben > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hvriedel at gmail.com Tue Apr 4 07:20:42 2017 From: hvriedel at gmail.com (Herbert Valerio Riedel) Date: Tue, 04 Apr 2017 07:20:42 +0000 Subject: GHC 8.2.1-rc1 source tarball availability In-Reply-To: References: <877ffpu3ax.fsf@smart-cactus.org> <87r3184xc2.fsf@ben-laptop.smart-cactus.org> Message-ID: $ tar xOf ghc-8.2.0.20170404-src.tar.xz ghc-8.2.0.20170404/GIT_COMMIT_ID ; echo d67f0471cd3584c5a548ff30c9023b599b598d3e On Tue, Apr 4, 2017 at 9:01 AM Alan & Kim Zimmerman wrote: > There is no tag in the source tree, which commit has been used? > > Alan > > On 4 April 2017 at 06:21, Ben Gamari wrote: > > > tl;dr: If you would like to produce a binary distribution for GHC > 8.2.1-rc1 then let me know, grab the source distribution and > start building. The binary distributions will be announced one > week from today. > > Hello GHC packagers, > > I am happy to announce the release of the 8.2.1-rc1 source distribution > to binary packagers. You will find the usual source artifacts at > > http://downloads.haskell.org/~ghc/8.2.1-rc1/ > > As usual, the sooner we can get the binary distributions together the > better, but I will hold off on announcing the distributions until next > Sunday to ensure we're all on the same page. It would be appreciated if > you could reply to this message confirming that you intend to offer a > binary distribution this release. > > Otherwise, let me know if you have any trouble building your > distribution. I have yet to push the ghc-8.2.1-rc1 tag in case we > encounter unexpected issues but all of my builds with this tarball > thusfar have gone swimmingly save a few known issues (namely #13426, > #13233, and #13509). > > Good luck and thanks for all of your work! > > Cheers, > > - Ben > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sgraf1337 at gmail.com Tue Apr 4 08:12:04 2017 From: sgraf1337 at gmail.com (Sebastian Graf) Date: Tue, 4 Apr 2017 10:12:04 +0200 Subject: Windows 10 Creators Update will break GHC (Was FW: GHC on Windows 10 15019+) Message-ID: Hi, when the Creators Update finally hits Windows 10 stable on April 11th (next week), every currently available GHC will be broken by it. See [1] for the previous discussion, [2] and [3] for the tickets discussing this. From what I read on the GHC 8.2 status wiki page, there might be an interval of some weeks where it will not be possible to compile a Haskell program on any up-to-date Windows 10 installation. I just wanted to raise awareness and offer myself to do guided grunt work (if there is any) to push out a hotfix/inplace patch release for GHC 8.0.2. Greetings, Sebastian [1] http://haskell.1045720.n5.nabble.com/FW-GHC-on-Windows-10-15019-td5853491.html [2] https://ghc.haskell.org/trac/ghc/ticket/13411 [3] https://ghc.haskell.org/trac/ghc/ticket/13472 -------------- next part -------------- An HTML attachment was scrubbed... URL: From sgraf1337 at gmail.com Tue Apr 4 08:14:19 2017 From: sgraf1337 at gmail.com (Sebastian Graf) Date: Tue, 4 Apr 2017 10:14:19 +0200 Subject: Windows 10 Creators Update will break GHC (Was FW: GHC on Windows 10 15019+) In-Reply-To: References: Message-ID: Also note that a minimal patch already exists which will ship in GHC 8.2: https://git.haskell.org/ghc.git/commitdiff/018ac7f4b2f7345e28d21fb1f99b44dbc79f6e85 It's 'just' a matter of packaging that up in a hotfix. On Tue, Apr 4, 2017 at 10:12 AM, Sebastian Graf wrote: > Hi, > > when the Creators Update finally hits Windows 10 stable on April 11th > (next week), every currently available GHC will be broken by it. > > See [1] for the previous discussion, [2] and [3] for the tickets > discussing this. From what I read on the GHC 8.2 status wiki page, there > might be an interval of some weeks where it will not be possible to compile > a Haskell program on any up-to-date Windows 10 installation. > > I just wanted to raise awareness and offer myself to do guided grunt work > (if there is any) to push out a hotfix/inplace patch release for GHC 8.0.2. > > Greetings, > Sebastian > > [1] http://haskell.1045720.n5.nabble.com/FW-GHC-on-Windows- > 10-15019-td5853491.html > [2] https://ghc.haskell.org/trac/ghc/ticket/13411 > [3] https://ghc.haskell.org/trac/ghc/ticket/13472 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Tue Apr 4 08:18:13 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Tue, 4 Apr 2017 08:18:13 +0000 Subject: Windows 10 Creators Update will break GHC (Was FW: GHC on Windows 10 15019+) In-Reply-To: References: Message-ID: Yikes that sounds bad! I just wanted to raise awareness and offer myself to do guided grunt work (if there is any) to push out a hotfix/inplace patch release for GHC 8.0.2. Help would be fantastic. Phyx, Ben, etc: do we have a plan? Simon From: ghc-devs [mailto:ghc-devs-bounces at haskell.org] On Behalf Of Sebastian Graf Sent: 04 April 2017 09:12 To: ghc-devs Subject: Windows 10 Creators Update will break GHC (Was FW: GHC on Windows 10 15019+) Hi, when the Creators Update finally hits Windows 10 stable on April 11th (next week), every currently available GHC will be broken by it. See [1] for the previous discussion, [2] and [3] for the tickets discussing this. From what I read on the GHC 8.2 status wiki page, there might be an interval of some weeks where it will not be possible to compile a Haskell program on any up-to-date Windows 10 installation. I just wanted to raise awareness and offer myself to do guided grunt work (if there is any) to push out a hotfix/inplace patch release for GHC 8.0.2. Greetings, Sebastian [1] http://haskell.1045720.n5.nabble.com/FW-GHC-on-Windows-10-15019-td5853491.html [2] https://ghc.haskell.org/trac/ghc/ticket/13411 [3] https://ghc.haskell.org/trac/ghc/ticket/13472 -------------- next part -------------- An HTML attachment was scrubbed... URL: From lonetiger at gmail.com Tue Apr 4 12:15:32 2017 From: lonetiger at gmail.com (Phyx) Date: Tue, 04 Apr 2017 12:15:32 +0000 Subject: Windows 10 Creators Update will break GHC (Was FW: GHC on Windows 10 15019+) In-Reply-To: References: Message-ID: We have discussed a few options on what to do, but haven't gotten a conclusion/concensus yet. Last we talked one option was presented to just re-tar the 7.10.3 and 8.0.2 tarballs with the fix. The gcc wrapper does not affect code generation of the compiler, it's just there to correct paths that are hard-coded in gcc because it was built to be used in MSYS2. 7.10.3 would be needed since we still support bootstrapping using it. This would be the easiest solution. Ben what do you think about changing the already released tarballs? Or would you prefer a full new release? On Tue, 4 Apr 2017, 09:18 Simon Peyton Jones via ghc-devs, < ghc-devs at haskell.org> wrote: > Yikes that sounds bad! > > > > I just wanted to raise awareness and offer myself to do guided grunt work > (if there is any) to push out a hotfix/inplace patch release for GHC 8.0.2. > > > > Help would be fantastic. Phyx, Ben, etc: do we have a plan? > > > > Simon > > > > *From:* ghc-devs [mailto:ghc-devs-bounces at haskell.org] *On Behalf Of *Sebastian > Graf > *Sent:* 04 April 2017 09:12 > *To:* ghc-devs > *Subject:* Windows 10 Creators Update will break GHC (Was FW: GHC on > Windows 10 15019+) > > > > Hi, > > > > when the Creators Update finally hits Windows 10 stable on April 11th > (next week), every currently available GHC will be broken by it. > > > > See [1] for the previous discussion, [2] and [3] for the tickets > discussing this. From what I read on the GHC 8.2 status wiki page, there > might be an interval of some weeks where it will not be possible to compile > a Haskell program on any up-to-date Windows 10 installation. > > > > I just wanted to raise awareness and offer myself to do guided grunt work > (if there is any) to push out a hotfix/inplace patch release for GHC 8.0.2. > > > > Greetings, > > Sebastian > > > > [1] > http://haskell.1045720.n5.nabble.com/FW-GHC-on-Windows-10-15019-td5853491.html > > [2] https://ghc.haskell.org/trac/ghc/ticket/13411 > > [3] https://ghc.haskell.org/trac/ghc/ticket/13472 > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ryan.trinkle at gmail.com Tue Apr 4 12:25:15 2017 From: ryan.trinkle at gmail.com (Ryan Trinkle) Date: Tue, 4 Apr 2017 08:25:15 -0400 Subject: Windows 10 Creators Update will break GHC (Was FW: GHC on Windows 10 15019+) In-Reply-To: References: Message-ID: I would strongly urge that any new tarballs be released under new names, and that the old tarballs be kept in place. Changing existing tarball URLs silently causes breakage for Nix, and, I would imagine, for any other build systems that are particularly concerned with reproducibility. Nix packages based on GHC use an SHA-256 hash of the tarball to check it when downloading, which of course will be broken by any change, no matter how minor. Of course, this doesn't necessarily impact the version number directly. On Tue, Apr 4, 2017 at 8:15 AM, Phyx wrote: > We have discussed a few options on what to do, but haven't gotten a > conclusion/concensus yet. > > Last we talked one option was presented to just re-tar the 7.10.3 and > 8.0.2 tarballs with the fix. The gcc wrapper does not affect code > generation of the compiler, it's just there to correct paths that are > hard-coded in gcc because it was built to be used in MSYS2. > > 7.10.3 would be needed since we still support bootstrapping using it. This > would be the easiest solution. > > Ben what do you think about changing the already released tarballs? Or > would you prefer a full new release? > > On Tue, 4 Apr 2017, 09:18 Simon Peyton Jones via ghc-devs, < > ghc-devs at haskell.org> wrote: > >> Yikes that sounds bad! >> >> >> >> I just wanted to raise awareness and offer myself to do guided grunt work >> (if there is any) to push out a hotfix/inplace patch release for GHC 8.0.2. >> >> >> >> Help would be fantastic. Phyx, Ben, etc: do we have a plan? >> >> >> >> Simon >> >> >> >> *From:* ghc-devs [mailto:ghc-devs-bounces at haskell.org] *On Behalf Of *Sebastian >> Graf >> *Sent:* 04 April 2017 09:12 >> *To:* ghc-devs >> *Subject:* Windows 10 Creators Update will break GHC (Was FW: GHC on >> Windows 10 15019+) >> >> >> >> Hi, >> >> >> >> when the Creators Update finally hits Windows 10 stable on April 11th >> (next week), every currently available GHC will be broken by it. >> >> >> >> See [1] for the previous discussion, [2] and [3] for the tickets >> discussing this. From what I read on the GHC 8.2 status wiki page, there >> might be an interval of some weeks where it will not be possible to compile >> a Haskell program on any up-to-date Windows 10 installation. >> >> >> >> I just wanted to raise awareness and offer myself to do guided grunt work >> (if there is any) to push out a hotfix/inplace patch release for GHC 8.0.2. >> >> >> >> Greetings, >> >> Sebastian >> >> >> >> [1] http://haskell.1045720.n5.nabble.com/FW-GHC-on-Windows- >> 10-15019-td5853491.html >> >> [2] https://ghc.haskell.org/trac/ghc/ticket/13411 >> >> [3] https://ghc.haskell.org/trac/ghc/ticket/13472 >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Tue Apr 4 13:57:39 2017 From: ben at well-typed.com (Ben Gamari) Date: Tue, 04 Apr 2017 09:57:39 -0400 Subject: GHC 8.2.1-rc1 source tarball availability In-Reply-To: References: <877ffpu3ax.fsf@smart-cactus.org> <87r3184xc2.fsf@ben-laptop.smart-cactus.org> Message-ID: <87k27046ng.fsf@ben-laptop.smart-cactus.org> Alan & Kim Zimmerman writes: > There is no tag in the source tree, which commit has been used? > As I mention in the message, I intentionally held off on pushing the tag to avoid the need to force push in case this commit ends up being bad. The tarball was produced from d67f0471cd3584c5a548ff30c9023b599b598d3e. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at smart-cactus.org Tue Apr 4 14:11:06 2017 From: ben at smart-cactus.org (Ben Gamari) Date: Tue, 04 Apr 2017 10:11:06 -0400 Subject: Windows 10 Creators Update will break GHC (Was FW: GHC on Windows 10 15019+) In-Reply-To: References: Message-ID: <87efx84611.fsf@ben-laptop.smart-cactus.org> Phyx writes: > We have discussed a few options on what to do, but haven't gotten a > conclusion/concensus yet. > Tamar and I have discussed this on IRC in the passed. I tend to agree that we will need to, at the very least, put out a new 8.0.2. I'm not sure about 7.10. Moreover, there is reportedly a non-negligible fraction of users still using 7.8 releases, although I'm not keen on going back that far given that this was pre-submodules. > Last we talked one option was presented to just re-tar the 7.10.3 and 8.0.2 > tarballs with the fix. The gcc wrapper does not affect code generation of > the compiler, it's just there to correct paths that are hard-coded in gcc > because it was built to be used in MSYS2. > > 7.10.3 would be needed since we still support bootstrapping using it. This > would be the easiest solution. > > Ben what do you think about changing the already released tarballs? Or > would you prefer a full new release? > We certainly can't change the existing tarballs. However, we could just push out a 7.10.4 (or perhaps 7.10.3b, I'm unsure) release with binary distributions for Windows only. There are a few unrelated patches on the ghc-7.10 branch that weren't present in 7.10.3, but I believe they are quite low-risk. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at smart-cactus.org Tue Apr 4 14:12:08 2017 From: ben at smart-cactus.org (Ben Gamari) Date: Tue, 04 Apr 2017 10:12:08 -0400 Subject: Windows 10 Creators Update will break GHC (Was FW: GHC on Windows 10 15019+) In-Reply-To: References: Message-ID: <87bmsc45zb.fsf@ben-laptop.smart-cactus.org> Ryan Trinkle writes: > I would strongly urge that any new tarballs be released under new names, > and that the old tarballs be kept in place. Changing existing tarball URLs > silently causes breakage for Nix, and, I would imagine, for any other build > systems that are particularly concerned with reproducibility. Nix packages > based on GHC use an SHA-256 hash of the tarball to check it when > downloading, which of course will be broken by any change, no matter how > minor. > I completely agree here; I avoid updating existing artifacts whenever possible. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From george.colpitts at gmail.com Tue Apr 4 21:00:04 2017 From: george.colpitts at gmail.com (George Colpitts) Date: Tue, 04 Apr 2017 21:00:04 +0000 Subject: GHC 8.2.1-rc1 source tarball availability In-Reply-To: <87r3184xc2.fsf@ben-laptop.smart-cactus.org> References: <877ffpu3ax.fsf@smart-cactus.org> <87r3184xc2.fsf@ben-laptop.smart-cactus.org> Message-ID: I don't know how to produce a binary distribution but I did do a build on Mac OS 10.12.4 with XCode 8.3. It went flawlessly and subjectively seemed much faster than previous builds. I then did a cabal install of hlint and verified that hlint worked. On Tue, Apr 4, 2017 at 1:22 AM Ben Gamari wrote: > > tl;dr: If you would like to produce a binary distribution for GHC > 8.2.1-rc1 then let me know, grab the source distribution and > start building. The binary distributions will be announced one > week from today. > > Hello GHC packagers, > > I am happy to announce the release of the 8.2.1-rc1 source distribution > to binary packagers. You will find the usual source artifacts at > > http://downloads.haskell.org/~ghc/8.2.1-rc1/ > > As usual, the sooner we can get the binary distributions together the > better, but I will hold off on announcing the distributions until next > Sunday to ensure we're all on the same page. It would be appreciated if > you could reply to this message confirming that you intend to offer a > binary distribution this release. > > Otherwise, let me know if you have any trouble building your > distribution. I have yet to push the ghc-8.2.1-rc1 tag in case we > encounter unexpected issues but all of my builds with this tarball > thusfar have gone swimmingly save a few known issues (namely #13426, > #13233, and #13509). > > Good luck and thanks for all of your work! > > Cheers, > > - Ben > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam at adamsteen.com.au Wed Apr 5 07:39:18 2017 From: adam at adamsteen.com.au (Adam Steen) Date: Wed, 5 Apr 2017 15:39:18 +0800 Subject: Setting Up a OpenBSD System for Building GHC Wiki Page Message-ID: Hi All I have created the Building/Preparation/OpenBSD Wiki Page, i have not yet link it to Building/Preparation page yet, but am looking for a review Setting Up a OpenBSD System for Building GHC Cheers Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Wed Apr 5 15:06:00 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Wed, 5 Apr 2017 15:06:00 +0000 Subject: Windows build Message-ID: Dear Windows folk, Here's my testsuite output for a clean build on Windows last night. Rather a lot of failures - can we reduce them? Simon SUMMARY for test run started at Wed Apr 5 01:37:30 2017 GMTST 0:50:08 spent to go through 5822 total tests, which gave rise to 16777 test cases, of which 11077 were skipped 52 had missing libraries 5486 expected passes 152 expected failures 17 caused framework failures 0 caused framework warnings 0 unexpected passes 9 unexpected failures 1 unexpected stat failures Unexpected failures: ./backpack/cabal/bkpcabal07/bkpcabal07.run bkpcabal07 [bad exit code] (normal) ./backpack/cabal/bkpcabal06/bkpcabal06.run bkpcabal06 [bad exit code] (normal) ./plugins/T10420.run T10420 [bad exit code] (normal) ./plugins/plugins07.run plugins07 [bad exit code] (normal) ./plugins/T10294a.run T10294a [bad exit code] (normal) ./plugins/T11244.run T11244 [bad stderr] (normal) ./th/T13366.run T13366 [bad exit code] (normal) ./th/T13366.run T13366 [bad exit code] (ext-interp) ../../libraries/base/tests/IO/readwrite002.run readwrite002 [bad stdout] (normal) Unexpected stat failures: perf/compiler/T5837.run T5837 [stat too good] (normal) Framework failures: . registry001 [duplicate] (There are multiple tests with this name) . helloworld [duplicate] (There are multiple tests with this name) . lasterror [duplicate] (There are multiple tests with this name) . T4452 [duplicate] (There are multiple tests with this name) . PokeTZI [duplicate] (There are multiple tests with this name) . HandleConversion [duplicate] (There are multiple tests with this name) ./indexed-types/should_fail/T13102/T13102.run T13102 [runTest] (Unhandled exception: [Errno 90] Directory not empty: '/c/Users/simonpj/AppData/Local/Temp/ghctest-1gr0gzv9/test spaces/./indexed-types/should_fail/T13102/T13102.run/orphan/dist/package.conf.inplace') ./plugins/T10420.run T10420 [normal] (pre_cmd failed: 2) ./plugins/plugins07.run plugins07 [normal] (pre_cmd failed: 2) ./plugins/T10294a.run T10294a [normal] (pre_cmd failed: 2) ./plugins/T11244.run T11244 [normal] (pre_cmd failed: 2) ./safeHaskell/check/pkg01/safePkg01.run safePkg01 [runTest] (Unhandled exception: [Errno 90] Directory not empty: '/c/Users/simonpj/AppData/Local/Temp/ghctest-1gr0gzv9/test spaces/./safeHaskell/check/pkg01/safePkg01.run/pdb.safePkg01/install/x86_64-windows-ghc-8.3.20170324/safePkg01-1.0-EBB2cBBfiph67c3CwCtZOx') ./safeHaskell/check/pkg01/ImpSafeOnly06.run ImpSafeOnly06 [runTest] (Unhandled exception: [Errno 90] Directory not empty: '/c/Users/simonpj/AppData/Local/Temp/ghctest-1gr0gzv9/test spaces/./safeHaskell/check/pkg01/ImpSafeOnly06.run/pdb.ImpSafeOnly06/install/x86_64-windows-ghc-8.3.20170324/safePkg01-1.0-EBB2cBBfiph67c3CwCtZOx') ./safeHaskell/check/pkg01/ImpSafeOnly08.run ImpSafeOnly08 [runTest] (Unhandled exception: [Errno 90] Directory not empty: '/c/Users/simonpj/AppData/Local/Temp/ghctest-1gr0gzv9/test spaces/./safeHaskell/check/pkg01/ImpSafeOnly08.run/pdb.ImpSafeOnly08/install/x86_64-windows-ghc-8.3.20170324') ./simplCore/should_compile/T7702.run T7702 [runTest] (Unhandled exception: [Errno 90] Directory not empty: '/c/Users/simonpj/AppData/Local/Temp/ghctest-1gr0gzv9/test spaces/./simplCore/should_compile/T7702.run/T7702plugin/pkg.T7702/dist/package.conf.inplace') ../../libraries/Win32/tests/registry001.run registry001 [runTest] (Unhandled exception: [Errno 2] No such file or directory: '/c/Users/simonpj/AppData/Local/Temp/ghctest-1gr0gzv9/test spaces/../../libraries/Win32/tests/registry001.run/registry001.o') ../../libraries/base/tests/IO/hDuplicateTo001.run hDuplicateTo001 [runTest] (Unhandled exception: [Errno 90] Directory not empty: '/c/Users/simonpj/AppData/Local/Temp/ghctest-1gr0gzv9/test spaces/../../libraries/base/tests/IO/hDuplicateTo001.run') == Start post-testsuite package check GHC package manager version 8.3.20170324 Timestamp 2017-04-04 22:31:56.0162608 UTC for C:\code\HEAD\inplace\lib\package.conf.d\package.cache using cache: C:\code\HEAD\inplace\lib\package.conf.d\package.cache db stack: ["C:\\Users\\simonpj\\AppData\\Roaming\\ghc\\x86_64-mingw32-8.3.20170324\\package.conf.d","C:\\code\\HEAD\\inplace\\lib\\package.conf.d"] flag db stack: ["C:\\Users\\simonpj\\AppData\\Roaming\\ghc\\x86_64-mingw32-8.3.20170324\\package.conf.d","C:\\code\\HEAD\\inplace\\lib\\package.conf.d"] == End post-testsuite package check ------------------------------------------------------------------- Oops! Looks like you have some unexpected test results or framework failures. Please fix them before pushing/sending patches. -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Wed Apr 5 15:29:44 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Wed, 5 Apr 2017 15:29:44 +0000 Subject: linux build broken Message-ID: My linux build is broken. See below. Help! Simon libraries/time/ghc.mk:4: libraries/time/dist-install/build/.depend-v-dyn.haskell: No such file or directory "rm" -f libraries/time/dist-install/build/.depend-v-dyn.haskell.tmp "inplace/bin/ghc-stage1" -M -static -O0 -H64m -Wall -fllvm-fill-undef-with-garbage -Werror -Wcpp-undef -this-unit-id time-1.8.0.1 -hide-all-packages -i -ilibraries/time/lib -ilibraries/time/dist-install/build -Ilibraries/time/dist-install/build -ilibraries/time/dist-install/build/./autogen -Ilibraries/time/dist-install/build/./autogen -Ilibraries/time/lib/include -optP-DLANGUAGE_Rank2Types -optP-DLANGUAGE_DeriveDataTypeable -optP-DLANGUAGE_StandaloneDeriving -optP-include -optPlibraries/time/dist-install/build/./autogen/cabal_macros.h -package-id base-4.10.0.0 -package-id deepseq-1.4.3.0 -Wall -fwarn-tabs -XHaskell2010 -XCPP -XRank2Types -XDeriveDataTypeable -XStandaloneDeriving -O -dcore-lint -dno-debug-output -ticky -no-user-package-db -rtsopts -Wno-deprecated-flags -Wnoncanonical-monad-instances -odir libraries/time/dist-install/build -hidir libraries/time/dist-install/build -stubdir libraries/time/dist-install/build -dep-makefile libraries/time/dist-install/build/.depend-v-dyn.haskell.tmp -dep-suffix "" -dep-suffix "dyn_" -include-pkg-deps libraries/time/lib/Data/Time/Calendar.hs libraries/time/lib/Data/Time/Calendar/MonthDay.hs libraries/time/lib/Data/Time/Calendar/OrdinalDate.hs libraries/time/lib/Data/Time/Calendar/WeekDate.hs libraries/time/lib/Data/Time/Calendar/Julian.hs libraries/time/lib/Data/Time/Calendar/Easter.hs libraries/time/lib/Data/Time/Clock.hs libraries/time/lib/Data/Time/Clock/System.hs libraries/time/lib/Data/Time/Clock/POSIX.hs libraries/time/lib/Data/Time/Clock/TAI.hs libraries/time/lib/Data/Time/LocalTime.hs libraries/time/lib/Data/Time/Format.hs libraries/time/lib/Data/Time.hs libraries/time/lib/Data/Time/Calendar/Private.hs libraries/time/lib/Data/Time/Calendar/Days.hs libraries/time/lib/Data/Time/Calendar/Gregorian.hs libraries/time/lib/Data/Time/Calendar/JulianYearDay.hs libraries/time/lib/Data/Time/Clock/Internal/DiffTime.hs libraries/time/lib/Data/Time/Clock/Internal/AbsoluteTime.hs libraries/time/lib/Data/Time/Clock/Internal/NominalDiffTime.hs libraries/time/lib/Data/Time/Clock/Internal/POSIXTime.hs libraries/time/lib/Data/Time/Clock/Internal/UniversalTime.hs libraries/time/lib/Data/Time/Clock/Internal/SystemTime.hs libraries/time/lib/Data/Time/Clock/Internal/UTCTime.hs libraries/time/lib/Data/Time/Clock/Internal/CTimeval.hs libraries/time/dist-install/build/Data/Time/Clock/Internal/CTimespec.hs libraries/time/lib/Data/Time/Clock/Internal/UTCDiff.hs libraries/time/lib/Data/Time/LocalTime/Internal/TimeZone.hs libraries/time/lib/Data/Time/LocalTime/Internal/TimeOfDay.hs libraries/time/lib/Data/Time/LocalTime/Internal/LocalTime.hs libraries/time/lib/Data/Time/LocalTime/Internal/ZonedTime.hs libraries/time/lib/Data/Time/Format/Parse.hs libraries/time/lib/Data/Time/Format/Locale.hs libraries/time/lib/Data/Time/Clock/Internal/SystemTime.hs:22:0: error: error: "HAVE_CLOCK_GETTIME" is not defined [-Werror=undef] | 22 | #elif HAVE_CLOCK_GETTIME | ^ libraries/time/lib/Data/Time/Clock/Internal/SystemTime.hs:70:0: error: error: "HAVE_CLOCK_GETTIME" is not defined [-Werror=undef] | 70 | #elif HAVE_CLOCK_GETTIME | ^ cc1: all warnings being treated as errors `gcc' failed in phase `C pre-processor'. (Exit code: 1) make[1]: *** [libraries/time/dist-install/build/.depend-v-dyn.haskell] Error 1 make: *** [all] Error 2 simonpj at cam-05-unx:~/5builds/HEAD-5$ -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Wed Apr 5 15:31:43 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Wed, 5 Apr 2017 15:31:43 +0000 Subject: linux again Message-ID: PS: a from-scratch build fell over like this (below). the time-library problem in my other message happened when I said 'sh validate -fast -no-clean'; that is, with -no-clean. Which is odd. Simoin checking for __gmpz_powm_sec in -lgmp... yes checking whether to use in-tree GMP... no checking gmp.h usability... yes checking gmp.h presence... yes checking for gmp.h... yes checking GMP version... 5.0.2 configure: creating ./config.status config.status: creating integer-gmp.buildinfo config.status: creating gmp/config.mk config.status: creating include/HsIntegerGmp.h configure: WARNING: unrecognized options: --with-compiler "/home/simonpj/5builds/HEAD-5/inplace/bin/ghc-pkg" update -v0 --force libraries/integer-gmp/dist-install/inplace-pkg-config integer-gmp-1.0.0.1: Warning: haddock-interfaces: /5playpen/simonpj/HEAD-5/libraries/integer-gmp/dist-install/doc/html/integer-gmp/integer-gmp.haddock doesn't exist or isn't a file <> echo "compiler_stage2_depfile_haskell_EXISTS = YES" >> compiler/stage2/build/.depend-v-dyn.haskell.tmp for dir in compiler/stage2/build/./ compiler/stage2/build/CodeGen/ compiler/stage2/build/CodeGen/Platform/ compiler/stage2/build/Dwarf/ compiler/stage2/build/Hoopl/ compiler/stage2/build/Llvm/ compiler/stage2/build/LlvmCodeGen/ compiler/stage2/build/PPC/ compiler/stage2/build/RegAlloc/ compiler/stage2/build/RegAlloc/Graph/ compiler/stage2/build/RegAlloc/Linear/ compiler/stage2/build/RegAlloc/Linear/PPC/ compiler/stage2/build/RegAlloc/Linear/SPARC/ compiler/stage2/build/RegAlloc/Linear/X86/ compiler/stage2/build/RegAlloc/Linear/X86_64/ compiler/stage2/build/SPARC/ compiler/stage2/build/SPARC/CodeGen/ compiler/stage2/build/SysTools/ compiler/stage2/build/Vectorise/ compiler/stage2/build/Vectorise/Builtins/ compiler/stage2/build/Vectorise/Generic/ compiler/stage2/build/Vectorise/Monad/ compiler/stage2/build/Vectorise/Type/ compiler/stage2/build/Vectorise/Utils/ compiler/stage2/build/X86/; do if test ! -d $dir; then mkdir -p $dir; fi done grep -v ' : [a-zA-Z]:/' compiler/stage2/build/.depend-v-dyn.haskell.tmp > compiler/stage2/build/.depend-v-dyn.haskell.tmp2 sed -e '/hs$/ p' -e '/hs$/ s/o /hi /g' -e '/hs$/ s/:/ : %hi: %o /' -e '/hs$/ s/^/$(eval $(call hi-rule,/' -e '/hs$/ s/$/))/' -e '/hs-boot$/ p' -e '/hs-boot$/ s/o-boot /hi-boot /g' -e '/hs-boot$/ s/:/ : %hi-boot: %o-boot /' -e '/hs-boot$/ s/^/$(eval $(call hi-rule,/' -e '/hs-boot$/ s/$/))/' compiler/stage2/build/.depend-v-dyn.haskell.tmp2 > compiler/stage2/build/.depend-v-dyn.haskell make: *** [all] Error 2 -------------- next part -------------- An HTML attachment was scrubbed... URL: From ggreif at gmail.com Wed Apr 5 15:41:24 2017 From: ggreif at gmail.com (Gabor Greif) Date: Wed, 5 Apr 2017 17:41:24 +0200 Subject: linux build broken In-Reply-To: References: Message-ID: I see this too :-( I have checked in a fix (please pull!), but libraries/time still needs this tweak: index 6027cdf..b87d302 100644 --- a/lib/Data/Time/Clock/Internal/SystemTime.hs +++ b/lib/Data/Time/Clock/Internal/SystemTime.hs @@ -19,7 +19,7 @@ import Data.Time.Clock.Internal.DiffTime #ifdef mingw32_HOST_OS import qualified System.Win32.Time as Win32 -#elif HAVE_CLOCK_GETTIME +#elif defined(HAVE_CLOCK_GETTIME) import Data.Time.Clock.Internal.CTimespec import Foreign.C.Types (CTime(..), CLong(..)) #else @@ -67,7 +67,7 @@ getSystemTime = do getTime_resolution = 100E-9 -- 100ns getTAISystemTime = Nothing -#elif HAVE_CLOCK_GETTIME +#elif defined(HAVE_CLOCK_GETTIME) -- Use hi-res clock_gettime timespecToSystemTime :: CTimespec -> SystemTime Cheers, Gabor On 4/5/17, Simon Peyton Jones via ghc-devs wrote: > My linux build is broken. See below. Help! > Simon > > libraries/time/ghc.mk:4: > libraries/time/dist-install/build/.depend-v-dyn.haskell: No such file or > directory > "rm" -f libraries/time/dist-install/build/.depend-v-dyn.haskell.tmp > "inplace/bin/ghc-stage1" -M -static -O0 -H64m -Wall > -fllvm-fill-undef-with-garbage -Werror -Wcpp-undef -this-unit-id > time-1.8.0.1 -hide-all-packages -i -ilibraries/time/lib > -ilibraries/time/dist-install/build -Ilibraries/time/dist-install/build > -ilibraries/time/dist-install/build/./autogen > -Ilibraries/time/dist-install/build/./autogen -Ilibraries/time/lib/include > -optP-DLANGUAGE_Rank2Types -optP-DLANGUAGE_DeriveDataTypeable > -optP-DLANGUAGE_StandaloneDeriving -optP-include > -optPlibraries/time/dist-install/build/./autogen/cabal_macros.h -package-id > base-4.10.0.0 -package-id deepseq-1.4.3.0 -Wall -fwarn-tabs -XHaskell2010 > -XCPP -XRank2Types -XDeriveDataTypeable -XStandaloneDeriving -O -dcore-lint > -dno-debug-output -ticky -no-user-package-db -rtsopts -Wno-deprecated-flags > -Wnoncanonical-monad-instances -odir libraries/time/dist-install/build > -hidir libraries/time/dist-install/build -stubdir > libraries/time/dist-install/build -dep-makefile > libraries/time/dist-install/build/.depend-v-dyn.haskell.tmp -dep-suffix "" > -dep-suffix "dyn_" -include-pkg-deps > libraries/time/lib/Data/Time/Calendar.hs > libraries/time/lib/Data/Time/Calendar/MonthDay.hs > libraries/time/lib/Data/Time/Calendar/OrdinalDate.hs > libraries/time/lib/Data/Time/Calendar/WeekDate.hs > libraries/time/lib/Data/Time/Calendar/Julian.hs > libraries/time/lib/Data/Time/Calendar/Easter.hs > libraries/time/lib/Data/Time/Clock.hs > libraries/time/lib/Data/Time/Clock/System.hs > libraries/time/lib/Data/Time/Clock/POSIX.hs > libraries/time/lib/Data/Time/Clock/TAI.hs > libraries/time/lib/Data/Time/LocalTime.hs > libraries/time/lib/Data/Time/Format.hs libraries/time/lib/Data/Time.hs > libraries/time/lib/Data/Time/Calendar/Private.hs > libraries/time/lib/Data/Time/Calendar/Days.hs > libraries/time/lib/Data/Time/Calendar/Gregorian.hs > libraries/time/lib/Data/Time/Calendar/JulianYearDay.hs > libraries/time/lib/Data/Time/Clock/Internal/DiffTime.hs > libraries/time/lib/Data/Time/Clock/Internal/AbsoluteTime.hs > libraries/time/lib/Data/Time/Clock/Internal/NominalDiffTime.hs > libraries/time/lib/Data/Time/Clock/Internal/POSIXTime.hs > libraries/time/lib/Data/Time/Clock/Internal/UniversalTime.hs > libraries/time/lib/Data/Time/Clock/Internal/SystemTime.hs > libraries/time/lib/Data/Time/Clock/Internal/UTCTime.hs > libraries/time/lib/Data/Time/Clock/Internal/CTimeval.hs > libraries/time/dist-install/build/Data/Time/Clock/Internal/CTimespec.hs > libraries/time/lib/Data/Time/Clock/Internal/UTCDiff.hs > libraries/time/lib/Data/Time/LocalTime/Internal/TimeZone.hs > libraries/time/lib/Data/Time/LocalTime/Internal/TimeOfDay.hs > libraries/time/lib/Data/Time/LocalTime/Internal/LocalTime.hs > libraries/time/lib/Data/Time/LocalTime/Internal/ZonedTime.hs > libraries/time/lib/Data/Time/Format/Parse.hs > libraries/time/lib/Data/Time/Format/Locale.hs > > libraries/time/lib/Data/Time/Clock/Internal/SystemTime.hs:22:0: error: > error: "HAVE_CLOCK_GETTIME" is not defined [-Werror=undef] > | > 22 | #elif HAVE_CLOCK_GETTIME > | ^ > > libraries/time/lib/Data/Time/Clock/Internal/SystemTime.hs:70:0: error: > error: "HAVE_CLOCK_GETTIME" is not defined [-Werror=undef] > | > 70 | #elif HAVE_CLOCK_GETTIME > | ^ > cc1: all warnings being treated as errors > `gcc' failed in phase `C pre-processor'. (Exit code: 1) > make[1]: *** [libraries/time/dist-install/build/.depend-v-dyn.haskell] Error > 1 > make: *** [all] Error 2 > simonpj at cam-05-unx:~/5builds/HEAD-5$ > From ben at smart-cactus.org Wed Apr 5 16:00:05 2017 From: ben at smart-cactus.org (Ben Gamari) Date: Wed, 05 Apr 2017 12:00:05 -0400 Subject: linux build broken In-Reply-To: References: Message-ID: <87pogq3kvu.fsf@ben-laptop.smart-cactus.org> Simon Peyton Jones via ghc-devs writes: > My linux build is broken. See below. Help! All of this is due to the recent CPP cleanup, unfortunately. I think I will revert it for now and we can work out the issues elsewhere. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ggreif at gmail.com Wed Apr 5 16:22:11 2017 From: ggreif at gmail.com (Gabor Greif) Date: Wed, 5 Apr 2017 18:22:11 +0200 Subject: linux build broken In-Reply-To: <87pogq3kvu.fsf@ben-laptop.smart-cactus.org> References: <87pogq3kvu.fsf@ben-laptop.smart-cactus.org> Message-ID: Ben, I am compiling with a libraries/time fix now, looks good. I have pushed it to https://github.com/ggreif/packages-time/tree/ghc maybe you can review it and make it official somehow. Cheers, Gabor On 4/5/17, Ben Gamari wrote: > Simon Peyton Jones via ghc-devs writes: > >> My linux build is broken. See below. Help! > > All of this is due to the recent CPP cleanup, unfortunately. I think > I will revert it for now and we can work out the issues elsewhere. > > Cheers, > > - Ben > From karel.gardas at centrum.cz Wed Apr 5 18:58:41 2017 From: karel.gardas at centrum.cz (Karel Gardas) Date: Wed, 05 Apr 2017 20:58:41 +0200 Subject: Setting Up a OpenBSD System for Building GHC Wiki Page In-Reply-To: References: Message-ID: <58E53E61.7070802@centrum.cz> LGTM, although I've not needed to do any python business you do, but IMHO does not hurt anyway. Thanks, Karel On 04/ 5/17 09:39 AM, Adam Steen wrote: > Hi All > > I have created the Building/Preparation/OpenBSD Wiki Page, i have not > yet link it to Building/Preparation > page yet, > but am looking for a review > > Setting Up a OpenBSD System for Building GHC > > > Cheers > Adam > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > From slyich at gmail.com Wed Apr 5 20:01:56 2017 From: slyich at gmail.com (Sergei Trofimovich) Date: Wed, 5 Apr 2017 21:01:56 +0100 Subject: Setting Up a OpenBSD System for Building GHC Wiki Page In-Reply-To: <58E53E61.7070802@centrum.cz> References: <58E53E61.7070802@centrum.cz> Message-ID: <20170405210156.5760e4b6@sf> On Wed, 05 Apr 2017 20:58:41 +0200 Karel Gardas wrote: Thanks for starting the page Adam! Trying to wire older python look unusual. ghc-HEAD should work fine with python3. When I tried to build ghc on an fresh OpenBSD install I had to install the following on top: - pkg_add py-sphinx [optional, to get docs] - pkg_add libiconv -- Sergei -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: Цифровая подпись OpenPGP URL: From karel.gardas at centrum.cz Wed Apr 5 20:19:56 2017 From: karel.gardas at centrum.cz (Karel Gardas) Date: Wed, 05 Apr 2017 22:19:56 +0200 Subject: Setting Up a OpenBSD System for Building GHC Wiki Page In-Reply-To: <20170405210156.5760e4b6@sf> References: <58E53E61.7070802@centrum.cz> <20170405210156.5760e4b6@sf> Message-ID: <58E5516C.6060104@centrum.cz> On 04/ 5/17 10:01 PM, Sergei Trofimovich wrote: > - pkg_add libiconv Indeed, I completely overlooked this missing in pkg_add line. Anyway, the configure is the correct one --with-iconv*... Thanks for correction Sergei, Karel From kili at outback.escape.de Wed Apr 5 20:38:01 2017 From: kili at outback.escape.de (Matthias Kilian) Date: Wed, 5 Apr 2017 22:38:01 +0200 Subject: Setting Up a OpenBSD System for Building GHC Wiki Page In-Reply-To: References: Message-ID: <20170405203801.GA14065@nutty.outback.escape.de> Hi Adam, On Wed, Apr 05, 2017 at 03:39:18PM +0800, Adam Steen wrote: > I have created the Building/Preparation/OpenBSD Wiki Page, i have not yet > link it to Building/Preparation > page yet, but > am looking for a review > > Setting Up a OpenBSD System for Building GHC > Nice to see that some people actively work on ghc on OpenBSD (and apologies for me beeing such a slacker keeping the ghc package for OpenBSD up to date). FWIW, you don't have to pass the very exact package name to pkg_add, like 'autoconf-2.69p2' or ''automake-1.15p0'. There's a relatively new concept in pkg_tools called 'branches' (see pkg_add(1) manpage). You just have to replace the '-' separating the stem and the version by a '%', e.g. pkg_add autoconf%2.69 automake%1.15 Then you'll get autoconf-2.69 and aotmake-1.15 regardless of what's the current REVISION (aka package patchleve) ot the package. Ciao, Kili From ben at well-typed.com Wed Apr 5 21:39:15 2017 From: ben at well-typed.com (Ben Gamari) Date: Wed, 05 Apr 2017 17:39:15 -0400 Subject: Phabricator upgrade tonight Message-ID: <877f2y356k.fsf@ben-laptop.smart-cactus.org> Unless someone objects I will be performing an upgrade of Phabricator tonight. I'll likely start around 23:00 EST (4:00 UTC). As usual, Phabricator may be down for anywhere from 10 minutes to a few hours, depending upon how things go. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From juhpetersen at gmail.com Thu Apr 6 00:42:21 2017 From: juhpetersen at gmail.com (Jens Petersen) Date: Thu, 6 Apr 2017 09:42:21 +0900 Subject: GHC 8.2.1-rc1 source tarball availability In-Reply-To: <87r3184xc2.fsf@ben-laptop.smart-cactus.org> References: <877ffpu3ax.fsf@smart-cactus.org> <87r3184xc2.fsf@ben-laptop.smart-cactus.org> Message-ID: On 4 April 2017 at 13:21, Ben Gamari wrote: > I am happy to announce the release of the 8.2.1-rc1 source distribution > to binary packagers. > It seems to build okay for me on Fedora 26 so far. But the testsuite completely failed in timeout: see https://ghc.haskell.org/trac/ghc/ticket/13534 Cheers, Jens -------------- next part -------------- An HTML attachment was scrubbed... URL: From george.colpitts at gmail.com Thu Apr 6 01:17:58 2017 From: george.colpitts at gmail.com (George Colpitts) Date: Thu, 06 Apr 2017 01:17:58 +0000 Subject: GHC 8.2.1-rc1 source tarball availability In-Reply-To: References: <877ffpu3ax.fsf@smart-cactus.org> <87r3184xc2.fsf@ben-laptop.smart-cactus.org> Message-ID: I'd like to run the testsuite on macOS but I am having trouble following the documentation at https://ghc.haskell.org/trac/ghc/wiki/Building/RunningTests/Running Do I need to download something in addition to the source tarball or am I making some mistake? Following is what I tried: pwd /Users/gcolpitts/Downloads/ghc-8.2.0.20170404/libffi # doc says: The commands on this page can all be executed from the testsuite directory. bash-3.2$ find . -name testsuite ./libffi/build/testsuite ./libffi/build/x86_64-apple-darwin/testsuite bash-3.2$ pushd libffi/build/x86_64-apple-darwin/testsuite ~/Downloads/ghc-8.2.0.20170404/libffi/build/x86_64-apple-darwin/testsuite ~/Downloads/ghc-8.2.0.20170404 bash-3.2$ make test make: *** No rule to make target `test'. Stop. bash-3.2$ popd ~/Downloads/ghc-8.2.0.20170404 bash-3.2$ pushd libffi/build/testsuite ~/Downloads/ghc-8.2.0.20170404/libffi/build/testsuite ~/Downloads/ghc-8.2.0.20170404 bash-3.2$ make test make: *** No rule to make target `test'. Stop. bash-3.2$ popd ~/Downloads/ghc-8.2.0.20170404 bash-3.2$ make test /Applications/Xcode.app/Contents/Developer/usr/bin/make -C testsuite/tests CLEANUP=1 SUMMARY_FILE=../../testsuite_summary.txt make: *** testsuite/tests: No such file or directory. Stop. make: *** [test] Error 2 Thanks George On Wed, Apr 5, 2017 at 9:43 PM Jens Petersen wrote: > On 4 April 2017 at 13:21, Ben Gamari wrote: > > I am happy to announce the release of the 8.2.1-rc1 source distribution > to binary packagers. > > > It seems to build okay for me on Fedora 26 so far. > > But the testsuite completely failed in timeout: see > https://ghc.haskell.org/trac/ghc/ticket/13534 > > Cheers, Jens > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From allbery.b at gmail.com Thu Apr 6 01:20:04 2017 From: allbery.b at gmail.com (Brandon Allbery) Date: Wed, 5 Apr 2017 21:20:04 -0400 Subject: GHC 8.2.1-rc1 source tarball availability In-Reply-To: References: <877ffpu3ax.fsf@smart-cactus.org> <87r3184xc2.fsf@ben-laptop.smart-cactus.org> Message-ID: On Wed, Apr 5, 2017 at 9:17 PM, George Colpitts wrote: > /Users/gcolpitts/Downloads/ghc-8.2.0.20170404/libffi > I don't think you want to be in the libffi subdirectory; any test suite there would be specific to libffi and require you to follow its directions, not the ghc testsuite directions. -- brandon s allbery kf8nh sine nomine associates allbery.b at gmail.com ballbery at sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Thu Apr 6 03:00:19 2017 From: ben at well-typed.com (Ben Gamari) Date: Wed, 05 Apr 2017 23:00:19 -0400 Subject: Phabricator upgrade tonight In-Reply-To: <877f2y356k.fsf@ben-laptop.smart-cactus.org> References: <877f2y356k.fsf@ben-laptop.smart-cactus.org> Message-ID: <87wpay1br0.fsf@ben-laptop.smart-cactus.org> Ben Gamari writes: > Unless someone objects I will be performing an upgrade of Phabricator > tonight. I'll likely start around 23:00 EST (4:00 UTC). As usual, > Phabricator may be down for anywhere from 10 minutes to a few hours, > depending upon how things go. > As there has been no objection I'll be bringing down Phabricator in about ten minutes for the upgrade. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From adam at adamsteen.com.au Thu Apr 6 03:04:13 2017 From: adam at adamsteen.com.au (Adam Steen) Date: Thu, 6 Apr 2017 11:04:13 +0800 Subject: Setting Up a OpenBSD System for Building GHC Wiki Page In-Reply-To: <20170405203801.GA14065@nutty.outback.escape.de> References: <20170405203801.GA14065@nutty.outback.escape.de> Message-ID: Good Morning All Thanks for the replies @Karel After more testing it looks like i can remove the python business. @Sergei and @Karel libiconv is a prerequisite of ghc and is installed automatically, do you think this should be explicit? @Matthias I will have to have a read about branches and will update the wiki shortly. Cheers Adam On 6 April 2017 at 04:38, Matthias Kilian wrote: > Hi Adam, > > On Wed, Apr 05, 2017 at 03:39:18PM +0800, Adam Steen wrote: >> I have created the Building/Preparation/OpenBSD Wiki Page, i have not yet >> link it to Building/Preparation >> page yet, but >> am looking for a review >> >> Setting Up a OpenBSD System for Building GHC >> > > Nice to see that some people actively work on ghc on OpenBSD (and > apologies for me beeing such a slacker keeping the ghc package for > OpenBSD up to date). > > FWIW, you don't have to pass the very exact package name to pkg_add, > like 'autoconf-2.69p2' or ''automake-1.15p0'. There's a relatively new > concept in pkg_tools called 'branches' (see pkg_add(1) manpage). > > You just have to replace the '-' separating the stem and the version by > a '%', e.g. > > pkg_add autoconf%2.69 automake%1.15 > > Then you'll get autoconf-2.69 and aotmake-1.15 regardless of what's the > current REVISION (aka package patchleve) ot the package. > > Ciao, > Kili On Thu, Apr 6, 2017 at 4:38 AM, Matthias Kilian wrote: > Hi Adam, > > On Wed, Apr 05, 2017 at 03:39:18PM +0800, Adam Steen wrote: >> I have created the Building/Preparation/OpenBSD Wiki Page, i have not yet >> link it to Building/Preparation >> page yet, but >> am looking for a review >> >> Setting Up a OpenBSD System for Building GHC >> > > Nice to see that some people actively work on ghc on OpenBSD (and > apologies for me beeing such a slacker keeping the ghc package for > OpenBSD up to date). > > FWIW, you don't have to pass the very exact package name to pkg_add, > like 'autoconf-2.69p2' or ''automake-1.15p0'. There's a relatively new > concept in pkg_tools called 'branches' (see pkg_add(1) manpage). > > You just have to replace the '-' separating the stem and the version by > a '%', e.g. > > pkg_add autoconf%2.69 automake%1.15 > > Then you'll get autoconf-2.69 and aotmake-1.15 regardless of what's the > current REVISION (aka package patchleve) ot the package. > > Ciao, > Kili From ben at well-typed.com Thu Apr 6 03:32:08 2017 From: ben at well-typed.com (Ben Gamari) Date: Wed, 05 Apr 2017 23:32:08 -0400 Subject: Phabricator upgrade tonight In-Reply-To: <87wpay1br0.fsf@ben-laptop.smart-cactus.org> References: <877f2y356k.fsf@ben-laptop.smart-cactus.org> <87wpay1br0.fsf@ben-laptop.smart-cactus.org> Message-ID: <87shlm1a9z.fsf@ben-laptop.smart-cactus.org> Ben Gamari writes: > Ben Gamari writes: > >> Unless someone objects I will be performing an upgrade of Phabricator >> tonight. I'll likely start around 23:00 EST (4:00 UTC). As usual, >> Phabricator may be down for anywhere from 10 minutes to a few hours, >> depending upon how things go. >> > As there has been no objection I'll be bringing down Phabricator in > about ten minutes for the upgrade. > Phabricator is once again up. As usual, let me know if anything has broken. I should note that thanks to mpickering's valiant efforts the GHC Trac Tickets Differential field is now once again working. Thanks Matthew! Chers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From david at well-typed.com Thu Apr 6 05:10:46 2017 From: david at well-typed.com (David Feuer) Date: Thu, 06 Apr 2017 01:10:46 -0400 Subject: GHC 8.2.1-rc1 source tarball availability Message-ID: I'm not sure why you're trying to run things from the libffi directory. As far as I know, the test suite is normally run from ghc/testsuite. David FeuerWell-Typed, LLP -------- Original message --------From: George Colpitts Date: 4/5/17 9:17 PM (GMT-05:00) To: Jens Petersen , Ben Gamari Cc: GHC developers Subject: Re: GHC 8.2.1-rc1 source tarball availability I'd like to run the testsuite on macOS but I am having trouble following the documentation at https://ghc.haskell.org/trac/ghc/wiki/Building/RunningTests/Running Do I need to download something in addition to the source tarball or am I making some mistake? Following is what I tried:  pwd/Users/gcolpitts/Downloads/ghc-8.2.0.20170404/libffi# doc says: The commands on this page can all be executed from the testsuite directory.bash-3.2$ find . -name testsuite./libffi/build/testsuite./libffi/build/x86_64-apple-darwin/testsuitebash-3.2$ pushd libffi/build/x86_64-apple-darwin/testsuite~/Downloads/ghc-8.2.0.20170404/libffi/build/x86_64-apple-darwin/testsuite ~/Downloads/ghc-8.2.0.20170404bash-3.2$ make testmake: *** No rule to make target `test'.  Stop.bash-3.2$ popd~/Downloads/ghc-8.2.0.20170404bash-3.2$ pushd libffi/build/testsuite~/Downloads/ghc-8.2.0.20170404/libffi/build/testsuite ~/Downloads/ghc-8.2.0.20170404bash-3.2$ make testmake: *** No rule to make target `test'.  Stop.bash-3.2$ popd~/Downloads/ghc-8.2.0.20170404bash-3.2$ make test/Applications/Xcode.app/Contents/Developer/usr/bin/make -C testsuite/tests CLEANUP=1 SUMMARY_FILE=../../testsuite_summary.txtmake: *** testsuite/tests: No such file or directory.  Stop.make: *** [test] Error 2 ThanksGeorge On Wed, Apr 5, 2017 at 9:43 PM Jens Petersen wrote: On 4 April 2017 at 13:21, Ben Gamari wrote: I am happy to announce the release of the 8.2.1-rc1 source distribution to binary packagers. It seems to build okay for me on Fedora 26 so far. But the testsuite completely failed in timeout: see https://ghc.haskell.org/trac/ghc/ticket/13534 Cheers, Jens _______________________________________________ ghc-devs mailing list ghc-devs at haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From george.colpitts at gmail.com Thu Apr 6 13:39:48 2017 From: george.colpitts at gmail.com (George Colpitts) Date: Thu, 06 Apr 2017 13:39:48 +0000 Subject: testsuite not in GHC 8.2.1-rc1 source tarball ? In-Reply-To: References: Message-ID: Thanks Brandon After downloading the source tarball and doing a build successfully I wanted to run the testsuite. You write - As far as I know, the test suite is normally run from ghc/testsuite. That directory doesn't exist for me: - pwd - /Users/gcolpitts/Downloads/ghc-8.2.0.20170404/ghc - bash-3.2$ ls testsuite - ls: testsuite: No such file or directory - bash-3.2$ so I guess the source tarball doesn't contain it and those who do a build can't test their build with the testsuite. I was hoping I could do that. I didn't think the libffi directories were the right place to run from but they were only testsuite directories that the find command gave me. Thanks again George On Thu, Apr 6, 2017 at 2:10 AM David Feuer wrote: I'm not sure why you're trying to run things from the libffi directory. As far as I know, the test suite is normally run from ghc/testsuite. David Feuer Well-Typed, LLP -------- Original message -------- From: George Colpitts Date: 4/5/17 9:17 PM (GMT-05:00) To: Jens Petersen , Ben Gamari Cc: GHC developers Subject: Re: GHC 8.2.1-rc1 source tarball availability I'd like to run the testsuite on macOS but I am having trouble following the documentation at https://ghc.haskell.org/trac/ghc/wiki/Building/RunningTests/Running Do I need to download something in addition to the source tarball or am I making some mistake? Following is what I tried: pwd /Users/gcolpitts/Downloads/ghc-8.2.0.20170404/libffi # doc says: The commands on this page can all be executed from the testsuite directory. bash-3.2$ find . -name testsuite ./libffi/build/testsuite ./libffi/build/x86_64-apple-darwin/testsuite bash-3.2$ pushd libffi/build/x86_64-apple-darwin/testsuite ~/Downloads/ghc-8.2.0.20170404/libffi/build/x86_64-apple-darwin/testsuite ~/Downloads/ghc-8.2.0.20170404 bash-3.2$ make test make: *** No rule to make target `test'. Stop. bash-3.2$ popd ~/Downloads/ghc-8.2.0.20170404 bash-3.2$ pushd libffi/build/testsuite ~/Downloads/ghc-8.2.0.20170404/libffi/build/testsuite ~/Downloads/ghc-8.2.0.20170404 bash-3.2$ make test make: *** No rule to make target `test'. Stop. bash-3.2$ popd ~/Downloads/ghc-8.2.0.20170404 bash-3.2$ make test /Applications/Xcode.app/Contents/Developer/usr/bin/make -C testsuite/tests CLEANUP=1 SUMMARY_FILE=../../testsuite_summary.txt make: *** testsuite/tests: No such file or directory. Stop. make: *** [test] Error 2 Thanks George On Wed, Apr 5, 2017 at 9:43 PM Jens Petersen wrote: On 4 April 2017 at 13:21, Ben Gamari wrote: I am happy to announce the release of the 8.2.1-rc1 source distribution to binary packagers. It seems to build okay for me on Fedora 26 so far. But the testsuite completely failed in timeout: see https://ghc.haskell.org/trac/ghc/ticket/13534 Cheers, Jens _______________________________________________ ghc-devs mailing list ghc-devs at haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at well-typed.com Thu Apr 6 13:58:01 2017 From: david at well-typed.com (David Feuer) Date: Thu, 06 Apr 2017 09:58:01 -0400 Subject: testsuite not in GHC 8.2.1-rc1 source tarball ? Message-ID: That's not really too surprising to me. The test suite is primarily intended for GHC developers, and at present only works reliably when GHC is compiled for validation. Including it in the distribution would force users who had no use for it to pay for it anyway. David FeuerWell-Typed, LLP -------- Original message --------From: George Colpitts Date: 4/6/17 9:39 AM (GMT-05:00) To: David Feuer , Jens Petersen , Ben Gamari Cc: GHC developers Subject: Re: testsuite not in GHC 8.2.1-rc1 source tarball ? Thanks Brandon After downloading the source tarball and doing a build successfully I wanted to run the testsuite. You writeAs far as I know, the test suite is normally run from ghc/testsuite. That directory doesn't exist for me:  pwd /Users/gcolpitts/Downloads/ghc-8.2.0.20170404/ghc bash-3.2$ ls testsuite ls: testsuite: No such file or directory bash-3.2$  so I guess the source tarball doesn't contain it and those who do a build can't test their build with the testsuite. I was hoping I could do that. I didn't think the libffi directories were the right place to run from but they were only testsuite directories that the find command gave me. Thanks againGeorge On Thu, Apr 6, 2017 at 2:10 AM David Feuer wrote: I'm not sure why you're trying to run things from the libffi directory. As far as I know, the test suite is normally run from ghc/testsuite. David FeuerWell-Typed, LLP -------- Original message --------From: George Colpitts Date: 4/5/17 9:17 PM (GMT-05:00) To: Jens Petersen , Ben Gamari Cc: GHC developers Subject: Re: GHC 8.2.1-rc1 source tarball availability I'd like to run the testsuite on macOS but I am having trouble following the documentation at https://ghc.haskell.org/trac/ghc/wiki/Building/RunningTests/Running Do I need to download something in addition to the source tarball or am I making some mistake? Following is what I tried:  pwd/Users/gcolpitts/Downloads/ghc-8.2.0.20170404/libffi# doc says: The commands on this page can all be executed from the testsuite directory.bash-3.2$ find . -name testsuite./libffi/build/testsuite./libffi/build/x86_64-apple-darwin/testsuitebash-3.2$ pushd libffi/build/x86_64-apple-darwin/testsuite~/Downloads/ghc-8.2.0.20170404/libffi/build/x86_64-apple-darwin/testsuite ~/Downloads/ghc-8.2.0.20170404bash-3.2$ make testmake: *** No rule to make target `test'.  Stop.bash-3.2$ popd~/Downloads/ghc-8.2.0.20170404bash-3.2$ pushd libffi/build/testsuite~/Downloads/ghc-8.2.0.20170404/libffi/build/testsuite ~/Downloads/ghc-8.2.0.20170404bash-3.2$ make testmake: *** No rule to make target `test'.  Stop.bash-3.2$ popd~/Downloads/ghc-8.2.0.20170404bash-3.2$ make test/Applications/Xcode.app/Contents/Developer/usr/bin/make -C testsuite/tests CLEANUP=1 SUMMARY_FILE=../../testsuite_summary.txtmake: *** testsuite/tests: No such file or directory.  Stop.make: *** [test] Error 2 ThanksGeorge On Wed, Apr 5, 2017 at 9:43 PM Jens Petersen wrote: On 4 April 2017 at 13:21, Ben Gamari wrote: I am happy to announce the release of the 8.2.1-rc1 source distribution to binary packagers. It seems to build okay for me on Fedora 26 so far. But the testsuite completely failed in timeout: see https://ghc.haskell.org/trac/ghc/ticket/13534 Cheers, Jens _______________________________________________ ghc-devs mailing list ghc-devs at haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From george.colpitts at gmail.com Thu Apr 6 15:17:15 2017 From: george.colpitts at gmail.com (George Colpitts) Date: Thu, 06 Apr 2017 15:17:15 +0000 Subject: testsuite not in GHC 8.2.1-rc1 source tarball ? In-Reply-To: References: Message-ID: Thanks for the clarification. I'm all set. On Thu, Apr 6, 2017 at 10:58 AM David Feuer wrote: > That's not really too surprising to me. The test suite is primarily > intended for GHC developers, and at present only works reliably when GHC is > compiled for validation. Including it in the distribution would force users > who had no use for it to pay for it anyway. > > > > David Feuer > Well-Typed, LLP > > -------- Original message -------- > From: George Colpitts > Date: 4/6/17 9:39 AM (GMT-05:00) > To: David Feuer , Jens Petersen < > juhpetersen at gmail.com>, Ben Gamari > Cc: GHC developers > Subject: Re: testsuite not in GHC 8.2.1-rc1 source tarball ? > > Thanks Brandon > > After downloading the source tarball and doing a build successfully I > wanted to run the testsuite. > > You write > > - As far as I know, the test suite is normally run from ghc/testsuite. > > That directory doesn't exist for me: > > > - pwd > - /Users/gcolpitts/Downloads/ghc-8.2.0.20170404/ghc > - bash-3.2$ ls testsuite > - ls: testsuite: No such file or directory > - bash-3.2$ > > > so I guess the source tarball doesn't contain it and those who do a build > can't test their build with the testsuite. I was hoping I could do that. > > I didn't think the libffi directories were the right place to run from but > they were only testsuite directories that the find command gave me. > > Thanks again > George > > > > On Thu, Apr 6, 2017 at 2:10 AM David Feuer wrote: > > I'm not sure why you're trying to run things from the libffi directory. As > far as I know, the test suite is normally run from ghc/testsuite. > > > > David Feuer > Well-Typed, LLP > > -------- Original message -------- > From: George Colpitts > Date: 4/5/17 9:17 PM (GMT-05:00) > To: Jens Petersen , Ben Gamari > > Cc: GHC developers > Subject: Re: GHC 8.2.1-rc1 source tarball availability > > I'd like to run the testsuite on macOS but I am having trouble following > the documentation at > https://ghc.haskell.org/trac/ghc/wiki/Building/RunningTests/Running > > Do I need to download something in addition to the source tarball or am I > making some mistake? > > Following is what I tried: > > pwd > /Users/gcolpitts/Downloads/ghc-8.2.0.20170404/libffi > # doc says: The commands on this page can all be executed from the > testsuite directory. > bash-3.2$ find . -name testsuite > ./libffi/build/testsuite > ./libffi/build/x86_64-apple-darwin/testsuite > bash-3.2$ pushd libffi/build/x86_64-apple-darwin/testsuite > ~/Downloads/ghc-8.2.0.20170404/libffi/build/x86_64-apple-darwin/testsuite > ~/Downloads/ghc-8.2.0.20170404 > bash-3.2$ make test > make: *** No rule to make target `test'. Stop. > bash-3.2$ popd > ~/Downloads/ghc-8.2.0.20170404 > bash-3.2$ pushd libffi/build/testsuite > ~/Downloads/ghc-8.2.0.20170404/libffi/build/testsuite > ~/Downloads/ghc-8.2.0.20170404 > bash-3.2$ make test > make: *** No rule to make target `test'. Stop. > bash-3.2$ popd > ~/Downloads/ghc-8.2.0.20170404 > bash-3.2$ make test > /Applications/Xcode.app/Contents/Developer/usr/bin/make -C testsuite/tests > CLEANUP=1 SUMMARY_FILE=../../testsuite_summary.txt > make: *** testsuite/tests: No such file or directory. Stop. > make: *** [test] Error 2 > > Thanks > George > > > On Wed, Apr 5, 2017 at 9:43 PM Jens Petersen > wrote: > > On 4 April 2017 at 13:21, Ben Gamari wrote: > > I am happy to announce the release of the 8.2.1-rc1 source distribution > to binary packagers. > > > It seems to build okay for me on Fedora 26 so far. > > But the testsuite completely failed in timeout: see > https://ghc.haskell.org/trac/ghc/ticket/13534 > > Cheers, Jens > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Thu Apr 6 17:01:42 2017 From: ben at well-typed.com (Ben Gamari) Date: Thu, 06 Apr 2017 13:01:42 -0400 Subject: testsuite not in GHC 8.2.1-rc1 source tarball ? In-Reply-To: References: Message-ID: <87o9w91nd5.fsf@ben-laptop.smart-cactus.org> George Colpitts writes: > Thanks Brandon > > After downloading the source tarball and doing a build successfully I > wanted to run the testsuite. > You can indeed run the testsuite. However, note that the testsuite is not included in the "-src" tarball to keep the distribution size down. There is a separate "-testsuite" tarball which includes the testsuite/ subtree. This can be extracted into the parent directory of the source tree. You should find this tarball in the usual place [1]. Cheers, - Ben [1] https://downloads.haskell.org/~ghc/8.2.1-rc1/ghc-8.2.0.20170404-testsuite.tar.xz -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From george.colpitts at gmail.com Thu Apr 6 21:16:29 2017 From: george.colpitts at gmail.com (George Colpitts) Date: Thu, 06 Apr 2017 21:16:29 +0000 Subject: testsuite not in GHC 8.2.1-rc1 source tarball ? In-Reply-To: <87o9w91nd5.fsf@ben-laptop.smart-cactus.org> References: <87o9w91nd5.fsf@ben-laptop.smart-cactus.org> Message-ID: Great, thanks Ben! On Thu, Apr 6, 2017 at 2:01 PM Ben Gamari wrote: > George Colpitts writes: > > > Thanks Brandon > > > > After downloading the source tarball and doing a build successfully I > > wanted to run the testsuite. > > > You can indeed run the testsuite. > > However, note that the testsuite is not included in the "-src" tarball > to keep the distribution size down. There is a separate "-testsuite" > tarball which includes the testsuite/ subtree. This can be extracted > into the parent directory of the source tree. You should find this > tarball in the usual place [1]. > > Cheers, > > - Ben > > > [1] > https://downloads.haskell.org/~ghc/8.2.1-rc1/ghc-8.2.0.20170404-testsuite.tar.xz > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chrisdone at gmail.com Fri Apr 7 12:55:22 2017 From: chrisdone at gmail.com (Christopher Done) Date: Fri, 7 Apr 2017 14:55:22 +0200 Subject: GHC Core simplifier question: turn off all passes? Message-ID: Hi all, Just checking, if I want to see the Haskell code desugared to core BEFORE any simplification passes, is this the way to do it? bash-3.2$ cat > X.hs module X where it = (\x -> x * 2) 123 bash-3.2$ stack exec -- ghc -ddump-simpl X.hs -fforce-recomp -fmax-simplifier-iterations=0 -O0 [1 of 1] Compiling X ( X.hs, X.o ) ==================== Tidy Core ==================== Result size of Tidy Core = {terms: 11, types: 3, coercions: 0} -- RHS size: {terms: 4, types: 1, coercions: 0} it :: Integer [GblId, Str=DmdType] it = * @ Integer GHC.Num.$fNumInteger 123 2 I was a bit surprised that the lambda is collapsed already with optimizations turned off and the simplifier iterations set to 0. I changed this to (\x -> x * x) and it produced a let instead. Is this just one of the things the HS->Core desugarer decides to do? Ciao! From rahulmutt at gmail.com Fri Apr 7 13:05:22 2017 From: rahulmutt at gmail.com (Rahul Muttineni) Date: Fri, 7 Apr 2017 18:35:22 +0530 Subject: GHC Core simplifier question: turn off all passes? In-Reply-To: References: Message-ID: Hi Chris, I think you're looking for -ddump-ds which outputs the Core just after the desugarer is done. -ddump-simpl gives the Core after the simplifier is run. Hope that helps, Rahul On Fri, Apr 7, 2017 at 6:25 PM, Christopher Done wrote: > Hi all, > > Just checking, if I want to see the Haskell code desugared to core > BEFORE any simplification passes, is this the way to do it? > > bash-3.2$ cat > X.hs > module X where it = (\x -> x * 2) 123 > bash-3.2$ stack exec -- ghc -ddump-simpl X.hs -fforce-recomp > -fmax-simplifier-iterations=0 -O0 > [1 of 1] Compiling X ( X.hs, X.o ) > > ==================== Tidy Core ==================== > Result size of Tidy Core = {terms: 11, types: 3, coercions: 0} > > -- RHS size: {terms: 4, types: 1, coercions: 0} > it :: Integer > [GblId, Str=DmdType] > it = * @ Integer GHC.Num.$fNumInteger 123 2 > > I was a bit surprised that the lambda is collapsed already with > optimizations turned off and the simplifier iterations set to 0. I > changed this to (\x -> x * x) and it produced a let instead. Is this > just one of the things the HS->Core desugarer decides to do? > > Ciao! > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -- Rahul Muttineni -------------- next part -------------- An HTML attachment was scrubbed... URL: From rae at cs.brynmawr.edu Fri Apr 7 13:09:58 2017 From: rae at cs.brynmawr.edu (Richard Eisenberg) Date: Fri, 7 Apr 2017 09:09:58 -0400 Subject: GHC Core simplifier question: turn off all passes? In-Reply-To: References: Message-ID: I second Rahul in that -ddump-ds is what you want. But the desugarer actually does a quick "simple-optimization" pass before -ddump-ds. If you really want the raw desugarer output, you'll need -ddump-ds *and* a DEBUG build of GHC. See the code in Desugar here: https://github.com/ghc/ghc/blob/master/compiler/deSugar/Desugar.hs#L159 Richard > On Apr 7, 2017, at 9:05 AM, Rahul Muttineni wrote: > > Hi Chris, > > I think you're looking for -ddump-ds which outputs the Core just after the desugarer is done. -ddump-simpl gives the Core after the simplifier is run. > > Hope that helps, > Rahul > > On Fri, Apr 7, 2017 at 6:25 PM, Christopher Done > wrote: > Hi all, > > Just checking, if I want to see the Haskell code desugared to core > BEFORE any simplification passes, is this the way to do it? > > bash-3.2$ cat > X.hs > module X where it = (\x -> x * 2) 123 > bash-3.2$ stack exec -- ghc -ddump-simpl X.hs -fforce-recomp > -fmax-simplifier-iterations=0 -O0 > [1 of 1] Compiling X ( X.hs, X.o ) > > ==================== Tidy Core ==================== > Result size of Tidy Core = {terms: 11, types: 3, coercions: 0} > > -- RHS size: {terms: 4, types: 1, coercions: 0} > it :: Integer > [GblId, Str=DmdType] > it = * @ Integer GHC.Num.$fNumInteger 123 2 > > I was a bit surprised that the lambda is collapsed already with > optimizations turned off and the simplifier iterations set to 0. I > changed this to (\x -> x * x) and it produced a let instead. Is this > just one of the things the HS->Core desugarer decides to do? > > Ciao! > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > > -- > Rahul Muttineni > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From chrisdone at gmail.com Fri Apr 7 13:24:34 2017 From: chrisdone at gmail.com (Christopher Done) Date: Fri, 7 Apr 2017 15:24:34 +0200 Subject: GHC Core simplifier question: turn off all passes? In-Reply-To: References: Message-ID: Indeed, it seems that even with -ddump-ds that there's a replacement of immediatelly-applied-lambdas ((\x->x) a => a). I'm using the GHC API (but running the exe with flags reproduced my problem), so perhaps I could call `desugarModule` with all the simplifier rules deleted or something? --- Output with -ddump-ds bash-3.2$ stack exec -- ghc -ddump-ds X.hs -fforce-recomp -fmax-simplifier-iterations=0 -O0 [1 of 1] Compiling X ( X.hs, X.o ) ==================== Desugar (after optimization) ==================== Result size of Desugar (after optimization) = {terms: 11, types: 3, coercions: 0} -- RHS size: {terms: 4, types: 1, coercions: 0} it :: Integer [LclIdX, Str=DmdType] it = * @ Integer GHC.Num.$fNumInteger 123 2 -- RHS size: {terms: 5, types: 0, coercions: 0} X.$trModule :: GHC.Types.Module [LclIdX, Str=DmdType] X.$trModule = GHC.Types.Module (GHC.Types.TrNameS "main"#) (GHC.Types.TrNameS "X"#) On 7 April 2017 at 15:09, Richard Eisenberg wrote: > I second Rahul in that -ddump-ds is what you want. But the desugarer > actually does a quick "simple-optimization" pass before -ddump-ds. If you > really want the raw desugarer output, you'll need -ddump-ds *and* a DEBUG > build of GHC. See the code in Desugar here: > https://github.com/ghc/ghc/blob/master/compiler/deSugar/Desugar.hs#L159 > > Richard > > On Apr 7, 2017, at 9:05 AM, Rahul Muttineni wrote: > > Hi Chris, > > I think you're looking for -ddump-ds which outputs the Core just after the > desugarer is done. -ddump-simpl gives the Core after the simplifier is run. > > Hope that helps, > Rahul > > On Fri, Apr 7, 2017 at 6:25 PM, Christopher Done > wrote: >> >> Hi all, >> >> Just checking, if I want to see the Haskell code desugared to core >> BEFORE any simplification passes, is this the way to do it? >> >> bash-3.2$ cat > X.hs >> module X where it = (\x -> x * 2) 123 >> bash-3.2$ stack exec -- ghc -ddump-simpl X.hs -fforce-recomp >> -fmax-simplifier-iterations=0 -O0 >> [1 of 1] Compiling X ( X.hs, X.o ) >> >> ==================== Tidy Core ==================== >> Result size of Tidy Core = {terms: 11, types: 3, coercions: 0} >> >> -- RHS size: {terms: 4, types: 1, coercions: 0} >> it :: Integer >> [GblId, Str=DmdType] >> it = * @ Integer GHC.Num.$fNumInteger 123 2 >> >> I was a bit surprised that the lambda is collapsed already with >> optimizations turned off and the simplifier iterations set to 0. I >> changed this to (\x -> x * x) and it produced a let instead. Is this >> just one of the things the HS->Core desugarer decides to do? >> >> Ciao! >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > > > -- > Rahul Muttineni > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > From chrisdone at gmail.com Fri Apr 7 13:28:08 2017 From: chrisdone at gmail.com (Christopher Done) Date: Fri, 7 Apr 2017 15:28:08 +0200 Subject: GHC Core simplifier question: turn off all passes? In-Reply-To: References: Message-ID: Thanks all, I'll try looking at the innards of the desugarer! On 7 April 2017 at 15:24, Christopher Done wrote: > Indeed, it seems that even with -ddump-ds that there's a replacement > of immediatelly-applied-lambdas ((\x->x) a => a). > > I'm using the GHC API (but running the exe with flags reproduced my > problem), so perhaps I could call `desugarModule` with all the > simplifier rules deleted or something? > > --- > Output with -ddump-ds > > bash-3.2$ stack exec -- ghc -ddump-ds X.hs -fforce-recomp > -fmax-simplifier-iterations=0 -O0 > [1 of 1] Compiling X ( X.hs, X.o ) > > ==================== Desugar (after optimization) ==================== > Result size of Desugar (after optimization) > = {terms: 11, types: 3, coercions: 0} > > -- RHS size: {terms: 4, types: 1, coercions: 0} > it :: Integer > [LclIdX, Str=DmdType] > it = * @ Integer GHC.Num.$fNumInteger 123 2 > > -- RHS size: {terms: 5, types: 0, coercions: 0} > X.$trModule :: GHC.Types.Module > [LclIdX, Str=DmdType] > X.$trModule = > GHC.Types.Module > (GHC.Types.TrNameS "main"#) (GHC.Types.TrNameS "X"#) > > On 7 April 2017 at 15:09, Richard Eisenberg wrote: >> I second Rahul in that -ddump-ds is what you want. But the desugarer >> actually does a quick "simple-optimization" pass before -ddump-ds. If you >> really want the raw desugarer output, you'll need -ddump-ds *and* a DEBUG >> build of GHC. See the code in Desugar here: >> https://github.com/ghc/ghc/blob/master/compiler/deSugar/Desugar.hs#L159 >> >> Richard >> >> On Apr 7, 2017, at 9:05 AM, Rahul Muttineni wrote: >> >> Hi Chris, >> >> I think you're looking for -ddump-ds which outputs the Core just after the >> desugarer is done. -ddump-simpl gives the Core after the simplifier is run. >> >> Hope that helps, >> Rahul >> >> On Fri, Apr 7, 2017 at 6:25 PM, Christopher Done >> wrote: >>> >>> Hi all, >>> >>> Just checking, if I want to see the Haskell code desugared to core >>> BEFORE any simplification passes, is this the way to do it? >>> >>> bash-3.2$ cat > X.hs >>> module X where it = (\x -> x * 2) 123 >>> bash-3.2$ stack exec -- ghc -ddump-simpl X.hs -fforce-recomp >>> -fmax-simplifier-iterations=0 -O0 >>> [1 of 1] Compiling X ( X.hs, X.o ) >>> >>> ==================== Tidy Core ==================== >>> Result size of Tidy Core = {terms: 11, types: 3, coercions: 0} >>> >>> -- RHS size: {terms: 4, types: 1, coercions: 0} >>> it :: Integer >>> [GblId, Str=DmdType] >>> it = * @ Integer GHC.Num.$fNumInteger 123 2 >>> >>> I was a bit surprised that the lambda is collapsed already with >>> optimizations turned off and the simplifier iterations set to 0. I >>> changed this to (\x -> x * x) and it produced a let instead. Is this >>> just one of the things the HS->Core desugarer decides to do? >>> >>> Ciao! >>> _______________________________________________ >>> ghc-devs mailing list >>> ghc-devs at haskell.org >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> >> >> >> >> -- >> Rahul Muttineni >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> >> From rae at cs.brynmawr.edu Fri Apr 7 13:36:25 2017 From: rae at cs.brynmawr.edu (Richard Eisenberg) Date: Fri, 7 Apr 2017 09:36:25 -0400 Subject: GHC Core simplifier question: turn off all passes? In-Reply-To: References: Message-ID: <53D45E7F-B3B8-4F60-BBD7-DEB88D82A36F@cs.brynmawr.edu> > On Apr 7, 2017, at 9:24 AM, Christopher Done wrote: > > all the > simplifier rules deleted or something? I'm not sure what you mean here by "simplifier rules deleted". The "simple-optimization" pass doesn't consult any user-specified RULES or anything -- it just does a few substitutions, etc. I don't think there's a way to disable it. Indeed, it's necessary in order for RULES to work, as the simple-optimizer shuffles RULES' left-hand sides to enable matching to succeed later on. Richard From alfredo.dinapoli at gmail.com Fri Apr 7 15:29:36 2017 From: alfredo.dinapoli at gmail.com (Alfredo Di Napoli) Date: Fri, 7 Apr 2017 17:29:36 +0200 Subject: Where do I start if I would like help improve GHC compilation times? Message-ID: Hey folks, maybe I’m setting up for something too ambitious for me, but I would like to take an active stance to the overlasting “GHC compilation times are terrible” matter, instead of simply stare at the screen with despair whenever GHC compiles a sufficiently large Haskell program ;) To make this even more interesting, I have never contributed to GHC either! The max I have pushed myself into was 2 years ago when I successfully built GHC head from source and tried to fix an Haddock “easy” ticket I don’t even recall (full disclosure, eventually I didn’t :D ). Specifically, I would love community recommendations & guidance about: 1. Is this simply too daunting for somebody like me? Maybe is better to first start contributing more regularly, take confidence with the code base AND then move forward? 2. Are compilation times largely dependant from the target platform (I’m on Darwin) or there is something which can be done “globally” so that the benefits can be experienced by everybody? 3. Is there any recommended workflow to profile GHC compilation times? Is there any build flavour one should prefer when doing so? (Maybe the full, slowest one?) Thanks in advance for taking the time reading this mail, and have a nice weekend! Alfredo -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at smart-cactus.org Fri Apr 7 16:22:50 2017 From: ben at smart-cactus.org (Ben Gamari) Date: Fri, 07 Apr 2017 12:22:50 -0400 Subject: Where do I start if I would like help improve GHC compilation times? In-Reply-To: References: Message-ID: <87fuhk192d.fsf@ben-laptop.smart-cactus.org> Alfredo Di Napoli writes: > Hey folks, > Hi Alfredo! First, thanks for writing. More eyes looking at GHC's compiler performance is badly needed. > maybe I’m setting up for something too ambitious for me, but I would like > to take an active stance to the overlasting “GHC compilation times are > terrible” matter, instead of simply stare at the screen with despair > whenever GHC compiles a sufficiently large Haskell program ;) > > To make this even more interesting, I have never contributed to GHC either! > The max I have pushed myself into was 2 years ago when I successfully built > GHC head from source and tried to fix an Haddock “easy” ticket I don’t even > recall (full disclosure, eventually I didn’t :D ). > > Specifically, I would love community recommendations & guidance about: > > 1. Is this simply too daunting for somebody like me? Maybe is better to > first start contributing more regularly, take confidence with the code base > AND then move forward? > As with any software project, it is possible to treat the compiler as a black box, throw a profiler at it and see what hotspots show up. This gives you a place to focus your effort, allowing you to learn a small area and broaden your knowledge as necessary. However, I think it's fair to say that you will be significantly more productive if you first develop a basic understanding of the compilation pipeline. I'd recommend having a look at the GHC Commentary [1] for a start. I think it also helps to have a rough idea of what "slow" means to you. I find it is quite helpful if you have a particular program which you feel compiles more slowly than you would like (especially if it even compiles slowly with -O0, since then much less of the compiler is involved in compilation). Another approach is to look for programs whose compilation time has regressed over the course of GHC releases. It is not hard to find these examples and it is often possible to bisect your way back to the regressing commit. Also, note that I have collected some notes pertaining to compiler performance on the Wiki [2]. Here you will find a number of tickets of interest (as well a some rough themes which I've noticed), some nofib results which might guide your efforts, as well as a list of some fixes which have been committed in the past. [1] https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler [2] https://ghc.haskell.org/trac/ghc/wiki/Performance/Compiler > 2. Are compilation times largely dependant from the target platform (I’m on > Darwin) or there is something which can be done “globally” so that the > benefits can be experienced by everybody? > There are some external considerations (e.g. the platform's compiler and linking toolchain) which contribute to GHC's runtime. For instance, it is known that the BFD ld linker implementation that many Linux distributions use by default is a great deal slower than it could be. This particular issue has come up recently and I'm currently working on allowing us to use the more performant gold linker when available. However, I think it's fair to say that for most programs GHC's runtime is largely independent of platform. I would invite you to try compiling a package which you consider GHC to compile "slowly" with GHC's -v flag (and GHC 8.0.1 or newer). This will give you a rough breakdown of where time is spent. For many packages you will find that the simplifier and/or typechecker dominate, followed (often distantly) by native code generation. Of these steps native code generation is the only one with a strong platform dependence. > 3. Is there any recommended workflow to profile GHC compilation times? Is > there any build flavour one should prefer when doing so? (Maybe the full, > slowest one?) > There are a few options here: * As of GHC 8.0 the compiler will output timing and allocation information for its various stages if run with -v. This can be extremely helpful to get a high-level picture of where the compiler is spending its time while compiling your program. This is almost always the right place to start. * As with any Haskell program, the cost centre profiler can be used to characterize the memory and CPU behavior of various parts of the compiler. GHC's source tree includes a "prof" build flavour which builds the compiler with profiling enabled. However it only includes a handful of cost-centres and is best used when you already have a rough idea where you are looking and can add further cost-centres to drill down to your hotspot. Simply enabling -fprof-exported across the entire tree just doesn't work in my experience: not only is the resulting compiler quite slow, but the profile you get is far too unwieldy to learn from. * Occassionally the ticky-ticky profiler can be helpful in identifying allocation hotspots without the full overhead of the cost-centre profiler. * In principle our newly-stable DWARF debug information can be used for profiling, although this is still a work in progress and requires a patched GHC for best results. It's probably best to stick to the more traditional profiling mechanisms for now. Anyways, I hope this helps. Always feel free to get in touch with me personally (IRC and email are both great) if you would like to discuss particular issues. Thanks again for your interest! Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From alfredo.dinapoli at gmail.com Fri Apr 7 16:30:16 2017 From: alfredo.dinapoli at gmail.com (Alfredo Di Napoli) Date: Fri, 7 Apr 2017 18:30:16 +0200 Subject: Where do I start if I would like help improve GHC compilation times? In-Reply-To: <87fuhk192d.fsf@ben-laptop.smart-cactus.org> References: <87fuhk192d.fsf@ben-laptop.smart-cactus.org> Message-ID: Hey Ben, thanks for the quite exhaustive reply! I’m on the go right now, but I promise to get back to you with a meaningful reply later this weekend ;) Alfredo On 7 April 2017 at 18:22, Ben Gamari wrote: > Alfredo Di Napoli writes: > > > Hey folks, > > > Hi Alfredo! > > First, thanks for writing. More eyes looking at GHC's compiler > performance is badly needed. > > > maybe I’m setting up for something too ambitious for me, but I would like > > to take an active stance to the overlasting “GHC compilation times are > > terrible” matter, instead of simply stare at the screen with despair > > whenever GHC compiles a sufficiently large Haskell program ;) > > > > To make this even more interesting, I have never contributed to GHC > either! > > The max I have pushed myself into was 2 years ago when I successfully > built > > GHC head from source and tried to fix an Haddock “easy” ticket I don’t > even > > recall (full disclosure, eventually I didn’t :D ). > > > > Specifically, I would love community recommendations & guidance about: > > > > 1. Is this simply too daunting for somebody like me? Maybe is better to > > first start contributing more regularly, take confidence with the code > base > > AND then move forward? > > > As with any software project, it is possible to treat the compiler as a > black box, throw a profiler at it and see what hotspots show up. This > gives you a place to focus your effort, allowing you to learn a small > area and broaden your knowledge as necessary. > > However, I think it's fair to say that you will be significantly more > productive if you first develop a basic understanding of the compilation > pipeline. I'd recommend having a look at the GHC Commentary [1] for a > start. > > I think it also helps to have a rough idea of what "slow" means to you. > I find it is quite helpful if you have a particular program which you > feel compiles more slowly than you would like (especially if it even > compiles slowly with -O0, since then much less of the compiler is > involved in compilation). Another approach is to look for programs whose > compilation time has regressed over the course of GHC releases. It is > not hard to find these examples and it is often possible to bisect your > way back to the regressing commit. > > Also, note that I have collected some notes pertaining to compiler > performance on the Wiki [2]. Here you will find a number of tickets of > interest (as well a some rough themes which I've noticed), some nofib > results which might guide your efforts, as well as a list of some > fixes which have been committed in the past. > > [1] https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler > [2] https://ghc.haskell.org/trac/ghc/wiki/Performance/Compiler > > > 2. Are compilation times largely dependant from the target platform (I’m > on > > Darwin) or there is something which can be done “globally” so that the > > benefits can be experienced by everybody? > > > There are some external considerations (e.g. the platform's compiler and > linking toolchain) which contribute to GHC's runtime. For instance, it > is known that the BFD ld linker implementation that many Linux > distributions use by default is a great deal slower than it could be. > This particular issue has come up recently and I'm currently working on > allowing us to use the more performant gold linker when available. > > However, I think it's fair to say that for most programs GHC's runtime > is largely independent of platform. I would invite you to try compiling > a package which you consider GHC to compile "slowly" with GHC's -v flag > (and GHC 8.0.1 or newer). This will give you a rough breakdown of where > time is spent. For many packages you will find that the simplifier > and/or typechecker dominate, followed (often distantly) by native code > generation. Of these steps native code generation is the only one with a > strong platform dependence. > > > 3. Is there any recommended workflow to profile GHC compilation times? Is > > there any build flavour one should prefer when doing so? (Maybe the full, > > slowest one?) > > > There are a few options here: > > * As of GHC 8.0 the compiler will output timing and allocation > information for its various stages if run with -v. This can be > extremely helpful to get a high-level picture of where the compiler > is spending its time while compiling your program. This is almost > always the right place to start. > > * As with any Haskell program, the cost centre profiler can be used to > characterize the memory and CPU behavior of various parts of the > compiler. > > GHC's source tree includes a "prof" build flavour which builds the > compiler with profiling enabled. However it only includes a handful > of cost-centres and is best used when you already have a rough idea > where you are looking and can add further cost-centres to drill down > to your hotspot. > > Simply enabling -fprof-exported across the entire tree just doesn't > work in my experience: not only is the resulting compiler quite slow, > but the profile you get is far too unwieldy to learn from. > > * Occassionally the ticky-ticky profiler can be helpful in identifying > allocation hotspots without the full overhead of the cost-centre > profiler. > > * In principle our newly-stable DWARF debug information can be used for > profiling, although this is still a work in progress and requires a > patched GHC for best results. It's probably best to stick to the more > traditional profiling mechanisms for now. > > Anyways, I hope this helps. Always feel free to get in touch with me > personally (IRC and email are both great) if you would like to discuss > particular issues. Thanks again for your interest! > > Cheers, > > - Ben > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rae at cs.brynmawr.edu Fri Apr 7 19:16:02 2017 From: rae at cs.brynmawr.edu (Richard Eisenberg) Date: Fri, 7 Apr 2017 15:16:02 -0400 Subject: testsuite failures on Mac Message-ID: Hi devs, On today's HEAD, I built from a cleaned tree and then ran the testsuite. I got gobs of errors. Here's one: --- ./T2713.run/T2713.stderr.normalised 2017-04-07 15:13:22.000000000 -0400 +++ ./T2713.run/T2713.comp.stderr.normalised 2017-04-07 15:13:22.000000000 -0400 @@ -1,8 +1,4 @@ - -T2713.hs:11:10: - The fixity signature for ‘.*.’ lacks an accompanying binding - (The fixity signature must be given where ‘.*.’ is declared) - -T2713.hs:12:1: - The type signature for ‘f’ lacks an accompanying binding - (The type signature must be given where ‘f’ is declared) +ghc-iserv.bin: +lookupSymbol failed in relocateSection (RELOC_GOT) +/Users/rae/ghc/ghc/libraries/integer-gmp/dist-install/build/HSinteger--o: unknown symbol `___gmp_rands' +ghc: unable to load package `integer--' *** unexpected failure for T2713(ext-interp) Although I have minor changes in some type-y stuff, nothing that should cause problems with ghc-iserv. Any thoughts? I'm on a Mac. Thanks! Richard From ben at smart-cactus.org Fri Apr 7 19:27:33 2017 From: ben at smart-cactus.org (Ben Gamari) Date: Fri, 07 Apr 2017 15:27:33 -0400 Subject: testsuite failures on Mac In-Reply-To: References: Message-ID: <87bms810ii.fsf@ben-laptop.smart-cactus.org> Richard Eisenberg writes: > Hi devs, > > On today's HEAD, I built from a cleaned tree and then ran the > testsuite. How clean is clean? distclean? Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From rae at cs.brynmawr.edu Fri Apr 7 20:00:19 2017 From: rae at cs.brynmawr.edu (Richard Eisenberg) Date: Fri, 7 Apr 2017 16:00:19 -0400 Subject: testsuite failures on Mac In-Reply-To: <87bms810ii.fsf@ben-laptop.smart-cactus.org> References: <87bms810ii.fsf@ben-laptop.smart-cactus.org> Message-ID: <8A7F4C12-2139-4C1D-BA20-BCB27F65A779@cs.brynmawr.edu> > On Apr 7, 2017, at 3:27 PM, Ben Gamari wrote: > > How clean is clean? distclean? maintainer-clean. I never settle for anything less. :) Richard From ben at smart-cactus.org Fri Apr 7 20:15:27 2017 From: ben at smart-cactus.org (Ben Gamari) Date: Fri, 07 Apr 2017 16:15:27 -0400 Subject: testsuite failures on Mac In-Reply-To: <8A7F4C12-2139-4C1D-BA20-BCB27F65A779@cs.brynmawr.edu> References: <87bms810ii.fsf@ben-laptop.smart-cactus.org> <8A7F4C12-2139-4C1D-BA20-BCB27F65A779@cs.brynmawr.edu> Message-ID: <8760ig0yao.fsf@ben-laptop.smart-cactus.org> Richard Eisenberg writes: >> On Apr 7, 2017, at 3:27 PM, Ben Gamari wrote: >> >> How clean is clean? distclean? > > maintainer-clean. I never settle for anything less. :) > Hmm, that is quite peculiar. Harbormaster is currently behind master by three commits, but the last completely build finished successfully. Moreover, I've never seen any issue quite like what you report. Are you using the GMP shipped with integer-gmp or your system's GMP? If the latter, have you updated or otherwise touched it recently? Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From george.colpitts at gmail.com Sat Apr 8 12:17:14 2017 From: george.colpitts at gmail.com (George Colpitts) Date: Sat, 08 Apr 2017 12:17:14 +0000 Subject: testsuite not in GHC 8.2.1-rc1 source tarball ? In-Reply-To: <87o9w91nd5.fsf@ben-laptop.smart-cactus.org> References: <87o9w91nd5.fsf@ben-laptop.smart-cactus.org> Message-ID: Are there any guidelines on how much memory is required to run the tests? On my Mac I have 12 GB of RAM and a few hundred gigabytes of free disk space. The first time I did - make THREADS=4 test and I got a message - Your system has run out of application memory so I reran with - make test and left it running unattended. My machine crashed about an hour later. Thanks On Thu, Apr 6, 2017 at 2:01 PM Ben Gamari wrote: > George Colpitts writes: > > > Thanks Brandon > > > > After downloading the source tarball and doing a build successfully I > > wanted to run the testsuite. > > > You can indeed run the testsuite. > > However, note that the testsuite is not included in the "-src" tarball > to keep the distribution size down. There is a separate "-testsuite" > tarball which includes the testsuite/ subtree. This can be extracted > into the parent directory of the source tree. You should find this > tarball in the usual place [1]. > > Cheers, > > - Ben > > > [1] > https://downloads.haskell.org/~ghc/8.2.1-rc1/ghc-8.2.0.20170404-testsuite.tar.xz > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Sun Apr 9 00:00:10 2017 From: ben at well-typed.com (Ben Gamari) Date: Sat, 08 Apr 2017 20:00:10 -0400 Subject: testsuite not in GHC 8.2.1-rc1 source tarball ? In-Reply-To: References: <87o9w91nd5.fsf@ben-laptop.smart-cactus.org> Message-ID: <87wpauzbzp.fsf@ben-laptop.smart-cactus.org> George Colpitts writes: > Are there any guidelines on how much memory is required to run the tests? > On my Mac I have 12 GB of RAM and a few hundred gigabytes of free disk > space. The first time I did > > > - make THREADS=4 test > > and I got a message > > > - Your system has run out of application memory > > so I reran with > > - make test > > and left it running unattended. My machine crashed about an hour later. > Well, unfortunately one of the changes that just barely missesd the window for -rc1 was a fix to a rather serious memory leak in the compiler (#13426). I suspect this is the reason you are seeing such high memory footprints. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From mle+hs at mega-nerd.com Sun Apr 9 02:13:01 2017 From: mle+hs at mega-nerd.com (Erik de Castro Lopo) Date: Sun, 9 Apr 2017 12:13:01 +1000 Subject: testsuite not in GHC 8.2.1-rc1 source tarball ? In-Reply-To: <87wpauzbzp.fsf@ben-laptop.smart-cactus.org> References: <87o9w91nd5.fsf@ben-laptop.smart-cactus.org> <87wpauzbzp.fsf@ben-laptop.smart-cactus.org> Message-ID: <20170409121301.c4d247016bf1f51b703ec55a@mega-nerd.com> Ben Gamari wrote: > Well, unfortunately one of the changes that just barely missesd the > window for -rc1 was a fix to a rather serious memory leak in the compiler > (#13426). Thats going to fixed for rc2? Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/ From ben at smart-cactus.org Sun Apr 9 02:36:17 2017 From: ben at smart-cactus.org (Ben Gamari) Date: Sat, 08 Apr 2017 22:36:17 -0400 Subject: testsuite not in GHC 8.2.1-rc1 source tarball ? In-Reply-To: <20170409121301.c4d247016bf1f51b703ec55a@mega-nerd.com> References: <87o9w91nd5.fsf@ben-laptop.smart-cactus.org> <87wpauzbzp.fsf@ben-laptop.smart-cactus.org> <20170409121301.c4d247016bf1f51b703ec55a@mega-nerd.com> Message-ID: <87o9w6z4ri.fsf@ben-laptop.smart-cactus.org> Erik de Castro Lopo writes: > Ben Gamari wrote: > >> Well, unfortunately one of the changes that just barely missesd the >> window for -rc1 was a fix to a rather serious memory leak in the compiler >> (#13426). > > Thats going to fixed for rc2? > Yes, the patch is already in the ghc-8.2 branch. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From alfredo.dinapoli at gmail.com Sun Apr 9 09:37:09 2017 From: alfredo.dinapoli at gmail.com (Alfredo Di Napoli) Date: Sun, 9 Apr 2017 11:37:09 +0200 Subject: Where do I start if I would like help improve GHC compilation times? In-Reply-To: References: <87fuhk192d.fsf@ben-laptop.smart-cactus.org> Message-ID: Hey Ben, as promised I’m back to you with something more articulated and hopefully meaningful. I do hear you perfectly — probably trying to dive head-first into this without at least a rough understanding of the performance hotspots or the GHC overall architecture is going to do me more harm than good (I get the overall picture and I’m aware of the different stages of the GHC compilation pipeline, but it’s far from saying I’m proficient with the architecture as whole). I have also read a couple of years ago the GHC chapter on the “Architeture of Open Source Applications” book, but I don’t know how much that is still relevant. If it is, I guess I should refresh my memory. I’m currently trying to move on 2 fronts — please advice if I’m a fool flogging a dead horse or if I have any hope of getting anything done ;) 1. I’m trying to treat indeed the compiler as a black block (as you adviced) trying to build a sufficiently large program where GHC is not “as fast as I would like” (I know that’s a very lame definition of “slow”, hehe). In particular, I have built the stage2 compiler with the “prof” flavour as you suggested, and I have chosen 2 examples as a reference “benchmark” for performance; DynFlags.hs (which seems to have been mentioned multiple times as a GHC perf killer) and the highlighting-kate package as posted here: https://ghc.haskell.org/trac/ghc/ticket/9221 . The idea would be to compile those with -v +RTS -p -hc -RTS enabled, look at the output from the .prof file AND the `-v` flag, find any hotspot, try to change something, recompile, observe diff, rinse and repeat. Do you think I have any hope of making progress this way? In particular, I think compiling DynFlags.hs is a bit of a dead-end; I whipped up this buggy script which escalated into a Behemoth which is compiling pretty much half of the compiler once again :D ``` #!/usr/bin/env bash ../ghc/inplace/bin/ghc-stage2 --make -j8 -v +RTS -A256M -qb0 -p -h \ -RTS -DSTAGE=2 -I../ghc/includes -I../ghc/compiler -I../ghc/compiler/stage2 \ -I../ghc/compiler/stage2/build \ -i../ghc/compiler/utils:../ghc/compiler/types:../ghc/compiler/typecheck:../ghc/compiler/basicTypes \ -i../ghc/compiler/main:../ghc/compiler/profiling:../ghc/compiler/coreSyn:../ghc/compiler/iface:../ghc/compiler/prelude \ -i../ghc/compiler/stage2/build:../ghc/compiler/simplStg:../ghc/compiler/cmm:../ghc/compiler/parser:../ghc/compiler/hsSyn \ -i../ghc/compiler/ghci:../ghc/compiler/deSugar:../ghc/compiler/simplCore:../ghc/compile/specialise \ -fforce-recomp -c $@ ``` I’m running it with `./dynflags.sh ../ghc/compiler/main/DynFlags.hs` but it’s taking a lot to compile (20+ mins on my 2014 mac Pro) because it’s pulling in half of the compiler anyway :D I tried to reuse the .hi files from my stage2 compilation but I failed (GHC was complaining about interface file mismatch). Short story short, I don’t think it will be a very agile way to proceed. Am I right? Do you have any recommendation in such sense? Do I have any hope to compile DynFlags.hs in a way which would make this perf investigation feasible? The second example (the highlighting-kate package) seems much more promising. It takes maybe 1-2 mins on my machine, which is enough to take a look at the perf output. Do you think I should follow this second lead? In principle any 50+ modules package I think would do (better if with a lot of TH ;) ) but this seems like a low-entry barrier start. 2. The second path I’m exploring is simply to take a less holistic approach and try to dive in into a performance ticket like the ones listed here: https://www.reddit.com/r/haskell/comments/45q90s/is_anything_being_done_to_remedy_the_soul/czzq6an/ Maybe some are very specific, but it seems like fixing small things and move forward could help giving me understanding of different sub-parts of GHC, which seems less intimidating than the black-box approach. In conclusion, what do you think is the best approach, 1 or 2, both or none? ;) Thank you! Alfredo On 7 April 2017 at 18:30, Alfredo Di Napoli wrote: > Hey Ben, > > thanks for the quite exhaustive reply! I’m on the go right now, but I > promise to get back to you with a meaningful reply later this weekend ;) > > Alfredo > > On 7 April 2017 at 18:22, Ben Gamari wrote: > >> Alfredo Di Napoli writes: >> >> > Hey folks, >> > >> Hi Alfredo! >> >> First, thanks for writing. More eyes looking at GHC's compiler >> performance is badly needed. >> >> > maybe I’m setting up for something too ambitious for me, but I would >> like >> > to take an active stance to the overlasting “GHC compilation times are >> > terrible” matter, instead of simply stare at the screen with despair >> > whenever GHC compiles a sufficiently large Haskell program ;) >> > >> > To make this even more interesting, I have never contributed to GHC >> either! >> > The max I have pushed myself into was 2 years ago when I successfully >> built >> > GHC head from source and tried to fix an Haddock “easy” ticket I don’t >> even >> > recall (full disclosure, eventually I didn’t :D ). >> > >> > Specifically, I would love community recommendations & guidance about: >> > >> > 1. Is this simply too daunting for somebody like me? Maybe is better to >> > first start contributing more regularly, take confidence with the code >> base >> > AND then move forward? >> > >> As with any software project, it is possible to treat the compiler as a >> black box, throw a profiler at it and see what hotspots show up. This >> gives you a place to focus your effort, allowing you to learn a small >> area and broaden your knowledge as necessary. >> >> However, I think it's fair to say that you will be significantly more >> productive if you first develop a basic understanding of the compilation >> pipeline. I'd recommend having a look at the GHC Commentary [1] for a >> start. >> >> I think it also helps to have a rough idea of what "slow" means to you. >> I find it is quite helpful if you have a particular program which you >> feel compiles more slowly than you would like (especially if it even >> compiles slowly with -O0, since then much less of the compiler is >> involved in compilation). Another approach is to look for programs whose >> compilation time has regressed over the course of GHC releases. It is >> not hard to find these examples and it is often possible to bisect your >> way back to the regressing commit. >> >> Also, note that I have collected some notes pertaining to compiler >> performance on the Wiki [2]. Here you will find a number of tickets of >> interest (as well a some rough themes which I've noticed), some nofib >> results which might guide your efforts, as well as a list of some >> fixes which have been committed in the past. >> >> [1] https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler >> [2] https://ghc.haskell.org/trac/ghc/wiki/Performance/Compiler >> >> > 2. Are compilation times largely dependant from the target platform >> (I’m on >> > Darwin) or there is something which can be done “globally” so that the >> > benefits can be experienced by everybody? >> > >> There are some external considerations (e.g. the platform's compiler and >> linking toolchain) which contribute to GHC's runtime. For instance, it >> is known that the BFD ld linker implementation that many Linux >> distributions use by default is a great deal slower than it could be. >> This particular issue has come up recently and I'm currently working on >> allowing us to use the more performant gold linker when available. >> >> However, I think it's fair to say that for most programs GHC's runtime >> is largely independent of platform. I would invite you to try compiling >> a package which you consider GHC to compile "slowly" with GHC's -v flag >> (and GHC 8.0.1 or newer). This will give you a rough breakdown of where >> time is spent. For many packages you will find that the simplifier >> and/or typechecker dominate, followed (often distantly) by native code >> generation. Of these steps native code generation is the only one with a >> strong platform dependence. >> >> > 3. Is there any recommended workflow to profile GHC compilation times? >> Is >> > there any build flavour one should prefer when doing so? (Maybe the >> full, >> > slowest one?) >> > >> There are a few options here: >> >> * As of GHC 8.0 the compiler will output timing and allocation >> information for its various stages if run with -v. This can be >> extremely helpful to get a high-level picture of where the compiler >> is spending its time while compiling your program. This is almost >> always the right place to start. >> >> * As with any Haskell program, the cost centre profiler can be used to >> characterize the memory and CPU behavior of various parts of the >> compiler. >> >> GHC's source tree includes a "prof" build flavour which builds the >> compiler with profiling enabled. However it only includes a handful >> of cost-centres and is best used when you already have a rough idea >> where you are looking and can add further cost-centres to drill down >> to your hotspot. >> >> Simply enabling -fprof-exported across the entire tree just doesn't >> work in my experience: not only is the resulting compiler quite slow, >> but the profile you get is far too unwieldy to learn from. >> >> * Occassionally the ticky-ticky profiler can be helpful in identifying >> allocation hotspots without the full overhead of the cost-centre >> profiler. >> >> * In principle our newly-stable DWARF debug information can be used for >> profiling, although this is still a work in progress and requires a >> patched GHC for best results. It's probably best to stick to the more >> traditional profiling mechanisms for now. >> >> Anyways, I hope this helps. Always feel free to get in touch with me >> personally (IRC and email are both great) if you would like to discuss >> particular issues. Thanks again for your interest! >> >> Cheers, >> >> - Ben >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rwbarton at gmail.com Sun Apr 9 20:42:49 2017 From: rwbarton at gmail.com (Reid Barton) Date: Sun, 9 Apr 2017 16:42:49 -0400 Subject: Where do I start if I would like help improve GHC compilation times? In-Reply-To: References: <87fuhk192d.fsf@ben-laptop.smart-cactus.org> Message-ID: Building modules from GHC itself is a little tricky and DynFlags is extra tricky since it is involved in import cycles. Here is what I do: * Copy DynFlags.hs somewhere outside the tree (for your present purposes, it is no longer part of the compiler, but just some module to be provided as input). * Get rid of all the {-# SOURCE #-} pragmas on imports to turn them into ordinary, non-boot file imports. * Build with ".../ghc/inplace/bin/ghc-stage2 DynFlags -package ghc -I.../ghc/compiler/stage2" plus whatever other options you want (e.g., probably "-fforce-recomp -O +RTS -s" at a minimum). By using "-package ghc" you compile DynFlags against the version of ghc that you have just built. * This will result in some type errors, because DynFlags imports some functions that expect arguments of type DynFlags. (This relates to the import cycles that we broke earlier.) Since you are building against the version of those functions from the ghc package, they expect the type ghc:DynFlags.DynFlags, but they are now receiving a value of type DynFlags from the main package. This is no big deal, just insert an unsafeCoerce wherever necessary (mostly in front of occurrences of "dflags") to get the compiler to stop complaining. This is not 100% faithful to the way DynFlags would actually be compiled during a GHC build, but the advantage of this method is that you don't have to worry about GHC doing any recompilation checking between the copy of DynFlags that you are testing on and the compiler's own modules. Regards, Reid Barton On Sun, Apr 9, 2017 at 5:37 AM, Alfredo Di Napoli wrote: > Hey Ben, > > as promised I’m back to you with something more articulated and hopefully > meaningful. I do hear you perfectly — probably trying to dive head-first > into this without at least a rough understanding of the performance hotspots > or the GHC overall architecture is going to do me more harm than good (I get > the overall picture and I’m aware of the different stages of the GHC > compilation pipeline, but it’s far from saying I’m proficient with the > architecture as whole). I have also read a couple of years ago the GHC > chapter on the “Architeture of Open Source Applications” book, but I don’t > know how much that is still relevant. If it is, I guess I should refresh my > memory. > > I’m currently trying to move on 2 fronts — please advice if I’m a fool > flogging a dead horse or if I have any hope of getting anything done ;) > > 1. I’m trying to treat indeed the compiler as a black block (as you adviced) > trying to build a sufficiently large program where GHC is not “as fast as I > would like” (I know that’s a very lame definition of “slow”, hehe). In > particular, I have built the stage2 compiler with the “prof” flavour as you > suggested, and I have chosen 2 examples as a reference “benchmark” for > performance; DynFlags.hs (which seems to have been mentioned multiple times > as a GHC perf killer) and the highlighting-kate package as posted here: > https://ghc.haskell.org/trac/ghc/ticket/9221 . The idea would be to compile > those with -v +RTS -p -hc -RTS enabled, look at the output from the .prof > file AND the `-v` flag, find any hotspot, try to change something, > recompile, observe diff, rinse and repeat. Do you think I have any hope of > making progress this way? In particular, I think compiling DynFlags.hs is a > bit of a dead-end; I whipped up this buggy script which escalated into a > Behemoth which is compiling pretty much half of the compiler once again :D > > ``` > #!/usr/bin/env bash > > ../ghc/inplace/bin/ghc-stage2 --make -j8 -v +RTS -A256M -qb0 -p -h \ > -RTS -DSTAGE=2 -I../ghc/includes -I../ghc/compiler -I../ghc/compiler/stage2 > \ > -I../ghc/compiler/stage2/build \ > -i../ghc/compiler/utils:../ghc/compiler/types:../ghc/compiler/typecheck:../ghc/compiler/basicTypes > \ > -i../ghc/compiler/main:../ghc/compiler/profiling:../ghc/compiler/coreSyn:../ghc/compiler/iface:../ghc/compiler/prelude > \ > -i../ghc/compiler/stage2/build:../ghc/compiler/simplStg:../ghc/compiler/cmm:../ghc/compiler/parser:../ghc/compiler/hsSyn > \ > -i../ghc/compiler/ghci:../ghc/compiler/deSugar:../ghc/compiler/simplCore:../ghc/compile/specialise > \ > -fforce-recomp -c $@ > ``` > > I’m running it with `./dynflags.sh ../ghc/compiler/main/DynFlags.hs` but > it’s taking a lot to compile (20+ mins on my 2014 mac Pro) because it’s > pulling in half of the compiler anyway :D I tried to reuse the .hi files > from my stage2 compilation but I failed (GHC was complaining about interface > file mismatch). Short story short, I don’t think it will be a very agile way > to proceed. Am I right? Do you have any recommendation in such sense? Do I > have any hope to compile DynFlags.hs in a way which would make this perf > investigation feasible? > > The second example (the highlighting-kate package) seems much more > promising. It takes maybe 1-2 mins on my machine, which is enough to take a > look at the perf output. Do you think I should follow this second lead? In > principle any 50+ modules package I think would do (better if with a lot of > TH ;) ) but this seems like a low-entry barrier start. > > 2. The second path I’m exploring is simply to take a less holistic approach > and try to dive in into a performance ticket like the ones listed here: > https://www.reddit.com/r/haskell/comments/45q90s/is_anything_being_done_to_remedy_the_soul/czzq6an/ > Maybe some are very specific, but it seems like fixing small things and move > forward could help giving me understanding of different sub-parts of GHC, > which seems less intimidating than the black-box approach. > > In conclusion, what do you think is the best approach, 1 or 2, both or none? > ;) > > Thank you! > > Alfredo > > On 7 April 2017 at 18:30, Alfredo Di Napoli > wrote: >> >> Hey Ben, >> >> thanks for the quite exhaustive reply! I’m on the go right now, but I >> promise to get back to you with a meaningful reply later this weekend ;) >> >> Alfredo >> >> On 7 April 2017 at 18:22, Ben Gamari wrote: >>> >>> Alfredo Di Napoli writes: >>> >>> > Hey folks, >>> > >>> Hi Alfredo! >>> >>> First, thanks for writing. More eyes looking at GHC's compiler >>> performance is badly needed. >>> >>> > maybe I’m setting up for something too ambitious for me, but I would >>> > like >>> > to take an active stance to the overlasting “GHC compilation times are >>> > terrible” matter, instead of simply stare at the screen with despair >>> > whenever GHC compiles a sufficiently large Haskell program ;) >>> > >>> > To make this even more interesting, I have never contributed to GHC >>> > either! >>> > The max I have pushed myself into was 2 years ago when I successfully >>> > built >>> > GHC head from source and tried to fix an Haddock “easy” ticket I don’t >>> > even >>> > recall (full disclosure, eventually I didn’t :D ). >>> > >>> > Specifically, I would love community recommendations & guidance about: >>> > >>> > 1. Is this simply too daunting for somebody like me? Maybe is better to >>> > first start contributing more regularly, take confidence with the code >>> > base >>> > AND then move forward? >>> > >>> As with any software project, it is possible to treat the compiler as a >>> black box, throw a profiler at it and see what hotspots show up. This >>> gives you a place to focus your effort, allowing you to learn a small >>> area and broaden your knowledge as necessary. >>> >>> However, I think it's fair to say that you will be significantly more >>> productive if you first develop a basic understanding of the compilation >>> pipeline. I'd recommend having a look at the GHC Commentary [1] for a >>> start. >>> >>> I think it also helps to have a rough idea of what "slow" means to you. >>> I find it is quite helpful if you have a particular program which you >>> feel compiles more slowly than you would like (especially if it even >>> compiles slowly with -O0, since then much less of the compiler is >>> involved in compilation). Another approach is to look for programs whose >>> compilation time has regressed over the course of GHC releases. It is >>> not hard to find these examples and it is often possible to bisect your >>> way back to the regressing commit. >>> >>> Also, note that I have collected some notes pertaining to compiler >>> performance on the Wiki [2]. Here you will find a number of tickets of >>> interest (as well a some rough themes which I've noticed), some nofib >>> results which might guide your efforts, as well as a list of some >>> fixes which have been committed in the past. >>> >>> [1] https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler >>> [2] https://ghc.haskell.org/trac/ghc/wiki/Performance/Compiler >>> >>> > 2. Are compilation times largely dependant from the target platform >>> > (I’m on >>> > Darwin) or there is something which can be done “globally” so that the >>> > benefits can be experienced by everybody? >>> > >>> There are some external considerations (e.g. the platform's compiler and >>> linking toolchain) which contribute to GHC's runtime. For instance, it >>> is known that the BFD ld linker implementation that many Linux >>> distributions use by default is a great deal slower than it could be. >>> This particular issue has come up recently and I'm currently working on >>> allowing us to use the more performant gold linker when available. >>> >>> However, I think it's fair to say that for most programs GHC's runtime >>> is largely independent of platform. I would invite you to try compiling >>> a package which you consider GHC to compile "slowly" with GHC's -v flag >>> (and GHC 8.0.1 or newer). This will give you a rough breakdown of where >>> time is spent. For many packages you will find that the simplifier >>> and/or typechecker dominate, followed (often distantly) by native code >>> generation. Of these steps native code generation is the only one with a >>> strong platform dependence. >>> >>> > 3. Is there any recommended workflow to profile GHC compilation times? >>> > Is >>> > there any build flavour one should prefer when doing so? (Maybe the >>> > full, >>> > slowest one?) >>> > >>> There are a few options here: >>> >>> * As of GHC 8.0 the compiler will output timing and allocation >>> information for its various stages if run with -v. This can be >>> extremely helpful to get a high-level picture of where the compiler >>> is spending its time while compiling your program. This is almost >>> always the right place to start. >>> >>> * As with any Haskell program, the cost centre profiler can be used to >>> characterize the memory and CPU behavior of various parts of the >>> compiler. >>> >>> GHC's source tree includes a "prof" build flavour which builds the >>> compiler with profiling enabled. However it only includes a handful >>> of cost-centres and is best used when you already have a rough idea >>> where you are looking and can add further cost-centres to drill down >>> to your hotspot. >>> >>> Simply enabling -fprof-exported across the entire tree just doesn't >>> work in my experience: not only is the resulting compiler quite slow, >>> but the profile you get is far too unwieldy to learn from. >>> >>> * Occassionally the ticky-ticky profiler can be helpful in identifying >>> allocation hotspots without the full overhead of the cost-centre >>> profiler. >>> >>> * In principle our newly-stable DWARF debug information can be used for >>> profiling, although this is still a work in progress and requires a >>> patched GHC for best results. It's probably best to stick to the more >>> traditional profiling mechanisms for now. >>> >>> Anyways, I hope this helps. Always feel free to get in touch with me >>> personally (IRC and email are both great) if you would like to discuss >>> particular issues. Thanks again for your interest! >>> >>> Cheers, >>> >>> - Ben >>> >> > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > From david at well-typed.com Sun Apr 9 21:04:41 2017 From: david at well-typed.com (David Feuer) Date: Sun, 09 Apr 2017 17:04:41 -0400 Subject: Where do I start if I would like help improve GHC compilation times? Message-ID: <6umejd873ugi8h2ulsv1mnxv.1491771742901@email.android.com> Be aware that some of the biggest performance problems with TH simply can't be fixed without changes to the TH language. For details, see Edward Yang's blog post: http://blog.ezyang.com/2016/07/what-template-haskell-gets-wrong-and-racket-gets-right/ There was a Reddit thread discussing that post at https://www.reddit.com/r/haskell/comments/4tfzah/what_template_haskell_gets_wrong_and_racket_gets/ David FeuerWell-Typed, LLP -------- Original message --------From: Alfredo Di Napoli Date: 4/9/17 5:37 AM (GMT-05:00) To: Ben Gamari Cc: ghc-devs at haskell.org Subject: Re: Where do I start if I would like help improve GHC compilation times? Hey Ben, as promised I’m back to you with something more articulated and hopefully meaningful. I do hear you perfectly — probably trying to dive head-first into this without at least a rough understanding of the performance hotspots or the GHC overall architecture is going to do me more harm than good (I get the overall picture and I’m aware of the different stages of the GHC compilation pipeline, but it’s far from saying I’m proficient with the architecture as whole). I have also read a couple of years ago the GHC chapter on the “Architeture of Open Source Applications” book, but I don’t know how much that is still relevant. If it is, I guess I should refresh my memory. I’m currently trying to move on 2 fronts — please advice if I’m a fool flogging a dead horse or if I have any hope of getting anything done ;) 1. I’m trying to treat indeed the compiler as a black block (as you adviced) trying to build a sufficiently large program where GHC is not “as fast as I would like” (I know that’s a very lame definition of “slow”, hehe). In particular, I have built the stage2 compiler with the “prof” flavour as you suggested, and I have chosen 2 examples as a reference “benchmark” for performance; DynFlags.hs (which seems to have been mentioned multiple times as a GHC perf killer) and the highlighting-kate package as posted here: https://ghc.haskell.org/trac/ghc/ticket/9221 . The idea would be to compile those with -v +RTS -p -hc -RTS enabled, look at the output from the .prof file AND the `-v` flag, find any hotspot, try to change something, recompile, observe diff, rinse and repeat. Do you think I have any hope of making progress this way? In particular, I think compiling DynFlags.hs is a bit of a dead-end; I whipped up this buggy script which escalated into a Behemoth which is compiling pretty much half of the compiler once again :D ```#!/usr/bin/env bash ../ghc/inplace/bin/ghc-stage2 --make -j8 -v +RTS -A256M -qb0 -p -h \-RTS -DSTAGE=2 -I../ghc/includes -I../ghc/compiler -I../ghc/compiler/stage2 \-I../ghc/compiler/stage2/build \-i../ghc/compiler/utils:../ghc/compiler/types:../ghc/compiler/typecheck:../ghc/compiler/basicTypes \-i../ghc/compiler/main:../ghc/compiler/profiling:../ghc/compiler/coreSyn:../ghc/compiler/iface:../ghc/compiler/prelude \-i../ghc/compiler/stage2/build:../ghc/compiler/simplStg:../ghc/compiler/cmm:../ghc/compiler/parser:../ghc/compiler/hsSyn \-i../ghc/compiler/ghci:../ghc/compiler/deSugar:../ghc/compiler/simplCore:../ghc/compile/specialise \-fforce-recomp -c $@``` I’m running it with `./dynflags.sh ../ghc/compiler/main/DynFlags.hs` but it’s taking a lot to compile (20+ mins on my 2014 mac Pro) because it’s pulling in half of the compiler anyway :D I tried to reuse the .hi files from my stage2 compilation but I failed (GHC was complaining about interface file mismatch). Short story short, I don’t think it will be a very agile way to proceed. Am I right? Do you have any recommendation in such sense? Do I have any hope to compile DynFlags.hs in a way which would make this perf investigation feasible? The second example (the highlighting-kate package) seems much more promising. It takes maybe 1-2 mins on my machine, which is enough to take a look at the perf output. Do you think I should follow this second lead? In principle any 50+ modules package I think would do (better if with a lot of TH ;) ) but this seems like a low-entry barrier start. 2. The second path I’m exploring is simply to take a less holistic approach and try to dive in into a performance ticket like the ones listed here: https://www.reddit.com/r/haskell/comments/45q90s/is_anything_being_done_to_remedy_the_soul/czzq6an/Maybe some are very specific, but it seems like fixing small things and move forward could help giving me understanding of different sub-parts of GHC, which seems less intimidating than the black-box approach. In conclusion, what do you think is the best approach, 1 or 2, both or none? ;) Thank you! Alfredo On 7 April 2017 at 18:30, Alfredo Di Napoli wrote: Hey Ben, thanks for the quite exhaustive reply! I’m on the go right now, but I promise to get back to you with a meaningful reply later this weekend ;) Alfredo On 7 April 2017 at 18:22, Ben Gamari wrote: Alfredo Di Napoli writes: > Hey folks, > Hi Alfredo! First, thanks for writing. More eyes looking at GHC's compiler performance is badly needed. > maybe I’m setting up for something too ambitious for me, but I would like > to take an active stance to the overlasting “GHC compilation times are > terrible” matter, instead of simply stare at the screen with despair > whenever GHC compiles a sufficiently large Haskell program ;) > > To make this even more interesting, I have never contributed to GHC either! > The max I have pushed myself into was 2 years ago when I successfully built > GHC head from source and tried to fix an Haddock “easy” ticket I don’t even > recall (full disclosure, eventually I didn’t :D ). > > Specifically, I would love community recommendations & guidance about: > > 1. Is this simply too daunting for somebody like me? Maybe is better to > first start contributing more regularly, take confidence with the code base > AND then move forward? > As with any software project, it is possible to treat the compiler as a black box, throw a profiler at it and see what hotspots show up. This gives you a place to focus your effort, allowing you to learn a small area and broaden your knowledge as necessary. However, I think it's fair to say that you will be significantly more productive if you first develop a basic understanding of the compilation pipeline. I'd recommend having a look at the GHC Commentary [1] for a start. I think it also helps to have a rough idea of what "slow" means to you. I find it is quite helpful if you have a particular program which you feel compiles more slowly than you would like (especially if it even compiles slowly with -O0, since then much less of the compiler is involved in compilation). Another approach is to look for programs whose compilation time has regressed over the course of GHC releases. It is not hard to find these examples and it is often possible to bisect your way back to the regressing commit. Also, note that I have collected some notes pertaining to compiler performance on the Wiki [2]. Here you will find a number of tickets of interest (as well a some rough themes which I've noticed), some nofib results which might guide your efforts, as well as a list of some fixes which have been committed in the past. [1] https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler [2] https://ghc.haskell.org/trac/ghc/wiki/Performance/Compiler > 2. Are compilation times largely dependant from the target platform (I’m on > Darwin) or there is something which can be done “globally” so that the > benefits can be experienced by everybody? > There are some external considerations (e.g. the platform's compiler and linking toolchain) which contribute to GHC's runtime. For instance, it is known that the BFD ld linker implementation that many Linux distributions use by default is a great deal slower than it could be. This particular issue has come up recently and I'm currently working on allowing us to use the more performant gold linker when available. However, I think it's fair to say that for most programs GHC's runtime is largely independent of platform. I would invite you to try compiling a package which you consider GHC to compile "slowly" with GHC's -v flag (and GHC 8.0.1 or newer). This will give you a rough breakdown of where time is spent. For many packages you will find that the simplifier and/or typechecker dominate, followed (often distantly) by native code generation. Of these steps native code generation is the only one with a strong platform dependence. > 3. Is there any recommended workflow to profile GHC compilation times? Is > there any build flavour one should prefer when doing so? (Maybe the full, > slowest one?) > There are a few options here:  * As of GHC 8.0 the compiler will output timing and allocation    information for its various stages if run with -v. This can be    extremely helpful to get a high-level picture of where the compiler    is spending its time while compiling your program. This is almost    always the right place to start.  * As with any Haskell program, the cost centre profiler can be used to    characterize the memory and CPU behavior of various parts of the    compiler.    GHC's source tree includes a "prof" build flavour which builds the    compiler with profiling enabled. However it only includes a handful    of cost-centres and is best used when you already have a rough idea    where you are looking and can add further cost-centres to drill down    to your hotspot.    Simply enabling -fprof-exported across the entire tree just doesn't    work in my experience: not only is the resulting compiler quite slow,    but the profile you get is far too unwieldy to learn from.  * Occassionally the ticky-ticky profiler can be helpful in identifying    allocation hotspots without the full overhead of the cost-centre    profiler.  * In principle our newly-stable DWARF debug information can be used for    profiling, although this is still a work in progress and requires a    patched GHC for best results. It's probably best to stick to the more    traditional profiling mechanisms for now. Anyways, I hope this helps. Always feel free to get in touch with me personally (IRC and email are both great) if you would like to discuss particular issues. Thanks again for your interest! Cheers, - Ben -------------- next part -------------- An HTML attachment was scrubbed... URL: From mail at nh2.me Sun Apr 9 23:54:18 2017 From: mail at nh2.me (=?UTF-8?Q?Niklas_Hamb=c3=bcchen?=) Date: Mon, 10 Apr 2017 01:54:18 +0200 Subject: Where do I start if I would like help improve GHC compilation times? In-Reply-To: References: Message-ID: <002c215c-a534-7c0f-2524-8f356c55e6a5@nh2.me> I have some suggestions for low hanging fruits in this effort. 1. Make ghc print more statistics on what it spending time on When I did the linking investigation recently (https://www.reddit.com/r/haskell/comments/63y43y/liked_linking_3x_faster_with_gold_link_10x_faster/) I noticed (with strace) that there are lots of interesting syscalls being made that you might not expect. For example, each time TH is used, shared libraries are loaded, and to determine the shared library paths, ghc shells out to `gcc --print-file-name`. Each such invocation takes 20 ms on my system, and I have 1000 invocations in my build. That's 20 seconds (out of 2 minutes build time) just asking gcc for paths. I recommend that for every call to an external GHC measures how long that call took, so that it can be asked to print a summary when it's done. That might give us lots of interesting things to optimize. For example, This would have made the long linker times totally obvious. At the end, I would love to know for each compilation (both one-shot as used in ghc's build system, and `ghc --make`): * What programs did it invoke and how long did they take * What files did it read and how long did that take * How long did it take to read all the `.hi` files in `ghc --make` * High level time summary (parsing, typechecking, codegen, .hi files, etc) That way we'll know at least what is slow, and don't have to resort to strace every time in order to obtain this basic answer. 2. Investigate if idiotic syscalls are being done and how much There's this concept I call "idiotic syscalls", which are syscalls of which you know from before that they won't contribute anything productive. For example, if you give a linker N many `-L` flags (library dirs) and M many `-l` flags (library names to link), it will try to `stat()` or `open()` N*M many files, out of which most are total rubbish, because we typically know what library is in what dir. Example: You pass `-L/usr/lib/opencv -L/usr/lib/imagemagick -L/usr/lib/blas -lopencv -limagemagick -lblas`. Then you you will get things like `open("/usr/lib/opencv/libimagemagick.so") = ENOENT` which makes no sense and obviously that file doesn't exist. This is a problem with the general "search path" concept; same happens for running executables searching through $PATH. Yes, nonexistent file opens fail fast, but in my typical ghc invocation I get millions of them (and we should at least measure how much time is wasted on them), and they clutter the strace output and make the real problems harder to investigate. We should check if we can create ways to give pass those files that do exist. 3. Add pure TemplateHaskell It is well known that TH is a problem for incremental compilation because it can have side effects and we must therefore be more conservative about when to recompile; when you see a `[TH]` in your `ghc --make` output, it's likely that time again. I believe this could be avoided by adding a variant of TH that forbids the use of the `runIO` function, and can thus not have side effects. Most TH does not need side effects, for example any form of code generation based on other data types (lenses, instances for whatever). If that was made "pure TH", we would not have to recompile when inputs to our TH functions change. Potentially this could even be determined automatically instead of adding a new variant of TH like was done for typed TH `$$()`, simply by inspecting what's in the TH and if we can decide there's no `runIO` in there, mark it as clean, otherwise as tainted. 4. Build ghc with `ghc --make` if possible This one might be controversial or impossible (others can likely tell us). Most Haskell code is built with `ghc --make`, not with the one-shot compilation system + make or Hadrian as as done in GHC's build system. Weirdly, often `ghc --make` scales much worse and has much worse incremental recompilation times than the one-shot mode, which doesn't make sense given that it has no process creation overhead, can do much better caching etc. I believe that if ghc or large parts of it (e.g. stage2) itself was built with `--make`, we would magically see --make become very good, simply we make the right people (GHC devs) suffer through it daily :D. I expect from this the solution of the `-j` slowness, GHC overhead reduction, faster interface file loads and so on. These are some ideas. Niklas From ben at well-typed.com Mon Apr 10 03:50:04 2017 From: ben at well-typed.com (Ben Gamari) Date: Sun, 09 Apr 2017 23:50:04 -0400 Subject: [ANNOUNCE] GHC 8.2.1 release candidate 1 Message-ID: <87lgr8zztf.fsf@ben-laptop.smart-cactus.org> Hello everyone, The GHC team is very pleased to announce the first candidate of the 8.2.1 release of the Glasgow Haskell Compiler. Source and binary distributions are available at https://downloads.haskell.org/~ghc/8.2.1-rc1/ This is the first of a few release candidates leading up the final 8.2.1 release. This release will feature, * A new type-indexed Typeable implementation * The long awaited Backpack * Deriving strategies for disambiguating DeriveAnyClass, GeneralizedNewtypeDeriving, and stock mechanisms * Overloaded record fields * Improved compiler performance * Better code generation through more robust tracking of join points * Compact regions for more efficient garbage collection and serialization * Better support for machines with non-uniform memory architectures * More robust support for levity (e.g. RuntimeRep) polymorphism * A simple interface for streaming eventlog data from live processes * Further refinement of DWARF support Unfortunately there are still a few known issues in this release, including a few compiler panics (#13233, #13509) and a memory leak in the simplifier (#13426) which may adversely affect heap sizes and compilation time for some modules. This memory leak unfortunately made it impossible to provide a 32-bit Windows distribution for this candidate; this will be resolved in -rc2. As always, please let us know if you have difficulty. Thanks to everyone who has contributed to this release! Happy testing, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From sean.westfall at gmail.com Mon Apr 10 04:26:55 2017 From: sean.westfall at gmail.com (Sean Westfall) Date: Sun, 9 Apr 2017 21:26:55 -0700 Subject: Setting Up a OpenBSD System for Building GHC Wiki Page In-Reply-To: References: <20170405203801.GA14065@nutty.outback.escape.de> Message-ID: Hi Adam Steen, I just chatted with you on https://github.com/commercialhaskell/stack/issues/416, I just added so more info for the wxallowed step to your wiki page -- hope you don't mind. I'm gonna post something to this mailing list in a minute about the trouble I've been having with building GHC on openBSD 6.0. Thanks, Sean On Wed, Apr 5, 2017 at 8:04 PM, Adam Steen wrote: > Good Morning All > > Thanks for the replies > > @Karel > After more testing it looks like i can remove the python business. > > @Sergei and @Karel > libiconv is a prerequisite of ghc and is installed automatically, do > you think this should be explicit? > > @Matthias > I will have to have a read about branches and will update the wiki shortly. > > Cheers > Adam > > On 6 April 2017 at 04:38, Matthias Kilian wrote: > > Hi Adam, > > > > On Wed, Apr 05, 2017 at 03:39:18PM +0800, Adam Steen wrote: > >> I have created the Building/Preparation/OpenBSD Wiki Page, i have not > yet > >> link it to Building/Preparation > >> page yet, > but > >> am looking for a review > >> > >> Setting Up a OpenBSD System for Building GHC > >> > > > > Nice to see that some people actively work on ghc on OpenBSD (and > > apologies for me beeing such a slacker keeping the ghc package for > > OpenBSD up to date). > > > > FWIW, you don't have to pass the very exact package name to pkg_add, > > like 'autoconf-2.69p2' or ''automake-1.15p0'. There's a relatively new > > concept in pkg_tools called 'branches' (see pkg_add(1) manpage). > > > > You just have to replace the '-' separating the stem and the version by > > a '%', e.g. > > > > pkg_add autoconf%2.69 automake%1.15 > > > > Then you'll get autoconf-2.69 and aotmake-1.15 regardless of what's the > > current REVISION (aka package patchleve) ot the package. > > > > Ciao, > > Kili > > On Thu, Apr 6, 2017 at 4:38 AM, Matthias Kilian > wrote: > > Hi Adam, > > > > On Wed, Apr 05, 2017 at 03:39:18PM +0800, Adam Steen wrote: > >> I have created the Building/Preparation/OpenBSD Wiki Page, i have not > yet > >> link it to Building/Preparation > >> page yet, > but > >> am looking for a review > >> > >> Setting Up a OpenBSD System for Building GHC > >> > > > > Nice to see that some people actively work on ghc on OpenBSD (and > > apologies for me beeing such a slacker keeping the ghc package for > > OpenBSD up to date). > > > > FWIW, you don't have to pass the very exact package name to pkg_add, > > like 'autoconf-2.69p2' or ''automake-1.15p0'. There's a relatively new > > concept in pkg_tools called 'branches' (see pkg_add(1) manpage). > > > > You just have to replace the '-' separating the stem and the version by > > a '%', e.g. > > > > pkg_add autoconf%2.69 automake%1.15 > > > > Then you'll get autoconf-2.69 and aotmake-1.15 regardless of what's the > > current REVISION (aka package patchleve) ot the package. > > > > Ciao, > > Kili > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -- Sean Westfall 2644 E. 4th St. Long Beach, CA 90802 ph: (808) 281-4695 em: sean.westfall at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam at adamsteen.com.au Mon Apr 10 05:23:34 2017 From: adam at adamsteen.com.au (Adam Steen) Date: Mon, 10 Apr 2017 13:23:34 +0800 Subject: Setting Up a OpenBSD System for Building GHC Wiki Page In-Reply-To: References: <20170405203801.GA14065@nutty.outback.escape.de> Message-ID: Hi Sean The more people using OpenBSD and GHC the better, please add as much information as you think necessary. Cheers Adam On Mon, Apr 10, 2017 at 12:26 PM, Sean Westfall wrote: > Hi Adam Steen, > I just chatted with you on > https://github.com/commercialhaskell/stack/issues/416, I just added so more > info for the wxallowed step to your wiki page -- hope you don't mind. I'm > gonna post something to this mailing list in a minute about the trouble I've > been having with building GHC on openBSD 6.0. > > Thanks, > Sean > > On Wed, Apr 5, 2017 at 8:04 PM, Adam Steen wrote: >> >> Good Morning All >> >> Thanks for the replies >> >> @Karel >> After more testing it looks like i can remove the python business. >> >> @Sergei and @Karel >> libiconv is a prerequisite of ghc and is installed automatically, do >> you think this should be explicit? >> >> @Matthias >> I will have to have a read about branches and will update the wiki >> shortly. >> >> Cheers >> Adam >> >> On 6 April 2017 at 04:38, Matthias Kilian wrote: >> > Hi Adam, >> > >> > On Wed, Apr 05, 2017 at 03:39:18PM +0800, Adam Steen wrote: >> >> I have created the Building/Preparation/OpenBSD Wiki Page, i have not >> >> yet >> >> link it to Building/Preparation >> >> page yet, >> >> but >> >> am looking for a review >> >> >> >> Setting Up a OpenBSD System for Building GHC >> >> >> > >> > Nice to see that some people actively work on ghc on OpenBSD (and >> > apologies for me beeing such a slacker keeping the ghc package for >> > OpenBSD up to date). >> > >> > FWIW, you don't have to pass the very exact package name to pkg_add, >> > like 'autoconf-2.69p2' or ''automake-1.15p0'. There's a relatively new >> > concept in pkg_tools called 'branches' (see pkg_add(1) manpage). >> > >> > You just have to replace the '-' separating the stem and the version by >> > a '%', e.g. >> > >> > pkg_add autoconf%2.69 automake%1.15 >> > >> > Then you'll get autoconf-2.69 and aotmake-1.15 regardless of what's the >> > current REVISION (aka package patchleve) ot the package. >> > >> > Ciao, >> > Kili >> >> On Thu, Apr 6, 2017 at 4:38 AM, Matthias Kilian >> wrote: >> > Hi Adam, >> > >> > On Wed, Apr 05, 2017 at 03:39:18PM +0800, Adam Steen wrote: >> >> I have created the Building/Preparation/OpenBSD Wiki Page, i have not >> >> yet >> >> link it to Building/Preparation >> >> page yet, >> >> but >> >> am looking for a review >> >> >> >> Setting Up a OpenBSD System for Building GHC >> >> >> > >> > Nice to see that some people actively work on ghc on OpenBSD (and >> > apologies for me beeing such a slacker keeping the ghc package for >> > OpenBSD up to date). >> > >> > FWIW, you don't have to pass the very exact package name to pkg_add, >> > like 'autoconf-2.69p2' or ''automake-1.15p0'. There's a relatively new >> > concept in pkg_tools called 'branches' (see pkg_add(1) manpage). >> > >> > You just have to replace the '-' separating the stem and the version by >> > a '%', e.g. >> > >> > pkg_add autoconf%2.69 automake%1.15 >> > >> > Then you'll get autoconf-2.69 and aotmake-1.15 regardless of what's the >> > current REVISION (aka package patchleve) ot the package. >> > >> > Ciao, >> > Kili >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > > > -- > > Sean Westfall > 2644 E. 4th St. > Long Beach, CA 90802 > ph: (808) 281-4695 > em: sean.westfall at gmail.com From m at tweag.io Mon Apr 10 07:13:02 2017 From: m at tweag.io (Boespflug, Mathieu) Date: Mon, 10 Apr 2017 09:13:02 +0200 Subject: [ANNOUNCE] GHC 8.2.1 release candidate 1 In-Reply-To: <87lgr8zztf.fsf@ben-laptop.smart-cactus.org> References: <87lgr8zztf.fsf@ben-laptop.smart-cactus.org> Message-ID: Hi Ben, this is great news! I'm particularly keen on learning more about two points you mentioned in your email: * Compiler performance: do you have any numbers to quantify what 8.0 vs 8.2 is likely to look like? How much has the work that's been done affect performance across the board? Or are these mostly pathological cases (e.g. ghc --make with high number of cores, large number of definitions in a module, large amount of let nesting in definitions, etc) * DWARF support: could you clarify at a very high-level what typical uses cases can be expected to work and which ones won't? Would be eager to read any resources you could point me at to help me understand what still needs to be done on this front. Many thanks, -- Mathieu Boespflug Founder at http://tweag.io. On 10 April 2017 at 05:50, Ben Gamari wrote: > > Hello everyone, > > The GHC team is very pleased to announce the first candidate of the > 8.2.1 release of the Glasgow Haskell Compiler. Source and binary > distributions are available at > > https://downloads.haskell.org/~ghc/8.2.1-rc1/ > > This is the first of a few release candidates leading up the final 8.2.1 > release. This release will feature, > > * A new type-indexed Typeable implementation > > * The long awaited Backpack > > * Deriving strategies for disambiguating DeriveAnyClass, > GeneralizedNewtypeDeriving, and stock mechanisms > > * Overloaded record fields > > * Improved compiler performance > > * Better code generation through more robust tracking of join points > > * Compact regions for more efficient garbage collection and serialization > > * Better support for machines with non-uniform memory architectures > > * More robust support for levity (e.g. RuntimeRep) polymorphism > > * A simple interface for streaming eventlog data from live processes > > * Further refinement of DWARF support > > Unfortunately there are still a few known issues in this release, > including a few compiler panics (#13233, #13509) and a memory leak in > the simplifier (#13426) which may adversely affect heap sizes and > compilation time for some modules. This memory leak unfortunately made > it impossible to provide a 32-bit Windows distribution for this > candidate; this will be resolved in -rc2. > > As always, please let us know if you have difficulty. Thanks to everyone > who has contributed to this release! > > Happy testing, > > - Ben > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lonetiger at gmail.com Mon Apr 10 12:42:56 2017 From: lonetiger at gmail.com (Phyx) Date: Mon, 10 Apr 2017 12:42:56 +0000 Subject: [ANNOUNCE] GHC 8.2.1 release candidate 1 In-Reply-To: References: <87lgr8zztf.fsf@ben-laptop.smart-cactus.org> Message-ID: Hi Ben, The windows build is unusable. The settings file has $TopDir expanded and hard-coded to the build path on drydock. Tamar On Mon, 10 Apr 2017, 08:14 Boespflug, Mathieu, wrote: > Hi Ben, > > this is great news! I'm particularly keen on learning more about two > points you mentioned in your email: > > * Compiler performance: do you have any numbers to quantify what 8.0 vs > 8.2 is likely to look like? How much has the work that's been done affect > performance across the board? Or are these mostly pathological cases (e.g. > ghc --make with high number of cores, large number of definitions in a > module, large amount of let nesting in definitions, etc) > * DWARF support: could you clarify at a very high-level what typical uses > cases can be expected to work and which ones won't? Would be eager to read > any resources you could point me at to help me understand what still needs > to be done on this front. > > Many thanks, > > -- > Mathieu Boespflug > Founder at http://tweag.io. > > On 10 April 2017 at 05:50, Ben Gamari wrote: > > > Hello everyone, > > The GHC team is very pleased to announce the first candidate of the > 8.2.1 release of the Glasgow Haskell Compiler. Source and binary > distributions are available at > > https://downloads.haskell.org/~ghc/8.2.1-rc1/ > > This is the first of a few release candidates leading up the final 8.2.1 > release. This release will feature, > > * A new type-indexed Typeable implementation > > * The long awaited Backpack > > * Deriving strategies for disambiguating DeriveAnyClass, > GeneralizedNewtypeDeriving, and stock mechanisms > > * Overloaded record fields > > * Improved compiler performance > > * Better code generation through more robust tracking of join points > > * Compact regions for more efficient garbage collection and serialization > > * Better support for machines with non-uniform memory architectures > > * More robust support for levity (e.g. RuntimeRep) polymorphism > > * A simple interface for streaming eventlog data from live processes > > * Further refinement of DWARF support > > Unfortunately there are still a few known issues in this release, > including a few compiler panics (#13233, #13509) and a memory leak in > the simplifier (#13426) which may adversely affect heap sizes and > compilation time for some modules. This memory leak unfortunately made > it impossible to provide a 32-bit Windows distribution for this > candidate; this will be resolved in -rc2. > > As always, please let us know if you have difficulty. Thanks to everyone > who has contributed to this release! > > Happy testing, > > - Ben > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Mon Apr 10 13:54:41 2017 From: ben at well-typed.com (Ben Gamari) Date: Mon, 10 Apr 2017 09:54:41 -0400 Subject: [ANNOUNCE] GHC 8.2.1 release candidate 1 In-Reply-To: References: <87lgr8zztf.fsf@ben-laptop.smart-cactus.org> Message-ID: <87inmcz7tq.fsf@ben-laptop.smart-cactus.org> Phyx writes: > Hi Ben, > > The windows build is unusable. The settings file has $TopDir expanded and > hard-coded to the build path on drydock. > Hmm, this is very odd and sounds like a build system bug since I used the same scripts as usual to generate this bindist. I'll investigate. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From alfredo.dinapoli at gmail.com Mon Apr 10 14:54:26 2017 From: alfredo.dinapoli at gmail.com (Alfredo Di Napoli) Date: Mon, 10 Apr 2017 16:54:26 +0200 Subject: Where do I start if I would like help improve GHC compilation times? In-Reply-To: <002c215c-a534-7c0f-2524-8f356c55e6a5@nh2.me> References: <002c215c-a534-7c0f-2524-8f356c55e6a5@nh2.me> Message-ID: Hey all, thanks a ton for the invaluable pointers. I’m now in the “I-dunno-what-I-am-doing” mode banging SCC annotations like there is no tomorrow, trying to spot any chance for some low-hanging-fruit algorithmic improvement (like using a sequence instead of a list, etc), and will come back to your suggestions as I will face the inevitable dead-end wall :D Alfredo On 10 April 2017 at 01:54, Niklas Hambüchen wrote: > I have some suggestions for low hanging fruits in this effort. > > 1. Make ghc print more statistics on what it spending time on > > When I did the linking investigation recently > (https://www.reddit.com/r/haskell/comments/63y43y/liked_ > linking_3x_faster_with_gold_link_10x_faster/) > I noticed (with strace) that there are lots of interesting syscalls > being made that you might not expect. For example, each time TH is used, > shared libraries are loaded, and to determine the shared library paths, > ghc shells out to `gcc --print-file-name`. Each such invocation takes 20 > ms on my system, and I have 1000 invocations in my build. That's 20 > seconds (out of 2 minutes build time) just asking gcc for paths. > > I recommend that for every call to an external GHC measures how long > that call took, so that it can be asked to print a summary when it's done. > > That might give us lots of interesting things to optimize. For example, > This would have made the long linker times totally obvious. > > At the end, I would love to know for each compilation (both one-shot as > used in ghc's build system, and `ghc --make`): > > * What programs did it invoke and how long did they take > * What files did it read and how long did that take > * How long did it take to read all the `.hi` files in `ghc --make` > * High level time summary (parsing, typechecking, codegen, .hi files, etc) > > That way we'll know at least what is slow, and don't have to resort to > strace every time in order to obtain this basic answer. > > 2. Investigate if idiotic syscalls are being done and how much > > There's this concept I call "idiotic syscalls", which are syscalls of > which you know from before that they won't contribute anything > productive. For example, if you give a linker N many `-L` flags (library > dirs) and M many `-l` flags (library names to link), it will try to > `stat()` or `open()` N*M many files, out of which most are total > rubbish, because we typically know what library is in what dir. > Example: You pass `-L/usr/lib/opencv -L/usr/lib/imagemagick > -L/usr/lib/blas -lopencv -limagemagick -lblas`. Then you you will get > things like `open("/usr/lib/opencv/libimagemagick.so") = ENOENT` which > makes no sense and obviously that file doesn't exist. This is a problem > with the general "search path" concept; same happens for running > executables searching through $PATH. Yes, nonexistent file opens fail > fast, but in my typical ghc invocation I get millions of them (and we > should at least measure how much time is wasted on them), and they > clutter the strace output and make the real problems harder to investigate. > We should check if we can create ways to give pass those files that do > exist. > > 3. Add pure TemplateHaskell > > It is well known that TH is a problem for incremental compilation > because it can have side effects and we must therefore be more > conservative about when to recompile; when you see a `[TH]` in your `ghc > --make` output, it's likely that time again. > > I believe this could be avoided by adding a variant of TH that forbids > the use of the `runIO` function, and can thus not have side effects. > > Most TH does not need side effects, for example any form of code > generation based on other data types (lenses, instances for whatever). > If that was made "pure TH", we would not have to recompile when inputs > to our TH functions change. > > Potentially this could even be determined automatically instead of > adding a new variant of TH like was done for typed TH `$$()`, simply by > inspecting what's in the TH and if we can decide there's no `runIO` in > there, mark it as clean, otherwise as tainted. > > 4. Build ghc with `ghc --make` if possible > > This one might be controversial or impossible (others can likely tell > us). Most Haskell code is built with `ghc --make`, not with the one-shot > compilation system + make or Hadrian as as done in GHC's build system. > Weirdly, often `ghc --make` scales much worse and has much worse > incremental recompilation times than the one-shot mode, which doesn't > make sense given that it has no process creation overhead, can do much > better caching etc. I believe that if ghc or large parts of it (e.g. > stage2) itself was built with `--make`, we would magically see --make > become very good, simply we make the right people (GHC devs) suffer > through it daily :D. I expect from this the solution of the `-j` > slowness, GHC overhead reduction, faster interface file loads and so on. > > These are some ideas. > > Niklas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Mon Apr 10 19:28:35 2017 From: ben at well-typed.com (Ben Gamari) Date: Mon, 10 Apr 2017 15:28:35 -0400 Subject: [ANNOUNCE] GHC 8.2.1 release candidate 1 In-Reply-To: References: <87lgr8zztf.fsf@ben-laptop.smart-cactus.org> Message-ID: <87fuhgysd8.fsf@ben-laptop.smart-cactus.org> "Boespflug, Mathieu" writes: > Hi Ben, > > this is great news! I'm particularly keen on learning more about two points > you mentioned in your email: > > * Compiler performance: do you have any numbers to quantify what 8.0 vs 8.2 > is likely to look like? I'm afraid the best I can provide at the moment is [1]. On closer inspection of these I'm a bit suspicious of the 8.0 numbers; I'll try to reproduce them (and characterize the current ghc-8.2 branch, which fixes a significant memory leak present in -rc1) shortly. That being said, there were a few major fixes in 8.2. > How much has the work that's been done affect performance across the > board? Or are these mostly pathological cases (e.g. ghc --make with > high number of cores, large number of definitions in a module, large > amount of let nesting in definitions, etc) The fixes span the gamut but I generally tried to concentrate on issues that looked like they might affect a large fraction of programs. I fixed at least one major regression present in 8.0 which significantly inflated allocations of typeclass instance matching. I've also done quite a bit of refactoring around our handling of names. These include improvements in interface file serialization efficiency and name lookup. These are just some of the major reworks; there are also numerous smaller fixes which I don't have time to cover here. Compilation performance has also generally improved as a result of some of the work in the simplifier. In particular, GHC now performs an early inlining phase which, quite surprisingly, tends to result in smaller programs earlier in simplification, reducing the amount of work that the compiler needs to carry out. Simon summarized some of his preliminary numbers here [2]. > * DWARF support: could you clarify at a very high-level what typical uses > cases can be expected to work and which ones won't? Would be eager to read > any resources you could point me at to help me understand what still needs > to be done on this front. > At this point if GHC compiles your program with -g you should have a pretty good assurance that the resulting DWARF information is reasonable. This means that users can use the ExecutionStack mechanism, RTS stack-trace functionality, and GDB without fear of the act of capturing a stacktrace leading to a crash. After starting to write more here, I realized that you will likely not be the last person to ask about this and updated the DWARF status page with additional discussion [3]. Let me know if you have any questions. Cheers, - Ben [1] https://gist.github.com/bgamari/fbd3e55047bd041d8208ebe820c0f493 [2] https://mail.haskell.org/pipermail/ghc-devs/2017-February/013818.html [3] https://ghc.haskell.org/trac/ghc/wiki/DWARF/Status -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From m at tweag.io Mon Apr 10 20:26:34 2017 From: m at tweag.io (Boespflug, Mathieu) Date: Mon, 10 Apr 2017 22:26:34 +0200 Subject: [ANNOUNCE] GHC 8.2.1 release candidate 1 In-Reply-To: <87fuhgysd8.fsf@ben-laptop.smart-cactus.org> References: <87lgr8zztf.fsf@ben-laptop.smart-cactus.org> <87fuhgysd8.fsf@ben-laptop.smart-cactus.org> Message-ID: Hi Ben, thank you for this summary. The DWARF status page is helpful. Something was unclear to me though. There are three main potential use cases for DWARF that I see: 1. debugging, possibly with gdb 2. stack traces on exceptions 3. stack sampling, which is a form of performance profiling. Forgive me for these naive questions, but... Is (1) possible at all at this point? If I compile GHC with all the right options as explained in the status page, do I magically get backtraces associated to all my exceptions? What should I do to get those? It's my understanding that (3) is possible and works fine, even when FFI code is invoked, but slower than for C programs due to running user-level code to inspect the stack for each sample. Is that right? Best, -- Mathieu Boespflug Founder at http://tweag.io. On 10 April 2017 at 21:28, Ben Gamari wrote: > "Boespflug, Mathieu" writes: > > > Hi Ben, > > > > this is great news! I'm particularly keen on learning more about two > points > > you mentioned in your email: > > > > * Compiler performance: do you have any numbers to quantify what 8.0 vs > 8.2 > > is likely to look like? > > I'm afraid the best I can provide at the moment is [1]. On closer > inspection of these I'm a bit suspicious of the 8.0 numbers; I'll try to > reproduce them (and characterize the current ghc-8.2 branch, which fixes > a significant memory leak present in -rc1) shortly. That being said, > there were a few major fixes in 8.2. > > > How much has the work that's been done affect performance across the > > board? Or are these mostly pathological cases (e.g. ghc --make with > > high number of cores, large number of definitions in a module, large > > amount of let nesting in definitions, etc) > > The fixes span the gamut but I generally tried to concentrate on issues > that looked like they might affect a large fraction of programs. I fixed > at least one major regression present in 8.0 which significantly > inflated allocations of typeclass instance matching. I've also done > quite a bit of refactoring around our handling of names. These include > improvements in interface file serialization efficiency and name lookup. > These are just some of the major reworks; there are also numerous > smaller fixes which I don't have time to cover here. > > Compilation performance has also generally improved as a result of some > of the work in the simplifier. In particular, GHC now performs an early > inlining phase which, quite surprisingly, tends to result in smaller > programs earlier in simplification, reducing the amount of work that the > compiler needs to carry out. Simon summarized some of his preliminary > numbers here [2]. > > > * DWARF support: could you clarify at a very high-level what typical uses > > cases can be expected to work and which ones won't? Would be eager to > read > > any resources you could point me at to help me understand what still > needs > > to be done on this front. > > > At this point if GHC compiles your program with -g you should have a > pretty good assurance that the resulting DWARF information is > reasonable. This means that users can use the ExecutionStack mechanism, > RTS stack-trace functionality, and GDB without fear of the act of > capturing a stacktrace leading to a crash. > > After starting to write more here, I realized that you will likely not > be the last person to ask about this and updated the DWARF status page > with additional discussion [3]. Let me know if you have any questions. > > Cheers, > > - Ben > > [1] https://gist.github.com/bgamari/fbd3e55047bd041d8208ebe820c0f493 > [2] https://mail.haskell.org/pipermail/ghc-devs/2017-February/013818.html > [3] https://ghc.haskell.org/trac/ghc/wiki/DWARF/Status > -------------- next part -------------- An HTML attachment was scrubbed... URL: From m at tweag.io Mon Apr 10 20:39:18 2017 From: m at tweag.io (Boespflug, Mathieu) Date: Mon, 10 Apr 2017 22:39:18 +0200 Subject: [ANNOUNCE] GHC 8.2.1 release candidate 1 In-Reply-To: <87fuhgysd8.fsf@ben-laptop.smart-cactus.org> References: <87lgr8zztf.fsf@ben-laptop.smart-cactus.org> <87fuhgysd8.fsf@ben-laptop.smart-cactus.org> Message-ID: > > * Compiler performance: do you have any numbers to quantify what 8.0 vs 8.2 > > is likely to look like? > > I'm afraid the best I can provide at the moment is [1]. On closer > inspection of these I'm a bit suspicious of the 8.0 numbers; I'll try to > reproduce them (and characterize the current ghc-8.2 branch, which fixes > a significant memory leak present in -rc1) shortly. That being said, > there were a few major fixes in 8.2. I'm not sure how to interpret these numbers. Looking at the compile time section, it seems to report an average 81% *increase* in compilation time, but compared to what - ghc-8.0 or ghc-7.4? From ben at well-typed.com Mon Apr 10 20:52:50 2017 From: ben at well-typed.com (Ben Gamari) Date: Mon, 10 Apr 2017 16:52:50 -0400 Subject: [ANNOUNCE] GHC 8.2.1 release candidate 1 In-Reply-To: References: <87lgr8zztf.fsf@ben-laptop.smart-cactus.org> <87fuhgysd8.fsf@ben-laptop.smart-cactus.org> Message-ID: <878tn8yogt.fsf@ben-laptop.smart-cactus.org> "Boespflug, Mathieu" writes: > Hi Ben, > > thank you for this summary. The DWARF status page is helpful. Something was > unclear to me though. There are three main potential use cases for DWARF > that I see: > > 1. debugging, possibly with gdb > 2. stack traces on exceptions > 3. stack sampling, which is a form of performance profiling. > > Forgive me for these naive questions, but... Is (1) possible at all at this > point? If you break into a program with gdb you will indeed get a usable stacktrace. However, you cannot probe in-scope bindings as you can in other languages. > If I compile GHC with all the right options as explained in the > status page, do I magically get backtraces associated to all my > exceptions? What should I do to get those? Ahh, thanks for pointing this out; this should be discussed on the status page. I have a proposal allowing this here [1] and there is a ticket here [2]. It turns out that this is actually a slightly tricky thing due to two facts, * stack unwinding can be expensive * exceptions (especially asynchronous exceptions) are sometimes used as a mechanism for normal control flow. Compatibility is quite tricky as under the current plan existing code would drop callstack information if it catches and rethrows an exception. I don't see any way to resolve this in a backwards compatible manner. > It's my understanding that (3) is possible and works fine, even when > FFI code is invoked, but slower than for C programs due to running > user-level code to inspect the stack for each sample. Is that right? It's possible with my statistical profiling patches, but these are currently not merged and the project is on hold due to lack of time. You can, however, use perf. --callgraph will be broken in Haskell code, however (e.g. you will only see the first stack frame). Cheers, - Ben [1] https://ghc.haskell.org/trac/ghc/wiki/Exceptions/StackTraces [2] https://ghc.haskell.org/trac/ghc/ticket/12096 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at well-typed.com Mon Apr 10 22:15:03 2017 From: ben at well-typed.com (Ben Gamari) Date: Mon, 10 Apr 2017 18:15:03 -0400 Subject: Where do I start if I would like help improve GHC compilation times? In-Reply-To: <002c215c-a534-7c0f-2524-8f356c55e6a5@nh2.me> References: <002c215c-a534-7c0f-2524-8f356c55e6a5@nh2.me> Message-ID: <87tw5vykns.fsf@ben-laptop.smart-cactus.org> Niklas Hambüchen writes: > I have some suggestions for low hanging fruits in this effort. Hi Niklas, Thanks for your suggestions, they are quite reasonable. > 1. Make ghc print more statistics on what it spending time on > > When I did the linking investigation recently > (https://www.reddit.com/r/haskell/comments/63y43y/liked_linking_3x_faster_with_gold_link_10x_faster/) > I noticed (with strace) that there are lots of interesting syscalls > being made that you might not expect. For example, each time TH is used, > shared libraries are loaded, and to determine the shared library paths, > ghc shells out to `gcc --print-file-name`. Each such invocation takes 20 > ms on my system, and I have 1000 invocations in my build. That's 20 > seconds (out of 2 minutes build time) just asking gcc for paths. > > I recommend that for every call to an external GHC measures how long > that call took, so that it can be asked to print a summary when it's done. Indeed. Since 8.0 GHC has had a scheme for tracking CPU time and allocations of various compiler phases. It would be great to implement a similar scheme for external tools. You can't necessarily reuse the existing infrastructure since you will want to use monotonic time instead of CPU time and allocations aren't relevant. > That might give us lots of interesting things to optimize. For example, > This would have made the long linker times totally obvious. I have a number of thoughts on this which I'll share in another message as they are largely orthogonal to the matter at hand. > At the end, I would love to know for each compilation (both one-shot as > used in ghc's build system, and `ghc --make`): > > * What programs did it invoke and how long did they take > * What files did it read and how long did that take > * How long did it take to read all the `.hi` files in `ghc --make` > * High level time summary (parsing, typechecking, codegen, .hi files, etc) We already have much of this. See D1959 for the relevant change. It would be great to extend this with more (optional) detail (e.g. measure individual interface file deserialization times). > That way we'll know at least what is slow, and don't have to resort to > strace every time in order to obtain this basic answer. Indeed. On the whole this would be a very nice, more-or-less self-contained project. > 2. Investigate if idiotic syscalls are being done and how much > > There's this concept I call "idiotic syscalls", which are syscalls of > which you know from before that they won't contribute anything > productive. For example, if you give a linker N many `-L` flags (library > dirs) and M many `-l` flags (library names to link), it will try to > `stat()` or `open()` N*M many files, out of which most are total > rubbish, because we typically know what library is in what dir. > Example: You pass `-L/usr/lib/opencv -L/usr/lib/imagemagick > -L/usr/lib/blas -lopencv -limagemagick -lblas`. Then you you will get > things like `open("/usr/lib/opencv/libimagemagick.so") = ENOENT` which > makes no sense and obviously that file doesn't exist. This is a problem > with the general "search path" concept; same happens for running > executables searching through $PATH. Yes, nonexistent file opens fail > fast, but in my typical ghc invocation I get millions of them (and we > should at least measure how much time is wasted on them), and they > clutter the strace output and make the real problems harder to investigate. > We should check if we can create ways to give pass those files that do > exist. Indeed, the matter of library search paths is actually a pretty bad one although largely a Cabal issue (see GHC #11587). > 3. Add pure TemplateHaskell > > It is well known that TH is a problem for incremental compilation > because it can have side effects and we must therefore be more > conservative about when to recompile; when you see a `[TH]` in your `ghc > --make` output, it's likely that time again. > > I believe this could be avoided by adding a variant of TH that forbids > the use of the `runIO` function, and can thus not have side effects. > > Most TH does not need side effects, for example any form of code > generation based on other data types (lenses, instances for whatever). > If that was made "pure TH", we would not have to recompile when inputs > to our TH functions change. > > Potentially this could even be determined automatically instead of > adding a new variant of TH like was done for typed TH `$$()`, simply by > inspecting what's in the TH and if we can decide there's no `runIO` in > there, mark it as clean, otherwise as tainted. Cross-compiling users would also greatly appreciate this. > 4. Build ghc with `ghc --make` if possible > > This one might be controversial or impossible (others can likely tell > us). Most Haskell code is built with `ghc --make`, not with the one-shot > compilation system + make or Hadrian as as done in GHC's build system. > Weirdly, often `ghc --make` scales much worse and has much worse > incremental recompilation times than the one-shot mode, which doesn't > make sense given that it has no process creation overhead, can do much > better caching etc. I believe that if ghc or large parts of it (e.g. > stage2) itself was built with `--make`, we would magically see --make > become very good, simply we make the right people (GHC devs) suffer > through it daily :D. I expect from this the solution of the `-j` > slowness, GHC overhead reduction, faster interface file loads and so on. > One thing I've wondered about is how we can make use of compact regions to try to minimize GC costs in long-lived GHC sessions (e.g. a --make invocation on a large project). This would likely help parallel performance as well. N.B. #9221 is essentially the catch-all ticket for -j scalability issues. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From m at tweag.io Mon Apr 10 22:25:23 2017 From: m at tweag.io (Boespflug, Mathieu) Date: Tue, 11 Apr 2017 00:25:23 +0200 Subject: [ANNOUNCE] GHC 8.2.1 release candidate 1 In-Reply-To: <878tn8yogt.fsf@ben-laptop.smart-cactus.org> References: <87lgr8zztf.fsf@ben-laptop.smart-cactus.org> <87fuhgysd8.fsf@ben-laptop.smart-cactus.org> <878tn8yogt.fsf@ben-laptop.smart-cactus.org> Message-ID: Hi Ben, it sounds like some of the remaining limitations around DWARF support (e.g. finishing the stack sampling work, local bindings in GDB, ...) could make for a good Haskell Summer of Code project. Have you considered writing this up as one or two project ideas here: https://summer.haskell.org/ideas.html? As HSoC sponsors, we'd be more than happy to see someone pick this up. Best, -- Mathieu Boespflug Founder at http://tweag.io. On 10 April 2017 at 22:52, Ben Gamari wrote: > "Boespflug, Mathieu" writes: > >> Hi Ben, >> >> thank you for this summary. The DWARF status page is helpful. Something was >> unclear to me though. There are three main potential use cases for DWARF >> that I see: >> >> 1. debugging, possibly with gdb >> 2. stack traces on exceptions >> 3. stack sampling, which is a form of performance profiling. >> >> Forgive me for these naive questions, but... Is (1) possible at all at this >> point? > > If you break into a program with gdb you will indeed get a usable > stacktrace. However, you cannot probe in-scope bindings as you can in > other languages. > >> If I compile GHC with all the right options as explained in the >> status page, do I magically get backtraces associated to all my >> exceptions? What should I do to get those? > > Ahh, thanks for pointing this out; this should be discussed on the > status page. I have a proposal allowing this here [1] and there is a > ticket here [2]. > > It turns out that this is actually a slightly tricky thing due to two > facts, > > * stack unwinding can be expensive > > * exceptions (especially asynchronous exceptions) are sometimes used > as a mechanism for normal control flow. > > Compatibility is quite tricky as under the current plan existing code > would drop callstack information if it catches and rethrows an > exception. I don't see any way to resolve this in a backwards > compatible manner. > >> It's my understanding that (3) is possible and works fine, even when >> FFI code is invoked, but slower than for C programs due to running >> user-level code to inspect the stack for each sample. Is that right? > > It's possible with my statistical profiling patches, but these are > currently not merged and the project is on hold due to lack of time. You > can, however, use perf. --callgraph will be broken in Haskell code, > however (e.g. you will only see the first stack frame). > > Cheers, > > - Ben > > [1] https://ghc.haskell.org/trac/ghc/wiki/Exceptions/StackTraces > [2] https://ghc.haskell.org/trac/ghc/ticket/12096 > From ben at well-typed.com Mon Apr 10 22:26:06 2017 From: ben at well-typed.com (Ben Gamari) Date: Mon, 10 Apr 2017 18:26:06 -0400 Subject: GHC and linker choice In-Reply-To: <002c215c-a534-7c0f-2524-8f356c55e6a5@nh2.me> References: <002c215c-a534-7c0f-2524-8f356c55e6a5@nh2.me> Message-ID: <87r30zyk5d.fsf@ben-laptop.smart-cactus.org> Niklas Hambüchen writes: > I have some suggestions for low hanging fruits in this effort. > > 1. Make ghc print more statistics on what it spending time on > > When I did the linking investigation recently > (https://www.reddit.com/r/haskell/comments/63y43y/liked_linking_3x_faster_with_gold_link_10x_faster/) > I noticed (with strace) that there are lots of interesting syscalls > being made that you might not expect. For example, each time TH is used, > shared libraries are loaded, and to determine the shared library paths, > ghc shells out to `gcc --print-file-name`. Each such invocation takes 20 > ms on my system, and I have 1000 invocations in my build. That's 20 > seconds (out of 2 minutes build time) just asking gcc for paths. > > I recommend that for every call to an external GHC measures how long > that call took, so that it can be asked to print a summary when it's done. > > That might give us lots of interesting things to optimize. For example, > This would have made the long linker times totally obvious. > For what it's worth, some of us have recognized for quite some time that BFD ld is a known slow spot in the compilation pipeline. However, up until now we have considered this to be more of an issue to be handled at the packaging level than in GHC issue. Currently, GHC relies on `gcc` to link and gcc has its own idea of the default linker. Many users (e.g. Debian packagers, users who are cross compiling) are quite unhappy when GHC coerces the system toolchain into changing its default behavior. Consequently we have maintained the status quo and silently wished that Linux distributions would start moving to gold by default. Sadly there continues to be little progress on this matter. However, given that so many users are now relying on GHC binary distributions and that BFD ld is an increasingly significant issue as dependency counts increase, I think the status quo is increasingly untenable. I have proposed (#13541) that we introduce configure logic to use gold when available. There are outstanding questions however * What interface (i.e. configure flags) do we expose to the user to allow them to turn on or off this logic? * Do we want to enable the logic by default in binary distributions to ensure that users see the benefits of gold if it is available? * Do we want to also enable it in source distributions as well? If yes then developers may be caught by surprise; if no then there will be an inconsistency between the default configuration of the source tree and binary distributions (although there is plenty of precedent for this). Anyways, let me know if you have thoughts on any of this. Hopefully we'll be able to get this done for 8.4. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at smart-cactus.org Mon Apr 10 22:47:25 2017 From: ben at smart-cactus.org (Ben Gamari) Date: Mon, 10 Apr 2017 18:47:25 -0400 Subject: Where do I start if I would like help improve GHC compilation times? In-Reply-To: References: <87fuhk192d.fsf@ben-laptop.smart-cactus.org> Message-ID: <87mvbnyj5u.fsf@ben-laptop.smart-cactus.org> Alfredo Di Napoli writes: > Hey Ben, > Hi Alfredo, Sorry for the late response! The email queue from the weekend was a bit longer than I would like. > as promised I’m back to you with something more articulated and hopefully > meaningful. I do hear you perfectly — probably trying to dive head-first > into this without at least a rough understanding of the performance > hotspots or the GHC overall architecture is going to do me more harm than > good (I get the overall picture and I’m aware of the different stages of > the GHC compilation pipeline, but it’s far from saying I’m proficient with > the architecture as whole). I have also read a couple of years ago the GHC > chapter on the “Architeture of Open Source Applications” book, but I don’t > know how much that is still relevant. If it is, I guess I should refresh my > memory. > It sounds like you have done a good amount of reading. That's great. Perhaps skimming the AOSA chapter again wouldn't hurt, but otherwise it's likely worthwhile diving in. > I’m currently trying to move on 2 fronts — please advice if I’m a fool > flogging a dead horse or if I have any hope of getting anything done ;) > > 1. I’m trying to treat indeed the compiler as a black block (as you > adviced) trying to build a sufficiently large program where GHC is not “as > fast as I would like” (I know that’s a very lame definition of “slow”, > hehe). In particular, I have built the stage2 compiler with the “prof” > flavour as you suggested, and I have chosen 2 examples as a reference > “benchmark” for performance; DynFlags.hs (which seems to have been > mentioned multiple times as a GHC perf killer) and the highlighting-kate > package as posted here: https://ghc.haskell.org/trac/ghc/ticket/9221 . Indeed, #9221 would be a very interesting ticket to look at. The highlighting-kate package is interesting in the context of that ticket as it has a very large amount of parallelism available. If you do want to look at #9221, note that the cost centre profiler may not provide the whole story. In particular, it has been speculated that the scaling issues may be due to either, * threads hitting a blackhole, resulting in blocking * the usual scaling limitations of GHC's stop-the-world GC The eventlog may be quite useful for characterising these. > The idea would be to compile those with -v +RTS -p -hc -RTS enabled, > look at the output from the .prof file AND the `-v` flag, find any > hotspot, try to change something, recompile, observe diff, rinse and > repeat. Do you think I have any hope of making progress this way? In > particular, I think compiling DynFlags.hs is a bit of a dead-end; I > whipped up this buggy script which > escalated into a Behemoth which is compiling pretty much half of the > compiler once again :D > > ``` > #!/usr/bin/env bash > > ../ghc/inplace/bin/ghc-stage2 --make -j8 -v +RTS -A256M -qb0 -p -h \ > -RTS -DSTAGE=2 -I../ghc/includes -I../ghc/compiler -I../ghc/compiler/stage2 > \ > -I../ghc/compiler/stage2/build \ > -i../ghc/compiler/utils:../ghc/compiler/types:../ghc/compiler/typecheck:../ghc/compiler/basicTypes > \ > -i../ghc/compiler/main:../ghc/compiler/profiling:../ghc/compiler/coreSyn:../ghc/compiler/iface:../ghc/compiler/prelude > \ > -i../ghc/compiler/stage2/build:../ghc/compiler/simplStg:../ghc/compiler/cmm:../ghc/compiler/parser:../ghc/compiler/hsSyn > \ > -i../ghc/compiler/ghci:../ghc/compiler/deSugar:../ghc/compiler/simplCore:../ghc/compile/specialise > \ > -fforce-recomp -c $@ > ``` > > I’m running it with `./dynflags.sh ../ghc/compiler/main/DynFlags.hs` but > it’s taking a lot to compile (20+ mins on my 2014 mac Pro) because it’s > pulling in half of the compiler anyway :D I tried to reuse the .hi files > from my stage2 compilation but I failed (GHC was complaining about > interface file mismatch). Short story short, I don’t think it will be a > very agile way to proceed. Am I right? Do you have any recommendation in > such sense? Do I have any hope to compile DynFlags.hs in a way which would > make this perf investigation feasible? > What I usually do in this case is just take the relevant `ghc` command line directly from the `make` output and execute it manually. I would imagine your debug cycle would look something like, * instrument the compiler * build stage1 * use stage2 to build DynFlags using the stage1 compiler (using a saved command line) * think * repeat This should only take a few minutes per iteration. > The second example (the highlighting-kate package) seems much more > promising. It takes maybe 1-2 mins on my machine, which is enough to take a > look at the perf output. Do you think I should follow this second lead? In > principle any 50+ modules package I think would do (better if with a lot of > TH ;) ) but this seems like a low-entry barrier start. > > 2. The second path I’m exploring is simply to take a less holistic approach > and try to dive in into a performance ticket like the ones listed here: > https://www.reddit.com/r/haskell/comments/45q90s/is_anything_being_done_to_remedy_the_soul/czzq6an/ > Maybe some are very specific, but it seems like fixing small things and > move forward could help giving me understanding of different sub-parts of > GHC, which seems less intimidating than the black-box approach. > Do you have any specific tickets from these lists that you found interesting? > In conclusion, what do you think is the best approach, 1 or 2, both or > none? ;) I would say that it largely depends upon what you feel most comfortable with. If you feel up for it, I think #9221 would be a nice, fairly self-contained, yet high-impact ticket which would be worth spending a few days diving further into. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From simonpj at microsoft.com Tue Apr 11 15:53:27 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Tue, 11 Apr 2017 15:53:27 +0000 Subject: submodules Message-ID: Devs I want to build a GHC from someone else repo; this one actually git at github.com:Tritlo/ghc.git. But when I clone it, and then do git submodule init; git submodule update, I get lots of git submodule update Cloning into '.arc-linters/arcanist-external-json-linter'... ERROR: Repository not found. fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists. Clone of 'git at github.com:Tritlo/arcanist-external-json-linter.git' into submodule path '.arc-linters/arcanist-external-json-linter' failed simonpj at cam-05-unx:~/code/ghc-holes$ What is the kosher way to do this? Thanks Simon -------------- next part -------------- An HTML attachment was scrubbed... URL: From rwbarton at gmail.com Tue Apr 11 16:01:23 2017 From: rwbarton at gmail.com (Reid Barton) Date: Tue, 11 Apr 2017 12:01:23 -0400 Subject: submodules In-Reply-To: References: Message-ID: Hi Simon, This happens because the locations of the submodules are specified using relative paths from the main GHC repository, but Tritlo has only made a fork of the main GHC repo, not all the submodules. I would do this: * Clone from the main GHC repo (including submodules) however you usually do it (e.g., git clone --recursive https://git.haskell.org/ghc.git) * Add Tritlo's ghc repo as a remote: git remote add tritlo git at github.com:Tritlo/ghc.git * Fetch from the new remote: git fetch tritlo * Check out the branch you want: git checkout tritlo/ Here "tritlo" is just a name for the remote within your local ghc checkout, so it can be anything you choose. Regards, Reid Barton On Tue, Apr 11, 2017 at 11:53 AM, Simon Peyton Jones via ghc-devs wrote: > Devs > > I want to build a GHC from someone else repo; this one actually > git at github.com:Tritlo/ghc.git. > > But when I clone it, and then do git submodule init; git submodule update, I > get lots of > > git submodule update > > Cloning into '.arc-linters/arcanist-external-json-linter'... > > ERROR: Repository not found. > > fatal: Could not read from remote repository. > > > > Please make sure you have the correct access rights > > and the repository exists. > > Clone of 'git at github.com:Tritlo/arcanist-external-json-linter.git' into > submodule path '.arc-linters/arcanist-external-json-linter' failed > > simonpj at cam-05-unx:~/code/ghc-holes$ > > What is the kosher way to do this? > > Thanks > > Simon > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > From simonpj at microsoft.com Tue Apr 11 16:29:15 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Tue, 11 Apr 2017 16:29:15 +0000 Subject: submodules In-Reply-To: References: Message-ID: Thanks Reid, Matthew. S | -----Original Message----- | From: Reid Barton [mailto:rwbarton at gmail.com] | Sent: 11 April 2017 17:01 | To: Simon Peyton Jones | Cc: ghc-devs | Subject: Re: submodules | | Hi Simon, | | This happens because the locations of the submodules are specified | using relative paths from the main GHC repository, but Tritlo has only | made a fork of the main GHC repo, not all the submodules. I would do | this: | | * Clone from the main GHC repo (including submodules) however you | usually do it (e.g., git clone --recursive | https://git.haskell.org/ghc.git) | * Add Tritlo's ghc repo as a remote: git remote add tritlo | git at github.com:Tritlo/ghc.git | * Fetch from the new remote: git fetch tritlo | * Check out the branch you want: git checkout tritlo/ | | Here "tritlo" is just a name for the remote within your local ghc | checkout, so it can be anything you choose. | | Regards, | Reid Barton | | On Tue, Apr 11, 2017 at 11:53 AM, Simon Peyton Jones via ghc-devs | wrote: | > Devs | > | > I want to build a GHC from someone else repo; this one actually | > git at github.com:Tritlo/ghc.git. | > | > But when I clone it, and then do git submodule init; git submodule | > update, I get lots of | > | > git submodule update | > | > Cloning into '.arc-linters/arcanist-external-json-linter'... | > | > ERROR: Repository not found. | > | > fatal: Could not read from remote repository. | > | > | > | > Please make sure you have the correct access rights | > | > and the repository exists. | > | > Clone of 'git at github.com:Tritlo/arcanist-external-json-linter.git' | > into submodule path '.arc-linters/arcanist-external-json-linter' | > failed | > | > simonpj at cam-05-unx:~/code/ghc-holes$ | > | > What is the kosher way to do this? | > | > Thanks | > | > Simon | > | > | > _______________________________________________ | > ghc-devs mailing list | > ghc-devs at haskell.org | > | https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.h | > askell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc- | devs&data=02%7C01%7Csimonpj%40microsoft.com%7C630b51f1147e4a4094ea08d4 | 80f40167%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6362752328637058 | 80&sdata=tw7pl0q6yMwSGGA2ggq5ZOI7H%2B%2FGVW2AozV7eObsyMU%3D&reserved=0 | > From ben at well-typed.com Tue Apr 11 20:08:01 2017 From: ben at well-typed.com (Ben Gamari) Date: Tue, 11 Apr 2017 16:08:01 -0400 Subject: [ANNOUNCE] GHC 8.2.1 release candidate 1 In-Reply-To: References: <87lgr8zztf.fsf@ben-laptop.smart-cactus.org> <87fuhgysd8.fsf@ben-laptop.smart-cactus.org> <878tn8yogt.fsf@ben-laptop.smart-cactus.org> Message-ID: <87efwyyafy.fsf@ben-laptop.smart-cactus.org> "Boespflug, Mathieu" writes: > Hi Ben, > > it sounds like some of the remaining limitations around DWARF support > (e.g. finishing the stack sampling work, local bindings in GDB, ...) > could make for a good Haskell Summer of Code project. Have you > considered writing this up as one or two project ideas here: > https://summer.haskell.org/ideas.html? As HSoC sponsors, we'd be more > than happy to see someone pick this up. > I haven't until now. To be honest I think the local-bindings-in-GDB issue is a hard problem and is likely well beyond a summer project. You would need not just the ability track the in-scope bindings in every context of the source program, but also the ability to reconstitute their values after simplification. It's also not clear to me how useful this feature would be given how much simplified code often differs from the code the user writes. However, I do see there might be room for a project on the statistical profiler itself or its associated tooling. We just need to come to a conclusion on which direction is most appropriate for GHC. For this having some concrete use-cases would be quite helpful. How do you envision using statistical profiling on Haskell projects? What is the minimal set of features that would make for a useful profiler? Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From m at tweag.io Tue Apr 11 21:39:57 2017 From: m at tweag.io (Boespflug, Mathieu) Date: Tue, 11 Apr 2017 23:39:57 +0200 Subject: [ANNOUNCE] GHC 8.2.1 release candidate 1 In-Reply-To: <87efwyyafy.fsf@ben-laptop.smart-cactus.org> References: <87lgr8zztf.fsf@ben-laptop.smart-cactus.org> <87fuhgysd8.fsf@ben-laptop.smart-cactus.org> <878tn8yogt.fsf@ben-laptop.smart-cactus.org> <87efwyyafy.fsf@ben-laptop.smart-cactus.org> Message-ID: > However, I do see there might be room for a project on the statistical > profiler itself or its associated tooling. We just need to come to a > conclusion on which direction is most appropriate for GHC. You mean - given a choice between somehow reusing perf or going down the route of fully custom tooling? > For this having some concrete use-cases would be quite helpful. How do > you envision using statistical profiling on Haskell projects? What is > the minimal set of features that would make for a useful profiler? That sounds like a good way to approach this. Here goes... I'd really prefer seeing a Haskell program as a black box, that I can profile using the same tools as C programs or native code generated from any other language. It shouldn't matter that the source is Haskell. In my ideal workflow, I have a *vanilla* Haskell program compiled with debug symbols by a *vanilla* GHC (no special ./configure options as prereqs), that I can hook up perf to, e.g. $ perf record -g ./mybinary Then I should be able to use perf report to analyze the results. Or indeed use existing pipelines to obtain other visualizations (flame graphs etc). I'm not particularly interested in integration with the event log, though others might have a need for that. I'm also interested in hotspot analysis, à la perf annotate. As Brendan Gregg says, "perf isn't some random tool: it's part of the Linux kernel, and is actively developed and enhanced." I need accurate and informative stack samples (no STG internal details in the output that I can't connect back to source locations) for programs that include all manner of FFI calls. Better still if time spent in the GC doesn't pollute my stack samples. The tricky part is that for flame graphs you need to sample stacks, and for that you need to teach perf how to collect that data somehow, since the C stack isn't used for haskell activation frames and we have a call-by-need evaluation strategy anyways. But the slow option you mention in the status page sounds okayish to me, and using eBPF to perform stack sampling entirely from the kernel looks like a promising direction. From simonpj at microsoft.com Wed Apr 12 15:25:50 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Wed, 12 Apr 2017 15:25:50 +0000 Subject: Join points and loopificaiton Message-ID: Luke Kavon Farvardin is here as an intern, mainly working on LLVM backend stuff. But he knows a lot about CPS etc and he'd read our paper. In talking about it, we realised a new Join Point Win! I've written it up in https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler/Loopification under "New idea: use join points". Let's do it! Ticket here: https://ghc.haskell.org/trac/ghc/ticket/13567 Simon -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjakway at nyu.edu Wed Apr 12 15:30:18 2017 From: tjakway at nyu.edu (Thomas Jakway) Date: Wed, 12 Apr 2017 11:30:18 -0400 Subject: Join points and loopificaiton In-Reply-To: <5af0aec6-9e80-eb6f-02e2-55ab7b7cc8b8@nyu.edu> References: <5af0aec6-9e80-eb6f-02e2-55ab7b7cc8b8@nyu.edu> Message-ID: <2b5c59bd-7b56-503c-76b4-acaab524463d@nyu.edu> woops, meant to reply list On 04/12/2017 11:29 AM, Thomas Jakway wrote: > > This would go great with https://ghc.haskell.org/trac/ghc/ticket/13051 > (which I have not abandoned by the way!) > > > On 04/12/2017 11:25 AM, Simon Peyton Jones via ghc-devs wrote: >> >> Luke >> >> Kavon Farvardin is here as an intern, mainly working on LLVM backend >> stuff. But he knows a lot about CPS etc and he'd read our paper. In >> talking about it, we realised a new Join Point Win! >> >> I’ve written it up in >> >> https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler/Loopification >> >> under “New idea: use join points”. Let’s do it! >> >> Ticket here: https://ghc.haskell.org/trac/ghc/ticket/13567 >> >> Simon >> >> >> >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Wed Apr 12 17:14:12 2017 From: ben at well-typed.com (Ben Gamari) Date: Wed, 12 Apr 2017 13:14:12 -0400 Subject: [ANNOUNCE] GHC 8.2.1 release candidate 1 In-Reply-To: References: <87lgr8zztf.fsf@ben-laptop.smart-cactus.org> <87fuhgysd8.fsf@ben-laptop.smart-cactus.org> <878tn8yogt.fsf@ben-laptop.smart-cactus.org> <87efwyyafy.fsf@ben-laptop.smart-cactus.org> Message-ID: <878tn5y2e3.fsf@ben-laptop.smart-cactus.org> "Boespflug, Mathieu" writes: >> However, I do see there might be room for a project on the statistical >> profiler itself or its associated tooling. We just need to come to a >> conclusion on which direction is most appropriate for GHC. > > You mean - given a choice between somehow reusing perf or going down > the route of fully custom tooling? > Right. >> For this having some concrete use-cases would be quite helpful. How do >> you envision using statistical profiling on Haskell projects? What is >> the minimal set of features that would make for a useful profiler? > > That sounds like a good way to approach this. Here goes... > > I'd really prefer seeing a Haskell program as a black box, that I can > profile using the same tools as C programs or native code generated > from any other language. It shouldn't matter that the source is > Haskell. In my ideal workflow, I have a *vanilla* Haskell program > compiled with debug symbols by a *vanilla* GHC (no special ./configure > options as prereqs), that I can hook up perf to, e.g. > > $ perf record -g ./mybinary > > Then I should be able to use perf report to analyze the results. Or > indeed use existing pipelines to obtain other visualizations (flame > graphs etc). > > I'm not particularly interested in integration with the event log, > though others might have a need for that. > > I'm also interested in hotspot analysis, à la perf annotate. > > As Brendan Gregg says, "perf isn't some random tool: it's part of the > Linux kernel, and is actively developed and enhanced." > > I need accurate and informative stack samples (no STG internal details > in the output that I can't connect back to source locations) for > programs that include all manner of FFI calls. Better still if time > spent in the GC doesn't pollute my stack samples. > > The tricky part is that for flame graphs you need to sample stacks, > and for that you need to teach perf how to collect that data somehow, > since the C stack isn't used for haskell activation frames and we have > a call-by-need evaluation strategy anyways. But the slow option you > mention in the status page sounds okayish to me, and using eBPF to > perform stack sampling entirely from the kernel looks like a promising > direction. Indeed that is the engineering side of the trickiness. However, there is also a theoretically difficult aspect which Peter articulates nicely in Chapter 2 of his thesis. I refer you to the thesis for the full explanation but, in short, reasoning about causality in lazy languages is quite difficult (which is why this whole endeavour was worthy of a thesis). This leads to some amount of impedance mismatch with existing tooling, which is the reason I started down the road of a Haskell-centric solution. To elaborate, while in an imperative language it is relatively easy to say "instruction at address $A arose from line $N in $FILE.c", this sort of unambiguous statement is not possible in Haskell. Instead, we have to be satisfied with *sets* of source locations. That is, "instruction at address $A arose from line $N in $FILE, and line $N' in $FILE', and ...". Unfortunately, essentially none of the existing profiling and debugging infrastructure (DWARF and perf included) was designed with this model in mind. In particular, DWARF's line information encoding requires that GHC choose a single location to attribute the cost of each instruction to. This is, as you might expect, a rather tricky choice to make and while GHC has a few heuristics, we will inevitably be wrong in some circumstances. From experience, I can say that attempting to analyse profiles naively, using only GHC's heuristically-guided guess of the appropriate source location, can lead to rather perplexing results. For this reason, GHC uses [1] DWARF's extensibility mechanisms to export an additional set of line information which can be consumed by Haskell-aware tooling which captures the full richness of GHC's source ticks. In section 5.8 of his thesis Peter proposes a scheme of "fuzzing" for making use of this location information. I haven't tried Peter's fuzzing approach, but I suspect it will make the profiler output significantly easier to reason about. All this being said, I totally agree that being able to use widely-known, well-maintained native tools is a significant advantage. This is a large part of why I put the profiling project on hold. I have wondered whether we might be able to provide a preprocessor which would take sample data from GHC's statistical profiler and export it to perf's own format, performing fuzzing and any other necessary Haskell-specific preprocessing. This would at least allow us to tap in to the ecosystem that has arisen around perf. Alternatively, we could try to contribute patches to perf's upstream, although I'm not sure how likely acceptance would be. Cheers, - Ben [1] https://phabricator.haskell.org/D1279#R?query=D1279 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From mail at nh2.me Wed Apr 12 22:46:03 2017 From: mail at nh2.me (=?UTF-8?Q?Niklas_Hamb=c3=bcchen?=) Date: Thu, 13 Apr 2017 00:46:03 +0200 Subject: GHC and linker choice In-Reply-To: <87r30zyk5d.fsf@ben-laptop.smart-cactus.org> References: <002c215c-a534-7c0f-2524-8f356c55e6a5@nh2.me> <87r30zyk5d.fsf@ben-laptop.smart-cactus.org> Message-ID: <72665bc1-5fc3-d33b-15cc-ef6ace13a7ce@nh2.me> Hi Ben, > For what it's worth, some of us have recognized for quite some time that > BFD ld is a known slow spot in the compilation pipeline. However, up > until now we have considered this to be more of an issue to be handled > at the packaging level than in GHC issue. Currently, GHC relies on `gcc` > to link and gcc has its own idea of the default linker. Many users (e.g. > Debian packagers, users who are cross compiling) are quite unhappy when > GHC coerces the system toolchain into changing its default behavior. > > Consequently we have maintained the status quo and silently wished that > Linux distributions would start moving to gold by default. Sadly there > continues to be little progress on this matter. This is a great point. I don't know if using BFD ld vs gold is a controversial move, given that they are both GNU projects. I would imagine if lld was in the game, that might be more controversial. I have the theory that both Haskell packagers and Linux distributions are simply conservative about changing the linker not because they don't want it, but because they are scared of breaking people's code, and especially in the packagers' case, that they don't feel expert enough to judge whether them making this move will be safe. I can imagine that they would rather welcome if the GHC devs made this decision, since after all they generate the linker invocations and have the knowledge to judge whether they are supposed to work. If a packager wants to *override* GHC's default behaviour, that should certainly be possible, but as a packager I would find it unfair if GHC pushed the burden of setting up the optimal defaults for my users on me. For a comparison with other programming languages: From tickets like https://github.com/golang/go/issues/1588 seems that go also uses gold by default when available. Rust switched to gold by default with https://github.com/rust-lang/rust/pull/29974, but reverted the commit with https://github.com/rust-lang/rust/pull/30913, with reasons being the linked tickets. The Rust tickets in particular have already discussed the various issues we may be running into regarding autodetection, cross compilation and so on, it may be good to get inspiration from there. > * Do we want to also enable it in source distributions as well? If yes > then developers may be caught by surprise I would recommend to do the same in source and binary distributions. The surprise should be minimised by properly advertising it as a key thing in the release notes. Packagers are supposed to read those, and in fact and most users of GHC love to read them for they contain many goodies. Another option is to not change the default or enable autodetection, but make it trivial to use gold for the user, e.g. by at least fixing the Cabal library (currently you need to put your linker options into a lot of places to get the desired results, see https://github.com/haskell/cabal/issues/4435, and even if you do, for some invocations Cabal just doesn't pass on what you tell it to do, see https://github.com/haskell/cabal/issues/4439). We could run this for a GHC release term, and also have cabal-install and stack recommend people that they should try to set some easy option to use the gold linker if available. If that works well, we could switch to autodetection the release after that. I haven't though much about whether this is actually a better idea than just enabling it where we can. Niklas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: OpenPGP digital signature URL: From carter.schonwald at gmail.com Thu Apr 13 02:37:57 2017 From: carter.schonwald at gmail.com (Carter Schonwald) Date: Thu, 13 Apr 2017 02:37:57 +0000 Subject: Join points and loopificaiton In-Reply-To: References: Message-ID: Ooo. I like this design idea. On Wed, Apr 12, 2017 at 11:26 AM Simon Peyton Jones via ghc-devs < ghc-devs at haskell.org> wrote: > Luke > > Kavon Farvardin is here as an intern, mainly working on LLVM backend > stuff. But he knows a lot about CPS etc and he'd read our paper. In > talking about it, we realised a new Join Point Win! > > I’ve written it up in > > https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler/Loopification > > under “New idea: use join points”. Let’s do it! > > Ticket here: https://ghc.haskell.org/trac/ghc/ticket/13567 > > Simon > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alfredo.dinapoli at gmail.com Thu Apr 13 13:28:54 2017 From: alfredo.dinapoli at gmail.com (Alfredo Di Napoli) Date: Thu, 13 Apr 2017 15:28:54 +0200 Subject: =?UTF-8?Q?ghc=2Dstage2_=E2=80=94interactive_segfaults_on_Mac_OS_X_10=2E1?= =?UTF-8?Q?1=2E6_=28build_flavour_=3D_prof=29?= Message-ID: Hey all, I’m trying to compile GHC HEAD (cloning from master) with the `prof` build flavour on a Mac OS X 10.11.6 machine and I have noticed that, despite ghc-stage2 works as expected, when invoked with —interactive it starts before crashing with a segmentation fault: ``` ☁ compiler [master] ⚡ ../inplace/bin/ghc-stage2 --interactive GHCi, version 8.3.20170413: http://www.haskell.org/ghc/ :? for help [1] 79176 segmentation fault ../inplace/bin/ghc-stage2 --interactive ``` Did it happen to somebody else or it’s just me? Shall I try throwing gdb at it to try and see what’s going on? Thanks, Alfredo -------------- next part -------------- An HTML attachment was scrubbed... URL: From mail at joachim-breitner.de Thu Apr 13 15:24:05 2017 From: mail at joachim-breitner.de (Joachim Breitner) Date: Thu, 13 Apr 2017 11:24:05 -0400 Subject: [Diffusion] [Raised Concern] rGHC751996e90a96: Kill off complications in CoreFVs In-Reply-To: References: <20170413033033.23405.80723.728D54DE@phabricator.haskell.org> Message-ID: <1492097045.1059.2.camel@joachim-breitner.de> [CC’ing ghc-dev in case others are also curious about how to navigate Gipeda.] Hi, Am Donnerstag, den 13.04.2017, 08:40 +0000 schrieb Simon Peyton Jones: > Thanks for noticing this!  I will investigate. >   > But where do you see this? > ·         at perf.haskell.org I see “GHC” and “Gipeda”.  It’s not > obvious which I want. You want the GHC Dashboard, as you guessed correctly. > ·         Clicking GHC shows me some “recent commits”.  The first > three are from a few hrs ago; then the next is 8 days ago. Can’t be > right! It only shows interesting commits. Interesting ones are the most recent ones and all with significant changes. If you want to see all, press the “=” button in the top-right corner. > ·         There is no key to tell me what the little symbols mean – I > may have known once but I don’t know.  A key would be terrific, or > even a link to a key. The symbols next to the three numbers on the right in the table? They are: * number of benchmarks * number of benchmarks improved * number of benchmarks regressed Maybe I should remove the first one, it is not very useful Hovering these icons show you which benchmarks changed, and by how much. > ·         Clicking on the offending patch (which does show), I see > the n-body complaint, which is good.  But there is also > “testsuite/unexpected stats, which is mysterious… clicking on it > yields no more info. There is a “view buildlog” link that I use to investigate such cases. It goes to https://raw.githubusercontent.com/nomeata/ghc-speed-logs/master/751996e90a964026a3f86853338f10c82db6b610.log which shows bytes allocated value is too low: (If this is because you have improved GHC, please update the test so that GHC doesn't regress again)     Expected    T13056(optasm) bytes allocated: 440548592 +/-5%     Lower bound T13056(optasm) bytes allocated: 418521162      Upper bound T13056(optasm) bytes allocated: 462576022      Actual      T13056(optasm) bytes allocated: 417666128      Deviation   T13056(optasm) bytes allocated:      -5.2 % *** unexpected stat test failure for T13056(optasm) Probably some creep, given how close it is to cut-off percentage. According to https://perf.haskell.org/ghc/#graph/tests/alloc/T13056 this has been stable in the last 50 commits (sorry, still no way to view these graph for any other time interval). > ·         If compile times went up, would that show up anywhere > (except in testsuite results)? Only if it changes the total time of running make, or if it affects the allocations of a compiler perf test case. We currently have no reliable compilation time benchmarks (which would require _compiling_ stuff NofibRun times). Greetings, Joachim -- Joachim “nomeata” Breitner   mail at joachim-breitner.de • https://www.joachim-breitner.de/   XMPP: nomeata at joachim-breitner.de • OpenPGP-Key: 0xF0FBF51F   Debian Developer: nomeata at debian.org -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From ben at smart-cactus.org Thu Apr 13 17:01:52 2017 From: ben at smart-cactus.org (Ben Gamari) Date: Thu, 13 Apr 2017 13:01:52 -0400 Subject: ghc-stage2 =?utf-8?Q?=E2=80=94interactive?= segfaults on Mac OS X 10.11.6 (build flavour = prof) In-Reply-To: References: Message-ID: <87r30wnsvz.fsf@ben-laptop.smart-cactus.org> Alfredo Di Napoli writes: > Hey all, > > I’m trying to compile GHC HEAD (cloning from master) with the `prof` build > flavour on a Mac OS X 10.11.6 machine and I have noticed that, despite > ghc-stage2 works as expected, when invoked with —interactive it starts > before crashing with a segmentation fault: > > ``` > ☁ compiler [master] ⚡ ../inplace/bin/ghc-stage2 --interactive > GHCi, version 8.3.20170413: http://www.haskell.org/ghc/ :? for help > [1] 79176 segmentation fault ../inplace/bin/ghc-stage2 --interactive > ``` > > Did it happen to somebody else or it’s just me? Shall I try throwing gdb at > it to try and see what’s going on? Hmm, interesting. I've not seen crashes like this locally nor in CI. It would be great if you could try to get some insight. Is this crash perfectly reproducible? It may be worth adding -dcore-lint to GhcStage2HcOpts to ensure the code we are producing is sane. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at well-typed.com Fri Apr 14 13:52:52 2017 From: ben at well-typed.com (Ben Gamari) Date: Fri, 14 Apr 2017 09:52:52 -0400 Subject: downloads.haskell.org unavailable for a bit Message-ID: <87h91rhz9n.fsf@ben-laptop.smart-cactus.org> Hello all, Unfortunately, due to a botched rsync, the tarballs on downloads.haskell.org will be unavailable for a few hours. I'm currently working on restoring the 8.0.2 tarballs first to ensure that they remain available to users. I will follow up with status updates as they come. Many apologies for the inconvenience. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at well-typed.com Fri Apr 14 14:37:00 2017 From: ben at well-typed.com (Ben Gamari) Date: Fri, 14 Apr 2017 10:37:00 -0400 Subject: downloads.haskell.org unavailable for a bit In-Reply-To: <87h91rhz9n.fsf@ben-laptop.smart-cactus.org> References: <87h91rhz9n.fsf@ben-laptop.smart-cactus.org> Message-ID: <87efwvhx83.fsf@ben-laptop.smart-cactus.org> Ben Gamari writes: > Hello all, > > Unfortunately, due to a botched rsync, the tarballs on > downloads.haskell.org will be unavailable for a few hours. I'm currently > working on restoring the 8.0.2 tarballs first to ensure that they remain > available to users. I will follow up with status updates as they come. > Many apologies for the inconvenience. > The 8.0.2 tarballs are now again available. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From alfredo.dinapoli at gmail.com Fri Apr 14 19:19:27 2017 From: alfredo.dinapoli at gmail.com (Alfredo Di Napoli) Date: Fri, 14 Apr 2017 21:19:27 +0200 Subject: =?UTF-8?Q?Re=3A_ghc=2Dstage2_=E2=80=94interactive_segfaults_on_Mac_OS_X_?= =?UTF-8?Q?10=2E11=2E6_=28build_flavour_=3D_prof=29?= In-Reply-To: <87r30wnsvz.fsf@ben-laptop.smart-cactus.org> References: <87r30wnsvz.fsf@ben-laptop.smart-cactus.org> Message-ID: Hey Ben, yes, it’s consistently reproducible. I have tried compiling GHC from scratch by adding `-dcore-lint` and `-debug` to GhcStage2HcOpts in my mk/ build.mk, but eventually the build process failed with: ld: library not found for -lHSrts_thr_debug_p Any idea what am I doing wrong? Next I’m going to try enabling `-DDEBUG` only as described here: https://ghc.haskell.org/trac/ghc/wiki/Debugging/Compiler To see if I get any further. Thanks! A. On 13 April 2017 at 19:01, Ben Gamari wrote: > Alfredo Di Napoli writes: > > > Hey all, > > > > I’m trying to compile GHC HEAD (cloning from master) with the `prof` > build > > flavour on a Mac OS X 10.11.6 machine and I have noticed that, despite > > ghc-stage2 works as expected, when invoked with —interactive it starts > > before crashing with a segmentation fault: > > > > ``` > > ☁ compiler [master] ⚡ ../inplace/bin/ghc-stage2 --interactive > > GHCi, version 8.3.20170413: http://www.haskell.org/ghc/ :? for help > > [1] 79176 segmentation fault ../inplace/bin/ghc-stage2 --interactive > > ``` > > > > Did it happen to somebody else or it’s just me? Shall I try throwing gdb > at > > it to try and see what’s going on? > > Hmm, interesting. I've not seen crashes like this locally nor in CI. It > would be great if you could try to get some insight. Is this crash > perfectly reproducible? > > It may be worth adding -dcore-lint to GhcStage2HcOpts to ensure the code > we are producing is sane. > > Cheers, > > - Ben > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alfredo.dinapoli at gmail.com Fri Apr 14 19:33:21 2017 From: alfredo.dinapoli at gmail.com (Alfredo Di Napoli) Date: Fri, 14 Apr 2017 21:33:21 +0200 Subject: =?UTF-8?Q?Re=3A_ghc=2Dstage2_=E2=80=94interactive_segfaults_on_Mac_OS_X_?= =?UTF-8?Q?10=2E11=2E6_=28build_flavour_=3D_prof=29?= In-Reply-To: References: <87r30wnsvz.fsf@ben-laptop.smart-cactus.org> Message-ID: Ok, I had success by removing “-debug” in favour of “-DDEBUG”. After compiling GHC I fired GDB and this is the output: Starting program: /Users/adinapoli/programming/haskell/ghc/inplace/lib/bin/ghc-stage2 -B/Users/adinapoli/programming/haskell/ghc/inplace/lib --interactive GHCi, version 8.3.20170413: http://www.haskell.org/ghc/ :? for help [New Thread 0x120f of process 19786] [New Thread 0x1403 of process 19786] [New Thread 0x1503 of process 19786] [New Thread 0x1603 of process 19786] Thread 1 received signal SIGSEGV, Segmentation fault. 0x0000000104cdd81a in ocInit_MachO () at rts/linker/MachO.c:141 141 if(NULL != oc->info->nlist) { Maybe it does ring a bell to any of you. In case not, I’m happy to continue digging. A. On 14 April 2017 at 21:19, Alfredo Di Napoli wrote: > Hey Ben, > > yes, it’s consistently reproducible. I have tried compiling GHC from > scratch by adding `-dcore-lint` and `-debug` to GhcStage2HcOpts in my mk/ > build.mk, but eventually the build process failed with: > > ld: library not found for -lHSrts_thr_debug_p > > Any idea what am I doing wrong? Next I’m going to try enabling `-DDEBUG` > only as described here: > > https://ghc.haskell.org/trac/ghc/wiki/Debugging/Compiler > > To see if I get any further. > > Thanks! > > A. > > > On 13 April 2017 at 19:01, Ben Gamari wrote: > >> Alfredo Di Napoli writes: >> >> > Hey all, >> > >> > I’m trying to compile GHC HEAD (cloning from master) with the `prof` >> build >> > flavour on a Mac OS X 10.11.6 machine and I have noticed that, despite >> > ghc-stage2 works as expected, when invoked with —interactive it starts >> > before crashing with a segmentation fault: >> > >> > ``` >> > ☁ compiler [master] ⚡ ../inplace/bin/ghc-stage2 --interactive >> > GHCi, version 8.3.20170413: http://www.haskell.org/ghc/ :? for help >> > [1] 79176 segmentation fault ../inplace/bin/ghc-stage2 --interactive >> > ``` >> > >> > Did it happen to somebody else or it’s just me? Shall I try throwing >> gdb at >> > it to try and see what’s going on? >> >> Hmm, interesting. I've not seen crashes like this locally nor in CI. It >> would be great if you could try to get some insight. Is this crash >> perfectly reproducible? >> >> It may be worth adding -dcore-lint to GhcStage2HcOpts to ensure the code >> we are producing is sane. >> >> Cheers, >> >> - Ben >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at smart-cactus.org Fri Apr 14 23:27:25 2017 From: ben at smart-cactus.org (Ben Gamari) Date: Fri, 14 Apr 2017 19:27:25 -0400 Subject: =?UTF-8?Q?Re=3A_ghc-stage2_=E2=80=94interactive_segfaults_on?= =?UTF-8?Q?_Mac_OS_X_10=2E11=2E6_=28build_flavour_=3D_prof=29?= In-Reply-To: References: <87r30wnsvz.fsf@ben-laptop.smart-cactus.org> Message-ID: <6A638216-C2D6-45B7-A207-87048371F4A9@smart-cactus.org> On April 14, 2017 3:33:21 PM EDT, Alfredo Di Napoli wrote: >Ok, I had success by removing “-debug” in favour of “-DDEBUG”. After >compiling GHC I fired GDB and this is the output: > >Starting program: >/Users/adinapoli/programming/haskell/ghc/inplace/lib/bin/ghc-stage2 >-B/Users/adinapoli/programming/haskell/ghc/inplace/lib --interactive >GHCi, version 8.3.20170413: http://www.haskell.org/ghc/ :? for help >[New Thread 0x120f of process 19786] >[New Thread 0x1403 of process 19786] >[New Thread 0x1503 of process 19786] >[New Thread 0x1603 of process 19786] > >Thread 1 received signal SIGSEGV, Segmentation fault. >0x0000000104cdd81a in ocInit_MachO () at rts/linker/MachO.c:141 >141 if(NULL != oc->info->nlist) { > >Maybe it does ring a bell to any of you. In case not, I’m happy to >continue >digging. > It doesn't ring a bell and I'm not near a computer at the moment, but it sounds like you are hot on the trail; that is a far more tractable error. I'll have a look at the code to see if I can see any obvious things to check when I get back home. From moritz.angermann at gmail.com Sat Apr 15 00:05:08 2017 From: moritz.angermann at gmail.com (Moritz Angermann) Date: Sat, 15 Apr 2017 08:05:08 +0800 Subject: =?utf-8?Q?Re:_ghc-stage2_=E2=80=94interactive_segfaults_on_Mac_O?= =?utf-8?Q?S_X_10.11.6_=28build_flavour_=3D_prof=29?= In-Reply-To: References: <87r30wnsvz.fsf@ben-laptop.smart-cactus.org> Message-ID: Hi, that looks pretty much like I broke that :-( Do you have a backtrace? The info structs should be filled properly by ocInit_MachO when loading object code. The nlist would point to the symbol names list. However I believe that either oc is already NULL, in which case you should have never reached that point, or info is NULL, in which case ocInit seems to not have been called. I'll try to reproduce this today. Cheers, Moritz Sent from my iPhone > On 15 Apr 2017, at 3:33 AM, Alfredo Di Napoli wrote: > > Ok, I had success by removing “-debug” in favour of “-DDEBUG”. After compiling GHC I fired GDB and this is the output: > > Starting program: /Users/adinapoli/programming/haskell/ghc/inplace/lib/bin/ghc-stage2 -B/Users/adinapoli/programming/haskell/ghc/inplace/lib --interactive > GHCi, version 8.3.20170413: http://www.haskell.org/ghc/ :? for help > [New Thread 0x120f of process 19786] > [New Thread 0x1403 of process 19786] > [New Thread 0x1503 of process 19786] > [New Thread 0x1603 of process 19786] > > Thread 1 received signal SIGSEGV, Segmentation fault. > 0x0000000104cdd81a in ocInit_MachO () at rts/linker/MachO.c:141 > 141 if(NULL != oc->info->nlist) { > > Maybe it does ring a bell to any of you. In case not, I’m happy to continue digging. > > A. > >> On 14 April 2017 at 21:19, Alfredo Di Napoli wrote: >> Hey Ben, >> >> yes, it’s consistently reproducible. I have tried compiling GHC from scratch by adding `-dcore-lint` and `-debug` to GhcStage2HcOpts in my mk/build.mk, but eventually the build process failed with: >> >> ld: library not found for -lHSrts_thr_debug_p >> >> Any idea what am I doing wrong? Next I’m going to try enabling `-DDEBUG` only as described here: >> >> https://ghc.haskell.org/trac/ghc/wiki/Debugging/Compiler >> >> To see if I get any further. >> >> Thanks! >> >> A. >> >> >>> On 13 April 2017 at 19:01, Ben Gamari wrote: >>> Alfredo Di Napoli writes: >>> >>> > Hey all, >>> > >>> > I’m trying to compile GHC HEAD (cloning from master) with the `prof` build >>> > flavour on a Mac OS X 10.11.6 machine and I have noticed that, despite >>> > ghc-stage2 works as expected, when invoked with —interactive it starts >>> > before crashing with a segmentation fault: >>> > >>> > ``` >>> > ☁ compiler [master] ⚡ ../inplace/bin/ghc-stage2 --interactive >>> > GHCi, version 8.3.20170413: http://www.haskell.org/ghc/ :? for help >>> > [1] 79176 segmentation fault ../inplace/bin/ghc-stage2 --interactive >>> > ``` >>> > >>> > Did it happen to somebody else or it’s just me? Shall I try throwing gdb at >>> > it to try and see what’s going on? >>> >>> Hmm, interesting. I've not seen crashes like this locally nor in CI. It >>> would be great if you could try to get some insight. Is this crash >>> perfectly reproducible? >>> >>> It may be worth adding -dcore-lint to GhcStage2HcOpts to ensure the code >>> we are producing is sane. >>> >>> Cheers, >>> >>> - Ben >>> >> > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Sat Apr 15 02:41:13 2017 From: ben at well-typed.com (Ben Gamari) Date: Fri, 14 Apr 2017 22:41:13 -0400 Subject: downloads.haskell.org unavailable for a bit In-Reply-To: <87efwvhx83.fsf@ben-laptop.smart-cactus.org> References: <87h91rhz9n.fsf@ben-laptop.smart-cactus.org> <87efwvhx83.fsf@ben-laptop.smart-cactus.org> Message-ID: <87a87iie9i.fsf@ben-laptop.smart-cactus.org> Ben Gamari writes: > Ben Gamari writes: > >> Hello all, >> >> Unfortunately, due to a botched rsync, the tarballs on >> downloads.haskell.org will be unavailable for a few hours. I'm currently >> working on restoring the 8.0.2 tarballs first to ensure that they remain >> available to users. I will follow up with status updates as they come. >> Many apologies for the inconvenience. >> > The 8.0.2 tarballs are now again available. > All of the tarballs are now once again available. Sorry again for the inconvenience! Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at well-typed.com Sat Apr 15 02:58:00 2017 From: ben at well-typed.com (Ben Gamari) Date: Fri, 14 Apr 2017 22:58:00 -0400 Subject: [ANNOUNCE] GHC tarballs for Windows 10 Creators Update Message-ID: <878tn2idhj.fsf@ben-laptop.smart-cactus.org> Hello GHC users, Over the next few weeks Microsoft will be pushing out an update (the so-called Creators Update) to Windows 10 which will break the GCC toolchain shipped with all GHC releases prior to the yet-to-be-released 8.2.1 (see Trac [1] for details). Over the last few weeks our resident Windows expert Tamar and I have been working on preparing a set of patched Windows tarballs with retrofitted GCC toolchains covering GHC 7.2.1 through 8.0.2. These tarballs are now available on downloads.haskell.org. The new tarballs are distinguished from the original releases with a `-win10` suffix. For instance, the 64-bit 8.0.2 binary distribution can be found at, https://downloads.haskell.org/~ghc/8.0.2/ghc-8.0.2-x86_64-unknown-mingw32-win10.tar.xz Other than an updated mingw/bin/gcc.exe, these are identical to the original releases. More details about the patched toolchain can be found in Tamar's ghc-compat repository [2]. As always, let me know if you encounter trouble. Finally, we should all thank Tamar for his work on this; without his effort GHC-on-Windows would be in a sad state indeed. Thanks Tamar! Cheers, - Ben [1] https://ghc.haskell.org/trac/ghc/ticket/13411 [2] https://github.com/Mistuke/ghc-compat -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From metaniklas at gmail.com Sat Apr 15 07:30:19 2017 From: metaniklas at gmail.com (Niklas Larsson) Date: Sat, 15 Apr 2017 09:30:19 +0200 Subject: [ANNOUNCE] GHC tarballs for Windows 10 Creators Update In-Reply-To: <878tn2idhj.fsf@ben-laptop.smart-cactus.org> References: <878tn2idhj.fsf@ben-laptop.smart-cactus.org> Message-ID: <5875A40F-1FCD-4335-B3D8-C471772C7346@gmail.com> Hi! That's great! I see that the download page at http://www.haskell.org/ghc/download still links to the old tarballs for 8.0.2. Will those links be updated? // Niklas > 15 apr. 2017 kl. 04:58 skrev Ben Gamari : > > > Hello GHC users, > > Over the next few weeks Microsoft will be pushing out an update (the > so-called Creators Update) to Windows 10 which will break the GCC > toolchain shipped with all GHC releases prior to the yet-to-be-released > 8.2.1 (see Trac [1] for details). > > Over the last few weeks our resident Windows expert Tamar and I have > been working on preparing a set of patched Windows tarballs with > retrofitted GCC toolchains covering GHC 7.2.1 through 8.0.2. These > tarballs are now available on downloads.haskell.org. The new tarballs > are distinguished from the original releases with a `-win10` suffix. For > instance, the 64-bit 8.0.2 binary distribution can be found at, > > https://downloads.haskell.org/~ghc/8.0.2/ghc-8.0.2-x86_64-unknown-mingw32-win10.tar.xz > > Other than an updated mingw/bin/gcc.exe, these are identical to the > original releases. More details about the patched toolchain can be found > in Tamar's ghc-compat repository [2]. As always, let me know if you > encounter trouble. > > Finally, we should all thank Tamar for his work on this; without his > effort GHC-on-Windows would be in a sad state indeed. Thanks Tamar! > > Cheers, > > - Ben > > > [1] https://ghc.haskell.org/trac/ghc/ticket/13411 > [2] https://github.com/Mistuke/ghc-compat > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From ben at well-typed.com Sat Apr 15 10:37:07 2017 From: ben at well-typed.com (Ben Gamari) Date: Sat, 15 Apr 2017 06:37:07 -0400 Subject: [ANNOUNCE] GHC tarballs for Windows 10 Creators Update In-Reply-To: <5875A40F-1FCD-4335-B3D8-C471772C7346@gmail.com> References: <878tn2idhj.fsf@ben-laptop.smart-cactus.org> <5875A40F-1FCD-4335-B3D8-C471772C7346@gmail.com> Message-ID: <8760i6hs8c.fsf@ben-laptop.smart-cactus.org> Niklas Larsson writes: > Hi! > > That's great! I see that the download page at > http://www.haskell.org/ghc/download still links to the old tarballs > for 8.0.2. Will those links be updated? > Yes, they will in due time. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From svenpanne at gmail.com Sat Apr 15 14:06:59 2017 From: svenpanne at gmail.com (Sven Panne) Date: Sat, 15 Apr 2017 16:06:59 +0200 Subject: [ANNOUNCE] GHC tarballs for Windows 10 Creators Update In-Reply-To: <878tn2idhj.fsf@ben-laptop.smart-cactus.org> References: <878tn2idhj.fsf@ben-laptop.smart-cactus.org> Message-ID: 2017-04-15 4:58 GMT+02:00 Ben Gamari : > [...} The new tarballs > are distinguished from the original releases with a `-win10` suffix. For > instance, the 64-bit 8.0.2 binary distribution can be found at, > > https://downloads.haskell.org/~ghc/8.0.2/ghc-8.0.2-x86_64- > unknown-mingw32-win10.tar.xz > [...] > I heavily rely on "stack" to install GHC version, so what is the intended way of telling "stack setup" about the suffix? If I understand things correctly, "stack setup" will install non-working versions on Windows with the Creators Update. -------------- next part -------------- An HTML attachment was scrubbed... URL: From allbery.b at gmail.com Sat Apr 15 16:14:13 2017 From: allbery.b at gmail.com (Brandon Allbery) Date: Sat, 15 Apr 2017 12:14:13 -0400 Subject: [ANNOUNCE] GHC tarballs for Windows 10 Creators Update In-Reply-To: References: <878tn2idhj.fsf@ben-laptop.smart-cactus.org> Message-ID: On Sat, Apr 15, 2017 at 10:06 AM, Sven Panne wrote: > I heavily rely on "stack" to install GHC version, so what is the intended > way of telling "stack setup" about the suffix? If I understand things > correctly, "stack setup" will install non-working versions on Windows with > the Creators Update. the stack devs will have to figure that out, but I'm sure they do their own distribution instead of relying on an "alien" one anyway. -- brandon s allbery kf8nh sine nomine associates allbery.b at gmail.com ballbery at sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net -------------- next part -------------- An HTML attachment was scrubbed... URL: From alfredo.dinapoli at gmail.com Mon Apr 17 17:44:44 2017 From: alfredo.dinapoli at gmail.com (Alfredo Di Napoli) Date: Mon, 17 Apr 2017 19:44:44 +0200 Subject: Where do I start if I would like help improve GHC compilation times? In-Reply-To: <87mvbnyj5u.fsf@ben-laptop.smart-cactus.org> References: <87fuhk192d.fsf@ben-laptop.smart-cactus.org> <87mvbnyj5u.fsf@ben-laptop.smart-cactus.org> Message-ID: Dear all, after sprinkling (ehm, littering) GHC source code with cost centres, I was not surprised to see that roughly 20% of the compilation time (as in .prof) was spent in the core gen/simplification process (10% of the total time) and on the asm code gen (another 10%). I have almost immediately abandoned the idea of try optimising some modules in simplCore (considering my 0-knowledge of GHC internals anyway..) but I have been dwelling on the following: Outputable.hs and Pretty.hs seems to be have been implemented making deliberate use of lists and concatenations between them, which left me wondering if there was room for optimisation there. I have found this interesting paper on the topic: https://www.cs.kent.ac.uk/pubs/2005/2062/content.pdf Now, it’s totally possible that this has been already tried (with no success) but judging from the original copyright of Pretty.hs (dated 2001), it seems it was written prior to the work of Olaf Chitil (the author of the paper). TL;DR I was thinking (even just as a fun exercise to learn more about GHC internals) to leverage the ideas of that paper and switch to a different implementation for `Doc` coupled with the use of lazy dequeues, which *might* increase the performances of the codegen and thus of the compiler overall. Am I fighting a strawman (or flogging a dead horse, pick your rethorical figure :D ) or is there even a tiny chance of this being actually useful? Have a nice evening, Alfredo On 11 April 2017 at 00:47, Ben Gamari wrote: > Alfredo Di Napoli writes: > > > Hey Ben, > > > Hi Alfredo, > > Sorry for the late response! The email queue from the weekend was a bit > longer than I would like. > > > as promised I’m back to you with something more articulated and hopefully > > meaningful. I do hear you perfectly — probably trying to dive head-first > > into this without at least a rough understanding of the performance > > hotspots or the GHC overall architecture is going to do me more harm than > > good (I get the overall picture and I’m aware of the different stages of > > the GHC compilation pipeline, but it’s far from saying I’m proficient > with > > the architecture as whole). I have also read a couple of years ago the > GHC > > chapter on the “Architeture of Open Source Applications” book, but I > don’t > > know how much that is still relevant. If it is, I guess I should refresh > my > > memory. > > > It sounds like you have done a good amount of reading. That's great. > Perhaps skimming the AOSA chapter again wouldn't hurt, but otherwise > it's likely worthwhile diving in. > > > I’m currently trying to move on 2 fronts — please advice if I’m a fool > > flogging a dead horse or if I have any hope of getting anything done ;) > > > > 1. I’m trying to treat indeed the compiler as a black block (as you > > adviced) trying to build a sufficiently large program where GHC is not > “as > > fast as I would like” (I know that’s a very lame definition of “slow”, > > hehe). In particular, I have built the stage2 compiler with the “prof” > > flavour as you suggested, and I have chosen 2 examples as a reference > > “benchmark” for performance; DynFlags.hs (which seems to have been > > mentioned multiple times as a GHC perf killer) and the highlighting-kate > > package as posted here: https://ghc.haskell.org/trac/ghc/ticket/9221 . > > Indeed, #9221 would be a very interesting ticket to look at. The > highlighting-kate package is interesting in the context of that ticket > as it has a very large amount of parallelism available. > > If you do want to look at #9221, note that the cost centre profiler may > not provide the whole story. In particular, it has been speculated that > the scaling issues may be due to either, > > * threads hitting a blackhole, resulting in blocking > > * the usual scaling limitations of GHC's stop-the-world GC > > The eventlog may be quite useful for characterising these. > > > The idea would be to compile those with -v +RTS -p -hc -RTS enabled, > > look at the output from the .prof file AND the `-v` flag, find any > > hotspot, try to change something, recompile, observe diff, rinse and > > repeat. Do you think I have any hope of making progress this way? In > > particular, I think compiling DynFlags.hs is a bit of a dead-end; I > > whipped up this buggy script which > > escalated into a Behemoth which is compiling pretty much half of the > > compiler once again :D > > > > ``` > > #!/usr/bin/env bash > > > > ../ghc/inplace/bin/ghc-stage2 --make -j8 -v +RTS -A256M -qb0 -p -h \ > > -RTS -DSTAGE=2 -I../ghc/includes -I../ghc/compiler > -I../ghc/compiler/stage2 > > \ > > -I../ghc/compiler/stage2/build \ > > -i../ghc/compiler/utils:../ghc/compiler/types:../ghc/ > compiler/typecheck:../ghc/compiler/basicTypes > > \ > > -i../ghc/compiler/main:../ghc/compiler/profiling:../ghc/ > compiler/coreSyn:../ghc/compiler/iface:../ghc/compiler/prelude > > \ > > -i../ghc/compiler/stage2/build:../ghc/compiler/ > simplStg:../ghc/compiler/cmm:../ghc/compiler/parser:../ghc/compiler/hsSyn > > \ > > -i../ghc/compiler/ghci:../ghc/compiler/deSugar:../ghc/ > compiler/simplCore:../ghc/compile/specialise > > \ > > -fforce-recomp -c $@ > > ``` > > > > I’m running it with `./dynflags.sh ../ghc/compiler/main/DynFlags.hs` but > > it’s taking a lot to compile (20+ mins on my 2014 mac Pro) because it’s > > pulling in half of the compiler anyway :D I tried to reuse the .hi files > > from my stage2 compilation but I failed (GHC was complaining about > > interface file mismatch). Short story short, I don’t think it will be a > > very agile way to proceed. Am I right? Do you have any recommendation in > > such sense? Do I have any hope to compile DynFlags.hs in a way which > would > > make this perf investigation feasible? > > > What I usually do in this case is just take the relevant `ghc` command > line directly from the `make` output and execute it manually. I would > imagine your debug cycle would look something like, > > * instrument the compiler > * build stage1 > * use stage2 to build DynFlags using the stage1 compiler (using a saved > command line) > * think > * repeat > > This should only take a few minutes per iteration. > > > The second example (the highlighting-kate package) seems much more > > promising. It takes maybe 1-2 mins on my machine, which is enough to > take a > > look at the perf output. Do you think I should follow this second lead? > In > > principle any 50+ modules package I think would do (better if with a lot > of > > TH ;) ) but this seems like a low-entry barrier start. > > > > 2. The second path I’m exploring is simply to take a less holistic > approach > > and try to dive in into a performance ticket like the ones listed here: > > https://www.reddit.com/r/haskell/comments/45q90s/is_ > anything_being_done_to_remedy_the_soul/czzq6an/ > > Maybe some are very specific, but it seems like fixing small things and > > move forward could help giving me understanding of different sub-parts of > > GHC, which seems less intimidating than the black-box approach. > > > Do you have any specific tickets from these lists that you found > interesting? > > > In conclusion, what do you think is the best approach, 1 or 2, both or > > none? ;) > > I would say that it largely depends upon what you feel most comfortable > with. If you feel up for it, I think #9221 would be a nice, fairly > self-contained, yet high-impact ticket which would be worth spending a > few days diving further into. > > Cheers, > > - Ben > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From marlowsd at gmail.com Tue Apr 18 10:01:07 2017 From: marlowsd at gmail.com (Simon Marlow) Date: Tue, 18 Apr 2017 11:01:07 +0100 Subject: Where do I start if I would like help improve GHC compilation times? In-Reply-To: References: <87fuhk192d.fsf@ben-laptop.smart-cactus.org> <87mvbnyj5u.fsf@ben-laptop.smart-cactus.org> Message-ID: Pretty-printing the asm is a likely contender for optimisation, however the problem is not the pretty-printing per se. We don't actually use any of the backtracking stuff when printing asm, since there's no point nicely indenting things or wrapping lines. The overhead is probably all in the layer of data structure that we generate in Pretty before it gets dumped into raw bytes. Using a ByteString Builder instead might yield some improvement. Cheers Simon On 17 April 2017 at 18:44, Alfredo Di Napoli wrote: > Dear all, > > after sprinkling (ehm, littering) GHC source code with cost centres, I was > not surprised to see that roughly 20% of the compilation time (as in .prof) > was spent in the core gen/simplification process (10% of the total time) > and on the asm code gen (another 10%). > > I have almost immediately abandoned the idea of try optimising some > modules in simplCore (considering my 0-knowledge of GHC internals anyway..) > but I have been dwelling on the following: Outputable.hs and Pretty.hs > seems to be have been implemented making deliberate use of lists and > concatenations between them, which left me wondering if there was room for > optimisation there. I have found this interesting paper on the topic: > > https://www.cs.kent.ac.uk/pubs/2005/2062/content.pdf > > Now, it’s totally possible that this has been already tried (with no > success) but judging from the original copyright of Pretty.hs (dated 2001), > it seems it was written prior to the work of Olaf Chitil (the author of the > paper). > > TL;DR I was thinking (even just as a fun exercise to learn more about GHC > internals) to leverage the ideas of that paper and switch to a different > implementation for `Doc` coupled with the use of lazy dequeues, which > *might* increase the performances of the codegen and thus of the compiler > overall. Am I fighting a strawman (or flogging a dead horse, pick your > rethorical figure :D ) or is there even a tiny chance of this being > actually useful? > > Have a nice evening, > > Alfredo > > On 11 April 2017 at 00:47, Ben Gamari wrote: > >> Alfredo Di Napoli writes: >> >> > Hey Ben, >> > >> Hi Alfredo, >> >> Sorry for the late response! The email queue from the weekend was a bit >> longer than I would like. >> >> > as promised I’m back to you with something more articulated and >> hopefully >> > meaningful. I do hear you perfectly — probably trying to dive head-first >> > into this without at least a rough understanding of the performance >> > hotspots or the GHC overall architecture is going to do me more harm >> than >> > good (I get the overall picture and I’m aware of the different stages of >> > the GHC compilation pipeline, but it’s far from saying I’m proficient >> with >> > the architecture as whole). I have also read a couple of years ago the >> GHC >> > chapter on the “Architeture of Open Source Applications” book, but I >> don’t >> > know how much that is still relevant. If it is, I guess I should >> refresh my >> > memory. >> > >> It sounds like you have done a good amount of reading. That's great. >> Perhaps skimming the AOSA chapter again wouldn't hurt, but otherwise >> it's likely worthwhile diving in. >> >> > I’m currently trying to move on 2 fronts — please advice if I’m a fool >> > flogging a dead horse or if I have any hope of getting anything done ;) >> > >> > 1. I’m trying to treat indeed the compiler as a black block (as you >> > adviced) trying to build a sufficiently large program where GHC is not >> “as >> > fast as I would like” (I know that’s a very lame definition of “slow”, >> > hehe). In particular, I have built the stage2 compiler with the “prof” >> > flavour as you suggested, and I have chosen 2 examples as a reference >> > “benchmark” for performance; DynFlags.hs (which seems to have been >> > mentioned multiple times as a GHC perf killer) and the highlighting-kate >> > package as posted here: https://ghc.haskell.org/trac/ghc/ticket/9221 . >> >> Indeed, #9221 would be a very interesting ticket to look at. The >> highlighting-kate package is interesting in the context of that ticket >> as it has a very large amount of parallelism available. >> >> If you do want to look at #9221, note that the cost centre profiler may >> not provide the whole story. In particular, it has been speculated that >> the scaling issues may be due to either, >> >> * threads hitting a blackhole, resulting in blocking >> >> * the usual scaling limitations of GHC's stop-the-world GC >> >> The eventlog may be quite useful for characterising these. >> >> > The idea would be to compile those with -v +RTS -p -hc -RTS enabled, >> > look at the output from the .prof file AND the `-v` flag, find any >> > hotspot, try to change something, recompile, observe diff, rinse and >> > repeat. Do you think I have any hope of making progress this way? In >> > particular, I think compiling DynFlags.hs is a bit of a dead-end; I >> > whipped up this buggy script which >> > escalated into a Behemoth which is compiling pretty much half of the >> > compiler once again :D >> > >> > ``` >> > #!/usr/bin/env bash >> > >> > ../ghc/inplace/bin/ghc-stage2 --make -j8 -v +RTS -A256M -qb0 -p -h \ >> > -RTS -DSTAGE=2 -I../ghc/includes -I../ghc/compiler >> -I../ghc/compiler/stage2 >> > \ >> > -I../ghc/compiler/stage2/build \ >> > -i../ghc/compiler/utils:../ghc/compiler/types:../ghc/compile >> r/typecheck:../ghc/compiler/basicTypes >> > \ >> > -i../ghc/compiler/main:../ghc/compiler/profiling:../ghc/comp >> iler/coreSyn:../ghc/compiler/iface:../ghc/compiler/prelude >> > \ >> > -i../ghc/compiler/stage2/build:../ghc/compiler/simplStg:../ >> ghc/compiler/cmm:../ghc/compiler/parser:../ghc/compiler/hsSyn >> > \ >> > -i../ghc/compiler/ghci:../ghc/compiler/deSugar:../ghc/compil >> er/simplCore:../ghc/compile/specialise >> > \ >> > -fforce-recomp -c $@ >> > ``` >> > >> > I’m running it with `./dynflags.sh ../ghc/compiler/main/DynFlags.hs` >> but >> > it’s taking a lot to compile (20+ mins on my 2014 mac Pro) because it’s >> > pulling in half of the compiler anyway :D I tried to reuse the .hi files >> > from my stage2 compilation but I failed (GHC was complaining about >> > interface file mismatch). Short story short, I don’t think it will be a >> > very agile way to proceed. Am I right? Do you have any recommendation in >> > such sense? Do I have any hope to compile DynFlags.hs in a way which >> would >> > make this perf investigation feasible? >> > >> What I usually do in this case is just take the relevant `ghc` command >> line directly from the `make` output and execute it manually. I would >> imagine your debug cycle would look something like, >> >> * instrument the compiler >> * build stage1 >> * use stage2 to build DynFlags using the stage1 compiler (using a saved >> command line) >> * think >> * repeat >> >> This should only take a few minutes per iteration. >> >> > The second example (the highlighting-kate package) seems much more >> > promising. It takes maybe 1-2 mins on my machine, which is enough to >> take a >> > look at the perf output. Do you think I should follow this second lead? >> In >> > principle any 50+ modules package I think would do (better if with a >> lot of >> > TH ;) ) but this seems like a low-entry barrier start. >> > >> > 2. The second path I’m exploring is simply to take a less holistic >> approach >> > and try to dive in into a performance ticket like the ones listed here: >> > https://www.reddit.com/r/haskell/comments/45q90s/is_anything >> _being_done_to_remedy_the_soul/czzq6an/ >> > Maybe some are very specific, but it seems like fixing small things and >> > move forward could help giving me understanding of different sub-parts >> of >> > GHC, which seems less intimidating than the black-box approach. >> > >> Do you have any specific tickets from these lists that you found >> interesting? >> >> > In conclusion, what do you think is the best approach, 1 or 2, both or >> > none? ;) >> >> I would say that it largely depends upon what you feel most comfortable >> with. If you feel up for it, I think #9221 would be a nice, fairly >> self-contained, yet high-impact ticket which would be worth spending a >> few days diving further into. >> >> Cheers, >> >> - Ben >> >> > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alfredo.dinapoli at gmail.com Tue Apr 18 10:17:29 2017 From: alfredo.dinapoli at gmail.com (Alfredo Di Napoli) Date: Tue, 18 Apr 2017 12:17:29 +0200 Subject: Where do I start if I would like help improve GHC compilation times? In-Reply-To: References: <87fuhk192d.fsf@ben-laptop.smart-cactus.org> <87mvbnyj5u.fsf@ben-laptop.smart-cactus.org> Message-ID: Hey Simon, thanks for chiming in. I had the same suspect as well, and in fact my first temptation was to use dlists to avoid costly concatenation (I guess a Builder shares pretty much the same idea, which is avoid right-appends). I eventually bailed out as that Pretty module had some functions like sepX or fillNBE (I might be wrong, going by memory only here) which needed to *inspect* the current [Doc] we were carrying around, and thus dlists couldn’t accomplish this in O(1), but forcing it back to a list was necessary. Am I right in thinking that using a Builder here will suffer the same malady? Ideally I would like constant time for both appends, left concat & inspection (as in pattern-matching inspection) but I don’t think what I’m asking exists, at least not in its functional declination anyway ;) That’s why I was thinking to give Deques (or more generally, Sequences) a go: they don’t have O(1) appends but at least can be inspected in O(1). Question has to be answered whether this will yield improvement or not, which I guess depends upon the ratio of inspection / appending we do in the pretty printing. In that case using dlists or builders might be better. Last but not least, although the current implementation doesn’t backtrack anyway, I’m wondering wether switching to a different representation for a Doc (namely a huge function composition of Token(s), as described in the paper) could be beneficial as well. Do you guys have any opinions? Yesterday I extracted Pretty.hs from the sourcetree and I’m now planning to produce a criterion benchmark and compare different implementations, althought it’s a bit hard to predict the real world usage as I don’t have access to a representative Doc document as produced by GHC, so my benchs could be all ill-founded. Alfredo On 18 April 2017 at 12:01, Simon Marlow wrote: > Pretty-printing the asm is a likely contender for optimisation, however > the problem is not the pretty-printing per se. We don't actually use any > of the backtracking stuff when printing asm, since there's no point nicely > indenting things or wrapping lines. The overhead is probably all in the > layer of data structure that we generate in Pretty before it gets dumped > into raw bytes. Using a ByteString Builder instead might yield some > improvement. > > Cheers > Simon > > On 17 April 2017 at 18:44, Alfredo Di Napoli > wrote: > >> Dear all, >> >> after sprinkling (ehm, littering) GHC source code with cost centres, I >> was not surprised to see that roughly 20% of the compilation time (as in >> .prof) was spent in the core gen/simplification process (10% of the total >> time) and on the asm code gen (another 10%). >> >> I have almost immediately abandoned the idea of try optimising some >> modules in simplCore (considering my 0-knowledge of GHC internals anyway..) >> but I have been dwelling on the following: Outputable.hs and Pretty.hs >> seems to be have been implemented making deliberate use of lists and >> concatenations between them, which left me wondering if there was room for >> optimisation there. I have found this interesting paper on the topic: >> >> https://www.cs.kent.ac.uk/pubs/2005/2062/content.pdf >> >> Now, it’s totally possible that this has been already tried (with no >> success) but judging from the original copyright of Pretty.hs (dated 2001), >> it seems it was written prior to the work of Olaf Chitil (the author of the >> paper). >> >> TL;DR I was thinking (even just as a fun exercise to learn more about GHC >> internals) to leverage the ideas of that paper and switch to a different >> implementation for `Doc` coupled with the use of lazy dequeues, which >> *might* increase the performances of the codegen and thus of the compiler >> overall. Am I fighting a strawman (or flogging a dead horse, pick your >> rethorical figure :D ) or is there even a tiny chance of this being >> actually useful? >> >> Have a nice evening, >> >> Alfredo >> >> On 11 April 2017 at 00:47, Ben Gamari wrote: >> >>> Alfredo Di Napoli writes: >>> >>> > Hey Ben, >>> > >>> Hi Alfredo, >>> >>> Sorry for the late response! The email queue from the weekend was a bit >>> longer than I would like. >>> >>> > as promised I’m back to you with something more articulated and >>> hopefully >>> > meaningful. I do hear you perfectly — probably trying to dive >>> head-first >>> > into this without at least a rough understanding of the performance >>> > hotspots or the GHC overall architecture is going to do me more harm >>> than >>> > good (I get the overall picture and I’m aware of the different stages >>> of >>> > the GHC compilation pipeline, but it’s far from saying I’m proficient >>> with >>> > the architecture as whole). I have also read a couple of years ago the >>> GHC >>> > chapter on the “Architeture of Open Source Applications” book, but I >>> don’t >>> > know how much that is still relevant. If it is, I guess I should >>> refresh my >>> > memory. >>> > >>> It sounds like you have done a good amount of reading. That's great. >>> Perhaps skimming the AOSA chapter again wouldn't hurt, but otherwise >>> it's likely worthwhile diving in. >>> >>> > I’m currently trying to move on 2 fronts — please advice if I’m a fool >>> > flogging a dead horse or if I have any hope of getting anything done ;) >>> > >>> > 1. I’m trying to treat indeed the compiler as a black block (as you >>> > adviced) trying to build a sufficiently large program where GHC is not >>> “as >>> > fast as I would like” (I know that’s a very lame definition of “slow”, >>> > hehe). In particular, I have built the stage2 compiler with the “prof” >>> > flavour as you suggested, and I have chosen 2 examples as a reference >>> > “benchmark” for performance; DynFlags.hs (which seems to have been >>> > mentioned multiple times as a GHC perf killer) and the >>> highlighting-kate >>> > package as posted here: https://ghc.haskell.org/trac/ghc/ticket/9221 . >>> >>> Indeed, #9221 would be a very interesting ticket to look at. The >>> highlighting-kate package is interesting in the context of that ticket >>> as it has a very large amount of parallelism available. >>> >>> If you do want to look at #9221, note that the cost centre profiler may >>> not provide the whole story. In particular, it has been speculated that >>> the scaling issues may be due to either, >>> >>> * threads hitting a blackhole, resulting in blocking >>> >>> * the usual scaling limitations of GHC's stop-the-world GC >>> >>> The eventlog may be quite useful for characterising these. >>> >>> > The idea would be to compile those with -v +RTS -p -hc -RTS enabled, >>> > look at the output from the .prof file AND the `-v` flag, find any >>> > hotspot, try to change something, recompile, observe diff, rinse and >>> > repeat. Do you think I have any hope of making progress this way? In >>> > particular, I think compiling DynFlags.hs is a bit of a dead-end; I >>> > whipped up this buggy script which >>> > escalated into a Behemoth which is compiling pretty much half of the >>> > compiler once again :D >>> > >>> > ``` >>> > #!/usr/bin/env bash >>> > >>> > ../ghc/inplace/bin/ghc-stage2 --make -j8 -v +RTS -A256M -qb0 -p -h \ >>> > -RTS -DSTAGE=2 -I../ghc/includes -I../ghc/compiler >>> -I../ghc/compiler/stage2 >>> > \ >>> > -I../ghc/compiler/stage2/build \ >>> > -i../ghc/compiler/utils:../ghc/compiler/types:../ghc/compile >>> r/typecheck:../ghc/compiler/basicTypes >>> > \ >>> > -i../ghc/compiler/main:../ghc/compiler/profiling:../ghc/comp >>> iler/coreSyn:../ghc/compiler/iface:../ghc/compiler/prelude >>> > \ >>> > -i../ghc/compiler/stage2/build:../ghc/compiler/simplStg:../g >>> hc/compiler/cmm:../ghc/compiler/parser:../ghc/compiler/hsSyn >>> > \ >>> > -i../ghc/compiler/ghci:../ghc/compiler/deSugar:../ghc/compil >>> er/simplCore:../ghc/compile/specialise >>> > \ >>> > -fforce-recomp -c $@ >>> > ``` >>> > >>> > I’m running it with `./dynflags.sh ../ghc/compiler/main/DynFlags.hs` >>> but >>> > it’s taking a lot to compile (20+ mins on my 2014 mac Pro) because it’s >>> > pulling in half of the compiler anyway :D I tried to reuse the .hi >>> files >>> > from my stage2 compilation but I failed (GHC was complaining about >>> > interface file mismatch). Short story short, I don’t think it will be a >>> > very agile way to proceed. Am I right? Do you have any recommendation >>> in >>> > such sense? Do I have any hope to compile DynFlags.hs in a way which >>> would >>> > make this perf investigation feasible? >>> > >>> What I usually do in this case is just take the relevant `ghc` command >>> line directly from the `make` output and execute it manually. I would >>> imagine your debug cycle would look something like, >>> >>> * instrument the compiler >>> * build stage1 >>> * use stage2 to build DynFlags using the stage1 compiler (using a saved >>> command line) >>> * think >>> * repeat >>> >>> This should only take a few minutes per iteration. >>> >>> > The second example (the highlighting-kate package) seems much more >>> > promising. It takes maybe 1-2 mins on my machine, which is enough to >>> take a >>> > look at the perf output. Do you think I should follow this second >>> lead? In >>> > principle any 50+ modules package I think would do (better if with a >>> lot of >>> > TH ;) ) but this seems like a low-entry barrier start. >>> > >>> > 2. The second path I’m exploring is simply to take a less holistic >>> approach >>> > and try to dive in into a performance ticket like the ones listed here: >>> > https://www.reddit.com/r/haskell/comments/45q90s/is_anything >>> _being_done_to_remedy_the_soul/czzq6an/ >>> > Maybe some are very specific, but it seems like fixing small things and >>> > move forward could help giving me understanding of different sub-parts >>> of >>> > GHC, which seems less intimidating than the black-box approach. >>> > >>> Do you have any specific tickets from these lists that you found >>> interesting? >>> >>> > In conclusion, what do you think is the best approach, 1 or 2, both or >>> > none? ;) >>> >>> I would say that it largely depends upon what you feel most comfortable >>> with. If you feel up for it, I think #9221 would be a nice, fairly >>> self-contained, yet high-impact ticket which would be worth spending a >>> few days diving further into. >>> >>> Cheers, >>> >>> - Ben >>> >>> >> >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From niteria at gmail.com Tue Apr 18 11:01:20 2017 From: niteria at gmail.com (Bartosz Nitka) Date: Tue, 18 Apr 2017 12:01:20 +0100 Subject: Where do I start if I would like help improve GHC compilation times? In-Reply-To: References: <87fuhk192d.fsf@ben-laptop.smart-cactus.org> <87mvbnyj5u.fsf@ben-laptop.smart-cactus.org> Message-ID: Hey Alfredo, Thanks for taking a look at this. I've taken a look at this before and collected some notes here: https://github.com/niteria/notes/blob/master/PrettyPrintingGHC.md#problems Based on my investigations I concluded that there are places where pretty-printing Asm and Cmm gets accidentally quadratic. If I remember correctly the core of the problem was that even in the "fast" mode the pretty printer was computing the lengths of subtrees in `reduceDoc`, which in some cases made the operations quadratic. I've tried implementing a "just append bytes"-mode without `reduceDoc`, but what stopped me was my lack of understanding of different semantics for 3 kinds of empty Doc's. I think if you took the time to understand it, you could implement it in a way that's not quadratic, reaping a substantial performance win. Cheers, Bartosz 2017-04-18 11:17 GMT+01:00 Alfredo Di Napoli : > Hey Simon, > > thanks for chiming in. I had the same suspect as well, and in fact my > first temptation was to use dlists to avoid costly concatenation (I guess a > Builder shares pretty much the same idea, which is avoid right-appends). I > eventually bailed out as that Pretty module had some functions like sepX or > fillNBE (I might be wrong, going by memory only here) which needed to > *inspect* the current [Doc] we were carrying around, and thus dlists > couldn’t accomplish this in O(1), but forcing it back to a list was > necessary. Am I right in thinking that using a Builder here will suffer the > same malady? Ideally I would like constant time for both appends, left > concat & inspection (as in pattern-matching inspection) but I don’t think > what I’m asking exists, at least not in its functional declination anyway ;) > > That’s why I was thinking to give Deques (or more generally, Sequences) a > go: they don’t have O(1) appends but at least can be inspected in O(1). > Question has to be answered whether this will yield improvement or not, > which I guess depends upon the ratio of inspection / appending we do in the > pretty printing. In that case using dlists or builders might be better. > Last but not least, although the current implementation doesn’t backtrack > anyway, I’m wondering wether switching to a different representation for a > Doc (namely a huge function composition of Token(s), as described in the > paper) could be beneficial as well. > > Do you guys have any opinions? Yesterday I extracted Pretty.hs from the > sourcetree and I’m now planning to produce a criterion benchmark and > compare different implementations, althought it’s a bit hard to predict the > real world usage as I don’t have access to a representative Doc document as > produced by GHC, so my benchs could be all ill-founded. > > Alfredo > > On 18 April 2017 at 12:01, Simon Marlow wrote: > >> Pretty-printing the asm is a likely contender for optimisation, however >> the problem is not the pretty-printing per se. We don't actually use any >> of the backtracking stuff when printing asm, since there's no point nicely >> indenting things or wrapping lines. The overhead is probably all in the >> layer of data structure that we generate in Pretty before it gets dumped >> into raw bytes. Using a ByteString Builder instead might yield some >> improvement. >> >> Cheers >> Simon >> >> On 17 April 2017 at 18:44, Alfredo Di Napoli >> wrote: >> >>> Dear all, >>> >>> after sprinkling (ehm, littering) GHC source code with cost centres, I >>> was not surprised to see that roughly 20% of the compilation time (as in >>> .prof) was spent in the core gen/simplification process (10% of the total >>> time) and on the asm code gen (another 10%). >>> >>> I have almost immediately abandoned the idea of try optimising some >>> modules in simplCore (considering my 0-knowledge of GHC internals anyway..) >>> but I have been dwelling on the following: Outputable.hs and Pretty.hs >>> seems to be have been implemented making deliberate use of lists and >>> concatenations between them, which left me wondering if there was room for >>> optimisation there. I have found this interesting paper on the topic: >>> >>> https://www.cs.kent.ac.uk/pubs/2005/2062/content.pdf >>> >>> Now, it’s totally possible that this has been already tried (with no >>> success) but judging from the original copyright of Pretty.hs (dated 2001), >>> it seems it was written prior to the work of Olaf Chitil (the author of the >>> paper). >>> >>> TL;DR I was thinking (even just as a fun exercise to learn more about >>> GHC internals) to leverage the ideas of that paper and switch to a >>> different implementation for `Doc` coupled with the use of lazy dequeues, >>> which *might* increase the performances of the codegen and thus of the >>> compiler overall. Am I fighting a strawman (or flogging a dead horse, pick >>> your rethorical figure :D ) or is there even a tiny chance of this being >>> actually useful? >>> >>> Have a nice evening, >>> >>> Alfredo >>> >>> On 11 April 2017 at 00:47, Ben Gamari wrote: >>> >>>> Alfredo Di Napoli writes: >>>> >>>> > Hey Ben, >>>> > >>>> Hi Alfredo, >>>> >>>> Sorry for the late response! The email queue from the weekend was a bit >>>> longer than I would like. >>>> >>>> > as promised I’m back to you with something more articulated and >>>> hopefully >>>> > meaningful. I do hear you perfectly — probably trying to dive >>>> head-first >>>> > into this without at least a rough understanding of the performance >>>> > hotspots or the GHC overall architecture is going to do me more harm >>>> than >>>> > good (I get the overall picture and I’m aware of the different stages >>>> of >>>> > the GHC compilation pipeline, but it’s far from saying I’m proficient >>>> with >>>> > the architecture as whole). I have also read a couple of years ago >>>> the GHC >>>> > chapter on the “Architeture of Open Source Applications” book, but I >>>> don’t >>>> > know how much that is still relevant. If it is, I guess I should >>>> refresh my >>>> > memory. >>>> > >>>> It sounds like you have done a good amount of reading. That's great. >>>> Perhaps skimming the AOSA chapter again wouldn't hurt, but otherwise >>>> it's likely worthwhile diving in. >>>> >>>> > I’m currently trying to move on 2 fronts — please advice if I’m a fool >>>> > flogging a dead horse or if I have any hope of getting anything done >>>> ;) >>>> > >>>> > 1. I’m trying to treat indeed the compiler as a black block (as you >>>> > adviced) trying to build a sufficiently large program where GHC is >>>> not “as >>>> > fast as I would like” (I know that’s a very lame definition of “slow”, >>>> > hehe). In particular, I have built the stage2 compiler with the “prof” >>>> > flavour as you suggested, and I have chosen 2 examples as a reference >>>> > “benchmark” for performance; DynFlags.hs (which seems to have been >>>> > mentioned multiple times as a GHC perf killer) and the >>>> highlighting-kate >>>> > package as posted here: https://ghc.haskell.org/trac/ghc/ticket/9221 >>>> . >>>> >>>> Indeed, #9221 would be a very interesting ticket to look at. The >>>> highlighting-kate package is interesting in the context of that ticket >>>> as it has a very large amount of parallelism available. >>>> >>>> If you do want to look at #9221, note that the cost centre profiler may >>>> not provide the whole story. In particular, it has been speculated that >>>> the scaling issues may be due to either, >>>> >>>> * threads hitting a blackhole, resulting in blocking >>>> >>>> * the usual scaling limitations of GHC's stop-the-world GC >>>> >>>> The eventlog may be quite useful for characterising these. >>>> >>>> > The idea would be to compile those with -v +RTS -p -hc -RTS enabled, >>>> > look at the output from the .prof file AND the `-v` flag, find any >>>> > hotspot, try to change something, recompile, observe diff, rinse and >>>> > repeat. Do you think I have any hope of making progress this way? In >>>> > particular, I think compiling DynFlags.hs is a bit of a dead-end; I >>>> > whipped up this buggy script which >>>> > escalated into a Behemoth which is compiling pretty much half of the >>>> > compiler once again :D >>>> > >>>> > ``` >>>> > #!/usr/bin/env bash >>>> > >>>> > ../ghc/inplace/bin/ghc-stage2 --make -j8 -v +RTS -A256M -qb0 -p -h \ >>>> > -RTS -DSTAGE=2 -I../ghc/includes -I../ghc/compiler >>>> -I../ghc/compiler/stage2 >>>> > \ >>>> > -I../ghc/compiler/stage2/build \ >>>> > -i../ghc/compiler/utils:../ghc/compiler/types:../ghc/compile >>>> r/typecheck:../ghc/compiler/basicTypes >>>> > \ >>>> > -i../ghc/compiler/main:../ghc/compiler/profiling:../ghc/comp >>>> iler/coreSyn:../ghc/compiler/iface:../ghc/compiler/prelude >>>> > \ >>>> > -i../ghc/compiler/stage2/build:../ghc/compiler/simplStg:../g >>>> hc/compiler/cmm:../ghc/compiler/parser:../ghc/compiler/hsSyn >>>> > \ >>>> > -i../ghc/compiler/ghci:../ghc/compiler/deSugar:../ghc/compil >>>> er/simplCore:../ghc/compile/specialise >>>> > \ >>>> > -fforce-recomp -c $@ >>>> > ``` >>>> > >>>> > I’m running it with `./dynflags.sh ../ghc/compiler/main/DynFlags.hs` >>>> but >>>> > it’s taking a lot to compile (20+ mins on my 2014 mac Pro) because >>>> it’s >>>> > pulling in half of the compiler anyway :D I tried to reuse the .hi >>>> files >>>> > from my stage2 compilation but I failed (GHC was complaining about >>>> > interface file mismatch). Short story short, I don’t think it will be >>>> a >>>> > very agile way to proceed. Am I right? Do you have any recommendation >>>> in >>>> > such sense? Do I have any hope to compile DynFlags.hs in a way which >>>> would >>>> > make this perf investigation feasible? >>>> > >>>> What I usually do in this case is just take the relevant `ghc` command >>>> line directly from the `make` output and execute it manually. I would >>>> imagine your debug cycle would look something like, >>>> >>>> * instrument the compiler >>>> * build stage1 >>>> * use stage2 to build DynFlags using the stage1 compiler (using a >>>> saved command line) >>>> * think >>>> * repeat >>>> >>>> This should only take a few minutes per iteration. >>>> >>>> > The second example (the highlighting-kate package) seems much more >>>> > promising. It takes maybe 1-2 mins on my machine, which is enough to >>>> take a >>>> > look at the perf output. Do you think I should follow this second >>>> lead? In >>>> > principle any 50+ modules package I think would do (better if with a >>>> lot of >>>> > TH ;) ) but this seems like a low-entry barrier start. >>>> > >>>> > 2. The second path I’m exploring is simply to take a less holistic >>>> approach >>>> > and try to dive in into a performance ticket like the ones listed >>>> here: >>>> > https://www.reddit.com/r/haskell/comments/45q90s/is_anything >>>> _being_done_to_remedy_the_soul/czzq6an/ >>>> > Maybe some are very specific, but it seems like fixing small things >>>> and >>>> > move forward could help giving me understanding of different >>>> sub-parts of >>>> > GHC, which seems less intimidating than the black-box approach. >>>> > >>>> Do you have any specific tickets from these lists that you found >>>> interesting? >>>> >>>> > In conclusion, what do you think is the best approach, 1 or 2, both or >>>> > none? ;) >>>> >>>> I would say that it largely depends upon what you feel most comfortable >>>> with. If you feel up for it, I think #9221 would be a nice, fairly >>>> self-contained, yet high-impact ticket which would be worth spending a >>>> few days diving further into. >>>> >>>> Cheers, >>>> >>>> - Ben >>>> >>>> >>> >>> _______________________________________________ >>> ghc-devs mailing list >>> ghc-devs at haskell.org >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >>> >>> >> > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alfredo.dinapoli at gmail.com Tue Apr 18 11:12:46 2017 From: alfredo.dinapoli at gmail.com (Alfredo Di Napoli) Date: Tue, 18 Apr 2017 13:12:46 +0200 Subject: Where do I start if I would like help improve GHC compilation times? In-Reply-To: References: <87fuhk192d.fsf@ben-laptop.smart-cactus.org> <87mvbnyj5u.fsf@ben-laptop.smart-cactus.org> Message-ID: Hey Bartosz, thanks for that, looks like I won the jackpot today ;) Sounds like your notes are an excellent starting point. Will try to get a better understanding of that module myself, to see if we can reap some perf win here, but the fact you “have been there before” looks like this is a promising path to take. Cheers! Alfredo On 18 April 2017 at 13:01, Bartosz Nitka wrote: > Hey Alfredo, > > Thanks for taking a look at this. I've taken a look at this before and > collected some notes here: https://github.com/niteria/notes/blob/master/ > PrettyPrintingGHC.md#problems > Based on my investigations I concluded that there are places where > pretty-printing Asm and Cmm gets accidentally quadratic. > If I remember correctly the core of the problem was that even in the > "fast" mode the pretty printer was computing the lengths of subtrees in > `reduceDoc`, which in some cases made the operations quadratic. > I've tried implementing a "just append bytes"-mode without `reduceDoc`, > but what stopped me was my lack of understanding of different semantics for > 3 kinds of empty Doc's. > I think if you took the time to understand it, you could implement it in a > way that's not quadratic, reaping a substantial performance win. > > Cheers, > Bartosz > > 2017-04-18 11:17 GMT+01:00 Alfredo Di Napoli : > >> Hey Simon, >> >> thanks for chiming in. I had the same suspect as well, and in fact my >> first temptation was to use dlists to avoid costly concatenation (I guess a >> Builder shares pretty much the same idea, which is avoid right-appends). I >> eventually bailed out as that Pretty module had some functions like sepX or >> fillNBE (I might be wrong, going by memory only here) which needed to >> *inspect* the current [Doc] we were carrying around, and thus dlists >> couldn’t accomplish this in O(1), but forcing it back to a list was >> necessary. Am I right in thinking that using a Builder here will suffer the >> same malady? Ideally I would like constant time for both appends, left >> concat & inspection (as in pattern-matching inspection) but I don’t think >> what I’m asking exists, at least not in its functional declination anyway ;) >> >> That’s why I was thinking to give Deques (or more generally, Sequences) a >> go: they don’t have O(1) appends but at least can be inspected in O(1). >> Question has to be answered whether this will yield improvement or not, >> which I guess depends upon the ratio of inspection / appending we do in the >> pretty printing. In that case using dlists or builders might be better. >> Last but not least, although the current implementation doesn’t backtrack >> anyway, I’m wondering wether switching to a different representation for a >> Doc (namely a huge function composition of Token(s), as described in the >> paper) could be beneficial as well. >> >> Do you guys have any opinions? Yesterday I extracted Pretty.hs from the >> sourcetree and I’m now planning to produce a criterion benchmark and >> compare different implementations, althought it’s a bit hard to predict the >> real world usage as I don’t have access to a representative Doc document as >> produced by GHC, so my benchs could be all ill-founded. >> >> Alfredo >> >> On 18 April 2017 at 12:01, Simon Marlow wrote: >> >>> Pretty-printing the asm is a likely contender for optimisation, however >>> the problem is not the pretty-printing per se. We don't actually use any >>> of the backtracking stuff when printing asm, since there's no point nicely >>> indenting things or wrapping lines. The overhead is probably all in the >>> layer of data structure that we generate in Pretty before it gets dumped >>> into raw bytes. Using a ByteString Builder instead might yield some >>> improvement. >>> >>> Cheers >>> Simon >>> >>> On 17 April 2017 at 18:44, Alfredo Di Napoli >> > wrote: >>> >>>> Dear all, >>>> >>>> after sprinkling (ehm, littering) GHC source code with cost centres, I >>>> was not surprised to see that roughly 20% of the compilation time (as in >>>> .prof) was spent in the core gen/simplification process (10% of the total >>>> time) and on the asm code gen (another 10%). >>>> >>>> I have almost immediately abandoned the idea of try optimising some >>>> modules in simplCore (considering my 0-knowledge of GHC internals anyway..) >>>> but I have been dwelling on the following: Outputable.hs and Pretty.hs >>>> seems to be have been implemented making deliberate use of lists and >>>> concatenations between them, which left me wondering if there was room for >>>> optimisation there. I have found this interesting paper on the topic: >>>> >>>> https://www.cs.kent.ac.uk/pubs/2005/2062/content.pdf >>>> >>>> Now, it’s totally possible that this has been already tried (with no >>>> success) but judging from the original copyright of Pretty.hs (dated 2001), >>>> it seems it was written prior to the work of Olaf Chitil (the author of the >>>> paper). >>>> >>>> TL;DR I was thinking (even just as a fun exercise to learn more about >>>> GHC internals) to leverage the ideas of that paper and switch to a >>>> different implementation for `Doc` coupled with the use of lazy dequeues, >>>> which *might* increase the performances of the codegen and thus of the >>>> compiler overall. Am I fighting a strawman (or flogging a dead horse, pick >>>> your rethorical figure :D ) or is there even a tiny chance of this being >>>> actually useful? >>>> >>>> Have a nice evening, >>>> >>>> Alfredo >>>> >>>> On 11 April 2017 at 00:47, Ben Gamari wrote: >>>> >>>>> Alfredo Di Napoli writes: >>>>> >>>>> > Hey Ben, >>>>> > >>>>> Hi Alfredo, >>>>> >>>>> Sorry for the late response! The email queue from the weekend was a bit >>>>> longer than I would like. >>>>> >>>>> > as promised I’m back to you with something more articulated and >>>>> hopefully >>>>> > meaningful. I do hear you perfectly — probably trying to dive >>>>> head-first >>>>> > into this without at least a rough understanding of the performance >>>>> > hotspots or the GHC overall architecture is going to do me more harm >>>>> than >>>>> > good (I get the overall picture and I’m aware of the different >>>>> stages of >>>>> > the GHC compilation pipeline, but it’s far from saying I’m >>>>> proficient with >>>>> > the architecture as whole). I have also read a couple of years ago >>>>> the GHC >>>>> > chapter on the “Architeture of Open Source Applications” book, but I >>>>> don’t >>>>> > know how much that is still relevant. If it is, I guess I should >>>>> refresh my >>>>> > memory. >>>>> > >>>>> It sounds like you have done a good amount of reading. That's great. >>>>> Perhaps skimming the AOSA chapter again wouldn't hurt, but otherwise >>>>> it's likely worthwhile diving in. >>>>> >>>>> > I’m currently trying to move on 2 fronts — please advice if I’m a >>>>> fool >>>>> > flogging a dead horse or if I have any hope of getting anything done >>>>> ;) >>>>> > >>>>> > 1. I’m trying to treat indeed the compiler as a black block (as you >>>>> > adviced) trying to build a sufficiently large program where GHC is >>>>> not “as >>>>> > fast as I would like” (I know that’s a very lame definition of >>>>> “slow”, >>>>> > hehe). In particular, I have built the stage2 compiler with the >>>>> “prof” >>>>> > flavour as you suggested, and I have chosen 2 examples as a reference >>>>> > “benchmark” for performance; DynFlags.hs (which seems to have been >>>>> > mentioned multiple times as a GHC perf killer) and the >>>>> highlighting-kate >>>>> > package as posted here: https://ghc.haskell.org/trac/ghc/ticket/9221 >>>>> . >>>>> >>>>> Indeed, #9221 would be a very interesting ticket to look at. The >>>>> highlighting-kate package is interesting in the context of that ticket >>>>> as it has a very large amount of parallelism available. >>>>> >>>>> If you do want to look at #9221, note that the cost centre profiler may >>>>> not provide the whole story. In particular, it has been speculated that >>>>> the scaling issues may be due to either, >>>>> >>>>> * threads hitting a blackhole, resulting in blocking >>>>> >>>>> * the usual scaling limitations of GHC's stop-the-world GC >>>>> >>>>> The eventlog may be quite useful for characterising these. >>>>> >>>>> > The idea would be to compile those with -v +RTS -p -hc -RTS enabled, >>>>> > look at the output from the .prof file AND the `-v` flag, find any >>>>> > hotspot, try to change something, recompile, observe diff, rinse and >>>>> > repeat. Do you think I have any hope of making progress this way? In >>>>> > particular, I think compiling DynFlags.hs is a bit of a dead-end; I >>>>> > whipped up this buggy script which >>>>> > escalated into a Behemoth which is compiling pretty much half of the >>>>> > compiler once again :D >>>>> > >>>>> > ``` >>>>> > #!/usr/bin/env bash >>>>> > >>>>> > ../ghc/inplace/bin/ghc-stage2 --make -j8 -v +RTS -A256M -qb0 -p -h \ >>>>> > -RTS -DSTAGE=2 -I../ghc/includes -I../ghc/compiler >>>>> -I../ghc/compiler/stage2 >>>>> > \ >>>>> > -I../ghc/compiler/stage2/build \ >>>>> > -i../ghc/compiler/utils:../ghc/compiler/types:../ghc/compile >>>>> r/typecheck:../ghc/compiler/basicTypes >>>>> > \ >>>>> > -i../ghc/compiler/main:../ghc/compiler/profiling:../ghc/comp >>>>> iler/coreSyn:../ghc/compiler/iface:../ghc/compiler/prelude >>>>> > \ >>>>> > -i../ghc/compiler/stage2/build:../ghc/compiler/simplStg:../g >>>>> hc/compiler/cmm:../ghc/compiler/parser:../ghc/compiler/hsSyn >>>>> > \ >>>>> > -i../ghc/compiler/ghci:../ghc/compiler/deSugar:../ghc/compil >>>>> er/simplCore:../ghc/compile/specialise >>>>> > \ >>>>> > -fforce-recomp -c $@ >>>>> > ``` >>>>> > >>>>> > I’m running it with `./dynflags.sh ../ghc/compiler/main/DynFlags.hs` >>>>> but >>>>> > it’s taking a lot to compile (20+ mins on my 2014 mac Pro) because >>>>> it’s >>>>> > pulling in half of the compiler anyway :D I tried to reuse the .hi >>>>> files >>>>> > from my stage2 compilation but I failed (GHC was complaining about >>>>> > interface file mismatch). Short story short, I don’t think it will >>>>> be a >>>>> > very agile way to proceed. Am I right? Do you have any >>>>> recommendation in >>>>> > such sense? Do I have any hope to compile DynFlags.hs in a way which >>>>> would >>>>> > make this perf investigation feasible? >>>>> > >>>>> What I usually do in this case is just take the relevant `ghc` command >>>>> line directly from the `make` output and execute it manually. I would >>>>> imagine your debug cycle would look something like, >>>>> >>>>> * instrument the compiler >>>>> * build stage1 >>>>> * use stage2 to build DynFlags using the stage1 compiler (using a >>>>> saved command line) >>>>> * think >>>>> * repeat >>>>> >>>>> This should only take a few minutes per iteration. >>>>> >>>>> > The second example (the highlighting-kate package) seems much more >>>>> > promising. It takes maybe 1-2 mins on my machine, which is enough to >>>>> take a >>>>> > look at the perf output. Do you think I should follow this second >>>>> lead? In >>>>> > principle any 50+ modules package I think would do (better if with a >>>>> lot of >>>>> > TH ;) ) but this seems like a low-entry barrier start. >>>>> > >>>>> > 2. The second path I’m exploring is simply to take a less holistic >>>>> approach >>>>> > and try to dive in into a performance ticket like the ones listed >>>>> here: >>>>> > https://www.reddit.com/r/haskell/comments/45q90s/is_anything >>>>> _being_done_to_remedy_the_soul/czzq6an/ >>>>> > Maybe some are very specific, but it seems like fixing small things >>>>> and >>>>> > move forward could help giving me understanding of different >>>>> sub-parts of >>>>> > GHC, which seems less intimidating than the black-box approach. >>>>> > >>>>> Do you have any specific tickets from these lists that you found >>>>> interesting? >>>>> >>>>> > In conclusion, what do you think is the best approach, 1 or 2, both >>>>> or >>>>> > none? ;) >>>>> >>>>> I would say that it largely depends upon what you feel most comfortable >>>>> with. If you feel up for it, I think #9221 would be a nice, fairly >>>>> self-contained, yet high-impact ticket which would be worth spending a >>>>> few days diving further into. >>>>> >>>>> Cheers, >>>>> >>>>> - Ben >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> ghc-devs mailing list >>>> ghc-devs at haskell.org >>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >>>> >>>> >>> >> >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at smart-cactus.org Tue Apr 18 12:21:15 2017 From: ben at smart-cactus.org (Ben Gamari) Date: Tue, 18 Apr 2017 08:21:15 -0400 Subject: Where do I start if I would like help improve GHC compilation times? In-Reply-To: References: <87fuhk192d.fsf@ben-laptop.smart-cactus.org> <87mvbnyj5u.fsf@ben-laptop.smart-cactus.org> Message-ID: <87vaq1hpok.fsf@ben-laptop.smart-cactus.org> Alfredo Di Napoli writes: > Hey Simon, > > thanks for chiming in. I had the same suspect as well, and in fact my first > temptation was to use dlists to avoid costly concatenation (I guess a > Builder shares pretty much the same idea, which is avoid right-appends). I > eventually bailed out as that Pretty module had some functions like sepX or > fillNBE (I might be wrong, going by memory only here) which needed to > *inspect* the current [Doc] we were carrying around, and thus dlists > couldn’t accomplish this in O(1), but forcing it back to a list was > necessary. Am I right in thinking that using a Builder here will suffer the > same malady? Ideally I would like constant time for both appends, left > concat & inspection (as in pattern-matching inspection) but I don’t think > what I’m asking exists, at least not in its functional declination anyway ;) > > That’s why I was thinking to give Deques (or more generally, Sequences) a > go: they don’t have O(1) appends but at least can be inspected in O(1). > Question has to be answered whether this will yield improvement or not, > which I guess depends upon the ratio of inspection / appending we do in the > pretty printing. In that case using dlists or builders might be better. > Last but not least, although the current implementation doesn’t backtrack > anyway, I’m wondering wether switching to a different representation for a > Doc (namely a huge function composition of Token(s), as described in the > paper) could be beneficial as well. > > Do you guys have any opinions? Yesterday I extracted Pretty.hs from the > sourcetree and I’m now planning to produce a criterion benchmark and > compare different implementations, althought it’s a bit hard to predict the > real world usage as I don’t have access to a representative Doc document as > produced by GHC, so my benchs could be all ill-founded. > Note that GHC's `Pretty` module is just a slightly modified version of the `pretty` package. The differences are fairly minimal (although important for performance): * It uses FastString in place of String, giving us fast `length` (in https://ghc.haskell.org/trac/ghc/ticket/8809#comment:60 I propose that we extend `pretty` with a typeclass for string types) * GHC's variant still has a known stack overflow bug that was fixed upstream. Unfortunately, compiler performance regressed when we attempted to port upstream's patch (#10735) Ideally we would fix these and just use the `pretty` library itself. In addition to these issues, it would be quite helpful if `pretty` gained a special-case for the infinite band-width case (which is what we use in the code generator). The point here is that we should need to do no layout computation in the infinite band case: merely place line breaks precisely where the user asks. This should result in a noticeable improvement in code generation performance (IIRC Bartosz noted rather significant amounts of time spent pretty-printing assembler). Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From alfredo.dinapoli at gmail.com Tue Apr 18 12:39:35 2017 From: alfredo.dinapoli at gmail.com (Alfredo Di Napoli) Date: Tue, 18 Apr 2017 14:39:35 +0200 Subject: Where do I start if I would like help improve GHC compilation times? In-Reply-To: <87vaq1hpok.fsf@ben-laptop.smart-cactus.org> References: <87fuhk192d.fsf@ben-laptop.smart-cactus.org> <87mvbnyj5u.fsf@ben-laptop.smart-cactus.org> <87vaq1hpok.fsf@ben-laptop.smart-cactus.org> Message-ID: Hey Ben, thanks for that. I didn’t realise I opened pretty much the Pandora’s box, hehe. I have now found (whilst reading Bartosz’s notes) the numerous tickets & discussions around the library, but what you say in this email neatly summarise the crux of the problem, as far as I could see. Rather than saying silly things, I would rather spend a bit of time reading everything that has been written by folks way smarter than me on the subject and get back to you folks later ;) Alfredo On 18 April 2017 at 14:21, Ben Gamari wrote: > Alfredo Di Napoli writes: > > > Hey Simon, > > > > thanks for chiming in. I had the same suspect as well, and in fact my > first > > temptation was to use dlists to avoid costly concatenation (I guess a > > Builder shares pretty much the same idea, which is avoid right-appends). > I > > eventually bailed out as that Pretty module had some functions like sepX > or > > fillNBE (I might be wrong, going by memory only here) which needed to > > *inspect* the current [Doc] we were carrying around, and thus dlists > > couldn’t accomplish this in O(1), but forcing it back to a list was > > necessary. Am I right in thinking that using a Builder here will suffer > the > > same malady? Ideally I would like constant time for both appends, left > > concat & inspection (as in pattern-matching inspection) but I don’t think > > what I’m asking exists, at least not in its functional declination > anyway ;) > > > > That’s why I was thinking to give Deques (or more generally, Sequences) a > > go: they don’t have O(1) appends but at least can be inspected in O(1). > > Question has to be answered whether this will yield improvement or not, > > which I guess depends upon the ratio of inspection / appending we do in > the > > pretty printing. In that case using dlists or builders might be better. > > Last but not least, although the current implementation doesn’t backtrack > > anyway, I’m wondering wether switching to a different representation for > a > > Doc (namely a huge function composition of Token(s), as described in the > > paper) could be beneficial as well. > > > > Do you guys have any opinions? Yesterday I extracted Pretty.hs from the > > sourcetree and I’m now planning to produce a criterion benchmark and > > compare different implementations, althought it’s a bit hard to predict > the > > real world usage as I don’t have access to a representative Doc document > as > > produced by GHC, so my benchs could be all ill-founded. > > > Note that GHC's `Pretty` module is just a slightly modified version of > the `pretty` package. The differences are fairly minimal (although > important for performance): > > * It uses FastString in place of String, giving us fast `length` (in > https://ghc.haskell.org/trac/ghc/ticket/8809#comment:60 > I propose that we extend `pretty` with a typeclass for string types) > > * GHC's variant still has a known stack overflow bug that was fixed > upstream. Unfortunately, compiler performance regressed when we > attempted to port upstream's patch (#10735) > > Ideally we would fix these and just use the `pretty` library itself. > > In addition to these issues, it would be quite helpful if `pretty` > gained a special-case for the infinite band-width case (which is what we > use in the code generator). The point here is that we should need to do > no layout computation in the infinite band case: merely place line > breaks precisely where the user asks. This should result in a noticeable > improvement in code generation performance (IIRC Bartosz noted rather > significant amounts of time spent pretty-printing assembler). > > Cheers, > > - Ben > -------------- next part -------------- An HTML attachment was scrubbed... URL: From johannes.waldmann at htwk-leipzig.de Tue Apr 18 18:16:02 2017 From: johannes.waldmann at htwk-leipzig.de (Johannes Waldmann) Date: Tue, 18 Apr 2017 20:16:02 +0200 Subject: Where do I start if I would like help improve GHC compilation times? Message-ID: <2c821580-302f-8708-a9bf-c8952219a7f3@htwk-leipzig.de> > it would be quite helpful if `pretty` > gained a special-case for the infinite band-width case Absolutely +1 this. Not "pretty" but similar: https://github.com/jwaldmann/wl-pprint-text/commit/7843d96db57ab01c0be336a5be0d7bca61902005 - J.W. From simonpj at microsoft.com Tue Apr 18 22:49:13 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Tue, 18 Apr 2017 22:49:13 +0000 Subject: [Diffusion] [Raised Concern] rGHC751996e90a96: Kill off complications in CoreFVs In-Reply-To: <1492097045.1059.2.camel@joachim-breitner.de> References: <20170413033033.23405.80723.728D54DE@phabricator.haskell.org> <1492097045.1059.2.camel@joachim-breitner.de> Message-ID: Thanks Joachim. But rather than giving me a person tutorial every six months, might it not be more efficient to make the web pages self explanatory? Eg by supplying a key, by signalling more clearly that only commits with significant changes are shown etc? Simon | -----Original Message----- | From: Joachim Breitner [mailto:mail at joachim-breitner.de] | Sent: 13 April 2017 16:24 | To: Simon Peyton Jones | Cc: ghc-devs | Subject: Re: [Diffusion] [Raised Concern] rGHC751996e90a96: Kill off | complications in CoreFVs | | [CC’ing ghc-dev in case others are also curious about how to navigate | Gipeda.] | | Hi, | | | Am Donnerstag, den 13.04.2017, 08:40 +0000 schrieb Simon Peyton Jones: | > Thanks for noticing this!  I will investigate. | > | > But where do you see this? | > ·         at perf.haskell.org I see “GHC” and “Gipeda”.  It’s not | > obvious which I want. | | You want the GHC Dashboard, as you guessed correctly. | | > ·         Clicking GHC shows me some “recent commits”.  The first | > three are from a few hrs ago; then the next is 8 days ago. Can’t be | > right! | | It only shows interesting commits. Interesting ones are the most recent | ones and all with significant changes. If you want to see all, press the | “=” button in the top-right corner. | | > ·         There is no key to tell me what the little symbols mean – I | > may have known once but I don’t know.  A key would be terrific, or | > even a link to a key. | | The symbols next to the three numbers on the right in the table? They | are: | * number of benchmarks | * number of benchmarks improved | * number of benchmarks regressed | Maybe I should remove the first one, it is not very useful | | Hovering these icons show you which benchmarks changed, and by how much. | | > ·         Clicking on the offending patch (which does show), I see the | > n-body complaint, which is good.  But there is also | > “testsuite/unexpected stats, which is mysterious… clicking on it | > yields no more info. | | There is a “view buildlog” link that I use to investigate such cases. | It goes to | https://raw.githubusercontent.com/nomeata/ghc-speed- | logs/master/751996e90a964026a3f86853338f10c82db6b610.log | which shows | | bytes allocated value is too low: | (If this is because you have improved GHC, please update the test so that | GHC doesn't regress again) |     Expected    T13056(optasm) bytes allocated: 440548592 +/-5% |     Lower bound T13056(optasm) bytes allocated: 418521162 |     Upper bound T13056(optasm) bytes allocated: 462576022 |     Actual      T13056(optasm) bytes allocated: 417666128 |     Deviation   T13056(optasm) bytes allocated:      -5.2 % | *** unexpected stat test failure for T13056(optasm) | | Probably some creep, given how close it is to cut-off percentage. | According to | https://perf.haskell.org/ghc/#graph/tests/alloc/T13056 | this has been stable in the last 50 commits (sorry, still no way to view | these graph for any other time interval). | | > ·         If compile times went up, would that show up anywhere | > (except in testsuite results)? | | Only if it changes the total time of running make, or if it affects the | allocations of a compiler perf test case. | | We currently have no reliable compilation time benchmarks (which would | require _compiling_ stuff NofibRun times). | | Greetings, | Joachim | | -- | Joachim “nomeata” Breitner |   mail at joachim-breitner.de • https://www.joachim-breitner.de/ |   XMPP: nomeata at joachim-breitner.de • OpenPGP-Key: 0xF0FBF51F |   Debian Developer: nomeata at debian.org From mail at joachim-breitner.de Tue Apr 18 23:07:12 2017 From: mail at joachim-breitner.de (Joachim Breitner) Date: Tue, 18 Apr 2017 19:07:12 -0400 Subject: [Diffusion] [Raised Concern] rGHC751996e90a96: Kill off complications in CoreFVs In-Reply-To: References: <20170413033033.23405.80723.728D54DE@phabricator.haskell.org> <1492097045.1059.2.camel@joachim-breitner.de> Message-ID: <1492556832.21788.1.camel@joachim-breitner.de> Hi, Am Dienstag, den 18.04.2017, 22:49 +0000 schrieb Simon Peyton Jones via ghc-devs: > Thanks Joachim.  But rather than giving me a person tutorial every > six months, might it not be more efficient to make the web pages self > explanatory?  Eg by supplying a key, by signalling more clearly that > only commits with significant changes are shown etc? Self-explanatory web-pages would be ideal, and I tried to make it as self-explanatory as my limited UI skills carry me. And before I add documentation (which gets out of date and does not get read anyways), I’d rather refine the UI. The missing commits are already hinted at with the somewhat thick dotted line that replaces then. Maybe it needs to be thicker, and possibly have a tooltip to explain it. Or I should try to recreate how GitHub visually indicates when only part of a diff is shown. Also, I don’t mind explaining it twice a year. This way I at least learn if people are still using the tool :-) Greetings, Joachim -- Joachim “nomeata” Breitner   mail at joachim-breitner.de • https://www.joachim-breitner.de/   XMPP: nomeata at joachim-breitner.de • OpenPGP-Key: 0xF0FBF51F   Debian Developer: nomeata at debian.org -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From moritz.angermann at gmail.com Thu Apr 20 10:15:43 2017 From: moritz.angermann at gmail.com (Moritz Angermann) Date: Thu, 20 Apr 2017 18:15:43 +0800 Subject: GHC cross compilation survey Message-ID: Dear friends, as some of you might have noticed, together with obsidian.systems, I’m trying to make cross compiling GHC a bit easier. To that extend, I’d like to invite you to fill out a short survey[1], so we understand the needs of the community better. Please follow the attached link and fill out the survey! Cheers, Moritz [1]: https://medium.com/@zw3rk/hello-world-a-cross-compilation-survey-890cb95029d7 From moritz.angermann at gmail.com Fri Apr 21 03:29:05 2017 From: moritz.angermann at gmail.com (Moritz Angermann) Date: Fri, 21 Apr 2017 11:29:05 +0800 Subject: GHC Build System & Cabal Message-ID: <6754D022-ABA3-4007-B46D-918C296FD914@gmail.com> Dear friends, I’m currently exploring the possibility to build two versions of base similar to how we build various ways of the rts. The reasons for doing this won’t add much to the discussion. Reading https://ghc.haskell.org/trac/ghc/wiki/Building/Architecture/Idiom/Cabal I find the following: > For the GHC build system we need to extract that metadata and use > it to build the package. which leaves me wondering why that is? Initially I expected we’d just shell out and let cabal do the configuration and building of .cabal’ised packages. But it seems to be the case that we sidestep cabal for building ghc. Can someone shed some light on the reasoning behind this? Cheers, Moritz From mail at joachim-breitner.de Fri Apr 21 03:58:14 2017 From: mail at joachim-breitner.de (Joachim Breitner) Date: Thu, 20 Apr 2017 23:58:14 -0400 Subject: GHC Build System & Cabal In-Reply-To: <6754D022-ABA3-4007-B46D-918C296FD914@gmail.com> References: <6754D022-ABA3-4007-B46D-918C296FD914@gmail.com> Message-ID: <1492747094.22438.1.camel@joachim-breitner.de> Hi, Am Freitag, den 21.04.2017, 11:29 +0800 schrieb Moritz Angermann: > Initially I expected we’d just > shell out and let cabal do the configuration and building of > .cabal’ised > packages.  But it seems to be the case that we sidestep cabal for > building > ghc. one reason, as far as I know, is increased parallelism; "make" can start building parts of ghc when the modules in base they depend on have already been built, even if base itself is not done yet. But you can build base using Cabal if you want to build it yourself, I do that here https://github.com/nomeata/veggies/blob/master/boot.sh#L91 for example. Greetings, Joachim -- Joachim “nomeata” Breitner   mail at joachim-breitner.de • https://www.joachim-breitner.de/   XMPP: nomeata at joachim-breitner.de • OpenPGP-Key: 0xF0FBF51F   Debian Developer: nomeata at debian.org -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From rae at cs.brynmawr.edu Mon Apr 24 01:13:31 2017 From: rae at cs.brynmawr.edu (Richard Eisenberg) Date: Sun, 23 Apr 2017 21:13:31 -0400 Subject: Harbormaster out to sea? Message-ID: I've posted a few diffs today, but Harbormaster seems stalled in checking them: - https://phabricator.haskell.org/D3424 - https://phabricator.haskell.org/D3487 - https://phabricator.haskell.org/D3490 Do you have any insight into this? Thanks! Richard From ben at smart-cactus.org Mon Apr 24 01:57:35 2017 From: ben at smart-cactus.org (Ben Gamari) Date: Sun, 23 Apr 2017 21:57:35 -0400 Subject: Harbormaster out to sea? In-Reply-To: References: Message-ID: <11C63386-6C22-4499-9A6B-51A96205C61D@smart-cactus.org> While I haven't checked, I suspect that the builders are simply busy as I've pushed a number of patches over the last few days. Cheers, - Ben -- Sent from my Android device with K-9 Mail. Please excuse my brevity. -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Mon Apr 24 12:31:04 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Mon, 24 Apr 2017 12:31:04 +0000 Subject: debug output Message-ID: mk/flavours/validate.mk has GhcStage2HcOpts = -O -dcore-lint GhcLibHcOpts = -O -dcore-lint How can I override these in my validate.mk? E.g. GhcLibHcOpts += -ddebug-output does not work. Thanks. One should really be able to override anything. Simon -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at smart-cactus.org Mon Apr 24 13:13:09 2017 From: ben at smart-cactus.org (Ben Gamari) Date: Mon, 24 Apr 2017 09:13:09 -0400 Subject: debug output In-Reply-To: References: Message-ID: <878tmqgd96.fsf@ben-laptop.smart-cactus.org> Simon Peyton Jones via ghc-devs writes: > mk/flavours/validate.mk has > > GhcStage2HcOpts = -O -dcore-lint > > GhcLibHcOpts = -O -dcore-lint > How can I override these in my validate.mk? E.g. > > GhcLibHcOpts += -ddebug-output > does not work. Are you sure this didn't work You added this line to mk/validate.mk, yes? Does mk/are-validating.mk exist? This seems to work for me. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From rae at cs.brynmawr.edu Mon Apr 24 13:29:26 2017 From: rae at cs.brynmawr.edu (Richard Eisenberg) Date: Mon, 24 Apr 2017 09:29:26 -0400 Subject: debug output In-Reply-To: <878tmqgd96.fsf@ben-laptop.smart-cactus.org> References: <878tmqgd96.fsf@ben-laptop.smart-cactus.org> Message-ID: Could the problem be that ghc doesn’t support a -ddebug-output flag? It has -dno-debug-output, but I don’t know of a way to reverse that decision. > On Apr 24, 2017, at 9:13 AM, Ben Gamari wrote: > > Simon Peyton Jones via ghc-devs writes: > >> mk/flavours/validate.mk has >> >> GhcStage2HcOpts = -O -dcore-lint >> >> GhcLibHcOpts = -O -dcore-lint >> How can I override these in my validate.mk? E.g. >> >> GhcLibHcOpts += -ddebug-output >> does not work. > > Are you sure this didn't work You added this line to mk/validate.mk, > yes? Does mk/are-validating.mk exist? This seems to work for me. > > Cheers, > > - Ben > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From astrohavoc at gmail.com Mon Apr 24 13:31:44 2017 From: astrohavoc at gmail.com (Shao Cheng) Date: Mon, 24 Apr 2017 21:31:44 +0800 Subject: Re-compiling wired-in packages Message-ID: Dear friends, I'm trying to implement a new codegen for ghc, starting by using the ghc api to compile modules to STG/Cmm. In order to support importing Prelude, I also need the STG/Cmm representations of wired-in packages like base, ghc-prim, etc, and I guess that means I need to re-compile them. I can roughly think of two approaches: * Load the `ModIface` of wired-in packages, which are tidied Core modules, and convert to STG/Cmm * Get the library sources from ghc repo and launch a regular compilation. I haven't succeeded in either approach. For the first one, I haven't found a way to convert `ModIface` of external package to regular compilation targets; for the second one, regular "cabal installs" won't do the trick, seems compiling wired-in packages require quite some magic. What is the preferred way of re-compiling wired-in packages and retrieving their STG/Cmm representations? Thank a lot. Cheers, Shao Cheng -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Mon Apr 24 13:51:15 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Mon, 24 Apr 2017 13:51:15 +0000 Subject: debug output In-Reply-To: References: <878tmqgd96.fsf@ben-laptop.smart-cactus.org> Message-ID: | Could the problem be that ghc doesn’t support a -ddebug-output flag? | It has -dno-debug-output, but I don’t know of a way to reverse that | decision. Yes, I think that's the problem. I was surprised -- I thought everything had a negation. Simon | -----Original Message----- | From: Richard Eisenberg [mailto:rae at cs.brynmawr.edu] | Sent: 24 April 2017 14:29 | To: Ben Gamari | Cc: Simon Peyton Jones ; ghc-devs | Subject: Re: debug output | | Could the problem be that ghc doesn’t support a -ddebug-output flag? | It has -dno-debug-output, but I don’t know of a way to reverse that | decision. | | > On Apr 24, 2017, at 9:13 AM, Ben Gamari | wrote: | > | > Simon Peyton Jones via ghc-devs writes: | > | >> mk/flavours/validate.mk has | >> | >> GhcStage2HcOpts = -O -dcore-lint | >> | >> GhcLibHcOpts = -O -dcore-lint | >> How can I override these in my validate.mk? E.g. | >> | >> GhcLibHcOpts += -ddebug-output | >> does not work. | > | > Are you sure this didn't work You added this line to mk/validate.mk, | > yes? Does mk/are-validating.mk exist? This seems to work for me. | > | > Cheers, | > | > - Ben | > _______________________________________________ | > ghc-devs mailing list | > ghc-devs at haskell.org | > | https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.h | > askell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc- | devs&data=02%7C01%7Csi | > | monpj%40microsoft.com%7C1e1f7299f4394d74e6ec08d48b15ede7%7C72f988bf86f | > | 141af91ab2d7cd011db47%7C1%7C0%7C636286373683248991&sdata=zsguOlRYp2hn5 | > TYLtU9p5BYxQBVe7eEh9yrFJ8tfMWI%3D&reserved=0 From mail at joachim-breitner.de Mon Apr 24 14:54:59 2017 From: mail at joachim-breitner.de (Joachim Breitner) Date: Mon, 24 Apr 2017 10:54:59 -0400 Subject: Re-compiling wired-in packages In-Reply-To: References: Message-ID: <1493045699.1140.7.camel@joachim-breitner.de> Hi, Am Montag, den 24.04.2017, 21:31 +0800 schrieb Shao Cheng: > I'm trying to implement a new codegen for ghc, starting by using the > ghc api to compile modules to STG/Cmm. In order to support importing > Prelude, I also need the STG/Cmm representations of wired-in packages > like base, ghc-prim, etc, and I guess that means I need to re-compile > them. I did just that recently, see https://github.com/nomeata/veggies for how I did it, in particular boot.sh. > * Load the `ModIface` of wired-in packages, which are tidied Core > modules, and convert to STG/Cmm What won’t work; the interface does not contain all the core, but only those of inlineable functions. > I haven't succeeded in either approach. For the first one, I haven't > found a way to convert `ModIface` of external package to regular > compilation targets; for the second one, regular "cabal installs" > won't do the trick, seems compiling wired-in packages require quite > some magic. Less than you would think! Use "ghc --make Setup.hs && ./Cabal" instead of the cabal tool, though. Greetings, Joachim -- Joachim “nomeata” Breitner   mail at joachim-breitner.de • https://www.joachim-breitner.de/   XMPP: nomeata at joachim-breitner.de • OpenPGP-Key: 0xF0FBF51F   Debian Developer: nomeata at debian.org -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From ben at smart-cactus.org Mon Apr 24 15:11:05 2017 From: ben at smart-cactus.org (Ben Gamari) Date: Mon, 24 Apr 2017 11:11:05 -0400 Subject: debug output In-Reply-To: References: <878tmqgd96.fsf@ben-laptop.smart-cactus.org> Message-ID: <93BA2854-71A0-4819-B8F2-2A7A68145444@smart-cactus.org> On April 24, 2017 9:29:26 AM EDT, Richard Eisenberg wrote: >Could the problem be that ghc doesn’t support a -ddebug-output flag? It >has -dno-debug-output, but I don’t know of a way to reverse that >decision. > >> On Apr 24, 2017, at 9:13 AM, Ben Gamari wrote: >> >> Simon Peyton Jones via ghc-devs writes: >> >>> mk/flavours/validate.mk has >>> >>> GhcStage2HcOpts = -O -dcore-lint >>> >>> GhcLibHcOpts = -O -dcore-lint >>> How can I override these in my validate.mk? E.g. >>> >>> GhcLibHcOpts += -ddebug-output >>> does not work. >> >> Are you sure this didn't work You added this line to mk/validate.mk, >> yes? Does mk/are-validating.mk exist? This seems to work for me. >> >> Cheers, >> >> - Ben >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs Good catch! From astrohavoc at gmail.com Tue Apr 25 11:30:44 2017 From: astrohavoc at gmail.com (Shao Cheng) Date: Tue, 25 Apr 2017 19:30:44 +0800 Subject: Re-compiling wired-in packages Message-ID: Hi, > I did just that recently, see > https://github.com/nomeata/veggies > for how I did it, in particular boot.sh. Thank you for your work! I looked at boot.sh, and tried the "ghc Setup.hs && ./Cabal" approach. It works for ghc-prim & integer-simple, but not base. Since porting gmp to my target platform is hard, I chose integer-simple, and "./Cabal build" produces the following error message for base: Preprocessing library for base-4.10.0.0.. Building library for base-4.10.0.0.. [ 1 of 238] Compiling GHC.IO[boot] ( GHC/IO.hs-boot, dist/build/GHC/IO.o-boot ) [ 2 of 238] Compiling GHC.Stack.Types ( GHC/Stack/Types.hs, dist/build/GHC/Stack/Types.o ) GHC/Stack/Types.hs:57:1: error: Bad interface file: /home/shaocheng/hana/lib/x86_64-linux-ghc-8.3.20170423/integer-simple-0.1.1.1-FieYyHVQ4yaKr3TU2vSN3a/GHC/Integer.hi Something is amiss; requested module integer-simple-0.1.1.1:GHC.Integer differs from name found in the interface file integer-simple:GHC.Integer (if these names look the same, try again with -dppr-debug) | 57 | import GHC.Integer () | ^^^^^^^^^^^^^^^^^^^^^ The compilation for integer-simple didn't produce error, and it is properly registered by ghc-pkg, so I've no idea why there's a bad interface file. Do you have any ideas on possible causes? Thanks a lot. Greetings, Shao Cheng -------------- next part -------------- An HTML attachment was scrubbed... URL: From mail at joachim-breitner.de Tue Apr 25 14:58:33 2017 From: mail at joachim-breitner.de (Joachim Breitner) Date: Tue, 25 Apr 2017 10:58:33 -0400 Subject: Re-compiling wired-in packages In-Reply-To: References: Message-ID: <1493132313.1703.3.camel@joachim-breitner.de> Hi, Am Dienstag, den 25.04.2017, 19:30 +0800 schrieb Shao Cheng: > > I did just that recently, see > > https://github.com/nomeata/veggies > > for how I did it, in particular boot.sh. > > Thank you for your work! I looked at boot.sh, and tried the "ghc > Setup.hs && ./Cabal" approach. It works for ghc-prim & integer- > simple, but not base. Since porting gmp to my target platform is > hard, I chose integer-simple, and "./Cabal build" produces the > following error message for base: > > Preprocessing library for base-4.10.0.0.. > Building library for base-4.10.0.0.. > [  1 of 238] Compiling GHC.IO[boot]     ( GHC/IO.hs-boot, > dist/build/GHC/IO.o-boot ) > [  2 of 238] Compiling GHC.Stack.Types  ( GHC/Stack/Types.hs, > dist/build/GHC/Stack/Types.o ) > > GHC/Stack/Types.hs:57:1: error: >     Bad interface file: /home/shaocheng/hana/lib/x86_64-linux-ghc- > 8.3.20170423/integer-simple-0.1.1.1- > FieYyHVQ4yaKr3TU2vSN3a/GHC/Integer.hi >         Something is amiss; requested module  integer-simple- > 0.1.1.1:GHC.Integer differs from name found in the interface file > integer-simple:GHC.Integer (if these names look the same, try again > with -dppr-debug) >    | > 57 | import GHC.Integer () >    | ^^^^^^^^^^^^^^^^^^^^^ > > The compilation for integer-simple didn't produce error, and it is > properly registered by ghc-pkg, so I've no idea why there's a bad > interface file. Do you have any ideas on possible causes? Thanks a > lot. In short: ghc (the library) will insist on using integer-gmp resp. integer-simple as configured at _build time_ of GHC. So if you want to use integer-simple, you need to build your own GHC with INTEGER_LIBRARY=integer-simple set in mk/build.mk Also see https://ghc.haskell.org/trac/ghc/ticket/13477 Greetings, Joachim -- Joachim “nomeata” Breitner   mail at joachim-breitner.de • https://www.joachim-breitner.de/   XMPP: nomeata at joachim-breitner.de • OpenPGP-Key: 0xF0FBF51F   Debian Developer: nomeata at debian.org -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From amindfv at gmail.com Thu Apr 27 15:18:17 2017 From: amindfv at gmail.com (amindfv at gmail.com) Date: Thu, 27 Apr 2017 10:18:17 -0500 Subject: Constraint solver iterations Message-ID: To build the 'vivid' package (on Hackage) with recent versions of GHC (I believe the problem is with 8.0+, didn't see it before then), I need to compile with -fconstraint-solver-iterations=0. (0 meaning unbounded) However, I noticed today that the GHC docs say "Typically one iteration suffices; so please yell if you find you need to set it higher than the default." The code (in Vivid.SynthDef.FromUA) uses a large amount of type family recursion. Is this not a case where I should be yelling? If I should be, I can provide a smaller repro case. If this is expected, are there techniques for rewriting the code in a way that'll fit under the default number of iterations? I've tried rewriting, making the number of iterations logarithmic instead of linear (e.g. by appending lists instead of cons-ing), but still hit the problem (even with an increased but bounded number of iteration steps). Thanks! Tom From simonpj at microsoft.com Thu Apr 27 14:25:22 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Thu, 27 Apr 2017 14:25:22 +0000 Subject: Constraint solver iterations In-Reply-To: References: Message-ID: I'd love to see the repo case. with -ddump-simpl-iterations -ddump-simpl-stats you'll see the result after each iteration, and what was done. I bet you get to the point where only one thing happens per iteration and that's bad. Make a ticket! Thanks Simon | -----Original Message----- | From: ghc-devs [mailto:ghc-devs-bounces at haskell.org] On Behalf Of | amindfv at gmail.com | Sent: 27 April 2017 16:18 | To: ghc-devs at haskell.org | Subject: Constraint solver iterations | | To build the 'vivid' package (on Hackage) with recent versions of GHC | (I believe the problem is with 8.0+, didn't see it before then), I | need to compile with -fconstraint-solver-iterations=0. (0 meaning | unbounded) | | However, I noticed today that the GHC docs say "Typically one | iteration suffices; so please yell if you find you need to set it | higher than the default." | | The code (in Vivid.SynthDef.FromUA) uses a large amount of type family | recursion. Is this not a case where I should be yelling? If I should | be, I can provide a smaller repro case. | | If this is expected, are there techniques for rewriting the code in a | way that'll fit under the default number of iterations? I've tried | rewriting, making the number of iterations logarithmic instead of | linear (e.g. by appending lists instead of cons-ing), but still hit | the problem (even with an increased but bounded number of iteration | steps). | | Thanks! | Tom | _______________________________________________ | ghc-devs mailing list | ghc-devs at haskell.org | https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.h | askell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc- | devs&data=02%7C01%7Csimonpj%40microsoft.com%7C897af52446cd4b1cefd608d4 | 8d785fa6%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6362889955277201 | 33&sdata=POYwck2lKRtnKCfC%2FNAASG8ZBvm2lFiuFaCCDminogc%3D&reserved=0 From maoe at foldr.in Thu Apr 27 20:26:58 2017 From: maoe at foldr.in (Mitsutoshi Aoe) Date: Fri, 28 Apr 2017 05:26:58 +0900 Subject: make install fails on master on macOS Sierra Message-ID: Hi ghc devs, I'm trying to build recent master (579bb766) on macOS Sierra but make install fails as follows: "/usr/local/ghc/ghc-HEAD/lib/ghc-8.3.20170426/bin/ghc-pkg" --force --global-package-db "/usr/local/ghc/ghc-HEAD/lib/ghc-8.3.20170426/package.conf.d" update rts/dist/package.conf.install Reading package info from "rts/dist/package.conf.install" ... done. : Warning: Unrecognized field exposed-modules on line 708 : Warning: Unrecognized field key on line 703 : Warning: Unrecognized field id on line 702 : Warning: Unrecognized field version on line 701 : Warning: Unrecognized field name on line 700 : invalid package identifier: : missing id field make[1]: *** [install_packages] Error 1 make: *** [install] Error 2 The package.conf.install in question is available here: https://gist.github.com/maoe/2cff85004ef8765937023425c53f2cc7. And my build.mk is here: https://gist.github.com/maoe/caee7b2dfd921def4e95580c202cd664 I can't reproduce this error on Linux. Does anyone know how to fix it? Regards, Mitsutoshi -------------- next part -------------- An HTML attachment was scrubbed... URL: From mle+hs at mega-nerd.com Thu Apr 27 21:11:06 2017 From: mle+hs at mega-nerd.com (Erik de Castro Lopo) Date: Fri, 28 Apr 2017 07:11:06 +1000 Subject: make install fails on master on macOS Sierra In-Reply-To: References: Message-ID: <20170428071106.1bde5fdf29181722d7e5a38b@mega-nerd.com> Mitsutoshi Aoe wrote: > I'm trying to build recent master (579bb766) on macOS Sierra but make install fails as follows: > > "/usr/local/ghc/ghc-HEAD/lib/ghc-8.3.20170426/bin/ghc-pkg" --force --global-package-db "/usr/local/ghc/ghc-HEAD/lib/ghc-8.3.20170426/package.conf.d" update rts/dist/package.conf.install > Reading package info from "rts/dist/package.conf.install" ... done. > : Warning: Unrecognized field exposed-modules on line 708 > : Warning: Unrecognized field key on line 703 > : Warning: Unrecognized field id on line 702 > : Warning: Unrecognized field version on line 701 > : Warning: Unrecognized field name on line 700 > : invalid package identifier: > : missing id field > make[1]: *** [install_packages] Error 1 > make: *** [install] Error 2 I *think* that may be an issue with using an older version of cabal. It should be 1.24.*. Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/ From simonpj at microsoft.com Fri Apr 28 08:49:54 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Fri, 28 Apr 2017 08:49:54 +0000 Subject: Pushing to nofib Message-ID: How can I push to the nofib repository? I get this: simonpj at cam-05-unx:~/code/HEAD-4/nofib$ git push fatal: remote error: access denied or repository not exported: /nofib.git Thanks Simon -------------- next part -------------- An HTML attachment was scrubbed... URL: From maoe at foldr.in Fri Apr 28 09:10:14 2017 From: maoe at foldr.in (Mitsutoshi Aoe) Date: Fri, 28 Apr 2017 18:10:14 +0900 Subject: make install fails on master on macOS Sierra In-Reply-To: <20170428071106.1bde5fdf29181722d7e5a38b@mega-nerd.com> References: <20170428071106.1bde5fdf29181722d7e5a38b@mega-nerd.com> Message-ID: Hi Erik, Thanks for the info. Is there a way to tell what version of Cabal was used to build GHC? I've tried the latest cabal-install from Hackage and from master on GitHub with no luck. Also the Cabal git submodule in the GHC checkout is pointed to 41f416bc2, which seems to be Cabal-2.0.0.0. Regards, Mitsutoshi On Apr 28, 2017 06:11 +0900, Erik de Castro Lopo , wrote: > Mitsutoshi Aoe wrote: > > > I'm trying to build recent master (579bb766) on macOS Sierra but make install fails as follows: > > > > "/usr/local/ghc/ghc-HEAD/lib/ghc-8.3.20170426/bin/ghc-pkg" --force --global-package-db "/usr/local/ghc/ghc-HEAD/lib/ghc-8.3.20170426/package.conf.d" update rts/dist/package.conf.install > > Reading package info from "rts/dist/package.conf.install" ... done. > > : Warning: Unrecognized field exposed-modules on line 708 > > : Warning: Unrecognized field key on line 703 > > : Warning: Unrecognized field id on line 702 > > : Warning: Unrecognized field version on line 701 > > : Warning: Unrecognized field name on line 700 > > : invalid package identifier: > > : missing id field > > make[1]: *** [install_packages] Error 1 > > make: *** [install] Error 2 > > I *think* that may be an issue with using an older version of cabal. > It should be 1.24.*. > > Erik > -- > ---------------------------------------------------------------------- > Erik de Castro Lopo > http://www.mega-nerd.com/ > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From mle+hs at mega-nerd.com Fri Apr 28 09:58:46 2017 From: mle+hs at mega-nerd.com (Erik de Castro Lopo) Date: Fri, 28 Apr 2017 19:58:46 +1000 Subject: make install fails on master on macOS Sierra In-Reply-To: References: <20170428071106.1bde5fdf29181722d7e5a38b@mega-nerd.com> Message-ID: <20170428195846.a266171844b6b7e14424c6f6@mega-nerd.com> Mitsutoshi Aoe wrote: > Hi Erik, > > Thanks for the info. Is there a way to tell what version of Cabal was used > to build GHC? I actually don't think that's relevant. > I've tried the latest cabal-install from Hackage and from master on GitHub > with no luck. Either of those should be fine. > Also the Cabal git submodule in the GHC checkout is pointed to 41f416bc2, > which seems to be Cabal-2.0.0.0. That is correct (and also not relevant). Since you are using the correct version of cabal, my guess at the problem being with the version of cabal is almost certainly wrong. At this point I would suggest doing a "make clean", then running the bootstrap script and configure again before rebuilding. Cheers, Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/ From ben at smart-cactus.org Fri Apr 28 13:07:32 2017 From: ben at smart-cactus.org (Ben Gamari) Date: Fri, 28 Apr 2017 09:07:32 -0400 Subject: Pushing to nofib In-Reply-To: References: Message-ID: <87efwcfzor.fsf@ben-laptop.smart-cactus.org> Simon Peyton Jones via ghc-devs writes: > How can I push to the nofib repository? I get this: > What does git remote -v say when run in the nofib/ directory? Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From kavon at farvard.in Fri Apr 28 13:27:40 2017 From: kavon at farvard.in (Kavon Farvardin) Date: Fri, 28 Apr 2017 14:27:40 +0100 Subject: NCG lowering of sqrt Message-ID: <336DB2F8-0FA8-4AFD-A05A-832C7AEE2E81@farvard.in> Given a Cmm expression such as (_c8Gq::F64) = call MO_F64_Sqrt(_s8oX::F64); // CmmUnsafeForeignCall the native code generator produces an actual call to the sqrt C function, which has the side-effect of causing all floating-point registers to be dumped as they are caller-saved. In the nbody benchmark, this is particularly bad for a rather hot piece of code (see below). Ideally the NCG would recognize this foreign call and instead use the `sqrtsd` SSE instruction when targeting x86-64. Does anyone know if the NCG can produce this instruction? I think it would be beneficial, as the below would turn into one or two instructions. Other math functions such as sin/cos require x87 FPU instructions, which as far as I know we're not using. ;;;;;;;;;;; ; NCG generates this in parts of the nbody benchmark ; to compute the sqrt ; subq $8,%rsp movsd %xmm9,176(%rsp) ; all floating-point registers movsd %xmm1,184(%rsp) ; are caller-saved in SysV ABI movsd %xmm2,192(%rsp) movsd %xmm3,200(%rsp) movq %rdi,208(%rsp) movq %rcx,216(%rsp) movq %rsi,224(%rsp) movsd %xmm4,232(%rsp) movsd %xmm5,240(%rsp) movsd %xmm6,248(%rsp) movsd %xmm7,256(%rsp) movsd %xmm8,264(%rsp) movsd %xmm11,272(%rsp) call _sqrt ;; the loads ;; below are interleaved ;; with computations addq $8,%rsp movsd 264(%rsp),%xmm1 movsd 240(%rsp),%xmm2 movsd 224(%rsp),%xmm2 movsd 232(%rsp),%xmm4 movq 200(%rsp),%rax movsd 248(%rsp),%xmm4 movsd 256(%rsp),%xmm4 movq 216(%rsp),%rcx movsd 192(%rsp),%xmm2 ~kavon From fryguybob at gmail.com Fri Apr 28 14:42:19 2017 From: fryguybob at gmail.com (Ryan Yates) Date: Fri, 28 Apr 2017 10:42:19 -0400 Subject: NCG lowering of sqrt In-Reply-To: <336DB2F8-0FA8-4AFD-A05A-832C7AEE2E81@farvard.in> References: <336DB2F8-0FA8-4AFD-A05A-832C7AEE2E81@farvard.in> Message-ID: Hi Kavon, I looked a bit and it does not appear that there is an SSE sqrt in the native code gen. It should be easy to add (see a similar addition here: https://phabricator.haskell.org/D3265). The x87 version was available for 32-bit. I think if you use the LLVM backend it will give you the SSE sqrt. Ryan On Fri, Apr 28, 2017 at 9:27 AM, Kavon Farvardin wrote: > Given a Cmm expression such as > > (_c8Gq::F64) = call MO_F64_Sqrt(_s8oX::F64); // CmmUnsafeForeignCall > > the native code generator produces an actual call to the sqrt C function, > which has the side-effect of causing all floating-point registers to be > dumped as they are caller-saved. In the nbody benchmark, this is > particularly bad for a rather hot piece of code (see below). > > Ideally the NCG would recognize this foreign call and instead use the > `sqrtsd` SSE instruction when targeting x86-64. > > Does anyone know if the NCG can produce this instruction? I think it would > be beneficial, as the below would turn into one or two instructions. > > Other math functions such as sin/cos require x87 FPU instructions, which > as far as I know we're not using. > > > ;;;;;;;;;;; > ; NCG generates this in parts of the nbody benchmark > ; to compute the sqrt > ; > subq $8,%rsp > movsd %xmm9,176(%rsp) ; all floating-point registers > movsd %xmm1,184(%rsp) ; are caller-saved in SysV ABI > movsd %xmm2,192(%rsp) > movsd %xmm3,200(%rsp) > movq %rdi,208(%rsp) > movq %rcx,216(%rsp) > movq %rsi,224(%rsp) > movsd %xmm4,232(%rsp) > movsd %xmm5,240(%rsp) > movsd %xmm6,248(%rsp) > movsd %xmm7,256(%rsp) > movsd %xmm8,264(%rsp) > movsd %xmm11,272(%rsp) > call _sqrt > ;; the loads > ;; below are interleaved > ;; with computations > addq $8,%rsp > movsd 264(%rsp),%xmm1 > movsd 240(%rsp),%xmm2 > movsd 224(%rsp),%xmm2 > movsd 232(%rsp),%xmm4 > movq 200(%rsp),%rax > movsd 248(%rsp),%xmm4 > movsd 256(%rsp),%xmm4 > movq 216(%rsp),%rcx > movsd 192(%rsp),%xmm2 > > > ~kavon > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at smart-cactus.org Fri Apr 28 15:01:26 2017 From: ben at smart-cactus.org (Ben Gamari) Date: Fri, 28 Apr 2017 11:01:26 -0400 Subject: NCG lowering of sqrt In-Reply-To: References: <336DB2F8-0FA8-4AFD-A05A-832C7AEE2E81@farvard.in> Message-ID: <87a870fuex.fsf@ben-laptop.smart-cactus.org> Ryan Yates writes: > Hi Kavon, > > I looked a bit and it does not appear that there is an SSE sqrt in the > native code gen. It should be easy to add (see a similar addition here: > https://phabricator.haskell.org/D3265). The x87 version was available for > 32-bit. I think if you use the LLVM backend it will give you the SSE sqrt. > Indeed. I pushed a starting point to D3508; I haven't validated it but there's a chance it will work. If not it would be great if someone could pick it up. Otherwise I'll return to it when I have time. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From simonpj at microsoft.com Fri Apr 28 15:47:04 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Fri, 28 Apr 2017 15:47:04 +0000 Subject: Pushing to nofib In-Reply-To: <87efwcfzor.fsf@ben-laptop.smart-cactus.org> References: <87efwcfzor.fsf@ben-laptop.smart-cactus.org> Message-ID: git remote -v origin git://git.haskell.org/nofib.git (fetch) origin git://git.haskell.org/nofib.git (push) simonpj at cam-05-unx:~/code/HEAD-4/nofib$ | -----Original Message----- | From: Ben Gamari [mailto:ben at smart-cactus.org] | Sent: 28 April 2017 14:08 | To: Simon Peyton Jones ; ghc-devs at haskell.org | Subject: Re: Pushing to nofib | | Simon Peyton Jones via ghc-devs writes: | | > How can I push to the nofib repository? I get this: | > | What does | | git remote -v | | say when run in the nofib/ directory? | | Cheers, | | - Ben From ben at smart-cactus.org Fri Apr 28 16:46:38 2017 From: ben at smart-cactus.org (Ben Gamari) Date: Fri, 28 Apr 2017 12:46:38 -0400 Subject: Pushing to nofib In-Reply-To: References: <87efwcfzor.fsf@ben-laptop.smart-cactus.org> Message-ID: <874lx8fpjl.fsf@ben-laptop.smart-cactus.org> Simon Peyton Jones writes: > git remote -v > origin git://git.haskell.org/nofib.git (fetch) > origin git://git.haskell.org/nofib.git (push) > simonpj at cam-05-unx:~/code/HEAD-4/nofib$ > Indeed that is the issue. git:// URLs are read-only. You'll need to add a new remote, git remote add upstream git at git.haskell.org:nofib Then you can push with, git push upstream Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From maoe at foldr.in Fri Apr 28 21:37:52 2017 From: maoe at foldr.in (Mitsutoshi Aoe) Date: Sat, 29 Apr 2017 06:37:52 +0900 Subject: make install fails on master on macOS Sierra In-Reply-To: <20170428195846.a266171844b6b7e14424c6f6@mega-nerd.com> References: <20170428071106.1bde5fdf29181722d7e5a38b@mega-nerd.com> <20170428195846.a266171844b6b7e14424c6f6@mega-nerd.com> Message-ID: <2e093e79-cac9-4815-bd86-958830c1f87b@Spark> > At this point I would suggest doing a "make clean", then running the > bootstrap script and configure again before rebuilding. I just tried building from a clean checkout with the same build.mk. ./boot ./configure --prefix=/usr/local/ghc/ghc-HEAD make -j4 make install This resulted in the same error: "/usr/local/ghc/ghc-HEAD/lib/ghc-8.3.20170428/bin/ghc-pkg" --force --global-package-db "/usr/local/ghc/ghc-HEAD/lib/ghc-8.3.20170428/package.conf.d" update rts/dist/package.conf.install Reading package info from "rts/dist/package.conf.install" ... done. : Warning: Unrecognized field exposed-modules on line 708 : Warning: Unrecognized field key on line 703 : Warning: Unrecognized field id on line 702 : Warning: Unrecognized field version on line 701 : Warning: Unrecognized field name on line 700 : invalid package identifier: : missing id field make[1]: *** [install_packages] Error 1 make: *** [install] Error 2 My cabal is 2.1.0.0 % cabal --version cabal-install version 2.1.0.0 compiled using version 2.1.0.0 of the Cabal library Regards, Mitsutoshi On Apr 28, 2017 18:59 +0900, Erik de Castro Lopo , wrote: > Mitsutoshi Aoe wrote: > > > Hi Erik, > > > > Thanks for the info. Is there a way to tell what version of Cabal was used > > to build GHC? > > I actually don't think that's relevant. > > > I've tried the latest cabal-install from Hackage and from master on GitHub > > with no luck. > > Either of those should be fine. > > > Also the Cabal git submodule in the GHC checkout is pointed to 41f416bc2, > > which seems to be Cabal-2.0.0.0. > > That is correct (and also not relevant). > > Since you are using the correct version of cabal, my guess at the problem > being with the version of cabal is almost certainly wrong. > > At this point I would suggest doing a "make clean", then running the > bootstrap script and configure again before rebuilding. > > Cheers, > Erik > -- > ---------------------------------------------------------------------- > Erik de Castro Lopo > http://www.mega-nerd.com/ > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: