From ben at well-typed.com Wed May 4 10:49:20 2016 From: ben at well-typed.com (Ben Gamari) Date: Wed, 04 May 2016 12:49:20 +0200 Subject: [Maintainer needed] Actively maintained Haddock are happy Haddock Message-ID: <874mae6qtr.fsf@smart-cactus.org> tl;dr. Our Haddock need a dedicated maintainer responsible for their care and feeding. Would you like to move forward a critical piece of infrastructure used on a daily basis by thousands of Haskellers? Let us know if so. Hello everyone, While for several years Mateusz Kowalczyk and Simon Hengel have done an admirable job in serving as comaintainers of the Haddock documentation tool, they have both indicated that they no longer have the time to continue to be active in this role. They both deserve our thanks for their service. While Matthew Pickering and I have tried to pick up some of the Haddock's maintenance load in the meantime, we really need a dedicated maintainer to ensure that Haddock continues to swim along healthfully. At the moment there are over 150 open tickets in the issue tracker [1], an open call for a stable release, and 4 patches in need of review. We currently lack the time to get on top of such a load. In particular, Haddock need someone to take ownership of the management of the project: this person would keep an eye on (and preferably pare down) the issue tracker (which typically isn't very active, happily), guide new contributors, and manage releases. This is an opportunity to drive forward a project used on a daily basis by a significant fraction of Haskellers. If you think you might be interested in serving in this role, please let us know. Thanks! Cheers, - Ben [1] https://github.com/haskell/haddock/pulls -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 472 bytes Desc: not available URL: From ben at well-typed.com Thu May 5 12:52:45 2016 From: ben at well-typed.com (Ben Gamari) Date: Thu, 05 May 2016 14:52:45 +0200 Subject: Looking for GHC compile-time performance tests Message-ID: <87r3dg650i.fsf@smart-cactus.org> tl;dr. Do you have a Haskell project which GHC now compiles more slowly than in the past? If you would like to see your program's compilation time improve in 8.2, reduce it to a minimal testcase, and provide it to us via Trac. Good news everyone, GHC 8.2 will be a release largely targetting consolidation instead of new features. In particular, we will focus on compiler performance issues. One of the challenges of this task is knowing in which ways GHC has regressed. It is currently hard to know how much of the apparent compiler performance regression is due to GHC getting slower and how much is due to users demanding more of it. To better understand the performance of our compiler, we would like to collect modern Haskell code samples that exhibit compile-time performance issues (either regressions or otherwise). For this we are asking for your help: we are looking for projects that, * are small-to-moderate-sized Haskell programs which you believe have regressed in compile time in the last few GHC releases * have minimal build dependencies, in the ideal case none outside of the standard GHC boot libraries * can be licensed for use under 3-clause BSD In some cases we may also be able to make use of examples with a few library dependencies that are necessary to demonstrate the performance issue (for instance, a suspected issue stemming from stream fusion requiring Conduit). For performance issues exhibiting non-linear scaling in some quantity (e.g. the size of a type in the program), it would be extremely helpful if the testcase exposed a "knob" demonstrating this scaling. So, if you would like to see your program's compilation time improve in GHC 8.2, put some time into reducing it to something minimal, submit it to us via a Trac ticket, and let us know in this thread. Thanks for your help! Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 472 bytes Desc: not available URL: From post at volker-wysk.de Thu May 5 19:19:01 2016 From: post at volker-wysk.de (Volker Wysk) Date: Thu, 05 May 2016 21:19:01 +0200 Subject: What happened to -optdep option? Message-ID: <2445391.fvxYkY92mg@desktop> Hello! I'm using GHC 7.10.3 after an upgrade. I have the following in my Makefile: depend_mod :: lib/abh ghc -M $(CFLAGS_GHC_0) -dep-makefile -optdepbuild/depend.tmp -dep-suffix "p_" \ $(foreach m, $(MOD_HS) $(PROG_HS), src/$(m).hs) \ $(foreach m, $(MOD_CHS) $(PROG_CHS), build/$(m).hs) ... This used to work. The previous GHC version was 7.8. Now GHC complains: -optdepbuild/depend.tmp: openBinaryFile: does not exist (No such file or directory) This looks like the -optdep argument isn't understood. There is no mention of it in the new/7.10.3 user manual. I don't know any longer what this "-optget" argument does, and it isn't in the documentation any longer. I googled the GHC web site. There it is mentioned that -optdep was deprecated, but almost all of the hits are very outdated (like ten years old). Can soneone tell me, how I should change my Makefile..? Greetings, Volker Wysk From mle+hs at mega-nerd.com Thu May 5 20:22:48 2016 From: mle+hs at mega-nerd.com (Erik de Castro Lopo) Date: Fri, 6 May 2016 06:22:48 +1000 Subject: Looking for GHC compile-time performance tests In-Reply-To: <87r3dg650i.fsf@smart-cactus.org> References: <87r3dg650i.fsf@smart-cactus.org> Message-ID: <20160506062248.08901f26a4e91ac410a5ce29@mega-nerd.com> Ben Gamari wrote: > So, if you would like to see your program's compilation time improve > in GHC 8.2, put some time into reducing it to something minimal, submit > it to us via a Trac ticket, and let us know in this thread. The vector package is probably a good candidate. Compling vector slowed down signicantly between 7.8 and 7.10, only to speed up again with 8.0. Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/ From ekmett at gmail.com Thu May 5 20:49:41 2016 From: ekmett at gmail.com (Edward Kmett) Date: Fri, 6 May 2016 06:49:41 +1000 Subject: Looking for GHC compile-time performance tests In-Reply-To: <20160506062248.08901f26a4e91ac410a5ce29@mega-nerd.com> References: <87r3dg650i.fsf@smart-cactus.org> <20160506062248.08901f26a4e91ac410a5ce29@mega-nerd.com> Message-ID: vector-algorithms has gotten slower to both compile and for users rather consistently during each release throughout the 7.x lifecycle. That may serve as a good torture test as well. > On May 6, 2016, at 6:22 AM, Erik de Castro Lopo wrote: > > Ben Gamari wrote: > >> So, if you would like to see your program's compilation time improve >> in GHC 8.2, put some time into reducing it to something minimal, submit >> it to us via a Trac ticket, and let us know in this thread. > > The vector package is probably a good candidate. Compling vector slowed > down signicantly between 7.8 and 7.10, only to speed up again with 8.0. > > Erik > -- > ---------------------------------------------------------------------- > Erik de Castro Lopo > http://www.mega-nerd.com/ > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users From david.feuer at gmail.com Thu May 5 21:02:06 2016 From: david.feuer at gmail.com (David Feuer) Date: Thu, 5 May 2016 17:02:06 -0400 Subject: Looking for GHC compile-time performance tests In-Reply-To: <20160506062248.08901f26a4e91ac410a5ce29@mega-nerd.com> References: <87r3dg650i.fsf@smart-cactus.org> <20160506062248.08901f26a4e91ac410a5ce29@mega-nerd.com> Message-ID: containers compile times have generally gotten slower from version to version. On Thu, May 5, 2016 at 4:22 PM, Erik de Castro Lopo wrote: > Ben Gamari wrote: > >> So, if you would like to see your program's compilation time improve >> in GHC 8.2, put some time into reducing it to something minimal, submit >> it to us via a Trac ticket, and let us know in this thread. > > The vector package is probably a good candidate. Compling vector slowed > down signicantly between 7.8 and 7.10, only to speed up again with 8.0. > > Erik > -- > ---------------------------------------------------------------------- > Erik de Castro Lopo > http://www.mega-nerd.com/ > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users From simonpj at microsoft.com Thu May 5 21:08:25 2016 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Thu, 5 May 2016 21:08:25 +0000 Subject: Looking for GHC compile-time performance tests In-Reply-To: References: <87r3dg650i.fsf@smart-cactus.org> <20160506062248.08901f26a4e91ac410a5ce29@mega-nerd.com> Message-ID: <2f2663dbeb664a1595311b1540004f27@DB4PR30MB030.064d.mgd.msft.net> Thanks. A repeatable test case would be incredibly helpful here. Simon | -----Original Message----- | From: Glasgow-haskell-users [mailto:glasgow-haskell-users- | bounces at haskell.org] On Behalf Of Edward Kmett | Sent: 05 May 2016 21:50 | To: Erik de Castro Lopo | Cc: glasgow-haskell-users at haskell.org | Subject: Re: Looking for GHC compile-time performance tests | | vector-algorithms has gotten slower to both compile and for users rather | consistently during each release throughout the 7.x lifecycle. That may serve | as a good torture test as well. | | > On May 6, 2016, at 6:22 AM, Erik de Castro Lopo | wrote: | > | > Ben Gamari wrote: | > | >> So, if you would like to see your program's compilation time improve | >> in GHC 8.2, put some time into reducing it to something minimal, submit | >> it to us via a Trac ticket, and let us know in this thread. | > | > The vector package is probably a good candidate. Compling vector slowed | > down signicantly between 7.8 and 7.10, only to speed up again with 8.0. | > | > Erik | > -- | > ---------------------------------------------------------------------- | > Erik de Castro Lopo | > https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.mega- | nerd.com%2f&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com%7c502efb8a671d48e | 03c2408d37526cd98%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=NO6ktHARs7KWr% | 2fFR9DJB3rAum%2bLblzbVFc5AeBi8Smg%3d | > _______________________________________________ | > Glasgow-haskell-users mailing list | > Glasgow-haskell-users at haskell.org | > | https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fmail.haskell. | org%2fcgi-bin%2fmailman%2flistinfo%2fglasgow-haskell- | users&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com%7c502efb8a671d48e03c240 | 8d37526cd98%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=f%2bspd5cb57tq15z4vF | TrUe2RPC2SXY6SAu9MPW8fYfw%3d | _______________________________________________ | Glasgow-haskell-users mailing list | Glasgow-haskell-users at haskell.org | https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fmail.haskell. | org%2fcgi-bin%2fmailman%2flistinfo%2fglasgow-haskell- | users&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com%7c502efb8a671d48e03c240 | 8d37526cd98%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=f%2bspd5cb57tq15z4vF | TrUe2RPC2SXY6SAu9MPW8fYfw%3d From carter.schonwald at gmail.com Thu May 5 21:32:21 2016 From: carter.schonwald at gmail.com (Carter Schonwald) Date: Thu, 5 May 2016 17:32:21 -0400 Subject: Looking for GHC compile-time performance tests In-Reply-To: <20160506062248.08901f26a4e91ac410a5ce29@mega-nerd.com> References: <87r3dg650i.fsf@smart-cactus.org> <20160506062248.08901f26a4e91ac410a5ce29@mega-nerd.com> Message-ID: It also helped that the test suite got added back to the main cabal. And the test suite took a WHILE, due to some sort of coercion seize blowup that I think was fixed in 8,0. But yes. Definitely vector build times decayed. Partly due to the shift to the general bundle rep, and partly due to ... Ghc builds getting slower On Thursday, May 5, 2016, Erik de Castro Lopo wrote: > Ben Gamari wrote: > > > So, if you would like to see your program's compilation time improve > > in GHC 8.2, put some time into reducing it to something minimal, submit > > it to us via a Trac ticket, and let us know in this thread. > > The vector package is probably a good candidate. Compling vector slowed > down signicantly between 7.8 and 7.10, only to speed up again with 8.0. > > Erik > -- > ---------------------------------------------------------------------- > Erik de Castro Lopo > http://www.mega-nerd.com/ > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bertram.felgenhauer at googlemail.com Thu May 5 22:35:01 2016 From: bertram.felgenhauer at googlemail.com (Bertram Felgenhauer) Date: Fri, 6 May 2016 00:35:01 +0200 Subject: What happened to -optdep option? In-Reply-To: <2445391.fvxYkY92mg@desktop> References: <2445391.fvxYkY92mg@desktop> Message-ID: <20160505223501.GA22271@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> Volker Wysk wrote: > Hello! > > I'm using GHC 7.10.3 after an upgrade. I have the following in my Makefile: > > depend_mod :: lib/abh > ghc -M $(CFLAGS_GHC_0) -dep-makefile -optdepbuild/depend.tmp -dep-suffix "p_" You should drop the -optdep here > \ > $(foreach m, $(MOD_HS) $(PROG_HS), src/$(m).hs) \ > $(foreach m, $(MOD_CHS) $(PROG_CHS), build/$(m).hs) > ... > > This used to work. The previous GHC version was 7.8. As a side effect of finally removing the -optdep-f flag, https://ghc.haskell.org/trac/ghc/ticket/2773 (another relevant commit is 994d5e3cfc32cb624b63683205dd8fd876171d0c) the code that strips -optdep from the filename, which was required to make -optdep-f -optdepfoo work, also got removed. The fact that -dep-makefile continued to strip -optdep was an accident in my view. Cheers, Bertram From post at volker-wysk.de Fri May 6 00:49:46 2016 From: post at volker-wysk.de (Volker Wysk) Date: Fri, 06 May 2016 02:49:46 +0200 Subject: What happened to -optdep option? In-Reply-To: <20160505223501.GA22271@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> References: <2445391.fvxYkY92mg@desktop> <20160505223501.GA22271@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> Message-ID: <4617413.LSK1WyEhSI@desktop> Hello! Am Freitag, 6. Mai 2016, 00:35:01 CEST schrieb Bertram Felgenhauer: > Volker Wysk wrote: > > Hello! > > > > I'm using GHC 7.10.3 after an upgrade. I have the following in my > > Makefile: > > > > depend_mod :: lib/abh > > > > ghc -M $(CFLAGS_GHC_0) -dep-makefile -optdepbuild/depend.tmp -dep-suffix > > "p_" > You should drop the -optdep here > > > \ > > > > $(foreach m, $(MOD_HS) $(PROG_HS), src/$(m).hs) \ > > $(foreach m, $(MOD_CHS) $(PROG_CHS), build/$(m).hs) > > > > ... > > > > This used to work. The previous GHC version was 7.8. > > As a side effect of finally removing the -optdep-f flag, > > https://ghc.haskell.org/trac/ghc/ticket/2773 > (another relevant commit is 994d5e3cfc32cb624b63683205dd8fd876171d0c) > > the code that strips -optdep from the filename, which was required to > make -optdep-f -optdepfoo work, also got removed. The fact that > -dep-makefile continued to strip -optdep was an accident in my view. I've fixed it by removing "-optdepbuild/depend.tmp" and replacing "-dep- makefile" with "-dep-makefile build/depend.tmp". Thanks. Volker From mle+hs at mega-nerd.com Fri May 6 09:04:13 2016 From: mle+hs at mega-nerd.com (Erik de Castro Lopo) Date: Fri, 6 May 2016 19:04:13 +1000 Subject: RFC: Removing the `-hb` profiling option Message-ID: <20160506190413.acc749602f72e83ce3f9420a@mega-nerd.com> Hi all, After a bit of rather tedious and frustrating debugging I came to the realisation that the code for the `-hb` profiling option is not thread safe. See https://ghc.haskell.org/trac/ghc/ticket/12019 This gives us an opportunity to simply remove it instead of fixing it. If there is anyone that thinks this future is really useful (ie more useful than the other profiling modes) then I'm willing to fix it. But if noone would miss it. I'd much rather remove it. Thoughts? Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/ From iavor.diatchki at gmail.com Fri May 6 16:41:04 2016 From: iavor.diatchki at gmail.com (Iavor Diatchki) Date: Fri, 6 May 2016 09:41:04 -0700 Subject: RFC: Removing the `-hb` profiling option In-Reply-To: <20160506190413.acc749602f72e83ce3f9420a@mega-nerd.com> References: <20160506190413.acc749602f72e83ce3f9420a@mega-nerd.com> Message-ID: I think that biographical profiling is quite nice! I wouldn't say that it is *more* useful than other modes of profiling, but it is certainly complementary, and helps give you an idea of what's going on. So I'd very much vote for fixing it rather than removing it. -Iavor On Fri, May 6, 2016 at 2:04 AM, Erik de Castro Lopo wrote: > Hi all, > > After a bit of rather tedious and frustrating debugging I came to the > realisation that the code for the `-hb` profiling option is not thread > safe. See https://ghc.haskell.org/trac/ghc/ticket/12019 > > This gives us an opportunity to simply remove it instead of fixing it. > If there is anyone that thinks this future is really useful (ie more > useful than the other profiling modes) then I'm willing to fix it. > But if noone would miss it. I'd much rather remove it. > > Thoughts? > > Erik > -- > ---------------------------------------------------------------------- > Erik de Castro Lopo > http://www.mega-nerd.com/ > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qdunkan at gmail.com Fri May 6 16:43:25 2016 From: qdunkan at gmail.com (Evan Laforge) Date: Fri, 6 May 2016 09:43:25 -0700 Subject: RFC: Removing the `-hb` profiling option In-Reply-To: <20160506190413.acc749602f72e83ce3f9420a@mega-nerd.com> References: <20160506190413.acc749602f72e83ce3f9420a@mega-nerd.com> Message-ID: I've used this a lot when looking for leaks. Especially combined with other things like -hbdrag -hc, so I'd be sad to see it go. Without it, how do you find lag and drag? On the other hand, the entire profiling system has been hard to use because of crashes (perhaps due to that thread-unsafety thing), but also because of lack of documentation, the fact that SCCs can drastically change the performance of the things they are trying to measure, and probably just personal incompetence. But that's not really directly relevant to -hb, except to say I'd rather fix it than delete it. On Fri, May 6, 2016 at 2:04 AM, Erik de Castro Lopo wrote: > Hi all, > > After a bit of rather tedious and frustrating debugging I came to the > realisation that the code for the `-hb` profiling option is not thread > safe. See https://ghc.haskell.org/trac/ghc/ticket/12019 > > This gives us an opportunity to simply remove it instead of fixing it. > If there is anyone that thinks this future is really useful (ie more > useful than the other profiling modes) then I'm willing to fix it. > But if noone would miss it. I'd much rather remove it. > > Thoughts? > > Erik > -- > ---------------------------------------------------------------------- > Erik de Castro Lopo > http://www.mega-nerd.com/ > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users From carter.schonwald at gmail.com Fri May 6 17:39:33 2016 From: carter.schonwald at gmail.com (Carter Schonwald) Date: Fri, 6 May 2016 13:39:33 -0400 Subject: RFC: Removing the `-hb` profiling option In-Reply-To: References: <20160506190413.acc749602f72e83ce3f9420a@mega-nerd.com> Message-ID: What about disallowing -hb on threaded builds? That does just sort of punt it a little bit, it a not quite satisfactory way perhaps On Friday, May 6, 2016, Evan Laforge wrote: > I've used this a lot when looking for leaks. Especially combined with > other things like -hbdrag -hc, so I'd be sad to see it go. Without > it, how do you find lag and drag? > > On the other hand, the entire profiling system has been hard to use > because of crashes (perhaps due to that thread-unsafety thing), but > also because of lack of documentation, the fact that SCCs can > drastically change the performance of the things they are trying to > measure, and probably just personal incompetence. But that's not > really directly relevant to -hb, except to say I'd rather fix it than > delete it. > > On Fri, May 6, 2016 at 2:04 AM, Erik de Castro Lopo > > wrote: > > Hi all, > > > > After a bit of rather tedious and frustrating debugging I came to the > > realisation that the code for the `-hb` profiling option is not thread > > safe. See https://ghc.haskell.org/trac/ghc/ticket/12019 > > > > This gives us an opportunity to simply remove it instead of fixing it. > > If there is anyone that thinks this future is really useful (ie more > > useful than the other profiling modes) then I'm willing to fix it. > > But if noone would miss it. I'd much rather remove it. > > > > Thoughts? > > > > Erik > > -- > > ---------------------------------------------------------------------- > > Erik de Castro Lopo > > http://www.mega-nerd.com/ > > _______________________________________________ > > Glasgow-haskell-users mailing list > > Glasgow-haskell-users at haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Fri May 6 17:58:19 2016 From: ben at well-typed.com (Ben Gamari) Date: Fri, 06 May 2016 19:58:19 +0200 Subject: [Maintainer needed] Actively maintained Haddock are happy Haddock In-Reply-To: References: <874mae6qtr.fsf@smart-cactus.org> Message-ID: <87zis3m5l0.fsf@smart-cactus.org> Sebastian writes: > Hello Ben, > Hi Sebastian, Sorry for the belated response. > If that's ok I'd be happy to take over the maintenance of Haddock. > Sure, that would be great. I can give you the keys in a bit. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 472 bytes Desc: not available URL: From davean at xkcd.com Fri May 6 18:02:09 2016 From: davean at xkcd.com (davean) Date: Fri, 6 May 2016 14:02:09 -0400 Subject: RFC: Removing the `-hb` profiling option In-Reply-To: References: <20160506190413.acc749602f72e83ce3f9420a@mega-nerd.com> Message-ID: On Fri, May 6, 2016 at 1:39 PM, Carter Schonwald wrote: > What about disallowing -hb on threaded builds? That does just sort of punt > it a little bit, it a not quite satisfactory way perhaps But the non-threaded runtime isn't really a first class citizen, is it? Consider http://hackage.haskell.org/package/base-4.8.2.0/docs/GHC-Event.html#v:getSystemEventManager which is Nothing on the non-threaded runtime. So you can't really just "run it unthreaded" to profile any program using that style of implementation. Even when you don't touch it directly, the program runs very differently. As for hb, I'd probably cry least about losing -hb but they all serve different, and in my opinion, important needs. -------------- next part -------------- An HTML attachment was scrubbed... URL: From spam at scientician.net Fri May 6 18:49:54 2016 From: spam at scientician.net (Bardur Arantsson) Date: Fri, 6 May 2016 20:49:54 +0200 Subject: RFC: Removing the `-hb` profiling option In-Reply-To: <20160506190413.acc749602f72e83ce3f9420a@mega-nerd.com> References: <20160506190413.acc749602f72e83ce3f9420a@mega-nerd.com> Message-ID: On 05/06/2016 11:04 AM, Erik de Castro Lopo wrote: > Hi all, > > After a bit of rather tedious and frustrating debugging I came to the > realisation that the code for the `-hb` profiling option is not thread > safe. See https://ghc.haskell.org/trac/ghc/ticket/12019 > > This gives us an opportunity to simply remove it instead of fixing it. > If there is anyone that thinks this future is really useful (ie more > useful than the other profiling modes) then I'm willing to fix it. > But if noone would miss it. I'd much rather remove it. > I can't say I've ever used the option, but just to inform discussion... Do you have any guesstimate[1] of how much effort it would take to fix vs. just removing? Unless there's a reasonably clear idea of how much effort there's going to be in it[2], I usually err on the side of removal *unless* a really good case can be made. Regards, [1] As ballpark as you like. Even a ratio would work, I think. [2] Which tends to not be the case for threading-related issues. From thomas.dubuisson at gmail.com Sat May 7 03:45:15 2016 From: thomas.dubuisson at gmail.com (Thomas DuBuisson) Date: Fri, 6 May 2016 20:45:15 -0700 Subject: Looking for GHC compile-time performance tests In-Reply-To: <87r3dg650i.fsf@smart-cactus.org> References: <87r3dg650i.fsf@smart-cactus.org> Message-ID: On Thu, May 5, 2016 at 5:52 AM, Ben Gamari wrote: > > So, if you would like to see your program's compilation time improve > in GHC 8.2, put some time into reducing it to something minimal, submit > it to us via a Trac ticket, and let us know in this thread. Please see https://ghc.haskell.org/trac/ghc/ticket/12028 for an example with no dependencies beyond commons (ex. bytestring). There is a 6x performance regression or 4x if you're willing to delete UNPACK pragmas. From thomas.dubuisson at gmail.com Sat May 7 19:56:33 2016 From: thomas.dubuisson at gmail.com (Thomas DuBuisson) Date: Sat, 7 May 2016 12:56:33 -0700 Subject: Looking for GHC compile-time performance tests In-Reply-To: References: <87r3dg650i.fsf@smart-cactus.org> Message-ID: Similarly, please see https://ghc.haskell.org/trac/ghc/ticket/12032 for a regression when using many top-level cases in equational style. I haven't looked at core, it's possible this issue is the same as ticket 12028. Also please note I am _not_ on the -dev list at present so that list is not getting my responses to this thread. -Thomas On Fri, May 6, 2016 at 8:45 PM, Thomas DuBuisson wrote: > On Thu, May 5, 2016 at 5:52 AM, Ben Gamari wrote: >> >> So, if you would like to see your program's compilation time improve >> in GHC 8.2, put some time into reducing it to something minimal, submit >> it to us via a Trac ticket, and let us know in this thread. > > > Please see https://ghc.haskell.org/trac/ghc/ticket/12028 for an > example with no dependencies beyond commons (ex. bytestring). There > is a 6x performance regression or 4x if you're willing to delete > UNPACK pragmas. From harendra.kumar at gmail.com Mon May 9 14:23:06 2016 From: harendra.kumar at gmail.com (Harendra Kumar) Date: Mon, 9 May 2016 19:53:06 +0530 Subject: suboptimal ghc code generation in IO vs equivalent pure code case Message-ID: I have a loop which runs millions of times. For some reason I have to run it in the IO monad. I noticed that when I convert the code from pure to IO monad the generated assembly code in essence is almost identical except one difference where it puts a piece of code in a separate block which is making a huge difference in performance (4-6x slower). I want to understand what makes GHC to generate code in this way and if there is anything that can be done at source level (or ghc option) to control that. The pure code looks like this: decomposeChars :: [Char] -> [Char] decomposeChars [] = [] decomposeChars [x] = case NFD.isDecomposable x of True -> decomposeChars (NFD.decomposeChar x) False -> [x] decomposeChars (x : xs) = decomposeChars [x] ++ decomposeChars xs The equivalent IO code is this: decomposeStrIO :: [Char] -> IO [Char] decomposeStrPtr !p = decomposeStrIO where decomposeStrIO [] = return [] decomposeStrIO [x] = do res <- NFD.isDecomposable p x case res of True -> decomposeStrIO (NFD.decomposeChar x) False -> return [x] decomposeStrIO (x : xs) = do s1 <- decomposeStrIO [x] s2 <- decomposeStrIO xs return (s1 ++ s2) The difference is in how the code corresponding to the call to the (++) operation is generated. In the pure case the (++) operation is inline in the main loop: _cn5N: movq $sat_sn2P_info,-48(%r12) movq %rax,-32(%r12) movq %rcx,-24(%r12) movq $:_con_info,-16(%r12) movq 16(%rbp),%rax movq %rax,-8(%r12) movq $GHC.Types.[]_closure+1,(%r12) leaq -48(%r12),%rsi leaq -14(%r12),%r14 addq $40,%rbp jmp GHC.Base.++_info In the IO monad version this code is placed in a separate block and a call is placed in the main loop: the main loop call site: _cn6A: movq $sat_sn3w_info,-24(%r12) movq 8(%rbp),%rax movq %rax,-8(%r12) movq %rbx,(%r12) leaq -24(%r12),%rbx addq $40,%rbp jmp *(%rbp) out of the line block - the code that was in the main loop in the previous case is now moved to this block (see label _cn5s below): sat_sn3w_info: _cn5p: leaq -16(%rbp),%rax cmpq %r15,%rax jb _cn5q _cn5r: addq $24,%r12 cmpq 856(%r13),%r12 ja _cn5t _cn5s: movq $stg_upd_frame_info,-16(%rbp) movq %rbx,-8(%rbp) movq 16(%rbx),%rax movq 24(%rbx),%rbx movq $:_con_info,-16(%r12) movq %rax,-8(%r12) movq $GHC.Types.[]_closure+1,(%r12) movq %rbx,%rsi leaq -14(%r12),%r14 addq $-16,%rbp jmp GHC.Base.++_info _cn5t: movq $24,904(%r13) _cn5q: jmp *-16(%r13) Except this difference the rest of the assembly looks pretty similar in both the cases. The corresponding dump-simpl output for the pure case: False -> ++ @ Char (GHC.Types.: @ Char ww_amuh (GHC.Types.[] @ Char)) (Data.Unicode.Internal.Normalization.decompose_$sdecompose ipv_smuv ipv1_smuD); And for the IO monad version: False -> case $sa1_sn0g ipv_smUT ipv1_smV6 ipv2_imWU of _ [Occ=Dead] { (# ipv4_XmXv, ipv5_XmXx #) -> (# ipv4_XmXv, ++ @ Char (GHC.Types.: @ Char sc_sn0b (GHC.Types.[] @ Char)) ipv5_XmXx #) }; The dump-simpl output is essentially the same except the difference due to the realworld token in the IO case. Why is the generated code different? I will appreciate if someone can throw some light on the reason or can point to the relevant ghc source to look at where this happens. I am using ghc-7.10.3 in native code generation mode (no llvm). Thanks, Harendra -------------- next part -------------- An HTML attachment was scrubbed... URL: From dan.doel at gmail.com Tue May 10 01:21:56 2016 From: dan.doel at gmail.com (Dan Doel) Date: Mon, 9 May 2016 21:21:56 -0400 Subject: suboptimal ghc code generation in IO vs equivalent pure code case In-Reply-To: References: Message-ID: I'm no expert on reading GHC's generated assembly. However, there may be a line you've overlooked in explaining the difference, namely: movq $stg_upd_frame_info,-16(%rbp) This appears only in the IO code, according to what you've pasted, and it appears to be pushing an update frame (I think). Update frames are used as part of lazy evaluation, to ensure that work only happens once, barring very short race conditions. So, at a guess, I would say that your IO-based code is creating extra updatable closures, which isn't free. It's sometimes difficult to see the difference at the core level. It will probably be clearer at the STG level, because the expression language is more disciplined there. But, for instance, your pure code tail calls (++), whereas your IO code returns an unboxed tuple with the same sort of expression that is in the pure tail call. However, complex expressions like that can't really be put in an unboxed tuple at the STG level, so what will happen is that the complex expression will be let (closure allocation), and the closure will be returned in the unboxed tuple. So that is the source of the difference. A more perspicuous picture would be something like: Pure: False -> let { l1 = : ww_amuh [] l2 = Data.Unicode.Internal.Normalization.decompose_$sdecompose ipv_smuv ipv1_smuD } in ++ l1 l2 IO: False -> case $sa1_sn0g ipv_smUT ipv1_smV6 ipv2_imWU of _ { (# ipv4_XmXv, ipv5_XmXx #) -> let { l1 = : sc_sn0b [] l3 = ++ l1 ipv5_XmXx } in (# ipv4_XmXv, l3 #) I can't say for certain that that's the only thing making a difference, but it might be one thing. -- Dan On Mon, May 9, 2016 at 10:23 AM, Harendra Kumar wrote: > I have a loop which runs millions of times. For some reason I have to run it > in the IO monad. I noticed that when I convert the code from pure to IO > monad the generated assembly code in essence is almost identical except one > difference where it puts a piece of code in a separate block which is making > a huge difference in performance (4-6x slower). > > I want to understand what makes GHC to generate code in this way and if > there is anything that can be done at source level (or ghc option) to > control that. > > The pure code looks like this: > > decomposeChars :: [Char] -> [Char] > > decomposeChars [] = [] > decomposeChars [x] = > case NFD.isDecomposable x of > True -> decomposeChars (NFD.decomposeChar x) > False -> [x] > decomposeChars (x : xs) = decomposeChars [x] ++ decomposeChars xs > > The equivalent IO code is this: > > decomposeStrIO :: [Char] -> IO [Char] > > decomposeStrPtr !p = decomposeStrIO > where > decomposeStrIO [] = return [] > decomposeStrIO [x] = do > res <- NFD.isDecomposable p x > case res of > True -> decomposeStrIO (NFD.decomposeChar x) > False -> return [x] > decomposeStrIO (x : xs) = do > s1 <- decomposeStrIO [x] > s2 <- decomposeStrIO xs > return (s1 ++ s2) > > The difference is in how the code corresponding to the call to the (++) > operation is generated. In the pure case the (++) operation is inline in the > main loop: > > _cn5N: > movq $sat_sn2P_info,-48(%r12) > movq %rax,-32(%r12) > movq %rcx,-24(%r12) > movq $:_con_info,-16(%r12) > movq 16(%rbp),%rax > movq %rax,-8(%r12) > movq $GHC.Types.[]_closure+1,(%r12) > leaq -48(%r12),%rsi > leaq -14(%r12),%r14 > addq $40,%rbp > jmp GHC.Base.++_info > > In the IO monad version this code is placed in a separate block and a call > is placed in the main loop: > > the main loop call site: > > _cn6A: > movq $sat_sn3w_info,-24(%r12) > movq 8(%rbp),%rax > movq %rax,-8(%r12) > movq %rbx,(%r12) > leaq -24(%r12),%rbx > addq $40,%rbp > jmp *(%rbp) > > out of the line block - the code that was in the main loop in the previous > case is now moved to this block (see label _cn5s below): > > sat_sn3w_info: > _cn5p: > leaq -16(%rbp),%rax > cmpq %r15,%rax > jb _cn5q > _cn5r: > addq $24,%r12 > cmpq 856(%r13),%r12 > ja _cn5t > _cn5s: > movq $stg_upd_frame_info,-16(%rbp) > movq %rbx,-8(%rbp) > movq 16(%rbx),%rax > movq 24(%rbx),%rbx > movq $:_con_info,-16(%r12) > movq %rax,-8(%r12) > movq $GHC.Types.[]_closure+1,(%r12) > movq %rbx,%rsi > leaq -14(%r12),%r14 > addq $-16,%rbp > jmp GHC.Base.++_info > _cn5t: > movq $24,904(%r13) > _cn5q: > jmp *-16(%r13) > > Except this difference the rest of the assembly looks pretty similar in both > the cases. The corresponding dump-simpl output for the pure case: > > False -> > ++ > @ Char > (GHC.Types.: @ Char ww_amuh (GHC.Types.[] @ Char)) > (Data.Unicode.Internal.Normalization.decompose_$sdecompose > ipv_smuv ipv1_smuD); > > And for the IO monad version: > > False -> > case $sa1_sn0g ipv_smUT ipv1_smV6 ipv2_imWU > of _ [Occ=Dead] { (# ipv4_XmXv, ipv5_XmXx #) -> > (# ipv4_XmXv, > ++ > @ Char > (GHC.Types.: @ Char sc_sn0b (GHC.Types.[] @ Char)) > ipv5_XmXx #) > }; > > The dump-simpl output is essentially the same except the difference due to > the realworld token in the IO case. Why is the generated code different? I > will appreciate if someone can throw some light on the reason or can point > to the relevant ghc source to look at where this happens. > > I am using ghc-7.10.3 in native code generation mode (no llvm). > > Thanks, > Harendra > > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users > From harendra.kumar at gmail.com Tue May 10 08:45:15 2016 From: harendra.kumar at gmail.com (Harendra Kumar) Date: Tue, 10 May 2016 14:15:15 +0530 Subject: suboptimal ghc code generation in IO vs equivalent pure code case In-Reply-To: References: Message-ID: Thanks Dan, that helped. I did notice and suspect the update frame and the unboxed tuple but given my limited knowledge about ghc/core/stg/cmm I was not sure what is going on. In fact I thought that the intermediate tuple should get optimized out since it is required only because of the realworld token which is not real. But it might be difficult to see that at this level? What you are saying may be true for the current implementation but in theory can we eliminate the intermediate closure? I observed that the same unboxed tuple results in an inline code for (:) constructor. I rewrote the source a bit differently which resulted in this core: case ipv1_amWI of _ [Occ=Dead] { False -> case a1_smWm xs_aloV ipv_amWH of _ [Occ=Dead] { (# ipv2_XmXf, ipv3_XmXh #) -> (# ipv2_XmXf, GHC.Types.: @ Char x_aloU ipv3_XmXh #) }; True -> case a1_smWm (NFD.decomposeChar x_aloU) ipv_amWH of _ [Occ=Dead] { (# ipv2_XmXf, ipv3_XmXh #) -> case a1_smWm xs_aloV ipv2_XmXf of _ [Occ=Dead] { (# ipv4_XmXk, ipv5_XmXm #) -> (# ipv4_XmXk, ++ @ Char ipv3_XmXh ipv5_XmXm #) } } } We can see that both True and the False case are returning an unboxed tuple. The only difference is (:) vs (++). But the generated assembly is different for both cases: (++) has a closure as before: sat_sn0Z_info: _cn2c: leaq -16(%rbp),%rax cmpq %r15,%rax jb _cn2d _cn2e: movq $stg_upd_frame_info,-16(%rbp) movq %rbx,-8(%rbp) movq 24(%rbx),%rsi movq 16(%rbx),%r14 addq $-16,%rbp jmp GHC.Base.++_info (:) is treated differently and generated inline as you can see below: _cn2I: movq $sat_sn0Z_info,-24(%r12) movq 8(%rbp),%rax movq %rax,-8(%r12) movq %rbx,(%r12) leaq -24(%r12),%rbx addq $16,%rbp jmp *(%rbp) . . . block_cn2u_info: _cn2u: addq $24,%r12 cmpq 856(%r13),%r12 ja _cn2C _cn2B: movq $:_con_info,-16(%r12) movq 8(%rbp),%rax movq %rax,-8(%r12) movq %rbx,(%r12) leaq -14(%r12),%rbx addq $24,%rbp jmp *(%rbp) So why is that? Why can't we generate (++) inline similar to (:)? How do we make this decision? Is there a theoretical reason for this or this is just an implementation artifact? Since the tuple is required only for passing the realworld token, ideally I wouldn't want it to make a difference to the generated code. Otherwise I would always have to worry about such performance side effects when using ST/IO. -harendra On 10 May 2016 at 06:51, Dan Doel wrote: > I'm no expert on reading GHC's generated assembly. However, there may > be a line you've overlooked in explaining the difference, namely: > > movq $stg_upd_frame_info,-16(%rbp) > > This appears only in the IO code, according to what you've pasted, and > it appears to be pushing an update frame (I think). Update frames are > used as part of lazy evaluation, to ensure that work only happens > once, barring very short race conditions. So, at a guess, I would say > that your IO-based code is creating extra updatable closures, which > isn't free. > > It's sometimes difficult to see the difference at the core level. It > will probably be clearer at the STG level, because the expression > language is more disciplined there. But, for instance, your pure code > tail calls (++), whereas your IO code returns an unboxed tuple with > the same sort of expression that is in the pure tail call. However, > complex expressions like that can't really be put in an unboxed tuple > at the STG level, so what will happen is that the complex expression > will be let (closure allocation), and the closure will be returned in > the unboxed tuple. So that is the source of the difference. A more > perspicuous picture would be something like: > > Pure: > False -> > let { > l1 = : ww_amuh [] > l2 = Data.Unicode.Internal.Normalization.decompose_$sdecompose > ipv_smuv ipv1_smuD > } in ++ l1 l2 > > > IO: > False -> > case $sa1_sn0g ipv_smUT ipv1_smV6 ipv2_imWU > of _ { (# ipv4_XmXv, ipv5_XmXx #) -> > let { > l1 = : sc_sn0b [] > l3 = ++ l1 ipv5_XmXx > } in (# ipv4_XmXv, l3 #) > > I can't say for certain that that's the only thing making a > difference, but it might be one thing. > > -- Dan > > > On Mon, May 9, 2016 at 10:23 AM, Harendra Kumar > wrote: > > I have a loop which runs millions of times. For some reason I have to > run it > > in the IO monad. I noticed that when I convert the code from pure to IO > > monad the generated assembly code in essence is almost identical except > one > > difference where it puts a piece of code in a separate block which is > making > > a huge difference in performance (4-6x slower). > > > > I want to understand what makes GHC to generate code in this way and if > > there is anything that can be done at source level (or ghc option) to > > control that. > > > > The pure code looks like this: > > > > decomposeChars :: [Char] -> [Char] > > > > decomposeChars [] = [] > > decomposeChars [x] = > > case NFD.isDecomposable x of > > True -> decomposeChars (NFD.decomposeChar x) > > False -> [x] > > decomposeChars (x : xs) = decomposeChars [x] ++ decomposeChars xs > > > > The equivalent IO code is this: > > > > decomposeStrIO :: [Char] -> IO [Char] > > > > decomposeStrPtr !p = decomposeStrIO > > where > > decomposeStrIO [] = return [] > > decomposeStrIO [x] = do > > res <- NFD.isDecomposable p x > > case res of > > True -> decomposeStrIO (NFD.decomposeChar x) > > False -> return [x] > > decomposeStrIO (x : xs) = do > > s1 <- decomposeStrIO [x] > > s2 <- decomposeStrIO xs > > return (s1 ++ s2) > > > > The difference is in how the code corresponding to the call to the (++) > > operation is generated. In the pure case the (++) operation is inline in > the > > main loop: > > > > _cn5N: > > movq $sat_sn2P_info,-48(%r12) > > movq %rax,-32(%r12) > > movq %rcx,-24(%r12) > > movq $:_con_info,-16(%r12) > > movq 16(%rbp),%rax > > movq %rax,-8(%r12) > > movq $GHC.Types.[]_closure+1,(%r12) > > leaq -48(%r12),%rsi > > leaq -14(%r12),%r14 > > addq $40,%rbp > > jmp GHC.Base.++_info > > > > In the IO monad version this code is placed in a separate block and a > call > > is placed in the main loop: > > > > the main loop call site: > > > > _cn6A: > > movq $sat_sn3w_info,-24(%r12) > > movq 8(%rbp),%rax > > movq %rax,-8(%r12) > > movq %rbx,(%r12) > > leaq -24(%r12),%rbx > > addq $40,%rbp > > jmp *(%rbp) > > > > out of the line block - the code that was in the main loop in the > previous > > case is now moved to this block (see label _cn5s below): > > > > sat_sn3w_info: > > _cn5p: > > leaq -16(%rbp),%rax > > cmpq %r15,%rax > > jb _cn5q > > _cn5r: > > addq $24,%r12 > > cmpq 856(%r13),%r12 > > ja _cn5t > > _cn5s: > > movq $stg_upd_frame_info,-16(%rbp) > > movq %rbx,-8(%rbp) > > movq 16(%rbx),%rax > > movq 24(%rbx),%rbx > > movq $:_con_info,-16(%r12) > > movq %rax,-8(%r12) > > movq $GHC.Types.[]_closure+1,(%r12) > > movq %rbx,%rsi > > leaq -14(%r12),%r14 > > addq $-16,%rbp > > jmp GHC.Base.++_info > > _cn5t: > > movq $24,904(%r13) > > _cn5q: > > jmp *-16(%r13) > > > > Except this difference the rest of the assembly looks pretty similar in > both > > the cases. The corresponding dump-simpl output for the pure case: > > > > False -> > > ++ > > @ Char > > (GHC.Types.: @ Char ww_amuh (GHC.Types.[] @ Char)) > > (Data.Unicode.Internal.Normalization.decompose_$sdecompose > > ipv_smuv ipv1_smuD); > > > > And for the IO monad version: > > > > False -> > > case $sa1_sn0g ipv_smUT ipv1_smV6 ipv2_imWU > > of _ [Occ=Dead] { (# ipv4_XmXv, ipv5_XmXx #) -> > > (# ipv4_XmXv, > > ++ > > @ Char > > (GHC.Types.: @ Char sc_sn0b (GHC.Types.[] @ Char)) > > ipv5_XmXx #) > > }; > > > > The dump-simpl output is essentially the same except the difference due > to > > the realworld token in the IO case. Why is the generated code different? > I > > will appreciate if someone can throw some light on the reason or can > point > > to the relevant ghc source to look at where this happens. > > > > I am using ghc-7.10.3 in native code generation mode (no llvm). > > > > Thanks, > > Harendra > > > > _______________________________________________ > > Glasgow-haskell-users mailing list > > Glasgow-haskell-users at haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dan.doel at gmail.com Thu May 12 00:11:50 2016 From: dan.doel at gmail.com (Dan Doel) Date: Wed, 11 May 2016 20:11:50 -0400 Subject: suboptimal ghc code generation in IO vs equivalent pure code case In-Reply-To: References: Message-ID: On Tue, May 10, 2016 at 4:45 AM, Harendra Kumar wrote: > Thanks Dan, that helped. I did notice and suspect the update frame and the > unboxed tuple but given my limited knowledge about ghc/core/stg/cmm I was > not sure what is going on. In fact I thought that the intermediate tuple > should get optimized out since it is required only because of the realworld > token which is not real. But it might be difficult to see that at this > level? The token exists as far as the STG level is concerned, I think, because that is the only thing ensuring that things happen in the right order. And the closure must be built to have properly formed STG. So optimizing away the closure building would have to happen at a level lower than STG, and I guess there is no such optimization. I'm not sure how easy it would be to do. > What you are saying may be true for the current implementation but in theory > can we eliminate the intermediate closure? I don't know. I'm a bit skeptical that building this one closure is the only thing causing your code to be a factor of 5 slower. For instance, another difference in the core is that the recursive call corresponding to the result s2 happens before allocating the additional closure. That is the case statement that unpacks the unboxed tuple. And the whole loop happens this way, so it is actually building a structure corresponding to the entire output list in memory rather eagerly. By contrast, your pure function is able to act in a streaming fashion, if consumed properly, where only enough of the result is built to keep driving the rest of the program. It probably runs in constant space, while your IO-based loop has a footprint linear in the size of the input list (in addition to having slightly more overhead per character because of the one extra thunk), because it is a more eager program. And having an asymptotically larger memory footprint is more likely to explain a 5x slowdown than allocating one extra thunk for each list concatenation, I think. (Of course, it could also be some other factor, as well.) You should probably run with +RTS -s (compiling with -rtsopts), and see if the IO version has a much larger maximum residency. Anyhow, this is fundamental to writing the algorithm using IO. It's an algorithm that's a sequence of steps that happen in order, the number of steps is proportional to the input list, and part of those steps is putting stuff in memory for the results. > So why is that? Why can't we generate (++) inline similar to (:)? How do we > make this decision? Is there a theoretical reason for this or this is just > an implementation artifact? The difference between these two is that (++) is a function call, and (:) is a constructor. Constructors are a special case, and don't need to have all the provisions that functions in general do. The best way to learn what the differences are is probably to read the paper about the STG machine. Hope this is a more fruitful lead, but it may be that there's not a lot that can be done, without doing some baroque things to your algorithm, because of the necessity of its using IO. -- Dan From harendra.kumar at gmail.com Sat May 14 11:40:15 2016 From: harendra.kumar at gmail.com (Harendra Kumar) Date: Sat, 14 May 2016 17:10:15 +0530 Subject: suboptimal ghc code generation in IO vs equivalent pure code case In-Reply-To: References: Message-ID: You are right about the way IO code is generated because of the ordering semantics. The IO version builds the list entirely in a recursive fashion before returning it while the pure code builds it lazily. I wrote very simple versions to keep things simpler: Pure version: f [] = [] f (x : xs) = x : f xs time 11.08 ms (10.86 ms .. 11.34 ms) Measured for a million elements in the list 104,041,264 bytes allocated in the heap 16,728 bytes copied during GC 35,992 bytes maximum residency (1 sample(s)) 21,352 bytes maximum slop 1 MB total memory in use (0 MB lost due to fragmentation) IO version: f [] = return [] f (x : xs) = do rest <- f xs return $ x : rest time 79.66 ms (75.49 ms .. 82.55 ms) 208,654,560 bytes allocated in the heap 121,045,336 bytes copied during GC 27,679,344 bytes maximum residency (8 sample(s)) 383,376 bytes maximum slop 66 MB total memory in use (0 MB lost due to fragmentation) Even though this is a small program not doing much and therefore enhancing even small differences to a great degree, I am not sure if I can completely explain the difference in slowness of the order of 7.5x by just the recursive vs lazy building of the list. I am wondering if there is anything that is worth further investigating and improving here. -harendra On 12 May 2016 at 05:41, Dan Doel wrote: > > On Tue, May 10, 2016 at 4:45 AM, Harendra Kumar > wrote: > > Thanks Dan, that helped. I did notice and suspect the update frame and the > > unboxed tuple but given my limited knowledge about ghc/core/stg/cmm I was > > not sure what is going on. In fact I thought that the intermediate tuple > > should get optimized out since it is required only because of the realworld > > token which is not real. But it might be difficult to see that at this > > level? > > The token exists as far as the STG level is concerned, I think, > because that is the only thing ensuring that things happen in the > right order. And the closure must be built to have properly formed > STG. So optimizing away the closure building would have to happen at a > level lower than STG, and I guess there is no such optimization. I'm > not sure how easy it would be to do. > > > What you are saying may be true for the current implementation but in theory > > can we eliminate the intermediate closure? > > I don't know. I'm a bit skeptical that building this one closure is > the only thing causing your code to be a factor of 5 slower. For > instance, another difference in the core is that the recursive call > corresponding to the result s2 happens before allocating the > additional closure. That is the case statement that unpacks the > unboxed tuple. And the whole loop happens this way, so it is actually > building a structure corresponding to the entire output list in memory > rather eagerly. > > By contrast, your pure function is able to act in a streaming fashion, > if consumed properly, where only enough of the result is built to keep > driving the rest of the program. It probably runs in constant space, > while your IO-based loop has a footprint linear in the size of the > input list (in addition to having slightly more overhead per character > because of the one extra thunk), because it is a more eager program. > > And having an asymptotically larger memory footprint is more likely to > explain a 5x slowdown than allocating one extra thunk for each list > concatenation, I think. (Of course, it could also be some other > factor, as well.) > > You should probably run with +RTS -s (compiling with -rtsopts), and > see if the IO version has a much larger maximum residency. > > Anyhow, this is fundamental to writing the algorithm using IO. It's an > algorithm that's a sequence of steps that happen in order, the number > of steps is proportional to the input list, and part of those steps is > putting stuff in memory for the results. > > > So why is that? Why can't we generate (++) inline similar to (:)? How do we > > make this decision? Is there a theoretical reason for this or this is just > > an implementation artifact? > > The difference between these two is that (++) is a function call, and > (:) is a constructor. Constructors are a special case, and don't need > to have all the provisions that functions in general do. The best way > to learn what the differences are is probably to read the paper about > the STG machine. > > Hope this is a more fruitful lead, but it may be that there's not a > lot that can be done, without doing some baroque things to your > algorithm, because of the necessity of its using IO. > > -- Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From harendra.kumar at gmail.com Sat May 14 18:31:02 2016 From: harendra.kumar at gmail.com (Harendra Kumar) Date: Sun, 15 May 2016 00:01:02 +0530 Subject: suboptimal ghc code generation in IO vs equivalent pure code case In-Reply-To: References: Message-ID: The difference seems to be entirely due to memory pressure. At list size 1000 both pure version and IO version perform equally. But as the size of the list increases the pure version scales linearly while the IO version degrades exponentially. Here are the execution times per list element in ns as the list size increases: Size of list Pure IO 1000 8.7 8.3 10000 8.7 18 100000 8.8 63 1000000 9.3 786 This seems to be due to increased GC activity in the IO case. The GC stats for list size 1 million are: IO case: %GC time 66.1% (61.1% elapsed) Pure case: %GC time 2.6% (3.3% elapsed) Not sure if there is a way to write this code in IO monad which can reduce this overhead. -harendra On 14 May 2016 at 17:10, Harendra Kumar wrote: > > You are right about the way IO code is generated because of the ordering semantics. The IO version builds the list entirely in a recursive fashion before returning it while the pure code builds it lazily. I wrote very simple versions to keep things simpler: > > Pure version: > > f [] = [] > f (x : xs) = x : f xs > > > time 11.08 ms (10.86 ms .. 11.34 ms) > Measured for a million elements in the list > > 104,041,264 bytes allocated in the heap > 16,728 bytes copied during GC > 35,992 bytes maximum residency (1 sample(s)) > 21,352 bytes maximum slop > 1 MB total memory in use (0 MB lost due to fragmentation) > > > IO version: > f [] = return [] > f (x : xs) = do > rest <- f xs > return $ x : rest > > time 79.66 ms (75.49 ms .. 82.55 ms) > > 208,654,560 bytes allocated in the heap > 121,045,336 bytes copied during GC > 27,679,344 bytes maximum residency (8 sample(s)) > 383,376 bytes maximum slop > 66 MB total memory in use (0 MB lost due to fragmentation) > > Even though this is a small program not doing much and therefore enhancing even small differences to a great degree, I am not sure if I can completely explain the difference in slowness of the order of 7.5x by just the recursive vs lazy building of the list. I am wondering if there is anything that is worth further investigating and improving here. > > -harendra > > On 12 May 2016 at 05:41, Dan Doel wrote: > > > > On Tue, May 10, 2016 at 4:45 AM, Harendra Kumar > > wrote: > > > Thanks Dan, that helped. I did notice and suspect the update frame and the > > > unboxed tuple but given my limited knowledge about ghc/core/stg/cmm I was > > > not sure what is going on. In fact I thought that the intermediate tuple > > > should get optimized out since it is required only because of the realworld > > > token which is not real. But it might be difficult to see that at this > > > level? > > > > The token exists as far as the STG level is concerned, I think, > > because that is the only thing ensuring that things happen in the > > right order. And the closure must be built to have properly formed > > STG. So optimizing away the closure building would have to happen at a > > level lower than STG, and I guess there is no such optimization. I'm > > not sure how easy it would be to do. > > > > > What you are saying may be true for the current implementation but in theory > > > can we eliminate the intermediate closure? > > > > I don't know. I'm a bit skeptical that building this one closure is > > the only thing causing your code to be a factor of 5 slower. For > > instance, another difference in the core is that the recursive call > > corresponding to the result s2 happens before allocating the > > additional closure. That is the case statement that unpacks the > > unboxed tuple. And the whole loop happens this way, so it is actually > > building a structure corresponding to the entire output list in memory > > rather eagerly. > > > > By contrast, your pure function is able to act in a streaming fashion, > > if consumed properly, where only enough of the result is built to keep > > driving the rest of the program. It probably runs in constant space, > > while your IO-based loop has a footprint linear in the size of the > > input list (in addition to having slightly more overhead per character > > because of the one extra thunk), because it is a more eager program. > > > > And having an asymptotically larger memory footprint is more likely to > > explain a 5x slowdown than allocating one extra thunk for each list > > concatenation, I think. (Of course, it could also be some other > > factor, as well.) > > > > You should probably run with +RTS -s (compiling with -rtsopts), and > > see if the IO version has a much larger maximum residency. > > > > Anyhow, this is fundamental to writing the algorithm using IO. It's an > > algorithm that's a sequence of steps that happen in order, the number > > of steps is proportional to the input list, and part of those steps is > > putting stuff in memory for the results. > > > > > So why is that? Why can't we generate (++) inline similar to (:)? How do we > > > make this decision? Is there a theoretical reason for this or this is just > > > an implementation artifact? > > > > The difference between these two is that (++) is a function call, and > > (:) is a constructor. Constructors are a special case, and don't need > > to have all the provisions that functions in general do. The best way > > to learn what the differences are is probably to read the paper about > > the STG machine. > > > > Hope this is a more fruitful lead, but it may be that there's not a > > lot that can be done, without doing some baroque things to your > > algorithm, because of the necessity of its using IO. > > > > -- Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.feuer at gmail.com Sat May 14 20:05:29 2016 From: david.feuer at gmail.com (David Feuer) Date: Sat, 14 May 2016 16:05:29 -0400 Subject: suboptimal ghc code generation in IO vs equivalent pure code case In-Reply-To: References: Message-ID: Well, a few weeks ago Bertram Felgenhauer came up with a version of IO that acts more like lazy ST. That could be just the thing. He placed it in the public domain/CC0 and told me I could put it up on Hackage if I want. I'll try to do that this week, but no promises. I could forward his email if you just want to try it out. On May 14, 2016 2:31 PM, "Harendra Kumar" wrote: > The difference seems to be entirely due to memory pressure. At list size > 1000 both pure version and IO version perform equally. But as the size of > the list increases the pure version scales linearly while the IO version > degrades exponentially. Here are the execution times per list element in ns > as the list size increases: > > Size of list Pure IO > 1000 8.7 8.3 > 10000 8.7 18 > 100000 8.8 63 > 1000000 9.3 786 > > This seems to be due to increased GC activity in the IO case. The GC stats > for list size 1 million are: > > IO case: %GC time 66.1% (61.1% elapsed) > Pure case: %GC time 2.6% (3.3% elapsed) > > Not sure if there is a way to write this code in IO monad which can reduce > this overhead. > > -harendra > > > On 14 May 2016 at 17:10, Harendra Kumar wrote: > > > > You are right about the way IO code is generated because of the ordering > semantics. The IO version builds the list entirely in a recursive fashion > before returning it while the pure code builds it lazily. I wrote very > simple versions to keep things simpler: > > > > Pure version: > > > > f [] = [] > > f (x : xs) = x : f xs > > > > > > time 11.08 ms (10.86 ms .. 11.34 ms) > > Measured for a million elements in the list > > > > 104,041,264 bytes allocated in the heap > > 16,728 bytes copied during GC > > 35,992 bytes maximum residency (1 sample(s)) > > 21,352 bytes maximum slop > > 1 MB total memory in use (0 MB lost due to fragmentation) > > > > > > IO version: > > f [] = return [] > > f (x : xs) = do > > rest <- f xs > > return $ x : rest > > > > time 79.66 ms (75.49 ms .. 82.55 ms) > > > > 208,654,560 bytes allocated in the heap > > 121,045,336 bytes copied during GC > > 27,679,344 bytes maximum residency (8 sample(s)) > > 383,376 bytes maximum slop > > 66 MB total memory in use (0 MB lost due to fragmentation) > > > > Even though this is a small program not doing much and therefore > enhancing even small differences to a great degree, I am not sure if I can > completely explain the difference in slowness of the order of 7.5x by just > the recursive vs lazy building of the list. I am wondering if there is > anything that is worth further investigating and improving here. > > > > -harendra > > > > On 12 May 2016 at 05:41, Dan Doel wrote: > > > > > > On Tue, May 10, 2016 at 4:45 AM, Harendra Kumar > > > wrote: > > > > Thanks Dan, that helped. I did notice and suspect the update frame > and the > > > > unboxed tuple but given my limited knowledge about ghc/core/stg/cmm > I was > > > > not sure what is going on. In fact I thought that the intermediate > tuple > > > > should get optimized out since it is required only because of the > realworld > > > > token which is not real. But it might be difficult to see that at > this > > > > level? > > > > > > The token exists as far as the STG level is concerned, I think, > > > because that is the only thing ensuring that things happen in the > > > right order. And the closure must be built to have properly formed > > > STG. So optimizing away the closure building would have to happen at a > > > level lower than STG, and I guess there is no such optimization. I'm > > > not sure how easy it would be to do. > > > > > > > What you are saying may be true for the current implementation but > in theory > > > > can we eliminate the intermediate closure? > > > > > > I don't know. I'm a bit skeptical that building this one closure is > > > the only thing causing your code to be a factor of 5 slower. For > > > instance, another difference in the core is that the recursive call > > > corresponding to the result s2 happens before allocating the > > > additional closure. That is the case statement that unpacks the > > > unboxed tuple. And the whole loop happens this way, so it is actually > > > building a structure corresponding to the entire output list in memory > > > rather eagerly. > > > > > > By contrast, your pure function is able to act in a streaming fashion, > > > if consumed properly, where only enough of the result is built to keep > > > driving the rest of the program. It probably runs in constant space, > > > while your IO-based loop has a footprint linear in the size of the > > > input list (in addition to having slightly more overhead per character > > > because of the one extra thunk), because it is a more eager program. > > > > > > And having an asymptotically larger memory footprint is more likely to > > > explain a 5x slowdown than allocating one extra thunk for each list > > > concatenation, I think. (Of course, it could also be some other > > > factor, as well.) > > > > > > You should probably run with +RTS -s (compiling with -rtsopts), and > > > see if the IO version has a much larger maximum residency. > > > > > > Anyhow, this is fundamental to writing the algorithm using IO. It's an > > > algorithm that's a sequence of steps that happen in order, the number > > > of steps is proportional to the input list, and part of those steps is > > > putting stuff in memory for the results. > > > > > > > So why is that? Why can't we generate (++) inline similar to (:)? > How do we > > > > make this decision? Is there a theoretical reason for this or this > is just > > > > an implementation artifact? > > > > > > The difference between these two is that (++) is a function call, and > > > (:) is a constructor. Constructors are a special case, and don't need > > > to have all the provisions that functions in general do. The best way > > > to learn what the differences are is probably to read the paper about > > > the STG machine. > > > > > > Hope this is a more fruitful lead, but it may be that there's not a > > > lot that can be done, without doing some baroque things to your > > > algorithm, because of the necessity of its using IO. > > > > > > -- Dan > > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From harendra.kumar at gmail.com Sat May 14 20:15:27 2016 From: harendra.kumar at gmail.com (Harendra Kumar) Date: Sun, 15 May 2016 01:45:27 +0530 Subject: suboptimal ghc code generation in IO vs equivalent pure code case In-Reply-To: References: Message-ID: On 15 May 2016 at 01:35, David Feuer wrote: > Well, a few weeks ago Bertram Felgenhauer came up with a version of IO > that acts more like lazy ST. That could be just the thing. He placed it in > the public domain/CC0 and told me I could put it up on Hackage if I want. > I'll try to do that this week, but no promises. I could forward his email > if you just want to try it out. > That's exactly what I was thinking about. Can you please forward it to me, I will try it out. Thanks, Harendra > On May 14, 2016 2:31 PM, "Harendra Kumar" > wrote: > >> The difference seems to be entirely due to memory pressure. At list size >> 1000 both pure version and IO version perform equally. But as the size of >> the list increases the pure version scales linearly while the IO version >> degrades exponentially. Here are the execution times per list element in ns >> as the list size increases: >> >> Size of list Pure IO >> 1000 8.7 8.3 >> 10000 8.7 18 >> 100000 8.8 63 >> 1000000 9.3 786 >> >> This seems to be due to increased GC activity in the IO case. The GC >> stats for list size 1 million are: >> >> IO case: %GC time 66.1% (61.1% elapsed) >> Pure case: %GC time 2.6% (3.3% elapsed) >> >> Not sure if there is a way to write this code in IO monad which can >> reduce this overhead. >> >> -harendra >> >> >> On 14 May 2016 at 17:10, Harendra Kumar wrote: >> > >> > You are right about the way IO code is generated because of the >> ordering semantics. The IO version builds the list entirely in a recursive >> fashion before returning it while the pure code builds it lazily. I wrote >> very simple versions to keep things simpler: >> > >> > Pure version: >> > >> > f [] = [] >> > f (x : xs) = x : f xs >> > >> > >> > time 11.08 ms (10.86 ms .. 11.34 ms) >> > Measured for a million elements in the list >> > >> > 104,041,264 bytes allocated in the heap >> > 16,728 bytes copied during GC >> > 35,992 bytes maximum residency (1 sample(s)) >> > 21,352 bytes maximum slop >> > 1 MB total memory in use (0 MB lost due to fragmentation) >> > >> > >> > IO version: >> > f [] = return [] >> > f (x : xs) = do >> > rest <- f xs >> > return $ x : rest >> > >> > time 79.66 ms (75.49 ms .. 82.55 ms) >> > >> > 208,654,560 bytes allocated in the heap >> > 121,045,336 bytes copied during GC >> > 27,679,344 bytes maximum residency (8 sample(s)) >> > 383,376 bytes maximum slop >> > 66 MB total memory in use (0 MB lost due to fragmentation) >> > >> > Even though this is a small program not doing much and therefore >> enhancing even small differences to a great degree, I am not sure if I can >> completely explain the difference in slowness of the order of 7.5x by just >> the recursive vs lazy building of the list. I am wondering if there is >> anything that is worth further investigating and improving here. >> > >> > -harendra >> > >> > On 12 May 2016 at 05:41, Dan Doel wrote: >> > > >> > > On Tue, May 10, 2016 at 4:45 AM, Harendra Kumar >> > > wrote: >> > > > Thanks Dan, that helped. I did notice and suspect the update frame >> and the >> > > > unboxed tuple but given my limited knowledge about ghc/core/stg/cmm >> I was >> > > > not sure what is going on. In fact I thought that the intermediate >> tuple >> > > > should get optimized out since it is required only because of the >> realworld >> > > > token which is not real. But it might be difficult to see that at >> this >> > > > level? >> > > >> > > The token exists as far as the STG level is concerned, I think, >> > > because that is the only thing ensuring that things happen in the >> > > right order. And the closure must be built to have properly formed >> > > STG. So optimizing away the closure building would have to happen at a >> > > level lower than STG, and I guess there is no such optimization. I'm >> > > not sure how easy it would be to do. >> > > >> > > > What you are saying may be true for the current implementation but >> in theory >> > > > can we eliminate the intermediate closure? >> > > >> > > I don't know. I'm a bit skeptical that building this one closure is >> > > the only thing causing your code to be a factor of 5 slower. For >> > > instance, another difference in the core is that the recursive call >> > > corresponding to the result s2 happens before allocating the >> > > additional closure. That is the case statement that unpacks the >> > > unboxed tuple. And the whole loop happens this way, so it is actually >> > > building a structure corresponding to the entire output list in memory >> > > rather eagerly. >> > > >> > > By contrast, your pure function is able to act in a streaming fashion, >> > > if consumed properly, where only enough of the result is built to keep >> > > driving the rest of the program. It probably runs in constant space, >> > > while your IO-based loop has a footprint linear in the size of the >> > > input list (in addition to having slightly more overhead per character >> > > because of the one extra thunk), because it is a more eager program. >> > > >> > > And having an asymptotically larger memory footprint is more likely to >> > > explain a 5x slowdown than allocating one extra thunk for each list >> > > concatenation, I think. (Of course, it could also be some other >> > > factor, as well.) >> > > >> > > You should probably run with +RTS -s (compiling with -rtsopts), and >> > > see if the IO version has a much larger maximum residency. >> > > >> > > Anyhow, this is fundamental to writing the algorithm using IO. It's an >> > > algorithm that's a sequence of steps that happen in order, the number >> > > of steps is proportional to the input list, and part of those steps is >> > > putting stuff in memory for the results. >> > > >> > > > So why is that? Why can't we generate (++) inline similar to (:)? >> How do we >> > > > make this decision? Is there a theoretical reason for this or this >> is just >> > > > an implementation artifact? >> > > >> > > The difference between these two is that (++) is a function call, and >> > > (:) is a constructor. Constructors are a special case, and don't need >> > > to have all the provisions that functions in general do. The best way >> > > to learn what the differences are is probably to read the paper about >> > > the STG machine. >> > > >> > > Hope this is a more fruitful lead, but it may be that there's not a >> > > lot that can be done, without doing some baroque things to your >> > > algorithm, because of the necessity of its using IO. >> > > >> > > -- Dan >> >> _______________________________________________ >> Glasgow-haskell-users mailing list >> Glasgow-haskell-users at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From twhitehead at gmail.com Sat May 14 20:21:38 2016 From: twhitehead at gmail.com (Tyson Whitehead) Date: Sat, 14 May 2016 16:21:38 -0400 Subject: suboptimal ghc code generation in IO vs equivalent pure code case In-Reply-To: References: Message-ID: <573788D2.40104@gmail.com> On 14/05/16 02:31 PM, Harendra Kumar wrote: > The difference seems to be entirely due to memory pressure. At list size 1000 both pure version and IO version perform equally. But as the size of the list increases the pure version scales linearly while the IO version degrades exponentially. Here are the execution times per list element in ns as the list size increases: > > Size of list Pure IO > 1000 8.7 8.3 > 10000 8.7 18 > 100000 8.8 63 > 1000000 9.3 786 > > This seems to be due to increased GC activity in the IO case. The GC stats for list size 1 million are: > > IO case: %GC time 66.1% (61.1% elapsed) > Pure case: %GC time 2.6% (3.3% elapsed) > > Not sure if there is a way to write this code in IO monad which can reduce this overhead. Something to be aware of is that GHC currently can't pass multiple return values in registers (that may not be a 100% accurate statement, but a reasonable high level summary, see ticket for details) https://ghc.haskell.org/trac/ghc/ticket/2289 This can bite you with with the IO monad as having to pass around the world state token turns single return values into multiple return values (i.e., the new state token plus the returned value). I haven't actually dug into your code to see if this is part of the problem, but figured I would mention it. Cheers! -Tyson From david.feuer at gmail.com Sat May 14 20:26:59 2016 From: david.feuer at gmail.com (David Feuer) Date: Sat, 14 May 2016 16:26:59 -0400 Subject: suboptimal ghc code generation in IO vs equivalent pure code case In-Reply-To: <573788D2.40104@gmail.com> References: <573788D2.40104@gmail.com> Message-ID: The state token is zero-width and should therefore be erased altogether in code generation. On May 14, 2016 4:21 PM, "Tyson Whitehead" wrote: > On 14/05/16 02:31 PM, Harendra Kumar wrote: > >> The difference seems to be entirely due to memory pressure. At list size >> 1000 both pure version and IO version perform equally. But as the size of >> the list increases the pure version scales linearly while the IO version >> degrades exponentially. Here are the execution times per list element in ns >> as the list size increases: >> >> Size of list Pure IO >> 1000 8.7 8.3 >> 10000 8.7 18 >> 100000 8.8 63 >> 1000000 9.3 786 >> >> This seems to be due to increased GC activity in the IO case. The GC >> stats for list size 1 million are: >> >> IO case: %GC time 66.1% (61.1% elapsed) >> Pure case: %GC time 2.6% (3.3% elapsed) >> >> Not sure if there is a way to write this code in IO monad which can >> reduce this overhead. >> > > Something to be aware of is that GHC currently can't pass multiple return > values in registers (that may not be a 100% accurate statement, but a > reasonable high level summary, see ticket for details) > > https://ghc.haskell.org/trac/ghc/ticket/2289 > > This can bite you with with the IO monad as having to pass around the > world state token turns single return values into multiple return values > (i.e., the new state token plus the returned value). > > I haven't actually dug into your code to see if this is part of the > problem, but figured I would mention it. > > Cheers! -Tyson > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From harendra.kumar at gmail.com Sat May 14 21:18:18 2016 From: harendra.kumar at gmail.com (Harendra Kumar) Date: Sun, 15 May 2016 02:48:18 +0530 Subject: suboptimal ghc code generation in IO vs equivalent pure code case In-Reply-To: References: <573788D2.40104@gmail.com> Message-ID: I have stared at the cmm and assembly quite a bit. Indeed there is no trace of a token in cmm and assembly as expected. Here is what is happening. In the IO case the entire original list is evaluated and unfolded on the stack first. During the recursion, the stack will have as many closure pointers as the size of the list, last element of the list being on top of the stack. When we finish recursing the original list, the stack unwinds and we start creating the closures for the new list in the reverse order. This is all pretty evident from the cmm dump output. This process retains a lot of heap and stack memory (proportional to the size of the list) which will require the GC to do a lot of walking, fixing and copying. I guess that's where the additional cost is coming from. When the list size increases this cost increases nonlinearly. That explains why at lower list sizes the IO version performs not just equal to but a tad better than the pure version because if GC overhead is not considered this code is in fact more efficient. -harendra On 15 May 2016 at 01:56, David Feuer wrote: > The state token is zero-width and should therefore be erased altogether in > code generation. > On May 14, 2016 4:21 PM, "Tyson Whitehead" wrote: > >> On 14/05/16 02:31 PM, Harendra Kumar wrote: >> >>> The difference seems to be entirely due to memory pressure. At list size >>> 1000 both pure version and IO version perform equally. But as the size of >>> the list increases the pure version scales linearly while the IO version >>> degrades exponentially. Here are the execution times per list element in ns >>> as the list size increases: >>> >>> Size of list Pure IO >>> 1000 8.7 8.3 >>> 10000 8.7 18 >>> 100000 8.8 63 >>> 1000000 9.3 786 >>> >>> This seems to be due to increased GC activity in the IO case. The GC >>> stats for list size 1 million are: >>> >>> IO case: %GC time 66.1% (61.1% elapsed) >>> Pure case: %GC time 2.6% (3.3% elapsed) >>> >>> Not sure if there is a way to write this code in IO monad which can >>> reduce this overhead. >>> >> >> Something to be aware of is that GHC currently can't pass multiple return >> values in registers (that may not be a 100% accurate statement, but a >> reasonable high level summary, see ticket for details) >> >> https://ghc.haskell.org/trac/ghc/ticket/2289 >> >> This can bite you with with the IO monad as having to pass around the >> world state token turns single return values into multiple return values >> (i.e., the new state token plus the returned value). >> >> I haven't actually dug into your code to see if this is part of the >> problem, but figured I would mention it. >> >> Cheers! -Tyson >> _______________________________________________ >> Glasgow-haskell-users mailing list >> Glasgow-haskell-users at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users >> > > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthony_clayden at clear.net.nz Mon May 16 01:25:25 2016 From: anthony_clayden at clear.net.nz (AntC) Date: Mon, 16 May 2016 01:25:25 +0000 (UTC) Subject: Magic classes for Overloaded Record Fields, overlaps, FunDeps References: <94809e65f6204690a6059842f7847c95@DB4PR30MB030.064d.mgd.msft.net> Message-ID: > Simon Peyton Jones microsoft.com> writes: > Hi Simon, I don't think there's an 'issue' in the sense fundeps can achieve something that type-families can't (or v.v.). It's more about elegance and ergonomics of the code to achieve it. (I'll try to avoid a question of judgment shading into a matter of taste ;-) > | > I have been vacillating between type families and fundeps for the ORF > | > classes. I hadn't fully appreciated this point about overlap, but I > | > think it is a reason to prefer fundeps, which is the direction in > | > which I'm leaning. I'd be grateful for feedback on this issue though! > ... > > | The difficulty remains that as soon as you want overlaps in such a way > | that takes you to FunDeps, it's very difficult to 'mix modes' with type > | families. > > Can one give a standalone explanation of the fundep/type-family/overlap > issue here? > Or is it explained on a wiki page? > Neither is it specifically about the Overloaded Record Fields design, nor anonymous records -- it's just that you need a meaty application like those to demonstrate the subtleties. I've taken some time to review what's explained where. I think most of it has come up before, scattered various places. I see 4 inter-related pieces. I'll volunteer to write these up, if somebody would like to tell me where. 1. ORF has chosen the `Has` class to be implemented using FunDeps, rather than type families. The motivation (for 'option 1') is documented on the wiki, pointing to the original design (with which you were involved). https://ghc.haskell.org/trac/ghc/wiki/Records/OverloadedRecordFields/Design Choosing FunDeps is right if we move on to anonymous records, which are bound to face some egregious overlaps, so... 2. Adam's point (and example) about overlap of instances wrt anonymous records, has been 'in the wind' before -- e.g. comment here https://typesandkinds.wordpress.com/2013/04/29/coincident-overlap-in-type- families/#comment-152 Detecting the unwanted ambiguity of instance satisfaction is discussed in the HList paper's `Lacks` constraint. 3. There's a clumsiness in type families faced with such examples. Really you want to put an equation in a closed type family to 'trap' or guard against the ambiguity. But in effect say "treat this as an error, not a proper instance". So the Instance Chain work (for example) has a 'fails' outcome. 4. When you've decided to implement some piece using FunDeps, It works OK to delegate some of the type-level calculations to (possibly closed) type families, per Adam's response to my q on this thread. But v.v. if you have some piece using type families, and then want to delegate to FunDeps because of nasty overlaps; that tends to get awkward. It's better than it was with the first releases of type families. AntC From simonpj at microsoft.com Mon May 16 10:00:09 2016 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Mon, 16 May 2016 10:00:09 +0000 Subject: suboptimal ghc code generation in IO vs equivalent pure code case In-Reply-To: References: <573788D2.40104@gmail.com> Message-ID: As Harendra has found, the biggest difference is probably that the IO version is necessarily strict, constructing the entire list (via the stack) before it returns, whereas the pure one is lazy, constructing the list only on demand. So the memory footprint of the lazy one will be asymptotically smaller (constant instead of linear in the list size). Simon From: Glasgow-haskell-users [mailto:glasgow-haskell-users-bounces at haskell.org] On Behalf Of Harendra Kumar Sent: 14 May 2016 22:18 To: David Feuer Cc: GHC users Subject: Re: suboptimal ghc code generation in IO vs equivalent pure code case I have stared at the cmm and assembly quite a bit. Indeed there is no trace of a token in cmm and assembly as expected. Here is what is happening. In the IO case the entire original list is evaluated and unfolded on the stack first. During the recursion, the stack will have as many closure pointers as the size of the list, last element of the list being on top of the stack. When we finish recursing the original list, the stack unwinds and we start creating the closures for the new list in the reverse order. This is all pretty evident from the cmm dump output. This process retains a lot of heap and stack memory (proportional to the size of the list) which will require the GC to do a lot of walking, fixing and copying. I guess that's where the additional cost is coming from. When the list size increases this cost increases nonlinearly. That explains why at lower list sizes the IO version performs not just equal to but a tad better than the pure version because if GC overhead is not considered this code is in fact more efficient. -harendra On 15 May 2016 at 01:56, David Feuer > wrote: The state token is zero-width and should therefore be erased altogether in code generation. On May 14, 2016 4:21 PM, "Tyson Whitehead" > wrote: On 14/05/16 02:31 PM, Harendra Kumar wrote: The difference seems to be entirely due to memory pressure. At list size 1000 both pure version and IO version perform equally. But as the size of the list increases the pure version scales linearly while the IO version degrades exponentially. Here are the execution times per list element in ns as the list size increases: Size of list Pure IO 1000 8.7 8.3 10000 8.7 18 100000 8.8 63 1000000 9.3 786 This seems to be due to increased GC activity in the IO case. The GC stats for list size 1 million are: IO case: %GC time 66.1% (61.1% elapsed) Pure case: %GC time 2.6% (3.3% elapsed) Not sure if there is a way to write this code in IO monad which can reduce this overhead. Something to be aware of is that GHC currently can't pass multiple return values in registers (that may not be a 100% accurate statement, but a reasonable high level summary, see ticket for details) https://ghc.haskell.org/trac/ghc/ticket/2289 This can bite you with with the IO monad as having to pass around the world state token turns single return values into multiple return values (i.e., the new state token plus the returned value). I haven't actually dug into your code to see if this is part of the problem, but figured I would mention it. Cheers! -Tyson _______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users at haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users _______________________________________________ Glasgow-haskell-users mailing list Glasgow-haskell-users at haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From voldermort at hotmail.com Mon May 16 12:39:07 2016 From: voldermort at hotmail.com (Jeremy) Date: Mon, 16 May 2016 05:39:07 -0700 (MST) Subject: TDNR without new operators or syntax changes Message-ID: <1463402347025-5835927.post@n5.nabble.com> Previous attempts to propose TDNR [1] have met with opposition over the accompanying proposal to change the syntax of the dot or add a new operator for postfix application. However, nothing about TDNR - other than certain motivating examples - actually requires changes to the syntax of Haskell or new operators. TDNR could be implemented as an extension which just give GHC a new way of disambiguating function names, and nothing else. This would still have some advantages: - Redundant module qualification no longer required. - Unqualified imports could be changed to a different module with the same interface (a very poor-man's backpack) without any other code changes. - People who want TDNR with postfix function application will only need to define a simple postfix operator. I would therefore like to propose TNDR without any syntax/prelude changes. [1] https://prime.haskell.org/wiki/TypeDirectedNameResolution -- View this message in context: http://haskell.1045720.n5.nabble.com/TDNR-without-new-operators-or-syntax-changes-tp5835927.html Sent from the Haskell - Glasgow-haskell-users mailing list archive at Nabble.com. From anthony_clayden at clear.net.nz Tue May 17 06:44:43 2016 From: anthony_clayden at clear.net.nz (AntC) Date: Tue, 17 May 2016 06:44:43 +0000 (UTC) Subject: TDNR without new operators or syntax changes References: <1463402347025-5835927.post@n5.nabble.com> Message-ID: > Jeremy hotmail.com> writes: Hi Jeremy, I feel your frustration at the slow evolution of records proposals. There are many reasons, including that there has been much debate and little consensus. > > Previous attempts to propose TDNR [1] have met with opposition over the > accompanying proposal to change the syntax of the dot or add a new operator > for postfix application. > > However, nothing about TDNR - other than certain motivating examples - > actually requires changes to the syntax of Haskell or new operators. ... You are possibly confusing parts of one records proposal with parts of TDNR. In https://ghc.haskell.org/trac/ghc/wiki/Records/ DeclaredOverloadedRecordFields/DotPostfix dot as postfix function apply could indeed be dispensed with. But that is not within a TDNR approach. > TDNR could be implemented as an extension which just give GHC > a new way of disambiguating function names, and nothing else. No. For TDNR GHC needs some syntactic signal to trigger disambiguation. The idea was the dot (or some other operator) says "here comes a field label". Without that signal, all GHC can see is a name -- could be a variable, a function, anything. > > I would therefore like to propose TNDR without any syntax/prelude changes. > I suspect that if you took the syntax away from TDNR, you'd have very little left. If there were a feasible records approach which needed no syntax/prelude changes, I rather think it would have been found by now. In terms of what's available now, have you looked at the lens approaches or Nikita's recent Anonymous Records? https://ghc.haskell.org/trac/ghc/wiki/Records/Volkov Have you looked at what's coming very shortly in GHC v8.0? https://ghc.haskell.org/trac/ghc/wiki/Records/OverloadedRecordFields I suspect Part 1: Duplicate Record Fields will go a long way towards disambinguating label names, with only an occasional need for explicit type signatures. How does that compare with TDNR minus syntax? AntC From voldermort at hotmail.com Tue May 17 07:21:28 2016 From: voldermort at hotmail.com (Jeremy) Date: Tue, 17 May 2016 00:21:28 -0700 (MST) Subject: TDNR without new operators or syntax changes In-Reply-To: References: <1463402347025-5835927.post@n5.nabble.com> Message-ID: <1463469688161-5835978.post@n5.nabble.com> AntC wrote > No. For TDNR GHC needs some syntactic signal to trigger disambiguation. > ... > I suspect that if you took the syntax away from TDNR, you'd have very > little left. To copy an example from the TDNR wiki page: module Foo where import Button( Button, reset ) as B import Canvas( Canvas, reset ) as C f :: Button -> Canvas -> IO () f b c = do { B.reset b; C.reset c } With syntaxless TDNR enabled, the last line could be: f b c = do { reset b; reset c } This requires no syntactic signal. The compiler will see two candidate definitions for reset, and in each case, pick the one which matches its argument. -- View this message in context: http://haskell.1045720.n5.nabble.com/TDNR-without-new-operators-or-syntax-changes-tp5835927p5835978.html Sent from the Haskell - Glasgow-haskell-users mailing list archive at Nabble.com. From anthony_clayden at clear.net.nz Wed May 18 09:28:06 2016 From: anthony_clayden at clear.net.nz (AntC) Date: Wed, 18 May 2016 09:28:06 +0000 (UTC) Subject: TDNR without new operators or syntax changes References: <1463402347025-5835927.post@n5.nabble.com> <1463469688161-5835978.post@n5.nabble.com> Message-ID: > > With syntaxless TDNR enabled, the last line could be: > > f b c = do { reset b; reset c } > Heck, I didn't think you meant something that radical. So bare name in a function application context is to need disambiguating. I think you'll find rather a lot of those in existing code. So this is a code-breaking change. The merit of adding an extension via new syntax, is that in the places you didn't and don't use the syntax, nothing breaks. Note that currently if you have in scope both a bare name and that same name qualified, the bare name applies, the compiler ignores the qualified one, unless you deliberately write in the qualification. How much are people deliberately playing with same name bare and qualified? Probably seldom. How much are people importing qualified so they don't have to worry about their local name clashing? Suddenly they'll have to start worrying. So I see difficulties. Perhaps TDNR's dot-apply would have suffered them too, but * it was expected that dot-suffixing would only be used for field access (and access to field-like functions) * it was expected that the argument to the function would be near by, so the compiler wouldn't have to look far for the type by which to resolve * the debate around TDNR didn't get into this much detail, because the dot syntax proposal met such a violent counter-reaction * so TDNR never got beyond a wiki page, AFAIK. One thing TDNR certainly suffers is that you still can't declare two distinct data types in the same module with the same field name. ORF Part 1 in GHC 8.0 at least allows that. Difficulties: overloaded functions (methods) don't have a specific data type argument, by which we could disambiguate the bare name. (Disambiguation is supposed to be lightweight, we don't want to go hunting for instances.) So a lot of the bare function names (eg imported from the Prelude) are going to run into trouble. In f $ g $ h x we have to disambiguate h to know its result type, before we can disambiguate g, before we can disambiguate f. Type improvement could rapidly get stuck. Ah! but those are not bare names in a function application context. So ($) contexts (and function composition (.) contexts) are going to behave different to f (g (h x)) But then your OP suggested users who want postfix apply could define their own operator. Presumably TDNR *is* to apply to that sort of function application(?) So to ($) or not? to (.) or not? to sections or not? to bare name functions supplied as arguments (to say map) or not? I think this is the point SPJ would ask you to write up a proposal on a wiki, to the same level of detail as all those other records proposals. AntC From voldermort at hotmail.com Wed May 18 13:18:39 2016 From: voldermort at hotmail.com (Jeremy) Date: Wed, 18 May 2016 06:18:39 -0700 (MST) Subject: TDNR without new operators or syntax changes In-Reply-To: References: <1463402347025-5835927.post@n5.nabble.com> <1463469688161-5835978.post@n5.nabble.com> Message-ID: <1463577519915-5836060.post@n5.nabble.com> AntC wrote > I think you'll find rather a lot of those in existing code. > So this is a code-breaking change. Could you give an example of existing code that would break? This certainly wasn't what I had in mind. -- View this message in context: http://haskell.1045720.n5.nabble.com/TDNR-without-new-operators-or-syntax-changes-tp5835927p5836060.html Sent from the Haskell - Glasgow-haskell-users mailing list archive at Nabble.com. From anthony_clayden at clear.net.nz Fri May 20 01:00:21 2016 From: anthony_clayden at clear.net.nz (AntC) Date: Fri, 20 May 2016 01:00:21 +0000 (UTC) Subject: TDNR without new operators or syntax changes References: <1463402347025-5835927.post@n5.nabble.com> <1463469688161-5835978.post@n5.nabble.com> <1463577519915-5836060.post@n5.nabble.com> Message-ID: > Jeremy hotmail.com> writes: > > AntC wrote > > I think you'll find rather a lot of those in existing code. > > So this is a code-breaking change. > > Could you give an example of existing code that would break? > This certainly wasn't what I had in mind. Then what do you have in mind? "Do not break existing code" is not a design. "Syntaxless TDNR" is not a design; it's not even a thing. I've given you two lengthy replies, with pointers to further material. You've given me two sentences, and a snippet of code indistinguishable from gazillions of bare name function calls in existing code. I urge you (for the third time) to look at ORF Part 1: Duplicate Record Fields. See the type-directed resolution of label names. AntC From bertram.felgenhauer at googlemail.com Sat May 21 15:14:19 2016 From: bertram.felgenhauer at googlemail.com (Bertram Felgenhauer) Date: Sat, 21 May 2016 17:14:19 +0200 Subject: TDNR without new operators or syntax changes In-Reply-To: References: <1463402347025-5835927.post@n5.nabble.com> <1463469688161-5835978.post@n5.nabble.com> Message-ID: <20160521151419.GA8247@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> AntC wrote: > > > > With syntaxless TDNR enabled, the last line could be: > > > > f b c = do { reset b; reset c } > > > > Heck, I didn't think you meant something that radical. > So bare name in a function application context is to need disambiguating. > > I think you'll find rather a lot of those in existing code. > So this is a code-breaking change. I don't understand your conclusion. The code above, in context, is currently illegal: There are two "reset" functions in scope, and the compiler will ask the programmer to specify which of them they intended to use. Jeremy's proposal, I believe, is that the compiler should pick /the/ possibility that type-checks (f had a type signature that would allow only one combination to work); . Note that this has nothing to do with record fields at all, except that they give rise to a compelling use case. (I'm not endorsing the proposal, just trying to clarify what it is.) Cheers, Bertram From ben at well-typed.com Sat May 21 15:18:35 2016 From: ben at well-typed.com (Ben Gamari) Date: Sat, 21 May 2016 17:18:35 +0200 Subject: [ANNOUNCE] GHC 8.0.1 is available! Message-ID: <8737pbxwvo.fsf@smart-cactus.org> =============================================== The Glasgow Haskell Compiler -- version 8.0.1 =============================================== The GHC developers are very pleased to announce the release of the first new super-major version of our Haskell compiler in six years, GHC 8.0.1. This release features dozens of exciting developments including, * A more refined interface for implicit call-stacks, allowing libraries to provide more helpful runtime error messages to users * The introduction of the DuplicateRecordFields language extension, allowing multiple record types to declare fields of the same name * Significant improvements in error message readability and content, including facilities for libraries to provide custom error messages, more aggressive warnings for fragile rewrite rules, and more helpful errors for missing imports * A rewritten and substantially more thorough pattern match checker, providing more precise exhaustiveness checking in GADT pattern matches * More reliable debugging information including experimental backtrace support, allowing better integration with traditional debugging tools * Support for desugaring do-notation to use Applicative combinators, allowing the intuitive do notation to be used in settings which previously required the direct use of Applicative combinators * The introduction of Strict and StrictData language extensions, allowing modules to be compiled with strict-by-default evaluation of bindings * Great improvements in portability, including more reliable linking on Windows, a new PPC64 code generator, support for the AIX operating system, unregisterised m68k support, and significant stabilization on ARM targets * A greatly improved user's guide, with beautiful and modern PDF and HTML output * Introduction of type application syntax, reducing the need for proxies * More complete support for pattern synonyms, including record pattern synonyms and the ability to export patterns "bundled" with a type, as you would a data constructor * Support for injective type families and recursive superclass relationships * An improved generics representation leveraging GHC's support for type-level literals * The TypeInType extension, which unifies types and kinds, allowing GHC to reason about kind equality and enabling promotion of more constructs to the type level * ...and more! A more thorough list of the changes included in this release can be found in the release notes, https://downloads.haskell.org/~ghc/8.0.1/docs/html/users_guide/8.0.1-notes.html As always, we have collected various points of interest for users of previous GHC releases on the GHC 8.0 migration page, https://ghc.haskell.org/trac/ghc/wiki/Migration/8.0 Please let us know if you encounter anything missing or unclear on this page. This release is the culmination of nearly eighteen months of effort by over one hundred contributors. We'd like to thank everyone who has contributed code, bug reports, and feedback over the past year. It's only because of their efforts that GHC continues to evolve. How to get it ~~~~~~~~~~~~~ Both the source tarball and binary distributions for a wide variety of platforms are available at, http://www.haskell.org/ghc/ Background ~~~~~~~~~~ Haskell is a standardized lazy functional programming language. The Glasgow Haskell Compiler (GHC) is a state-of-the-art programming suite for Haskell. Included is an optimising compiler generating efficient code for a variety of platforms, together with an interactive system for convenient, quick development. The distribution includes space and time profiling facilities, a large collection of libraries, and support for various language extensions, including concurrency, exceptions, and foreign language interfaces. GHC is distributed under a BSD-style open source license. Supported Platforms ~~~~~~~~~~~~~~~~~~~ The list of platforms we support, and the people responsible for them, can be found on the GHC wiki http://ghc.haskell.org/trac/ghc/wiki/Platforms Ports to other platforms are possible with varying degrees of difficulty. The Building Guide describes how to go about porting to a new platform: http://ghc.haskell.org/trac/ghc/wiki/Building Developers ~~~~~~~~~~ We welcome new contributors. Instructions on getting started with hacking on GHC are available from GHC's developer site, http://ghc.haskell.org/trac/ghc/ Community Resources ~~~~~~~~~~~~~~~~~~~ There are mailing lists for GHC users, develpoers, and monitoring bug tracker activity; to subscribe, use the web interfaces at http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-tickets There are several other Haskell and GHC-related mailing lists on www.haskell.org; for the full list, see https://mail.haskell.org/cgi-bin/mailman/listinfo Some GHC developers hang out on the #ghc and #haskell of the Freenode IRC network, too: http://www.haskell.org/haskellwiki/IRC_channel Please report bugs using our bug tracking system. Instructions on reporting bugs can be found here: http://www.haskell.org/ghc/reportabug -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 472 bytes Desc: not available URL: From anthony_clayden at clear.net.nz Sun May 22 04:53:41 2016 From: anthony_clayden at clear.net.nz (AntC) Date: Sun, 22 May 2016 04:53:41 +0000 (UTC) Subject: TDNR without new operators or syntax changes References: <1463402347025-5835927.post@n5.nabble.com> <1463469688161-5835978.post@n5.nabble.com> <20160521151419.GA8247@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> Message-ID: > Bertram Felgenhauer googlemail.com> writes: > ... > I don't understand your conclusion. Hi Bertram, I'm trying to tease out of Jeremy, what he thinks his proposal amounts to. Seeing as he's given very little clue so far. So if I've arrived at an incorrect conclusion, I want him to tell me where/what's gone wrong. The point, I believe, although Jeremy's not said, is exactly as you say: > Note that this has nothing to do with record fields at all, ... So let's modify Jeremy's code snippet > > > With syntaxless TDNR enabled, the last line could be: > > > > > > f b c = do { reset b; reset c } to something that's nothing to do with record fields: f b c = do { print b; print c } `print` is a bare name appearing in a function application context. It therefore needs disambiguating. It's type :: (Show a) => a -> IO () Are `f`s params of type `a`, or of type (Show a) => a ? No, they're String or Int or whatever. Compile fail. Now you might say `print` doesn't need disambiguating. Why not? Jeremy hasn't said. Perhaps Jeremy intends a rule: If there's only one version of the function in scope, choose that. If so, Jeremy hasn't said. "syntaxless TDNR" doesn't give me a clue. I don't think TDNR intended such a rule. Suppose I do Jeremy's job for him and presume such a rule. OK then. Second example: Suppose I don't like what's happening with the FTP library changes, specifically not the generalisation of `length`. So I write my own import qualified Data.List as D -- note the qualified length :: [a] -> Int. -- my version restricted to lists length = D.length Now I use length in a bare name function application ... (length [1, 2, 3, 4, 5]) ... Therefore length needs disambiguating. a) the disambiguator can't match argument type [Int] to [a]. b) even if it could, `length` remains ambiguous between my definition and the import. (As it happens, they're the same, but can the disambiguator tell that? Jeremy hasn't said what he thinks the disambiguator can do. But this is way beyond the original TDNR proposal.) > Jeremy's proposal, I believe, is that the compiler should pick /the/ > possibility that type-checks .. Well you're guessing what is the proposal, as much as I am. In both the examples above, neither type checks. (Or arguably both do if we allow polymorphic type checking. But should that allow both paramteric and ad-hoc polymorphic? Polymorphic checking is too generous for the way TDNR was spec'd.) > Note that this has nothing to do with record fields at all, > except that they give rise to a compelling use case. > So the compelling use case has given us ORF Part 1 in GHC 8.0. That has a clear antecedent in TDNR. But, pace your note, it applies _only_ for record fields. If you look at the implementation notes, the functions derived from field labels are specially flagged. So disambiguation applies only to them, not in general to bare function names. > (I'm not endorsing the proposal, ... Quite. > ... just trying to clarify what it is.) > Quite. And since 8.0 is officially released as of this weekend, I rather think "syntaxless TDNR", whatever it is/was, is stillborn. AntC From voldermort at hotmail.com Sun May 22 07:17:43 2016 From: voldermort at hotmail.com (Jeremy .) Date: Sun, 22 May 2016 07:17:43 +0000 Subject: TDNR without new operators or syntax changes Message-ID: Yes, that it indeed was I meant. AntC seems to be replying to a much more complicated/invasive proposal than what I had intended, apologies if I wasn't clear. (I see in retrospect that I may have misunderstood the original TDNR proposal, understandably leading to confusion.) 1. If the compiler encounters a term f a, and there is more than one definition for f in scope (after following all of the usual rules for qualified imports); 2. And exactly one of these definitions matches the type of a (or the expected type of f if given); 3. Then this is the definition to use. That is all, although point 2 could be extended to consider the return type of f or other arguments as well. Even with the extension on, it would have no effect on programs which compile without it. This has nothing to do with ORF, which relies on magic type classes (although there is some overlap in what it can achieve). The primary use-case I had in mind is disambiguating name clashes between local and/or unqualified imports. Cross-posting to cafe and libraries as this doesn't seem to have attracted a lot of interest in users. Maybe it's just a boring proposal, maybe I didn't post it to the right list. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lemming at henning-thielemann.de Sun May 22 08:03:31 2016 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Sun, 22 May 2016 10:03:31 +0200 (CEST) Subject: TDNR without new operators or syntax changes In-Reply-To: References: Message-ID: On Sun, 22 May 2016, Jeremy . wrote: > 1. If the compiler encounters a term f a, and there is more than one definition for f in scope (after following all of > the usual rules for qualified imports); > > 2. And exactly one of these definitions matches?the?type of a (or the expected type of f if given); > > 3. Then this is the definition to use. I know people are unhappy with Haskell's records and module system, but I still think that's because these language features are not used properly. Type classes are the tool to write generic code and reduce combinatoric explosion of functions and modules are a way to collect functions per type. Following this principle you give function names that make sense together with the module name like File.write or Channel.write. Then there is no need for the compiler to disambiguate unqualified identifiers and you keep the type and the module issues separated. Today, if you stick to the unqualified style, and I guess you want to import modules without explicit identifier list, too, then your code can break whenever the imported modules add identifiers. In order to prevent this you have to use tight package import version bounds in your Cabal file. Your proposal would not solve this problem. Type-driven name resolution may disambiguate identifiers in some case but not in all ones, thus you still need to use tight version bounds. The proposal seems to be a big complication. The compiler can no longer emit a message: "Type of function f mismatch", but it has to emit "If f means A.f, then the type error would be ..., but if it means B.f, then the type error would be ..." Or even worse: "If f means A.f and g means C.g, then the type error would be ... if f means B.f and g means C.g ... or f means A.f and g means D.g ..." The proposal also is more error-prone, because if you make a type-error this could be shadowed by a function in scope with a type that matches accidentally. A function with a very generic type and a generic name might be such an accidental match. That said, the more complicated the proposals become the more one should reconsider about whether one runs in the right direction. I think this is the case here. The simple answer could be: Switch to a cleaner coding style! From voldermort at hotmail.com Sun May 22 08:38:20 2016 From: voldermort at hotmail.com (Jeremy) Date: Sun, 22 May 2016 01:38:20 -0700 (MST) Subject: TDNR without new operators or syntax changes In-Reply-To: References: <1463402347025-5835927.post@n5.nabble.com> Message-ID: <1463906300679-5836376.post@n5.nabble.com> Henning Thielemann wrote > I know people are unhappy with Haskell's records and module system, but I > still think that's because these language features are not used properly. > Type classes are the tool to write generic code and reduce combinatoric > explosion of functions and modules are a way to collect functions per > type. Following this principle you give function names that make sense > together with the module name like File.write or Channel.write. Then there > is no need for the compiler to disambiguate unqualified identifiers and > you keep the type and the module issues separated. The issue is with importing third-party modules, where you can't combine them into sensible type classes without writing you own abstraction layer. And in some cases, the similarities are in name only, so type classes wouldn't work in any case. -- View this message in context: http://haskell.1045720.n5.nabble.com/TDNR-without-new-operators-or-syntax-changes-tp5835927p5836376.html Sent from the Haskell - Glasgow-haskell-users mailing list archive at Nabble.com. From bertram.felgenhauer at googlemail.com Sun May 22 12:41:29 2016 From: bertram.felgenhauer at googlemail.com (Bertram Felgenhauer) Date: Sun, 22 May 2016 14:41:29 +0200 Subject: TDNR without new operators or syntax changes In-Reply-To: References: Message-ID: <20160522124128.GB1592@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> Jeremy . wrote: > Yes, that it indeed was I meant. AntC seems to be replying to a much > more complicated/invasive proposal than what I had intended, apologies > if I wasn't clear. (I see in retrospect that I may have misunderstood > the original TDNR proposal, understandably leading to confusion.) Your client broke the thread (it didn't add any References or In-Reply-To headers, and in my case I received the haskell-cafe copy first, so matching by Subject didn't work either). In any case this paragraph wouldn't make sense to anyone on the haskell-cafe or libraries mailing lists. > 1. If the compiler encounters a term f a, and there is more than one > definition for f in scope (after following all of the usual rules for > qualified imports); > > 2. And exactly one of these definitions matches the type of a (or the > expected type of f if given); > > 3. Then this is the definition to use. I now find that Anthony's concerns are justified. The question is, what exactly does the type matching in step 2 involve? If it is a recursive call to the type-checker then you'll have a huge performance problem. If, on the other hand, you only take into account what is known about the type of a at a given time, then you need special treatment for unambiguous names or even trivial programs will fail to type-check, just as Anthony said. There's another concern: will type checking still be predictable? How do you explain type checking errors that arise from ambiguity to the user? > That is all, although point 2 could be extended to consider the return > type of f or other arguments as well. Even with the extension on, it > would have no effect on programs which compile without it. Right, so what is special about the first argument? Do you have any use case in mind that goes beyond record field selectors? Cheers, Bertram From voldermort at hotmail.com Sun May 22 15:01:07 2016 From: voldermort at hotmail.com (Jeremy) Date: Sun, 22 May 2016 08:01:07 -0700 (MST) Subject: TDNR without new operators or syntax changes In-Reply-To: <20160522124128.GB1592@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> References: <1463402347025-5835927.post@n5.nabble.com> <20160522124128.GB1592@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> Message-ID: <1463929267673-5836393.post@n5.nabble.com> Bertram Felgenhauer-2 wrote >> 1. If the compiler encounters a term f a, and there is more than one >> definition for f in scope (after following all of the usual rules for >> qualified imports); >> >> 2. And exactly one of these definitions matches the type of a (or the >> expected type of f if given); >> >> 3. Then this is the definition to use. > > I now find that Anthony's concerns are justified. The question is, what > exactly does the type matching in step 2 involve? If it is a recursive > call to the type-checker then you'll have a huge performance problem. I was concerned about this, but not knowing anything about how type-checking/inference is implemented, I wouldn't know if this is actually a problem or not. > If, on the other hand, you only take into account what is known about > the type of a at a given time, then you need special treatment for > unambiguous names or even trivial programs will fail to type-check, just > as Anthony said. Why special treatment for unambiguous names? They shouldn't be effected at all by this. -- View this message in context: http://haskell.1045720.n5.nabble.com/TDNR-without-new-operators-or-syntax-changes-tp5835927p5836393.html Sent from the Haskell - Glasgow-haskell-users mailing list archive at Nabble.com. From adam at well-typed.com Tue May 24 08:07:58 2016 From: adam at well-typed.com (Adam Gundry) Date: Tue, 24 May 2016 09:07:58 +0100 Subject: TDNR without new operators or syntax changes In-Reply-To: <1463929267673-5836393.post@n5.nabble.com> References: <1463402347025-5835927.post@n5.nabble.com> <20160522124128.GB1592@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> <1463929267673-5836393.post@n5.nabble.com> Message-ID: Thanks for starting this discussion! Having spent more time thinking about record field overloading than perhaps I should, here are some things to think about... On 22/05/16 16:01, Jeremy wrote: > Bertram Felgenhauer-2 wrote >>> 1. If the compiler encounters a term f a, and there is more than one >>> definition for f in scope (after following all of the usual rules for >>> qualified imports); >>> >>> 2. And exactly one of these definitions matches the type of a (or the >>> expected type of f if given); >>> >>> 3. Then this is the definition to use. >> >> I now find that Anthony's concerns are justified. The question is, what >> exactly does the type matching in step 2 involve? If it is a recursive >> call to the type-checker then you'll have a huge performance problem. > > I was concerned about this, but not knowing anything about how > type-checking/inference is implemented, I wouldn't know if this is actually > a problem or not. Unfortunately this is indeed a problem. When the type-checker encounters `f a`, it infers the type of `f`, ensures it is a function type `s -> t`, then checks that `a` has type `s`. But if there are multiple possible `f`s in scope, what should it do? Options include: 1. run the entire type-checking process for each possible type of `f` (this is almost certainly too slow); 2. give ad-hoc rules to look at type information from the context or the inferred type of the argument `a`; 3. introduce a name resolution constraint and defer it to the constraint solver (the same way that solving for typeclasses works); 4. require a type signature in a fixed location. Both 2 and 3 would need careful specification, as they run the risk of exposing the type-checking algorithm to the user, and changing the typing rules depending on what is in scope may be problematic. In particular, if `f` has a higher-rank type than bidirectional type inference is likely to break down. The DuplicateRecordFields approach is to make use of bidirectional type inference (a bit like 2, but without looking at the type of the argument) and otherwise require a type signature. This is carefully crafted to avoid needing to change existing typing rules. The wiki page [1] sets out some of the cases in which this does and doesn't work. Point 3 is rather like the magic classes in the OverloadedRecordFields proposal [2] (this isn't in GHC HEAD yet, but an early version is on Phab [3]). >> If, on the other hand, you only take into account what is known about >> the type of a at a given time, then you need special treatment for >> unambiguous names or even trivial programs will fail to type-check, just >> as Anthony said. > > Why special treatment for unambiguous names? They shouldn't be effected at > all by this. There are some design choices here as well, separately from the options above. Some possibilities are: A. leave unambiguous names alone, and only do the special thing for the ambiguous ones (this doesn't break existing code, but can lead to confusing errors if an import is changed to introduce or remove an ambiguity, especially for options 2 or 3 above); B. do the same thing for unambiguous names as ambiguous ones (but this may well break existing code, and is likely to restrict inference too much); C. require an explicit syntactic cue that the name should be treated specially (e.g. the original TDNR's use of dot, or the # sign in the OverloadedLabels part of ORF [4]). As you can see, there are quite a few complex trade-offs here. I suspect this is at least part of the reason for the slow progress on TDNR-like extensions. It will be interesting to see how DuplicateRecordFields is received! All the best, Adam [1] https://ghc.haskell.org/trac/ghc/wiki/Records/OverloadedRecordFields/DuplicateRecordFields [2] https://ghc.haskell.org/trac/ghc/wiki/Records/OverloadedRecordFields/MagicClasses [3] https://phabricator.haskell.org/D1687 [4] https://ghc.haskell.org/trac/ghc/wiki/Records/OverloadedRecordFields/OverloadedLabels -- Adam Gundry, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/ From voldermort at hotmail.com Wed May 25 11:55:00 2016 From: voldermort at hotmail.com (Peter) Date: Wed, 25 May 2016 04:55:00 -0700 (MST) Subject: Cannot install 8.0.1 from bindist Message-ID: <1464177300318-5836591.post@n5.nabble.com> Trying to install ghc-8.0.1-x86_64-centos67-linux.tar.xz with: ./configure --prefix={prefix} make install-strip gives me an error. The tail of the log is: Registering ghc-8.0.1... for f in '{prefix}/lib/ghc-8.0.1/package.conf.d'/*; do create () { touch "$1" && chmod 644 "$1" ; } && create "$f"; done /usr/bin/install -c -m 755 -d "{prefix}/bin" for i in utils/hp2ps/dist/build/tmp/hp2ps; do \ /usr/bin/install -c -m 755 -s $i "{prefix}/bin" ; \ done /usr/bin/install -c -m 755 -d "{prefix}/lib/ghc-8.0.1/bin" for i in inplace/lib/bin/ghc-split; do \ /usr/bin/install -c -m 755 $i "{prefix}/lib/ghc-8.0.1/bin"; \ done /usr/bin/install -c -m 755 -d "{prefix}/share/doc/ghc-8.0.1" /usr/bin/install -c -m 755 -d "{prefix}/share/doc/ghc-8.0.1/html" /usr/bin/install -c -m 644 docs/index.html "{prefix}/share/doc/ghc-8.0.1/html" /usr/bin/install -c -m 755 -d "{prefix}/share/doc/ghc-8.0.1/html/libraries" for i in libraries/dist-haddock/*; do \ /usr/bin/install -c -m 644 $i "{prefix}/share/doc/ghc-8.0.1/html/libraries/"; \ done /usr/bin/install -c -m 644 libraries/prologue.txt "{prefix}/share/doc/ghc-8.0.1/html/libraries/" /usr/bin/install -c -m 755 libraries/gen_contents_index "{prefix}/share/doc/ghc-8.0.1/html/libraries/" mkdir: cannot create directory `doc/aries/ghc-prim-0.5.0.0': No such file or directory Is there any way of knowing what the call to mkdir is relative to, and why it's trying to create the directory there? -- View this message in context: http://haskell.1045720.n5.nabble.com/Cannot-install-8-0-1-from-bindist-tp5836591.html Sent from the Haskell - Glasgow-haskell-users mailing list archive at Nabble.com. From allbery.b at gmail.com Wed May 25 12:48:48 2016 From: allbery.b at gmail.com (Brandon Allbery) Date: Wed, 25 May 2016 08:48:48 -0400 Subject: Cannot install 8.0.1 from bindist In-Reply-To: <1464177300318-5836591.post@n5.nabble.com> References: <1464177300318-5836591.post@n5.nabble.com> Message-ID: On Wed, May 25, 2016 at 7:55 AM, Peter wrote: > mkdir: cannot create directory `doc/aries/ghc-prim-0.5.0.0': No such file > or > directory > "doc/aries"? That looks like a "libr" went missing somehow. -- brandon s allbery kf8nh sine nomine associates allbery.b at gmail.com ballbery at sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net -------------- next part -------------- An HTML attachment was scrubbed... URL: From dan.doel at gmail.com Wed May 25 13:55:54 2016 From: dan.doel at gmail.com (Dan Doel) Date: Wed, 25 May 2016 09:55:54 -0400 Subject: TDNR without new operators or syntax changes In-Reply-To: References: <1463402347025-5835927.post@n5.nabble.com> <20160522124128.GB1592@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> <1463929267673-5836393.post@n5.nabble.com> Message-ID: As a supplement, here's a series of definitions to think about. The question is: what should happen in each section, and why? The more detailed the answer, the better. Definitions from previous sections are in scope in subsequent ones, for convenience. The examples are arranged in a slippery slope, but there may be more good, confusing examples that I missed. --- cut --- -- 1 -- f :: T -> Int f t = ... f :: U -> Int -> Char f u = ... t :: T t = ... u :: U u = ... i1 :: Int i1 = f t -- 2 -- g :: (T -> Int) -> Int g h = h t i2 :: Int i2 = g f -- 3 -- v :: T v = t v :: U v = u i3 :: Int i3 = f v -- 4 -- class C a where c :: a -> Int instance C T where c = f i4 :: Int i4 = c v -- 5 -- main = print $ f v --- cut --- -- Dan On Tue, May 24, 2016 at 4:07 AM, Adam Gundry wrote: > Thanks for starting this discussion! Having spent more time thinking > about record field overloading than perhaps I should, here are some > things to think about... > > > On 22/05/16 16:01, Jeremy wrote: >> Bertram Felgenhauer-2 wrote >>>> 1. If the compiler encounters a term f a, and there is more than one >>>> definition for f in scope (after following all of the usual rules for >>>> qualified imports); >>>> >>>> 2. And exactly one of these definitions matches the type of a (or the >>>> expected type of f if given); >>>> >>>> 3. Then this is the definition to use. >>> >>> I now find that Anthony's concerns are justified. The question is, what >>> exactly does the type matching in step 2 involve? If it is a recursive >>> call to the type-checker then you'll have a huge performance problem. >> >> I was concerned about this, but not knowing anything about how >> type-checking/inference is implemented, I wouldn't know if this is actually >> a problem or not. > > Unfortunately this is indeed a problem. When the type-checker encounters > `f a`, it infers the type of `f`, ensures it is a function type `s -> > t`, then checks that `a` has type `s`. But if there are multiple > possible `f`s in scope, what should it do? Options include: > > 1. run the entire type-checking process for each possible type of `f` > (this is almost certainly too slow); > > 2. give ad-hoc rules to look at type information from the context or > the inferred type of the argument `a`; > > 3. introduce a name resolution constraint and defer it to the > constraint solver (the same way that solving for typeclasses works); > > 4. require a type signature in a fixed location. > > Both 2 and 3 would need careful specification, as they run the risk of > exposing the type-checking algorithm to the user, and changing the > typing rules depending on what is in scope may be problematic. In > particular, if `f` has a higher-rank type than bidirectional type > inference is likely to break down. > > The DuplicateRecordFields approach is to make use of bidirectional type > inference (a bit like 2, but without looking at the type of the > argument) and otherwise require a type signature. This is carefully > crafted to avoid needing to change existing typing rules. The wiki page > [1] sets out some of the cases in which this does and doesn't work. > > Point 3 is rather like the magic classes in the OverloadedRecordFields > proposal [2] (this isn't in GHC HEAD yet, but an early version is on > Phab [3]). > > >>> If, on the other hand, you only take into account what is known about >>> the type of a at a given time, then you need special treatment for >>> unambiguous names or even trivial programs will fail to type-check, just >>> as Anthony said. >> >> Why special treatment for unambiguous names? They shouldn't be effected at >> all by this. > > There are some design choices here as well, separately from the options > above. Some possibilities are: > > A. leave unambiguous names alone, and only do the special thing for the > ambiguous ones (this doesn't break existing code, but can lead to > confusing errors if an import is changed to introduce or remove an > ambiguity, especially for options 2 or 3 above); > > B. do the same thing for unambiguous names as ambiguous ones (but this > may well break existing code, and is likely to restrict inference too much); > > C. require an explicit syntactic cue that the name should be treated > specially (e.g. the original TDNR's use of dot, or the # sign in the > OverloadedLabels part of ORF [4]). > > > As you can see, there are quite a few complex trade-offs here. I suspect > this is at least part of the reason for the slow progress on TDNR-like > extensions. It will be interesting to see how DuplicateRecordFields is > received! > > All the best, > > Adam > > > [1] > https://ghc.haskell.org/trac/ghc/wiki/Records/OverloadedRecordFields/DuplicateRecordFields > > [2] > https://ghc.haskell.org/trac/ghc/wiki/Records/OverloadedRecordFields/MagicClasses > > [3] https://phabricator.haskell.org/D1687 > > [4] > https://ghc.haskell.org/trac/ghc/wiki/Records/OverloadedRecordFields/OverloadedLabels > > > -- > Adam Gundry, Haskell Consultant > Well-Typed LLP, http://www.well-typed.com/ > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users From voldermort at hotmail.com Wed May 25 14:43:39 2016 From: voldermort at hotmail.com (Peter) Date: Wed, 25 May 2016 07:43:39 -0700 (MST) Subject: Cannot install 8.0.1 from bindist In-Reply-To: References: <1464177300318-5836591.post@n5.nabble.com> Message-ID: <1464187419251-5836603.post@n5.nabble.com> OK, looks like a weird bug in the script which was calling make - so not ghc. Sorry for the noise. -- View this message in context: http://haskell.1045720.n5.nabble.com/Cannot-install-8-0-1-from-bindist-tp5836591p5836603.html Sent from the Haskell - Glasgow-haskell-users mailing list archive at Nabble.com. From voldermort at hotmail.com Thu May 26 09:14:45 2016 From: voldermort at hotmail.com (Peter) Date: Thu, 26 May 2016 02:14:45 -0700 (MST) Subject: TDNR without new operators or syntax changes In-Reply-To: References: <1463402347025-5835927.post@n5.nabble.com> <20160522124128.GB1592@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> <1463929267673-5836393.post@n5.nabble.com> Message-ID: <1464254085674-5836657.post@n5.nabble.com> Thank you for feeding my thoughts. How would this do for a slightly more detailed definition? 1. If the compiler encounters an ambiguous function, it will temporarily give it the type a -> b, or the type declared in the signature if there is one. 2. Type inference completes as normal. 3. If the inferred or declared type for an ambiguous name is sufficient to disambiguate it, it will be bound to the correct definition. Dan Doel wrote > f :: T -> Int > f t = ... > > f :: U -> Int -> Char > f u = ... > > t :: T > t = ... > > u :: U > u = ... > > i1 :: Int > i1 = f t Solving for everything but f, we get f :: T -> Int. > g :: (T -> Int) -> Int > g h = h t > > i2 :: Int > i2 = g f Solving for everything but f, we get f :: T -> Int. > v :: T > v = t > > v :: U > v = u > > i3 :: Int > i3 = f v May not be solvable, would fail to disambiguate. -- View this message in context: http://haskell.1045720.n5.nabble.com/TDNR-without-new-operators-or-syntax-changes-tp5835927p5836657.html Sent from the Haskell - Glasgow-haskell-users mailing list archive at Nabble.com. From dan.doel at gmail.com Thu May 26 13:57:29 2016 From: dan.doel at gmail.com (Dan Doel) Date: Thu, 26 May 2016 09:57:29 -0400 Subject: TDNR without new operators or syntax changes In-Reply-To: <1464254085674-5836657.post@n5.nabble.com> References: <1463402347025-5835927.post@n5.nabble.com> <20160522124128.GB1592@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> <1463929267673-5836393.post@n5.nabble.com> <1464254085674-5836657.post@n5.nabble.com> Message-ID: On Thu, May 26, 2016 at 5:14 AM, Peter wrote: > Solving for everything but f, we get f :: T -> Int. So TDNR happens for things in function position (applied to something). > Solving for everything but f, we get f :: T -> Int. So TDNR happens for things in argument position. > May not be solvable, would fail to disambiguate. But there's exactly one combination of f and v definitions that will succeed with the right type. So why doesn't that happen? Here's an intermediate step: i1' x = f x :: Int What happens there? Another way to phrase the question is: why would TDNR only disambiguate based on argument types of functions and not on return types? Why would function types even be a factor at all? And to be honest, I don't really think the description of what the type checker would be doing is detailed enough. If there are two ambiguous candidates how does type checking proceed with 'the type in the declared signature' when there are two possible signatures which may be completely different? Is it doing backtracking search? How do you add backtracking search to GHC's inference algorithm? Etc. The later examples are designed to raise questions about this, too. I'm rather of the opinion that TDNR just isn't a good feature for most things. Implicit in the famous "How to Make Ad-hoc Polymorphism Less Ad-hoc" is that being ad-hoc is bad. And type classes fix that by turning overloading into something that happens via an agreed upon interface, with declared conventions, and which can be abstracted over well. But TDNR has none of that, so to my mind, it's not really desirable, except possibly in cases where there is no hope of being abstract (for instance, Agda does some TDNR for constructors in patterns, and there isn't much basis in the underlying theory for trying to abstract over doing induction on completely different types with similarly named pieces; so it's more acceptable). But also, for something as far reaching as doing TDNR for every ambiguous name, it's not terribly clear to me what a good algorithm even is, unless it's only good enough to handle really simple examples, and just doesn't work most of the time (and even then, I'm not sure it's been thought through enough to say that it's a simple addition to GHC's type checking). -- Dan From emax at chalmers.se Thu May 26 15:27:00 2016 From: emax at chalmers.se (Emil Axelsson) Date: Thu, 26 May 2016 17:27:00 +0200 Subject: Pattern synonyms and GADTs in GHC 8.0.1 Message-ID: <574715C4.8040900@chalmers.se> I have a problem where a pattern synonym doesn't provide the expected type refinement in GHC 8.0.1. > {-# LANGUAGE GADTs #-} > {-# LANGUAGE PatternSynonyms #-} > > data Exp a > where > Num :: (Eq a, Num a) => a -> Exp a > Add :: (Eq a, Num a) => Exp a -> Exp a -> Exp a > > pattern NumP a = Num a > > pattern AddP :: (Num a, Eq a) => Exp a -> Exp a -> Exp a > pattern AddP a b = Add a b > > simplifyP :: Exp a -> Exp a > simplifyP (AddP a (NumP 0)) = a > simplifyP a = a This gives the error ? No instance for (Eq a) arising from a pattern Possible fix: add (Eq a) to the context of the type signature for: simplifyP :: Exp a -> Exp a ? In the pattern: AddP a (NumP 0) In an equation for ?simplifyP?: simplifyP (AddP a (NumP 0)) = a If I remove the type signature for `AddP`, the code goes through. Unfortunately, in my real code I need the type signature in order to resolve overloading. GHC 7.10 didn't have this problem. Is this a bug? / Emil From simonpj at microsoft.com Thu May 26 16:46:22 2016 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Thu, 26 May 2016 16:46:22 +0000 Subject: Pattern synonyms and GADTs in GHC 8.0.1 In-Reply-To: <574715C4.8040900@chalmers.se> References: <574715C4.8040900@chalmers.se> Message-ID: GHC 8.0 swaps the order of provided vs required contexts in a pattern synonym signature. (7.10 was advertised as experimental). Thus: pattern AddP :: () => (Num a, Eq a) => Exp a -> Exp a -> Exp a Then it's fine Simon | -----Original Message----- | From: Glasgow-haskell-users [mailto:glasgow-haskell-users- | bounces at haskell.org] On Behalf Of Emil Axelsson | Sent: 26 May 2016 16:27 | To: glasgow-haskell-users | Subject: Pattern synonyms and GADTs in GHC 8.0.1 | | I have a problem where a pattern synonym doesn't provide the expected | type refinement in GHC 8.0.1. | | > {-# LANGUAGE GADTs #-} | > {-# LANGUAGE PatternSynonyms #-} | > | > data Exp a | > where | > Num :: (Eq a, Num a) => a -> Exp a | > Add :: (Eq a, Num a) => Exp a -> Exp a -> Exp a | > | > pattern NumP a = Num a | > | > pattern AddP :: (Num a, Eq a) => Exp a -> Exp a -> Exp a pattern | AddP | > a b = Add a b | > | > simplifyP :: Exp a -> Exp a | > simplifyP (AddP a (NumP 0)) = a | > simplifyP a = a | | This gives the error | | ? No instance for (Eq a) arising from a pattern | Possible fix: | add (Eq a) to the context of | the type signature for: | simplifyP :: Exp a -> Exp a | ? In the pattern: AddP a (NumP 0) | In an equation for ?simplifyP?: simplifyP (AddP a (NumP 0)) = a | | If I remove the type signature for `AddP`, the code goes through. | Unfortunately, in my real code I need the type signature in order to | resolve overloading. | | GHC 7.10 didn't have this problem. | | Is this a bug? | | / Emil | _______________________________________________ | Glasgow-haskell-users mailing list | Glasgow-haskell-users at haskell.org | https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fmail.h | askell.org%2fcgi-bin%2fmailman%2flistinfo%2fglasgow-haskell- | users&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com%7ccb54f042472240 | 7ed99608d3857a317a%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=xv8M7C | IC4zyT4Zgq3HnXiGzUA1Z0tltZpE%2fIYhYP8KQ%3d From emax at chalmers.se Thu May 26 18:59:51 2016 From: emax at chalmers.se (Emil Axelsson) Date: Thu, 26 May 2016 20:59:51 +0200 Subject: Pattern synonyms and GADTs in GHC 8.0.1 In-Reply-To: References: <574715C4.8040900@chalmers.se> Message-ID: <574747A7.1060509@chalmers.se> Ah, excellent! Thank you! However, it seems that `:t` gives the wrong type: *Main> :t AddP AddP :: (Num a, Eq a) => Exp a -> Exp a -> Exp a This type is reported whether or not I include the (correct) signature for `AddP`. `:i` is correct though: *Main> :i AddP pattern AddP :: () => (Num a, Eq a) => Exp a -> Exp a -> Exp a / Emil Den 2016-05-26 kl. 18:46, skrev Simon Peyton Jones: > GHC 8.0 swaps the order of provided vs required contexts in a pattern synonym signature. (7.10 was advertised as experimental). Thus: > > pattern AddP :: () => (Num a, Eq a) => Exp a -> Exp a -> Exp a > > > Then it's fine > > Simon > > | -----Original Message----- > | From: Glasgow-haskell-users [mailto:glasgow-haskell-users- > | bounces at haskell.org] On Behalf Of Emil Axelsson > | Sent: 26 May 2016 16:27 > | To: glasgow-haskell-users > | Subject: Pattern synonyms and GADTs in GHC 8.0.1 > | > | I have a problem where a pattern synonym doesn't provide the expected > | type refinement in GHC 8.0.1. > | > | > {-# LANGUAGE GADTs #-} > | > {-# LANGUAGE PatternSynonyms #-} > | > > | > data Exp a > | > where > | > Num :: (Eq a, Num a) => a -> Exp a > | > Add :: (Eq a, Num a) => Exp a -> Exp a -> Exp a > | > > | > pattern NumP a = Num a > | > > | > pattern AddP :: (Num a, Eq a) => Exp a -> Exp a -> Exp a pattern > | AddP > | > a b = Add a b > | > > | > simplifyP :: Exp a -> Exp a > | > simplifyP (AddP a (NumP 0)) = a > | > simplifyP a = a > | > | This gives the error > | > | ? No instance for (Eq a) arising from a pattern > | Possible fix: > | add (Eq a) to the context of > | the type signature for: > | simplifyP :: Exp a -> Exp a > | ? In the pattern: AddP a (NumP 0) > | In an equation for ?simplifyP?: simplifyP (AddP a (NumP 0)) = a > | > | If I remove the type signature for `AddP`, the code goes through. > | Unfortunately, in my real code I need the type signature in order to > | resolve overloading. > | > | GHC 7.10 didn't have this problem. > | > | Is this a bug? > | > | / Emil > | _______________________________________________ > | Glasgow-haskell-users mailing list > | Glasgow-haskell-users at haskell.org > | https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fmail.h > | askell.org%2fcgi-bin%2fmailman%2flistinfo%2fglasgow-haskell- > | users&data=01%7c01%7csimonpj%40064d.mgd.microsoft.com%7ccb54f042472240 > | 7ed99608d3857a317a%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=xv8M7C > | IC4zyT4Zgq3HnXiGzUA1Z0tltZpE%2fIYhYP8KQ%3d > From emax at chalmers.se Thu May 26 19:21:17 2016 From: emax at chalmers.se (Emil Axelsson) Date: Thu, 26 May 2016 21:21:17 +0200 Subject: Pattern synonyms and GADTs in GHC 8.0.1 In-Reply-To: <574747A7.1060509@chalmers.se> References: <574715C4.8040900@chalmers.se> <574747A7.1060509@chalmers.se> Message-ID: <57474CAD.5090301@chalmers.se> Aha, that's because `:t` operates on expressions, and when a pattern synonym is used as an expression the required and provided contexts are merged into one. Makes sense. / Emil Den 2016-05-26 kl. 20:59, skrev Emil Axelsson: > However, it seems that `:t` gives the wrong type: > > *Main> :t AddP > AddP :: (Num a, Eq a) => Exp a -> Exp a -> Exp a > > This type is reported whether or not I include the (correct) signature > for `AddP`. > > `:i` is correct though: > > *Main> :i AddP > pattern AddP :: () => (Num a, Eq a) => Exp a -> Exp a -> Exp a > > / Emil From juhpetersen at gmail.com Fri May 27 09:09:34 2016 From: juhpetersen at gmail.com (Jens Petersen) Date: Fri, 27 May 2016 18:09:34 +0900 Subject: [ANNOUNCE] GHC 8.0.1 is available! In-Reply-To: <8737pbxwvo.fsf@smart-cactus.org> References: <8737pbxwvo.fsf@smart-cactus.org> Message-ID: On 22 May 2016 at 00:18, Ben Gamari wrote: > The GHC developers are very pleased to announce the release of the first > new super-major version of our Haskell compiler in six years, GHC 8.0.1. > Congratulations on the release! I forgot to mention I built it for Fedora and RHEL/Centos 7 in my Copr repo: https://copr.fedorainfracloud.org/coprs/petersen/ghc-8.0.1/ If you run into any problems please report them to me. I also made a Fedora docker image with 8.0.1 tag that includes also cabal-install and stack: https://hub.docker.com/r/juhp/fedora-haskell-ghc/ Jens -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthony_clayden at clear.net.nz Sat May 28 05:33:19 2016 From: anthony_clayden at clear.net.nz (AntC) Date: Sat, 28 May 2016 05:33:19 +0000 (UTC) Subject: TDNR without new operators or syntax changes References: <1463402347025-5835927.post@n5.nabble.com> <20160522124128.GB1592@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> <1463929267673-5836393.post@n5.nabble.com> <1464254085674-5836657.post@n5.nabble.com> Message-ID: > Dan Doel gmail.com> writes: > >> On Thu, May 26, 2016 at 5:14 AM, Peter hotmail.com> wrote: >> Solving for everything but f, we get f :: T -> Int. > > So TDNR happens for things in function position (applied to something). Before we get carried away, TDNR doesn't happen at all. You're speculating about what might be able to happen? > > Solving for everything but f, we get f :: T -> Int. > > So TDNR happens for things in argument position. [By "things" there you mean functions.] TDNR as originally spec'd wouldn't try to solve this. (Because there's no dot-postfix invocation.) And my reading of DuplicateRecordFields is that won't either. There's a good reason. We're trying to solve for record field accessors. So unlike Dan's examples, there's a bunch of same-named functions: personId :: T -> Int personId :: U -> Int personId :: V -> Int personId's are always Ints. There's no point trying to work 'outside in', because it won't disambiguate anythng. > > May not be solvable, would fail to disambiguate. > > But there's exactly one combination of f and v definitions that will > succeed with the right type. So why doesn't that happen? Because in general you have to run the whole of type inference to solve this case. (That's Peter's step 2. in his earlier message.) But we can't run type inference until we've disambiguated all names. Chicken and egg. > ... Another way to phrase the question is: why would > TDNR only disambiguate based on argument types of functions > and not on return types? ... Because, per above, the return types are all the same (for same-named field accessors). > ... Is it doing backtracking search? > How do you add backtracking search to GHC's inference algorithm? Etc. No GHC does not now do backtracking search. No it couldn't be somehow bolted on. There's no guarantee that adding backtracking could resolve any more cases that can be solved now, because now we have hugely powerful inference honed for decades, and type system design to exploit it. > ... And type classes fix that by > turning overloading into something that happens via an agreed upon > interface, with declared conventions, and which can be abstracted over > well. ... Yes, so the full program for ORF is to make 'Magic Type Classes' seem to the type inferencer like regular class-based overloading. > But also, for something as far reaching as doing TDNR for every > ambiguous name, it's not terribly clear to me what a good algorithm > even is, unless it's only good enough to handle really simple > examples, and just doesn't work most of the time ... DuplicateRecordFields is aiming for the simple examples. If some case isn't simple enough, you can always add signatures until it is. AntC From qdunkan at gmail.com Sat May 28 06:47:56 2016 From: qdunkan at gmail.com (Evan Laforge) Date: Fri, 27 May 2016 23:47:56 -0700 Subject: TDNR without new operators or syntax changes In-Reply-To: References: <1463402347025-5835927.post@n5.nabble.com> <20160522124128.GB1592@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> <1463929267673-5836393.post@n5.nabble.com> <1464254085674-5836657.post@n5.nabble.com> Message-ID: As long as were back on this topic again (sort of), and just to satisfy my curiousity, what would happen if you tried to do records just like C structs? So e.g. a?b requires 'a' to be a record with a 'b' field, and is just one identifier, no functions involved, and 'b' is not a separate value. I could see how people might think it was not powerful enough or not very idiomatic, but is there something about haskell that would make it just not work? I didn't see that option on the increasingly all-encompassing Records wiki page. On Fri, May 27, 2016 at 10:33 PM, AntC wrote: >> Dan Doel gmail.com> writes: >> >>> On Thu, May 26, 2016 at 5:14 AM, Peter hotmail.com> wrote: >>> Solving for everything but f, we get f :: T -> Int. >> >> So TDNR happens for things in function position (applied to something). > > Before we get carried away, TDNR doesn't happen at all. > You're speculating about what might be able to happen? > >> > Solving for everything but f, we get f :: T -> Int. >> >> So TDNR happens for things in argument position. > > [By "things" there you mean functions.] > > TDNR as originally spec'd wouldn't try to solve this. > (Because there's no dot-postfix invocation.) > And my reading of DuplicateRecordFields is that won't either. > > There's a good reason. We're trying to solve for record field accessors. > So unlike Dan's examples, there's a bunch of same-named functions: > personId :: T -> Int > personId :: U -> Int > personId :: V -> Int > > personId's are always Ints. There's no point trying to work 'outside in', > because it won't disambiguate anythng. > >> > May not be solvable, would fail to disambiguate. >> >> But there's exactly one combination of f and v definitions that will >> succeed with the right type. So why doesn't that happen? > > Because in general you have to run the whole of type inference > to solve this case. > (That's Peter's step 2. in his earlier message.) > But we can't run type inference until we've disambiguated all names. > Chicken and egg. > >> ... Another way to phrase the question is: why would >> TDNR only disambiguate based on argument types of functions >> and not on return types? ... > > Because, per above, the return types are all the same > (for same-named field accessors). > >> ... Is it doing backtracking search? >> How do you add backtracking search to GHC's inference algorithm? Etc. > > No GHC does not now do backtracking search. > No it couldn't be somehow bolted on. > There's no guarantee that adding backtracking > could resolve any more cases that can be solved now, > because now we have hugely powerful inference honed for decades, > and type system design to exploit it. > >> ... And type classes fix that by >> turning overloading into something that happens via an agreed upon >> interface, with declared conventions, and which can be abstracted over >> well. ... > > Yes, so the full program for ORF is to make 'Magic Type Classes' > seem to the type inferencer like regular class-based overloading. > >> But also, for something as far reaching as doing TDNR for every >> ambiguous name, it's not terribly clear to me what a good algorithm >> even is, unless it's only good enough to handle really simple >> examples, and just doesn't work most of the time ... > > DuplicateRecordFields is aiming for the simple examples. > If some case isn't simple enough, > you can always add signatures until it is. > > > AntC > > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users From anthony_clayden at clear.net.nz Sat May 28 10:13:43 2016 From: anthony_clayden at clear.net.nz (AntC) Date: Sat, 28 May 2016 10:13:43 +0000 (UTC) Subject: TDNR without new operators or syntax changes References: <1463402347025-5835927.post@n5.nabble.com> <20160522124128.GB1592@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> <1463929267673-5836393.post@n5.nabble.com> <1464254085674-5836657.post@n5.nabble.com> Message-ID: > Evan Laforge gmail.com> writes: > ... what would happen if you tried to do records > just like C structs? So e.g. a?b requires 'a' to be a record with a > 'b' field, and is just one identifier, no functions involved, and 'b' > is not a separate value. Hi Evan, um, that's the original TDNR isn't it? http://hackage.haskell.org/trac/haskell-prime/wiki/ TypeDirectedNameResolution As per Jeremy's ref in the first post in this thread. If you're talking C then that blobby thing between a and b should be a dot(?) as postfix operator. And it is an operator, not "just one identifier". Then if you intend that dot bind tighter than function apply, but not as tight as Module name prefixes, you have the original TDNR. If you intend some operator symbol other than dot, there's a risk existing code is already using it, and/or it's not ASCII. And you're introducing an operator with very non-standard binding rules. (The reason SPJ chose dot, apart from that following other languages, is there's already a precedent for it being tight-binding in Module prefixes.) > > I didn't see that option on the increasingly all-encompassing > Records wiki page. > It's linked as Solution 2 on that records page, called "TDNR". Surely there can't be any records approaches that haven't been aired by now? And GHC's direction is set. AntC From voldermort at hotmail.com Sat May 28 20:11:45 2016 From: voldermort at hotmail.com (Peter) Date: Sat, 28 May 2016 13:11:45 -0700 (MST) Subject: TDNR without new operators or syntax changes In-Reply-To: References: <1463402347025-5835927.post@n5.nabble.com> <20160522124128.GB1592@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> <1463929267673-5836393.post@n5.nabble.com> <1464254085674-5836657.post@n5.nabble.com> Message-ID: <1464466305963-5836791.post@n5.nabble.com> A slightly more refined definition for disambiguation: 1) If a type signature has been supplied for an ambiguous name, GHC will attempt to disambiguate with the type signature alone. 2) If the name is a function applied to an explicit argument, and the type of the argument can be inferred without disambiguating anything (or is supplied as a type signature), then GCH will attempt to disambiguate by matching the type of the argument. This is sufficient to cover all of the motivating examples proposed for TDNR, and every case where I have personally wanted it. It does not require backtracing (although it may require two passes to disambiguate on the inferred type of an argument), and disambiguating one name will not depend on disambiguating any other name. -- View this message in context: http://haskell.1045720.n5.nabble.com/TDNR-without-new-operators-or-syntax-changes-tp5835927p5836791.html Sent from the Haskell - Glasgow-haskell-users mailing list archive at Nabble.com. From qdunkan at gmail.com Sat May 28 21:14:27 2016 From: qdunkan at gmail.com (Evan Laforge) Date: Sat, 28 May 2016 14:14:27 -0700 Subject: TDNR without new operators or syntax changes In-Reply-To: References: <1463402347025-5835927.post@n5.nabble.com> <20160522124128.GB1592@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> <1463929267673-5836393.post@n5.nabble.com> <1464254085674-5836657.post@n5.nabble.com> Message-ID: On Sat, May 28, 2016 at 3:13 AM, AntC wrote: >> Evan Laforge gmail.com> writes: > >> ... what would happen if you tried to do records >> just like C structs? So e.g. a?b requires 'a' to be a record with a >> 'b' field, and is just one identifier, no functions involved, and 'b' >> is not a separate value. > > Hi Evan, um, that's the original TDNR isn't it? > http://hackage.haskell.org/trac/haskell-prime/wiki/ > TypeDirectedNameResolution > As per Jeremy's ref in the first post in this thread. > > If you're talking C then that blobby thing between a and b > should be a dot(?) as postfix operator. > And it is an operator, not "just one identifier". That's why I was trying to emphasize "not an operator". TDNR is complicated is because the field name is a first class value, so it's a function applied to a record, and now we have to disambiguate the function. I didn't meant that. I guess C (and Java and all the others) does call "." an operator because it parses as one and has precedence, but it works with identifiers, not values, and has special grammar. It's not an operator from the haskell point of view. The C compiler isn't doing any overloading or disambiguation. As far as I understand it the implementation is straightforward. Could ghc do something similarly straightforward? It already does for qualified module names, and in the same way in M.a, ".a" is not a function that gets something out of an M. From anthony_clayden at clear.net.nz Sun May 29 00:42:19 2016 From: anthony_clayden at clear.net.nz (AntC) Date: Sun, 29 May 2016 00:42:19 +0000 (UTC) Subject: TDNR without new operators or syntax changes References: <1463402347025-5835927.post@n5.nabble.com> <20160522124128.GB1592@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> <1463929267673-5836393.post@n5.nabble.com> <1464254085674-5836657.post@n5.nabble.com> Message-ID: > Evan Laforge gmail.com> writes: > > That's why I was trying to emphasize "not an operator". > TDNR is complicated because ... >> Peter voldermort writes: >> A slightly more refined definition for disambiguation: ... Hi Evan, Peter, (and even James), I'm not seeing you're proposing anything that's significantly different to DuplicateRecordFields. That has the advantage we can use it now. If you are proposing something different, you need to explain in a lot more detail, so that we can see the advantages. So [ref Evan] even though a field name is a first-class function usually, DuplicateRecordFields only gets triggered where you use the bare name. [Ref Peter] I'm not seeing why you're talking about two passes, but that does not sound like a robust approach. (Can you be sure two passes is enough? If it is enough, why can't the second pass's logic get built into the first?) Yes DuplicateRecordFIelds is a bit of a hack. The proper solution (MagicTypeClasses) is still awaited. Can you explain why GHC should depart from that plan? AntC From voldermort at hotmail.com Sun May 29 07:38:25 2016 From: voldermort at hotmail.com (Peter) Date: Sun, 29 May 2016 00:38:25 -0700 (MST) Subject: TDNR without new operators or syntax changes In-Reply-To: References: <1463929267673-5836393.post@n5.nabble.com> <1464254085674-5836657.post@n5.nabble.com> Message-ID: <1464507505032-5836818.post@n5.nabble.com> AntC wrote > I'm not seeing you're proposing anything that's significantly different > to DuplicateRecordFields. That has the advantage we can use it now. > > If you are proposing something different, you need to explain > in a lot more detail, so that we can see the advantages. > > So [ref Evan] even though a field name is a first-class function usually, > DuplicateRecordFields only gets triggered where you use the bare name. > > [Ref Peter] I'm not seeing why you're talking about two passes, > but that does not sound like a robust approach. > (Can you be sure two passes is enough? > If it is enough, why can't the second pass's logic > get built into the first?) DRF only works for records, but brings polymorphism, duplicate definitions in the same module, and other goodies. This proposal will work for any function, but only helps with disambiguation. There is some overlap, but it is a different solution for different requirements. -- View this message in context: http://haskell.1045720.n5.nabble.com/TDNR-without-new-operators-or-syntax-changes-tp5835927p5836818.html Sent from the Haskell - Glasgow-haskell-users mailing list archive at Nabble.com.