From mail at joachim-breitner.de Fri Sep 1 11:47:03 2017 From: mail at joachim-breitner.de (Joachim Breitner) Date: Fri, 01 Sep 2017 12:47:03 +0100 Subject: How to get a heap visualization In-Reply-To: References: Message-ID: <1504266423.2496.2.camel@joachim-breitner.de> Hi, author of ghc-heap-view here. Am Mittwoch, den 30.08.2017, 18:34 +0300 schrieb Yitzchak Gale: > Getting ghc-vis to compile looks hopeless, for a number of reasons. > The dependencies on gtk and cairo are huge. Is that really a problem? > It hasn't been updated > on Hackage for a year and a half. It requires base < 4.9. GitHub is already ahead. I guess this just needs to be uploaded? https://github.com/def-/ghc-vis/blob/master/ghc-vis.cabal > I need to run > the visualizer either on a headless Ubuntu 16.04 server, or locally on > Windows. Ok, that is more tricky. > The heap scraper backend for ghc-vis, ghc-heap-view, looks usable, > and better supported than vacuum. But is there a quick and simple > visualizer for its output, without ghc-vis? Well, the :printHeap command that comes with it does “visualize” things as something resembling Haskell syntax: let x1 = "A Value"     x16 = True : False : x16 in (x1,x1,x16) I don’t know of anything more graphical besides ghc-vis, but you could roll your own, if you want to; you can use ghc-heap-view to get a graph using http://hackage.haskell.org/package/ghc-heap-view-0.5.9/docs/GHC-HeapView.html#v:buildHeapGraph and then visualize that as you like. Greetings, Joachim -- Joachim “nomeata” Breitner mail at joachim-breitner.de https://www.joachim-breitner.de/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From gale at sefer.org Sun Sep 3 16:43:00 2017 From: gale at sefer.org (Yitzchak Gale) Date: Sun, 3 Sep 2017 19:43:00 +0300 Subject: How to get a heap visualization In-Reply-To: <1504266423.2496.2.camel@joachim-breitner.de> References: <1504266423.2496.2.camel@joachim-breitner.de> Message-ID: Joachim, first and foremost, thanks for the awesome libraries ghc-vis and ghc-heap-view. The design trade-offs for ghc-vis do make sense if you think of it as a didactic tool. But as a debugging tool, the most important factor is that it should "Just Work", with no big builds, no fiddling, no googling. When you reach for a debugging tool, you are *already* in a debugging situation; you don't have patience for also debugging the tool. I wrote: >> Getting ghc-vis to compile looks hopeless, for a number of >> reasons..The dependencies on gtk and cairo are huge. You wrote: > Is that really a problem? Admittedly I haven't tried in quite a while. In the past, getting all the Haskell libraries and C libraries to match up and work together on any given platform at matching versions was a huge task. If nowadays you can just type "stack build" or "cabal install" (with new-build or in a sandbox) on even one platform and have it just work, reliably every time, month after month, that would be an amazing feat. But in my case, I need even more than that. I need it to "Just Work" on certain specific platforms, none of which have ever been known as the best for running GTK: Windows, and headless Ubuntu. >> The heap scraper backend for ghc-vis, ghc-heap-view, >> looks usable, and better supported than vacuum. >> But is there a quick and simple >> visualizer for its output, without ghc-vis? > Well, the :printHeap command that comes with it does > “visualize” things as something resembling Haskell syntax... > I don’t know of anything more graphical besides ghc-vis, > but you could roll your own, if you want to; you can use > ghc-heap-view to get a graph using buildHeapGraph > and then visualize that as you like. Yes, this is the approach I will probably take. I'll start with just printing the output into a text file. Maybe I can massage that and find what I need. Thanks, Yitz From conal at conal.net Fri Sep 15 01:45:09 2017 From: conal at conal.net (Conal Elliott) Date: Thu, 14 Sep 2017 18:45:09 -0700 Subject: Inhibiting the specialiser? Message-ID: Is there a GHC flag for inhibiting the specializer (but not all optimizations)? I'm seeing huge output from the Specialise phase killed at 4GB and growing. The output starts as follows: Result size of Specialise = {terms: 29,639, types: 10,921,552, coercions: 4,425,185} Sounds like a lot to me. Is it?. I get this behavior with -O2 and with -O. -- Conal -------------- next part -------------- An HTML attachment was scrubbed... URL: From harendra.kumar at gmail.com Fri Sep 15 03:30:44 2017 From: harendra.kumar at gmail.com (Harendra Kumar) Date: Fri, 15 Sep 2017 09:00:44 +0530 Subject: Inhibiting the specialiser? In-Reply-To: References: Message-ID: As per the GHC manual, it should be -fno-specialise for disabling all specialization, and -fno-cross-module-specialise for disabling only the specialization of imported INLINABLE functions. Both of these flags are "on" when using -O and -O2. -harendra On 15 September 2017 at 07:15, Conal Elliott wrote: > Is there a GHC flag for inhibiting the specializer (but not all > optimizations)? I'm seeing huge output from the Specialise phase killed at > 4GB and growing. The output starts as follows: > > Result size of Specialise > = {terms: 29,639, types: 10,921,552, coercions: 4,425,185} > > Sounds like a lot to me. Is it?. I get this behavior with -O2 and with -O. > > -- Conal > > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Fri Sep 15 10:23:34 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Fri, 15 Sep 2017 10:23:34 +0000 Subject: Inhibiting the specialiser? In-Reply-To: References: Message-ID: Did you try -fno-specialise? From: Glasgow-haskell-users [mailto:glasgow-haskell-users-bounces at haskell.org] On Behalf Of Conal Elliott Sent: 15 September 2017 02:45 To: glasgow-haskell-users at haskell.org Subject: Inhibiting the specialiser? Is there a GHC flag for inhibiting the specializer (but not all optimizations)? I'm seeing huge output from the Specialise phase killed at 4GB and growing. The output starts as follows: Result size of Specialise = {terms: 29,639, types: 10,921,552, coercions: 4,425,185} Sounds like a lot to me. Is it?. I get this behavior with -O2 and with -O. -- Conal -------------- next part -------------- An HTML attachment was scrubbed... URL: From conal at conal.net Fri Sep 15 14:41:20 2017 From: conal at conal.net (Conal Elliott) Date: Fri, 15 Sep 2017 07:41:20 -0700 Subject: Inhibiting the specialiser? In-Reply-To: References: Message-ID: Thanks! Exactly what I was looking for. On Thu, Sep 14, 2017 at 8:30 PM, Harendra Kumar wrote: > As per the GHC manual, it should be -fno-specialise for disabling all > specialization, and -fno-cross-module-specialise for disabling only the > specialization of imported INLINABLE functions. Both of these flags are > "on" when using -O and -O2. > > -harendra > > On 15 September 2017 at 07:15, Conal Elliott wrote: > >> Is there a GHC flag for inhibiting the specializer (but not all >> optimizations)? I'm seeing huge output from the Specialise phase killed at >> 4GB and growing. The output starts as follows: >> >> Result size of Specialise >> = {terms: 29,639, types: 10,921,552, coercions: 4,425,185} >> >> Sounds like a lot to me. Is it?. I get this behavior with -O2 and with -O. >> >> -- Conal >> >> _______________________________________________ >> Glasgow-haskell-users mailing list >> Glasgow-haskell-users at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From conal at conal.net Fri Sep 15 14:41:47 2017 From: conal at conal.net (Conal Elliott) Date: Fri, 15 Sep 2017 07:41:47 -0700 Subject: Inhibiting the specialiser? In-Reply-To: References: Message-ID: Thanks. I somehow didn't notice that flag. On Fri, Sep 15, 2017 at 3:23 AM, Simon Peyton Jones wrote: > Did you try -fno-specialise? > > > > *From:* Glasgow-haskell-users [mailto:glasgow-haskell-users- > bounces at haskell.org] *On Behalf Of *Conal Elliott > *Sent:* 15 September 2017 02:45 > *To:* glasgow-haskell-users at haskell.org > *Subject:* Inhibiting the specialiser? > > > > Is there a GHC flag for inhibiting the specializer (but not all > optimizations)? I'm seeing huge output from the Specialise phase killed at > 4GB and growing. The output starts as follows: > > > > Result size of Specialise > > = {terms: 29,639, types: 10,921,552, coercions: 4,425,185} > > > > Sounds like a lot to me. Is it?. I get this behavior with -O2 and with -O. > > > > -- Conal > -------------- next part -------------- An HTML attachment was scrubbed... URL: From conal at conal.net Mon Sep 18 20:05:09 2017 From: conal at conal.net (Conal Elliott) Date: Mon, 18 Sep 2017 13:05:09 -0700 Subject: Spurious recompilations when using a compiler plugin Message-ID: It appears that use of GHC plugins causes client code to get needlessly recompiled. (See Trac issues 12567 and 7414 .) It’s becoming more of a problem for usability of the plugin I’ve been developing, and I’m wondering what can be done. Some questions: - Is there any work in progress on fixing this situation? - Are there serious obstacles to fixing it? - Do plugin writers or users have any workarounds? Other insights appreciated. – Conal -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at smart-cactus.org Tue Sep 19 12:59:54 2017 From: ben at smart-cactus.org (Ben Gamari) Date: Tue, 19 Sep 2017 08:59:54 -0400 Subject: Spurious recompilations when using a compiler plugin In-Reply-To: References: Message-ID: <87tvzyyhbp.fsf@ben-laptop.smart-cactus.org> Conal Elliott writes: > It appears that use of GHC plugins causes client code to get needlessly > recompiled. (See Trac issues 12567 > and 7414 > .) It’s becoming more of a > problem for usability of the plugin > I’ve been developing, and > I’m wondering what can be done. Some questions: > > - Is there any work in progress on fixing this situation? > - Are there serious obstacles to fixing it? > - Do plugin writers or users have any workarounds? > I think the real question is what sort of interface do plugin authors need? I suspect there are a few distinct tasks here, * compute and record module implementation hashes in interface files * to include plugin implementation hashes in the recompilation check * to provide an interface allowing a plugin to compute a hash of its arguments which can be included into the recompilation check. One way of realising this would be to add a field like the following to Plugin, pluginHash :: [CommandLineOption] -> Maybe Fingerprint -- Nothing would denote "always rebuild" Would this help in your case? This would allow us to fix the TH problem in #7277 and fix the plugins problem in #7414 and #12567 in a nearly optimal way (assuming the plugin author is able to precisely define a hash). None of this is terribly difficult and given Nick's recent work on his row types plugin, it seems like it's getting more urgent. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From george.colpitts at gmail.com Tue Sep 19 14:06:41 2017 From: george.colpitts at gmail.com (George Colpitts) Date: Tue, 19 Sep 2017 14:06:41 +0000 Subject: Spurious recompilations when using a compiler plugin In-Reply-To: <87tvzyyhbp.fsf@ben-laptop.smart-cactus.org> References: <87tvzyyhbp.fsf@ben-laptop.smart-cactus.org> Message-ID: is it possible that this is also connected to https://ghc.haskell.org/trac/ghc/ticket/13604 ? On Tue, Sep 19, 2017 at 10:00 AM Ben Gamari wrote: > Conal Elliott writes: > > > It appears that use of GHC plugins causes client code to get needlessly > > recompiled. (See Trac issues 12567 > > and 7414 > > .) It’s becoming more of a > > problem for usability of the plugin > > I’ve been developing, > and > > I’m wondering what can be done. Some questions: > > > > - Is there any work in progress on fixing this situation? > > - Are there serious obstacles to fixing it? > > - Do plugin writers or users have any workarounds? > > > I think the real question is what sort of interface do plugin authors > need? > > I suspect there are a few distinct tasks here, > > * compute and record module implementation hashes in interface files > > * to include plugin implementation hashes in the recompilation check > > * to provide an interface allowing a plugin to compute a hash of its > arguments which can be included into the recompilation check. One way > of realising this would be to add a field like the following to Plugin, > > pluginHash :: [CommandLineOption] -> Maybe Fingerprint > -- Nothing would denote "always rebuild" > > Would this help in your case? > > This would allow us to fix the TH problem in #7277 and fix the plugins > problem in #7414 and #12567 in a nearly optimal way (assuming the plugin > author is able to precisely define a hash). > > None of this is terribly difficult and given Nick's recent work on his > row types plugin, it seems like it's getting more urgent. > > Cheers, > > - Ben > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at smart-cactus.org Tue Sep 19 15:34:17 2017 From: ben at smart-cactus.org (Ben Gamari) Date: Tue, 19 Sep 2017 11:34:17 -0400 Subject: Spurious recompilations when using a compiler plugin In-Reply-To: References: <87tvzyyhbp.fsf@ben-laptop.smart-cactus.org> Message-ID: <87lglaya6e.fsf@ben-laptop.smart-cactus.org> George Colpitts writes: > is it possible that this is also connected to > https://ghc.haskell.org/trac/ghc/ticket/13604 ? > In that it pertains to the recompilation checker, yes. I've tried to collect the recompilation checker bugs in play here under the RecompilationCheck keyword in Trac. It would be great to get some eyes on these issues as they are clearly causing real pain for some users. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at well-typed.com Wed Sep 20 02:09:42 2017 From: ben at well-typed.com (Ben Gamari) Date: Tue, 19 Sep 2017 22:09:42 -0400 Subject: How to get a heap visualization In-Reply-To: References: <1504266423.2496.2.camel@joachim-breitner.de> Message-ID: <8760cexgrd.fsf@ben-laptop.smart-cactus.org> Yitzchak Gale writes: > Joachim, first and foremost, thanks for the awesome libraries > ghc-vis and ghc-heap-view. > > The design trade-offs for ghc-vis do make sense if you think of > it as a didactic tool. But as a debugging tool, the most important > factor is that it should "Just Work", with no big builds, no fiddling, > no googling. When you reach for a debugging tool, you are > *already* in a debugging situation; you don't have patience > for also debugging the tool. > I should point out that there is a differential (D3055) by erikd from earlier this year which folded ghc-heap-view into GHC. The goal was to make heap-view a supported library which could be relied upon by third party tools. As far as I can remember most of the hard work is done; it just needed some finishing touches. Perhaps you would be interested in picking it up? Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From mle+hs at mega-nerd.com Wed Sep 20 02:37:04 2017 From: mle+hs at mega-nerd.com (Erik de Castro Lopo) Date: Wed, 20 Sep 2017 12:37:04 +1000 Subject: How to get a heap visualization In-Reply-To: <8760cexgrd.fsf@ben-laptop.smart-cactus.org> References: <1504266423.2496.2.camel@joachim-breitner.de> <8760cexgrd.fsf@ben-laptop.smart-cactus.org> Message-ID: <20170920123704.15c39ca003c95721f81c083c@mega-nerd.com> Ben Gamari wrote: > Yitzchak Gale writes: > > > Joachim, first and foremost, thanks for the awesome libraries > > ghc-vis and ghc-heap-view. > > > > The design trade-offs for ghc-vis do make sense if you think of > > it as a didactic tool. But as a debugging tool, the most important > > factor is that it should "Just Work", with no big builds, no fiddling, > > no googling. When you reach for a debugging tool, you are > > *already* in a debugging situation; you don't have patience > > for also debugging the tool. > > > I should point out that there is a differential (D3055) by erikd from > earlier this year which folded ghc-heap-view into GHC. The goal was to > make heap-view a supported library which could be relied upon by third > party tools. As far as I can remember most of the hard work is done; it > just needed some finishing touches. Perhaps you would be interested in > picking it up? Sorry if I gave the wrong impression, but IMO ghc-heap-view is a *long* way from being ready to integrate into GHC. ghc-heap-view currently supports most of the normal/simple heap object types. but has no support for more complex objects. Furthermore, levity polymorphism adds a whole new dimension to the problem. My estimate of the amount of work required to make ghc-heap-view cover all the heap objects was such that I abandoned the project. I would love to see someone pick it up and run with it. Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/ From ben at well-typed.com Wed Sep 20 17:36:37 2017 From: ben at well-typed.com (Ben Gamari) Date: Wed, 20 Sep 2017 13:36:37 -0400 Subject: How to get a heap visualization In-Reply-To: <20170920123704.15c39ca003c95721f81c083c@mega-nerd.com> References: <1504266423.2496.2.camel@joachim-breitner.de> <8760cexgrd.fsf@ben-laptop.smart-cactus.org> <20170920123704.15c39ca003c95721f81c083c@mega-nerd.com> Message-ID: <87zi9pw9ui.fsf@ben-laptop.smart-cactus.org> Erik de Castro Lopo writes: > Ben Gamari wrote: > >> I should point out that there is a differential (D3055) by erikd from >> earlier this year which folded ghc-heap-view into GHC. The goal was to >> make heap-view a supported library which could be relied upon by third >> party tools. As far as I can remember most of the hard work is done; it >> just needed some finishing touches. Perhaps you would be interested in >> picking it up? > > Sorry if I gave the wrong impression, but IMO ghc-heap-view is a *long* > way from being ready to integrate into GHC. ghc-heap-view currently supports > most of the normal/simple heap object types. but has no support for more > complex objects. Furthermore, levity polymorphism adds a whole new dimension > to the problem. My estimate of the amount of work required to make ghc-heap-view > cover all the heap objects was such that I abandoned the project. I would > love to see someone pick it up and run with it. > I'm not sure it's actually so far off. As far as I recall the big issues were, * teaching it to understand profiled and debugged info tables (annoying, but not difficult) * teaching it to understand unlifted fields As far as levity polymorphism is concerned, this shouldn't affect heap view at all as its simply a Core-level change; the STG machine and its associated heap layout doesn't change at all. It's easy for me to say having never worked on the patch, but I'd be surprised if it were more than a weekend away from completion. I should also mention that if you just need to poke around at the heap inside of gdb, you might consider using my GDB Python extension [1]. It's not terribly well documented, but I think it's pretty self-explanatory and I'm happy to add some documentation is there is demand. Cheers, - Ben [1] https://github.com/bgamari/ghc-utils/blob/master/ghc-gdb.py -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From trebla at vex.net Thu Sep 28 20:57:44 2017 From: trebla at vex.net (Albert Y. C. Lai) Date: Thu, 28 Sep 2017 16:57:44 -0400 Subject: Multiple C threads -> one unthreaded GHC RTS Message-ID: <0346f7e5-86be-6a03-b965-5e74b6987df9@vex.net> Suppose I have a C program with multiple C threads calling Haskell functions, maybe the same Haskell function too, and the GHC RTS used is the unthreaded one. What will go wrong? I ask because Cabal/cabal-install has recently added building of export-to-C shared libraries, i.e., the "foreign-library" section. It sees to all the technical details, very nice, I'm very happy about this addition, but it chooses the unthreaded GHC RTS. Thanks. From allbery.b at gmail.com Thu Sep 28 21:08:16 2017 From: allbery.b at gmail.com (Brandon Allbery) Date: Thu, 28 Sep 2017 17:08:16 -0400 Subject: Multiple C threads -> one unthreaded GHC RTS In-Reply-To: <0346f7e5-86be-6a03-b965-5e74b6987df9@vex.net> References: <0346f7e5-86be-6a03-b965-5e74b6987df9@vex.net> Message-ID: On Thu, Sep 28, 2017 at 4:57 PM, Albert Y. C. Lai wrote: > Suppose I have a C program with multiple C threads calling Haskell > functions, maybe the same Haskell function too, and the GHC RTS used is the > unthreaded one. What will go wrong? > > I ask because Cabal/cabal-install has recently added building of > export-to-C shared libraries, i.e., the "foreign-library" section. It sees > to all the technical details, very nice, I'm very happy about this > addition, but it chooses the unthreaded GHC RTS. > I don't expect the unthreaded RTS to be reentrant, so would expect you to need locking around calls into it to avoid the dreaded 'scheduler entered unsafely!' RTS error. And thread-local storage might be an issue. -- brandon s allbery kf8nh sine nomine associates allbery.b at gmail.com ballbery at sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net -------------- next part -------------- An HTML attachment was scrubbed... URL: